Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move issue pin to an standalone table for querying performance #33452

Draft
wants to merge 11 commits into
base: main
Choose a base branch
from

Conversation

lunny
Copy link
Member

@lunny lunny commented Jan 31, 2025

Noticed a SQL in gitea.com has a bigger load. It seems both is_pull and pin_order are not indexed columns in the database.

SELECT `id`, `repo_id`, `index`, `poster_id`, `original_author`, `original_author_id`, `name`, `content`, `content_version`, `milestone_id`, `priority`, `is_closed`, `is_pull`, `num_comments`, `ref`, `pin_order`, `deadline_unix`, `created_unix`, `updated_unix`, `closed_unix`, `is_locked`, `time_estimate` FROM `issue` WHERE (repo_id =?) AND (is_pull = 0) AND (pin_order > 0) ORDER BY pin_order

I came across a comment #24406 (comment) from @delvh , which presents a more reasonable approach. Based on this, this PR will migrate all issue and pull request pin data from the issue table to the issue_pin table. This change benefits larger Gitea instances by improving scalability and performance.

@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Jan 31, 2025
@pull-request-size pull-request-size bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Jan 31, 2025
@github-actions github-actions bot added modifies/api This PR adds API routes or modifies them modifies/go Pull requests that update Go code modifies/migrations labels Jan 31, 2025
@lunny lunny marked this pull request as ready for review February 3, 2025 21:14
@lunny lunny requested a review from delvh February 3, 2025 21:14
@GiteaBot GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Feb 5, 2025
@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Feb 6, 2025
@lunny lunny added the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Feb 6, 2025
@@ -341,6 +360,11 @@ func (issue *Issue) LoadAttributes(ctx context.Context) (err error) {
return issue.loadReactions(ctx)
}

// IsPinned returns if a Issue is pinned
func (issue *Issue) IsPinned() bool {
return issue.PinOrder != 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It needs to add an panic check: if pinOrderLoaded=false, panic to tell developers that there is a bug.

Copy link
Contributor

@wxiaoguang wxiaoguang Feb 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you'd like to use the convention that PinOrder > 0 means "pinned", you can use PinOrder=-1 means loaded, then no need to introduce another fragile pinOrderLoaded field. -1 also means non-existing, so it matches the meaning.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// Pin pins a Issue
func PinIssue(ctx context.Context, issue *Issue, user *user_model.User) error {
return db.WithTx(ctx, func(ctx context.Context) error {
pinnedIssuesNum, err := getPinnedIssuesNum(ctx, issue.RepoID, issue.IsPull)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changed logic is wrong.

It can't use count (from getPinnedIssuesNum) here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why it can't use count. getPinnedIssuesNum returned the records in the database which means how many issues have been pinned.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Order: 1, 2, 3
  • 2 is deleted
  • count() == 2, then pinnedIssuesNum + 1 == 3

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wxiaoguang wxiaoguang removed the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Feb 6, 2025
@wxiaoguang wxiaoguang marked this pull request as draft February 6, 2025 03:13
if _, err = db.GetEngine(ctx).Insert(&IssuePin{
RepoID: issue.RepoID,
IssueID: issue.ID,
IsPull: issue.IsPull,
Copy link
Contributor

@wxiaoguang wxiaoguang Feb 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see there is no default value for IsPull bool `xorm:"NOT NULL"`

Then if IsPull==false, will XORM generate the column for the SQL?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. modifies/api This PR adds API routes or modifies them modifies/go Pull requests that update Go code modifies/internal modifies/migrations size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants