Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some network cost of stats background worker can be avoided when infoschema v2 is enabled #59104

Open
winoros opened this issue Jan 21, 2025 · 2 comments
Labels
sig/planner SIG: Planner type/enhancement The issue or PR belongs to an enhancement.

Comments

@winoros
Copy link
Member

winoros commented Jan 21, 2025

Enhancement

Image

I'm taking this picture as an example. We can see that we just test whether this table exists and check whether it's a system table. But the TableInfoByID will try to get the whole table meta of the given table id and it's possible to read things from storage and then do the json decoding. There'll be huge costs once it needs to query from the storage.

We can avoid the possible network cost to get the whole table meta from storage.

@winoros winoros added sig/planner SIG: Planner type/enhancement The issue or PR belongs to an enhancement. labels Jan 21, 2025
@winoros
Copy link
Member Author

winoros commented Jan 22, 2025

All three major background jobs(Update Memory Objs, Dump Count/ModifyCount to Storage, GC stats of deleted tables) can save most of the network cost by avoiding reading the full table meta of tables.

@winoros
Copy link
Member Author

winoros commented Jan 22, 2025

By testing against #57844 (2025.01.22 version), both the Update job and the Dump Delta to Storage worker can reduce their running time from minutes to seconds.
As a side benefit, the total time of the rolling update will be saved too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/planner SIG: Planner type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

No branches or pull requests

1 participant