[benchmark] Don't obliterate validators as much when wrapping up #3474
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Mar 4, 2025
This stack of pull requests is managed by Graphite. Learn more about stacking. |
This was referenced Mar 4, 2025
ndr-ds
force-pushed
the
03-04-_benchmark_don_t_obliterate_validators_as_much_when_wrapping_up
branch
3 times, most recently
from
f1e337c to
25b99cc
Compare
ndr-ds
force-pushed
the
03-04-_benchmark_don_t_obliterate_validators_as_much_when_wrapping_up
branch
2 times, most recently
from
3c661dc to
4522a2d
Compare
ndr-ds
force-pushed
the
02-24-query_prometheus_metrics_from_benchmark
branch
from
2666a06 to
25bb2d7
Compare
ndr-ds
force-pushed
the
02-24-query_prometheus_metrics_from_benchmark
branch
from
25bb2d7 to
e14cf17
Compare
ndr-ds
force-pushed
the
03-04-_benchmark_don_t_obliterate_validators_as_much_when_wrapping_up
branch
from
4522a2d to
39d242f
Compare
ndr-ds
force-pushed
the
02-24-query_prometheus_metrics_from_benchmark
branch
from
e14cf17 to
8f5e1c6
Compare
ndr-ds
force-pushed
the
03-04-_benchmark_don_t_obliterate_validators_as_much_when_wrapping_up
branch
2 times, most recently
from
fa6b1a6 to
6af0689
Compare
ndr-ds
force-pushed
the
02-24-query_prometheus_metrics_from_benchmark
branch
from
8f5e1c6 to
32b2bce
Compare
ndr-ds
force-pushed
the
03-04-_benchmark_don_t_obliterate_validators_as_much_when_wrapping_up
branch
from
6af0689 to
4f147f4
Compare
ndr-ds
force-pushed
the
02-24-query_prometheus_metrics_from_benchmark
branch
2 times, most recently
from
26e8c6a to
5824eec
Compare
ndr-ds
force-pushed
the
03-04-_benchmark_don_t_obliterate_validators_as_much_when_wrapping_up
branch
from
4f147f4 to
1138bb5
Compare
ndr-ds
force-pushed
the
02-24-query_prometheus_metrics_from_benchmark
branch
from
5824eec to
73e32e8
Compare
ndr-ds
force-pushed
the
03-04-_benchmark_don_t_obliterate_validators_as_much_when_wrapping_up
branch
from
1138bb5 to
fdce5c3
Compare
ndr-ds
force-pushed
the
02-24-query_prometheus_metrics_from_benchmark
branch
from
73e32e8 to
bb69877
Compare
ndr-ds
force-pushed
the
03-04-_benchmark_don_t_obliterate_validators_as_much_when_wrapping_up
branch
from
fdce5c3 to
2f71e16
Compare
ndr-ds
force-pushed
the
02-24-query_prometheus_metrics_from_benchmark
branch
2 times, most recently
from
6caf011 to
76c77e7
Compare
ndr-ds
force-pushed
the
03-04-_benchmark_don_t_obliterate_validators_as_much_when_wrapping_up
branch
2 times, most recently
from
3f1bd6d to
a1b0081
Compare
ndr-ds
force-pushed
the
03-04-_benchmark_don_t_obliterate_validators_as_much_when_wrapping_up
branch
from
a1b0081 to
8c8f5f1
Compare
ndr-ds
force-pushed
the
02-24-query_prometheus_metrics_from_benchmark
branch
3 times, most recently
from
37b5af1 to
6508b5c
Compare
ndr-ds
force-pushed
the
03-04-_benchmark_don_t_obliterate_validators_as_much_when_wrapping_up
branch
from
8c8f5f1 to
ba2d316
Compare
ma2bd
approved these changes
Mar 5, 2025
ndr-ds
force-pushed
the
03-04-_benchmark_don_t_obliterate_validators_as_much_when_wrapping_up
branch
from
ba2d316 to
ea1a2f3
Compare
ndr-ds
force-pushed
the
02-24-query_prometheus_metrics_from_benchmark
branch
2 times, most recently
from
6651527 to
1e58768
Compare
ndr-ds
force-pushed
the
03-04-_benchmark_don_t_obliterate_validators_as_much_when_wrapping_up
branch
2 times, most recently
from
1e0167d to
b77161d
Compare
ndr-ds
force-pushed
the
02-24-query_prometheus_metrics_from_benchmark
branch
2 times, most recently
from
e016dce to
65d1362
Compare
ndr-ds
force-pushed
the
03-04-_benchmark_don_t_obliterate_validators_as_much_when_wrapping_up
branch
from
b77161d to
c877d00
Compare
ndr-ds
changed the base branch from
02-24-query_prometheus_metrics_from_benchmark
to
graphite-base/3474
ndr-ds
force-pushed
the
03-04-_benchmark_don_t_obliterate_validators_as_much_when_wrapping_up
branch
from
c877d00 to
1cc71a4
Compare
ndr-ds
deleted the
03-04-_benchmark_don_t_obliterate_validators_as_much_when_wrapping_up
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Right now if we have the
--close-chainsoption on, we'll close all the chains in parallel. If we don't, we'll process all the inboxes in parallel also before saving the benchmark chains to the wallet.This is not ideal because depending on how many chains we're using, this can pound the validators pretty hard, and drive latency up to the point of us getting a bunch of timeouts when wrapping things up.
Also, when latency spikes up too much, the validators take a while to recover.
Proposal
Introduce a
--wrap-up-max-in-flightargument to limit the concurrency when wrapping things upTest Plan
Tested locally, no timeouts anymore, and things actually go faster (if we timeout, we retry, and this can take a while until we actually eventually succeed within the timeout constraint)
Release Plan