[benchmarking] Refactor Benchmark #1533
Conversation
longbowlu
requested review from
Clay-Mysten,
huitseeker and
patrickkuo
as code owners
| /// Size of the Sui committee. Minimum size is 4 to tolerate one fault | ||
| #[clap(long, default_value = "10", global = true)] | ||
| /// Size of the Sui committee. | ||
| #[clap(long, default_value = "1", global = true)] |
There was a problem hiding this comment.
Why are we setting this to default = 1? Then someone who runs the bench with default values will get no sense of the overhead of running even with the smallest meaningful committee of 4 (in terms of sig verification etc).
There was a problem hiding this comment.
In setup, we spin out {committee_size} validators. However currently the benchmarking only supports one validator, hence it continuously looks for peers and gets connection errors. Soon we should be able to add support for multiple validators though.
For the default value, I'm changing it to 1 so we don't have the issue above. I can keep it at 10 or 4 but we need to specify --committee-size=1 for now. I would expect multi-validator approach come along as a fast follow so either way works for me!
There was a problem hiding this comment.
@gdanezis I'm keeping it @ 1 for now. Happy to update it any time!
…ading the transactions information (#1446) Co-authored-by: George Danezis <[email protected]>
There is no undefined address in the [addresses] section (e.g. AddressToBeFilledIn = "_") so having a [dev-addresses] section is not necessary.
- Made init rules static - The existence of the function is optional - The function must be named 'init' - The function must be private - The function can have a single parameter, &mut TxContext - Alternatively, the function can have zero parameters
|
@longbowlu is there any functional change or is it just a refactor? |
as we discussed, the PR also adds the ability to run a single validator as a process |
* Refactor Build index by common user workflow * Update index.md Fix paths missing Markdown file extension
Removing tutorial series until ready
Remove version from API link Capitalize REST
Remove version from API link Capitalize REST
This PR refactors the benchmark into several pieces:
ValidatorPreparerthat keeps tracks of validator/committee information and more importantly takes care of validator deployment and teardown.TransactionCreatorby moving some validator logic toValidatorPreparer. After this diff it will only focuses on transaction generationAlso introduce the ability to run validator locally as a process. Add args 'local-single-validator-process' and 'local-single-validator-thread' to distinguish the running mode. More modes to be added for distributed benchmarking,
Also Add "How-to" in
bench.rs.In the long term, there are more abstraction work can be done. e.g. fully split out benchmark coordinator, clients (load generators), validator, and more roles e.g. gateway in the future. Separating load generator from coordinator is beneficial in having distributed load generators to saturate network.