Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify ZK proof benches #191

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Conversation

fjarri
Copy link
Member

@fjarri fjarri commented Feb 14, 2025

Trying to flatten out the code and reduce repetitions in ZK proof benches, and also keep the criterion dependency in dev-dependencies only.

Currently implemented the approach with iter_custom() I mentioned in #173. This seems ok, except:

  • I have to repeat public parameter creation (perhaps can be avoided if they implement Clone?)
  • I have to repeat the iteration loop in every method

An alternative is to use iter_batched() and return a closure to execute. This is currently implemented for Aff-g proofs, for the sake of comparison. It is a little more unwieldy, but defers more work to criterion.

Other issues/plans:

  • I pre-calculate two Paillier keys and two RP params and save them in a LazyLock (since that's generally the slowest part of the setup). This means that tests lose a bit of variation, but I don't expect much variation on the Paillier key, since that would mean timing attacks are possible. Should we keep that?
  • I don't particularly like that we repeat the same setup code in the proof modules themselves, and in the benchmarks. Could we move the functions into the corresponding proof modules? Merge the code somehow?
  • Aff-g* is super slow. Takes 13 seconds to generate a single proof. Is it supposed to be like that? We only do it in the error round in Presigning, but still.

Copy link

codecov bot commented Feb 14, 2025

Codecov Report

Attention: Patch coverage is 0% with 391 lines in your changes missing coverage. Please review.

Project coverage is 91.60%. Comparing base (2587f5a) to head (e57854f).

Files with missing lines Patch % Lines
synedrion/src/private_benches.rs 0.00% 391 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #191      +/-   ##
==========================================
+ Coverage   89.33%   91.60%   +2.26%     
==========================================
  Files          39       39              
  Lines       10913    10643     -270     
==========================================
  Hits         9749     9749              
+ Misses       1164      894     -270     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dvdplm
Copy link
Contributor

dvdplm commented Feb 21, 2025

  • Aff-g* is super slow. Takes 13 seconds to generate a single proof. Is it supposed to be like that? We only do it in the error round in Presigning, but still.

Yeah, that's what I see too. Verification is also very slow:

Benchmarking AffG* proof/prove: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 122.0s.
AffG* proof/prove       time:   [11.964 s 12.042 s 12.141 s]
                        change: [-2.8507% -1.8706% -0.7880%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild
Benchmarking AffG* proof/verify: Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 232.5s.
AffG* proof/verify      time:   [10.791 s 10.854 s 10.945 s]
                        change: [+0.3281% +0.9628% +1.8262%] (p = 0.01 < 0.05)
                        Change within noise threshold.

I took a few samples while the benchmark is running, both during proving and verification and there's nothing funny going on in the benchmark themselves afaict: it is spending the time in side AffGStarProof::new and AffGStarProof::verify.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants