fix: events: Improve performance of event migration from V1 to V2 #11064
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related Issues
Fixes: #11056
Proposed Changes
Added a few temporary indices to help speed up the migration.
Also, after the migration finishes successfully, I perform some manually DB administration to free up disk space (reducing size on disk from 3GB to 0.5GB).
Additional Info
Test plan
I used the large events DB that @TippyFlitsUK had generated and shared with me (link) as a benchmark when testing out this PR. Ths db contained 5.9mio entries and 21.6mio event entries.
Using this events DB and starting lotus resulted in migration from V1 to V2 in about 55 seconds:
After the migration, the events DB contained 0,7mio (737494) entries and 2.7mio (2705234) event entries and the size on disk went from 3GB to 0.5GB.
Checklist
Before you mark the PR ready for review, please make sure that:
<PR type>: <area>: <change being made>
fix: mempool: Introduce a cache for valid signatures
PR type
: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, testarea
, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps