clears walletdb datastores during reset #3797
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
following a network reset the walletdb may have data that is incompatible with database schemas. for instance, if asset identifiers in transactions or notes change, then any datastores that encode that data must be migrated or cleared. we avoid deleting the walletdb during 'reset' so that we do not delete user keys. however, the node will fail to start if it needs to apply migrations on incompatible data or even if it needs to deserialize incompatible data in the course of deleting it. to resolve this we can use 'clear' on each walletdb datastore during the 'reset' command. clear will delete all keys within a range without deserializing the values. note that clear soft-deletes data that is cleanup up later during database compactions rather than deleting immediately. - clears each walletdb datastore, except accounts and meta, during 'reset' - adds missing datastores to cleanupDeletedAccounts
EvanJRichard
approved these changes
Apr 18, 2023
ironfish-cli/src/commands/reset.ts
Outdated
Comment on lines
83
to
102
| const node = await this.sdk.node() | ||
| const walletDb = node.wallet.walletDb | ||
|
|
||
| const walletDbStores: IDatabaseStore<{ | ||
| key: Readonly<unknown> | ||
| value: unknown | ||
| }>[] = [ | ||
| walletDb.decryptedNotes, | ||
| walletDb.nullifierToNoteHash, | ||
| walletDb.sequenceToNoteHash, | ||
| walletDb.nonChainNoteHashes, | ||
| walletDb.transactions, | ||
| walletDb.sequenceToTransactionHash, | ||
| walletDb.pendingTransactionHashes, | ||
| walletDb.timestampToTransactionHash, | ||
| walletDb.assets, | ||
| walletDb.nullifierToTransactionHash, | ||
| walletDb.unspentNoteHashes, | ||
| ] | ||
|
|
There was a problem hiding this comment.
this list isn't reused anywhere right? was wondering if it's worth setting it up as a constant (or something like that) somewhere, but if this is the only place it's used, no point
There was a problem hiding this comment.
It's used in walletDb.cleanupDeletedAccounts, too.
I think it's a good idea to define it as a walletDb property!
this is the list of stores that cache and/or index decrypted chain data that must be cleared when resetting the wallet or deleting an account
hughy
added a commit
that referenced
this pull request
Apr 19, 2023
adds changes from #3797 into 'mainnet' command. clears old chain data from the walletdb that cannot be migrated or deleted by key following changes from network reset.
hughy
added a commit
that referenced
this pull request
Apr 19, 2023
adds changes from #3797 into 'mainnet' command. clears old chain data from the walletdb that cannot be migrated or deleted by key following changes from network reset.
This was referenced Apr 19, 2023
NullSoldier
pushed a commit
that referenced
this pull request
Apr 20, 2023
adds changes from #3797 into 'mainnet' command. clears old chain data from the walletdb that cannot be migrated or deleted by key following changes from network reset.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
following a network reset the walletdb may have data that is incompatible with database schemas. for instance, if asset identifiers in transactions or notes change, then any datastores that encode that data must be migrated or cleared.
we avoid deleting the walletdb during 'reset' so that we do not delete user keys. however, the node will fail to start if it needs to apply migrations on incompatible data or even if it needs to deserialize incompatible data in the course of deleting it.
to resolve this we can use 'clear' on each walletdb datastore during the 'reset' command. clear will delete all keys within a range without deserializing the values.
note that clear soft-deletes data that is cleanup up later during database compactions rather than deleting immediately.
Testing Plan
manual testing:
ironfish resettested with pool account from phase 3 (~10k transactions and ~10k notes) and resetting wallet data using these changes took less than 10s.
Documentation
Does this change require any updates to the Iron Fish Docs (ex. the RPC API
Reference)? If yes, link a
related documentation pull request for the website.
Breaking Change
Is this a breaking change? If yes, add notes below on why this is breaking and
what additional work is required, if any.