startup shrink all slots has to avoid the snapshot slot

[jeffwashington]

Problem

If we shrink away zero lamport accounts with 1 ref count and 1 slot list at startup, then we could affect the startup bank hash calculation and cause the validator to fail to start.

Summary of Changes

Don't force shrink the storage/slot we're loading a snapshot from.

Fixes #

[HaoranYi]

Nice catch. lgtm!

[brooksprumo]

This would only affect the incremental accounts hash, right? Regular full accounts hash should not be affected since it does not include zero lamport accounts.

[HaoranYi]

This would only affect the incremental accounts hash, right? Regular full accounts hash should not be affected since it does not include zero lamport accounts.

By incremental accounts hash, you mean the accounts delta hashes part of the bank hash?
If yes, then I think so.

[brooksprumo]

By incremental accounts hash, you mean the accounts delta hashes part of the bank hash?

I mean the incremental accounts hash that happens for incremental snapshots. This one: solana-labs#30819

(sorry for editing your comment! that was a mistake)

[jeffwashington]

By incremental accounts hash, you mean the accounts delta hashes part of the bank hash?

I mean the incremental accounts hash that happens for incremental snapshots. This one: solana-labs#30819

(sorry for editing your comment! that was a mistake)

i think this could affect starting from a full or full + inc.

After untarring and building the index, we sometimes run shrink and clean before starting replay. We calculate the bank hash for the slot that was in the snapshot (whether full or incremental). The accounts delta hash is part of the bank hash. The accounts delta hash is all accounts in the storage for the bank's slot. This includes zero lamport accounts. Zero lamport accounts ARE included in the accounts delta hash. Changes coming soon to shrink can cause us to shrink a slot and remove accounts which are zero lamport and have 1 slot list entry and 1 ref count. If such an account gets removed from the storage for the bank's slot by shrink_all_slots here, then when we go through the storage and build the accounts delta hash, we will NOT have the zero lamport account anymore. This will mean we get a different accounts delta hash.

To differentiate: a full hash of all accounts skips zero lamport accounts. So, the existence or obliteration of a zero lamport account has no impact on a full hash of all accounts. But, a full snapshot can absolutely contain zero lamport accounts and even must (for the bank's slot at least). So much nuance and details here.

[brooksprumo]

Ah, yes, I forgot about the verification of the bank hash itself at startup, which recalculates the accounts delta hash. Thanks for that reminder.

So:

• for accounts delta hash: cannot shrink the snapshot slot (can shrink one slot older though)
• for incremental accounts hash: cannot remove zero lamport accounts in shrink (or anywhere) past the last full snapshot slot

[brooksprumo]