Question: Hard Spoon Solutions Practice?

[hackfisher]

Latest devnet stuck on following issue

2020-05-07 09:27:36 Error with block built on 0x937efeed9dfa375bdac396c3d278f034e8c5e8340e0c28d01e30c3ca8ae2da0f: ClientImport("Expected epoch change to happen at 0x1fd78e2705f55df0b2a06fbd4021f4355da6132b808cc23af3be29e0c1d24624, s264807276")

Suspect that this is caused by a runtime upgrade changing the epoch duration introduced by #436

We need further detail of the root cause and solutions to handle this case.

[hackfisher]

Might be related:
paritytech/substrate#5776
paritytech/substrate#5789

paritytech/substrate#5514 (review)

[andresilva]

Epoch length cannot currently be changed in BABE, sorry that this wasn't properly documented. We added some capabilities for dynamic configuration in those PRs but it still does not support epoch length changes (it is a bit more complicated).

[hackfisher]

https://hackmd.io/@lsaether/BJbdvZrpS?type=view

[hackfisher]

https://medium.com/polkadot-network/kusamas-first-adventure-2cd4f439a7a4

[hackfisher]

new sub-command export-state

paritytech/substrate#5842

[hackfisher]

You ask if there is a better way to do runtime upgrade

1. Here is the substrate seminar: https://youtu.be/0eNGZpNkJk4
2. This is Gav's runtime upgrade script for the kusama network. You can back up the block data of the current production environment, and then build a new chain, then you can run runtime upgrade on it for testing. For reference gav script: https://hackmd.io/mGgNZX0VT4S0UTaq89-_SQ
3. Substrate export-blocks and substrate import-blocks are your friends. It can be run on a single node after exporting the data and tested for migration. Of course, there are still blocks on mainnet. These cannot be tried. But this should also debug most of the problems

[drewstone]

Any standard practices with modifying exported state? I'm trying to change out the validator set after exporting a chain with a different validator set than desired.

[drewstone]

Ok modifying it MIGHT be as easy as just replacing all storage keys under specific modules with ones you intend. I created a new chainspec with new validaors and session info of course (blank at block 0) and replaced these keys with all keys in the exported state. Not sure what side-effects I'll run into but this should work well!

[aurexav]

From Alexander

There is an export-state command that will export the current state to a chainspec.

Do note that you cannot start a new chain from the unmodified chainspec if the delay between export and new start is too great as there will be timestamp issues in the consensus algorithm.

You can use the fork-off-substrate tool (https://github.com/maxsam4/fork-off-substrate) to create a new chain spec from your state and use that

[hackfisher]

https://github.com/maxsam4/fork-off-substrate

[hackfisher]

https://github.com/paritytech/substrate/blob/13bea77c6910f76ef7fa59fec347874c99b2170c/utils/frame/remote-externalities/src/lib.rs#L18

New remote externalities might be helpful to provide related solutions.

[hackfisher]

https://github.com/darwinia-network/darwinia/wiki/Substrate-Chain-State-Generate-Tools

[xcaptain]

https://github.com/maxsam4/fork-off-substrate

Does fork-off-substrate solve the ClientImport(Unexpected epoch change) error?

[aurexav]

Does fork-off-substrate solve the ClientImport(Unexpected epoch change) error?

We can not say it as "solve".
It just restarted an utterly new chain based on the old data.

Once you encounter ClientImport(Unexpected epoch change).
The chain is almost "dead".
Unless you modify the BABE protocol to rescue it.

[xcaptain]

Thanks, so the export-state command can't be used as a hard spoon solution, because the new chain can not run. I will try fork-off-substrate later to see if the fork chain can produce new blocks. I also asked at the substrate issue, seems like it's not easy to remove storage keys from exported genesis file.