Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Async backing enablement checklist #3226

Closed
13 of 14 tasks
alexggh opened this issue Feb 6, 2024 · 14 comments
Closed
13 of 14 tasks

Async backing enablement checklist #3226

alexggh opened this issue Feb 6, 2024 · 14 comments

Comments

@alexggh
Copy link
Contributor

alexggh commented Feb 6, 2024

The purpose of this ticket is to collate together all the needed things to be able to safely enable async backing on kusama and polkadot, in chronological order.

Kusama enablement

  • Kusama runtime-api supports async_backing params. Async backing is not enabled, but all the necessary subsystems become live and used in producing parachains blocks at 12s rate(sync_backing).
  • Kusama runtime deployed with fix 97df9dd from polkadot-sdk1.7.
  • Confirm pre-async backing parachains are not affected by a relay-chain with async backing enabled.
  • Confirm parachains that support async backing run correctly on a relay-chain with async backing disable.
  • Deploy system parachains with async backing enabled on Westend relay chain with async backing enabled and confirm they work correctly.
  • Deploy system parachains with async backing enabled on Rococo relay chain with async backing enabled and confirm they work correctly.
  • Enable async backing on kusama with the following steps Propose async backing parameter change on Kusama #2236.
  • Referendas for enabling async backing: https://kusama.subsquare.io/referenda/374 & https://kusama.subsquare.io/fellowship/referenda/70

Polkadot enablement

Other items

  • Create a reliable, relevant, repeatable benchmark.
@alexggh
Copy link
Contributor Author

alexggh commented Feb 6, 2024

FYI @rphmeier in case you want to add other things or you know if some of this stuff was already tested/checked.

@Polkadot-Forum
Copy link

This issue has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/async-backing-development-updates/6176/1

@alexggh
Copy link
Contributor Author

alexggh commented Feb 26, 2024

Confirm pre-async backing parachains are not affected by a relay-chain with async backing enabled.

Verified this, by running zombienets with asset hub parachains from polkadot 1.1.0 and polkadot 1.3.0

@alexggh alexggh moved this from Backlog to In Progress in parachains team board Mar 1, 2024
@alexggh
Copy link
Contributor Author

alexggh commented Mar 1, 2024

Confirm parachains that support async backing run correctly on a relay-chain with async backing disable.

Done, westend parachains, from 1.8.0 release the with async backing changes, got upgrade while async backing was disable on relay chain. They correctly produced blocks at 12s rate.

@alexggh
Copy link
Contributor Author

alexggh commented Mar 5, 2024

Deploy system parachains with async backing enabled on Westend relay chain with async backing enabled and confirm they work correctly.

Checked westend block times with async backing enabled things looked good, blocked times averages are around 6.7, but that's expected because one of the validator is kagome which does not seem to back things.

@alexggh
Copy link
Contributor Author

alexggh commented Mar 5, 2024

Ran subp2p-explorer for polkadot network, it discovered just 2/3 out of the all authorities, from them it seems 13(7%) are still running software that is not compatible with async-backing.

Discovered 193/300 authorities
uthority="1HCf6UeB9y6xpmdd7ZxphAXYLHvnEqXZMh6eXmwinkzwzx9" peer_id=PeerId("12D3KooWGLy7qSXW5CFHfkmiGnDRTnthHMVvhD733rE8ZXPaTijn") addresses="Parity Polkadot/v0.9.41-e203bfb396e (teufel-00-05)" version={"/ip4/145.239.254.33/tcp/30111/p2p/12D3KooWGLy7qSXW5CFHfkmiGnDRTnthHMVvhD733rE8ZXPaTijn"} 
authority="1WDfDKfcx3Yvoy5qGXGNS7Lv3eNZqzULbkT1ne3AyqQcvZL" peer_id=PeerId("12D3KooWR9ZQqixCCskgLs1iQMqFYYLsFdei1CQgg2TRkpuerU8r") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (Sv8fx5lh)" version={"/ip6/2605:6441::103:219:171:54/tcp/30333/p2p/12D3KooWR9ZQqixCCskgLs1iQMqFYYLsFdei1CQgg2TRkpuerU8r", "/ip4/103.219.171.54/tcp/30333/p2p/12D3KooWR9ZQqixCCskgLs1iQMqFYYLsFdei1CQgg2TRkpuerU8r"} 
authority="12sCPzrgTQwbmDXyFDW2LqUsRguzpr2nzpdm2KU5gEaUTPSi" peer_id=PeerId("12D3KooWMxvAqUDhvyZm4zn3tFHk58eMKgeTNhhSrMLu9icZCpD1") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (JjyYfep1)" version={"/ip4/162.19.234.29/tcp/30333/p2p/12D3KooWMxvAqUDhvyZm4zn3tFHk58eMKgeTNhhSrMLu9icZCpD1", "/ip6/2001:41d0:700:721d::/tcp/30333/p2p/12D3KooWMxvAqUDhvyZm4zn3tFHk58eMKgeTNhhSrMLu9icZCpD1"} 
authority="13BxUF8jUcmBgqJa4j1RHdwax1FYPRrTju48Ag5BtEhkWHM5" peer_id=PeerId("12D3KooWK4goNMpYkvgSVv7ibs7MBFtW69a3UQCuBrVqcB1LNsyr") addresses="Parity Polkadot/v0.9.41-e203bfb396e (teufel-02-04)" version={"/ip4/54.38.44.133/tcp/30111/p2p/12D3KooWK4goNMpYkvgSVv7ibs7MBFtW69a3UQCuBrVqcB1LNsyr"} 
authority="13JmqWwcBkGVyH3UW23Y6MAqbPwm1hUYQ31zBhdNayyy7XAG" peer_id=PeerId("12D3KooWKxFZYcBGqZGoDty3E8qekWiHqXE3vEdEFGtKFUBUkGyg") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (IJ4lBvM8)" version={"/ip4/51.210.155.189/tcp/30333/p2p/12D3KooWKxFZYcBGqZGoDty3E8qekWiHqXE3vEdEFGtKFUBUkGyg", "/ip6/2001:41d0:403:42bd::/tcp/30333/p2p/12D3KooWKxFZYcBGqZGoDty3E8qekWiHqXE3vEdEFGtKFUBUkGyg"} 
authority="14KfTnpc2iyTHJjjc1PyWvRVT1vLj1Ck46uheT1MoDu1SQiv" peer_id=PeerId("12D3KooWCt9P98Ef8DsSYfUxJsTTbzmEkwhrXyUuR7xAZbW4XXVZ") addresses="Parity Polkadot/v0.9.41-e203bfb396e (teufel-00-03)" version={"/ip4/146.59.47.240/tcp/30111/p2p/12D3KooWCt9P98Ef8DsSYfUxJsTTbzmEkwhrXyUuR7xAZbW4XXVZ"} 
authority="14SwX6chMbBpUfGWugNx6mQ5xy9jzVSfnmuyj2k5dWpzw1jA" peer_id=PeerId("12D3KooWDAUr785H2pKfRVYTC2DM4GKJAAifemVQHJHD4bcK5KjU") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (cjEn7shR)" version={"/ip4/57.129.1.125/tcp/30333/p2p/12D3KooWDAUr785H2pKfRVYTC2DM4GKJAAifemVQHJHD4bcK5KjU", "/ip6/2001:41d0:700:757d::/tcp/30333/p2p/12D3KooWDAUr785H2pKfRVYTC2DM4GKJAAifemVQHJHD4bcK5KjU"} 
authority="14rRrRZUhiMLKbAVZybbUps9mAPhE1mbtREh3oTe3VUAe5vq" peer_id=PeerId("12D3KooWKeWm4YowCzPvVZhB3TEfmVUkFSAhLTRcpHVgjVdeWckW") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (GKNqsQay)" version={"/ip4/141.94.241.119/tcp/30333/p2p/12D3KooWKeWm4YowCzPvVZhB3TEfmVUkFSAhLTRcpHVgjVdeWckW", "/ip6/2001:41d0:203:bf77::/tcp/30333/p2p/12D3KooWKeWm4YowCzPvVZhB3TEfmVUkFSAhLTRcpHVgjVdeWckW"} 
authority="15KChULvwa8wChG9wLwFQTvkXoh9GkgwakMY2ZKuRP4m5HJo" peer_id=PeerId("12D3KooWH58YJMSFzhRMpizqyDNZpfS59uyjMfyeWwCCdCLi2r8U") addresses="Parity Polkadot/v0.9.41-e203bfb396e (teufel-02-06)" version={"/ip4/54.38.44.133/tcp/30222/p2p/12D3KooWH58YJMSFzhRMpizqyDNZpfS59uyjMfyeWwCCdCLi2r8U", "/ip6/2001:41d0:203:2985::/tcp/30222/p2p/12D3KooWH58YJMSFzhRMpizqyDNZpfS59uyjMfyeWwCCdCLi2r8U"} 
authority="15PiSyKWRpjcgtdHZKdu7cpiSM7StseXAbizqkibgCrtRNPp" peer_id=PeerId("12D3KooWCr5qt3B8Sa8rQZHVnxtCoZymA4NpnUcwLPknb8DxBysB") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (FwId20nR)" version={"/ip4/162.19.235.85/tcp/30333/p2p/12D3KooWCr5qt3B8Sa8rQZHVnxtCoZymA4NpnUcwLPknb8DxBysB", "/ip6/2001:41d0:700:7355::/tcp/30333/p2p/12D3KooWCr5qt3B8Sa8rQZHVnxtCoZymA4NpnUcwLPknb8DxBysB"} 
authority="15Q99CmcDfg9NneKKVyYjQqg4rSyzQGxcDvdwXLB9jBoQ7hr" peer_id=PeerId("12D3KooWAxY9c1xbJyWuFZKBdE95yhVwesQCS4vrBEpPu8z9aA5v") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (CRYPTOSTAKE)" version={"/ip4/159.203.20.171/tcp/30333/p2p/12D3KooWAxY9c1xbJyWuFZKBdE95yhVwesQCS4vrBEpPu8z9aA5v"} 
authority="15uRgi6g9J2Na5eFX5VWenowAadT9jJ2K64pmURQwoFbw6Bu" peer_id=PeerId("12D3KooWDhH8U7TJmYH84v4u54iFax2xs2672QdgF5FVtrfu9Ryj") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (PMHsevNy)" version={"/ip6/2001:41d0:306:348d::/tcp/30333/p2p/12D3KooWDhH8U7TJmYH84v4u54iFax2xs2672QdgF5FVtrfu9Ryj", "/ip4/37.59.23.141/tcp/30333/p2p/12D3KooWDhH8U7TJmYH84v4u54iFax2xs2672QdgF5FVtrfu9Ryj"} 
authority="16G2Y2MiEkhASib1ogrTfdCxkFyG2bXMTuXoYXu3nHCYepvK" peer_id=PeerId("12D3KooWMAaJzcZiqDnV7QM7eCZYYbvWgzh28hSGAnhnDsQC6TVr") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (fabulous_ape)" version={"/ip4/51.222.74.49/tcp/30333/p2p/12D3KooWMAaJzcZiqDnV7QM7eCZYYbvWgzh28hSGAnhnDsQC6TVr"} 

@burdges
Copy link

burdges commented Mar 9, 2024

it discovered just 2/3 out of the all authorities

Really, how is that possible?

@alexggh
Copy link
Contributor Author

alexggh commented Mar 11, 2024

it discovered just 2/3 out of the all authorities

Really, how is that possible?

I used the tool from here: https://github.com/paritytech/data/issues/30, as far as I can tell subp2p-explorer can't connect to those nodes to query their version. I'll let @lexnv tell you why our tool can't connect to all of the nodes.

@alexggh
Copy link
Contributor Author

alexggh commented Apr 12, 2024

Known polkadot nodes that run a version behind the desired >=1.7, total number 23(8%) out of 286 discovered by subp2p-explorer.

authority="14uCDhfAcrwuCidcJ3qr8RHVFTKuXwtomxHaQN7RmKi9V7n" peer_id=PeerId("12D3KooWF4puQdrBtrYWQCtU8zjwFKSVVB3pkDwp8x5NxU8NYvGn") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (Ed4w37DS)" version={"/ip4/162.19.234.30/tcp/30333/p2p/12D3KooWF4puQdrBtrYWQCtU8zjwFKSVVB3pkDwp8x5NxU8NYvGn", "/ip6/2001:41d0:700:721e::/tcp/30333/p2p/12D3KooWF4puQdrBtrYWQCtU8zjwFKSVVB3pkDwp8x5NxU8NYvGn"} 
authority="1HCf6UeB9y6xpmdd7ZxphAXYLHvnEqXZMh6eXmwinkzwzx9" peer_id=PeerId("12D3KooWGLy7qSXW5CFHfkmiGnDRTnthHMVvhD733rE8ZXPaTijn") addresses="Parity Polkadot/v0.9.41-e203bfb396e (teufel-00-05)" version={"/ip4/145.239.254.33/tcp/30111/p2p/12D3KooWGLy7qSXW5CFHfkmiGnDRTnthHMVvhD733rE8ZXPaTijn"} 
authority="1KbA4NQP5T7nnXmtNDkWcF3UxPV8diaPBvKvxQG5fYn8Ti9" peer_id=PeerId("12D3KooWKtcuisE3Rd9UUYuQnzS3gGcQQuZAG2igwi4wjZtecmME") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (8ArIuAq3)" version={"/ip4/198.244.176.224/tcp/30333/p2p/12D3KooWKtcuisE3Rd9UUYuQnzS3gGcQQuZAG2igwi4wjZtecmME", "/ip6/2001:41d0:800:38e0::/tcp/30333/p2p/12D3KooWKtcuisE3Rd9UUYuQnzS3gGcQQuZAG2igwi4wjZtecmME"} 
authority="1WDfDKfcx3Yvoy5qGXGNS7Lv3eNZqzULbkT1ne3AyqQcvZL" peer_id=PeerId("12D3KooWR9ZQqixCCskgLs1iQMqFYYLsFdei1CQgg2TRkpuerU8r") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (Sv8fx5lh)" version={"/ip6/2605:6441::103:219:171:54/tcp/30333/p2p/12D3KooWR9ZQqixCCskgLs1iQMqFYYLsFdei1CQgg2TRkpuerU8r", "/ip4/103.219.171.54/tcp/30333/p2p/12D3KooWR9ZQqixCCskgLs1iQMqFYYLsFdei1CQgg2TRkpuerU8r"} 
authority="1eM8NGcKiAKhciip3ufizJMbeGNVy1WtjDweLfpQGXczzxm" peer_id=PeerId("12D3KooWHVDNuABpm6oZ8ZCuiE4MvnMsZTxrURm92ESDexj1YCTw") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (JapdGMIo)" version={"/ip6/2001:41d0:800:38df::/tcp/30333/p2p/12D3KooWHVDNuABpm6oZ8ZCuiE4MvnMsZTxrURm92ESDexj1YCTw", "/ip4/198.244.176.223/tcp/30333/p2p/12D3KooWHVDNuABpm6oZ8ZCuiE4MvnMsZTxrURm92ESDexj1YCTw"} 
authority="1hC9wTNFfsaRHJBxtZ3D9DjEcE7omuiZrkKmcdocmizW8EJ" peer_id=PeerId("12D3KooWLCqdkWQjgyQG1vp34NA3yRUqqUDgtwSbp9Dvdty4ATde") addresses="Parity Polkadot/v0.9.41-e203bfb396e (teufel-03-08)" version={"/ip4/51.89.7.22/tcp/30222/p2p/12D3KooWLCqdkWQjgyQG1vp34NA3yRUqqUDgtwSbp9Dvdty4ATde", "/ip6/2001:41d0:700:2c16::/tcp/30222/p2p/12D3KooWLCqdkWQjgyQG1vp34NA3yRUqqUDgtwSbp9Dvdty4ATde"} 
authority="1nTTxVCmuJVvQ3tjvJLpW3iB35fHjGXwdMwMq7KohvNTBd5" peer_id=PeerId("12D3KooWF95AH96LwU6kE8bJk55erRkGXmQkcK9VbEnbydwjvHLA") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (90B8wBEG)" version={"/ip4/198.244.176.226/tcp/30333/p2p/12D3KooWF95AH96LwU6kE8bJk55erRkGXmQkcK9VbEnbydwjvHLA", "/ip6/2001:41d0:800:38e2::/tcp/30333/p2p/12D3KooWF95AH96LwU6kE8bJk55erRkGXmQkcK9VbEnbydwjvHLA"} 
authority="123xPVeiMaSiWeZKHGRnB3uZQf5Fx12EEtuxeJaTua6Sgrax" peer_id=PeerId("12D3KooWKd1kA7yfXL5GuwMowBGg174wCs9raegxGwd99FTyRXvq") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (dg91XV4i)" version={"/ip6/2001:41d0:800:38b0::/tcp/30333/p2p/12D3KooWKd1kA7yfXL5GuwMowBGg174wCs9raegxGwd99FTyRXvq", "/ip4/198.244.176.176/tcp/30333/p2p/12D3KooWKd1kA7yfXL5GuwMowBGg174wCs9raegxGwd99FTyRXvq"} 
authority="126CwLfB7c2otNGpiCMXwASPvHtaemMcVEzW38zd38Ef913t" peer_id=PeerId("12D3KooWJPVNrXLTVrzczDJCm5Ni57R51JixmcYEEiyEab6dRw1f") addresses="Parity Polkadot/v0.9.41-e203bfb396e (teufel-03-07)" version={"/ip6/2001:41d0:700:2c16::/tcp/30111/p2p/12D3KooWJPVNrXLTVrzczDJCm5Ni57R51JixmcYEEiyEab6dRw1f", "/ip4/51.89.7.22/tcp/30111/p2p/12D3KooWJPVNrXLTVrzczDJCm5Ni57R51JixmcYEEiyEab6dRw1f"} 
authority="12HMDAq5E8Y28FLys2EiuHfJG4cGAX6o4CXVVLEwNULUz11G" peer_id=PeerId("12D3KooWMhcLznJWbCGwkBubPHr664GsSHKf7k8vyCA8CQouYtHY") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (awUkvZ1j)" version={"/ip6/2001:41d0:602:4f31::/tcp/30333/p2p/12D3KooWMhcLznJWbCGwkBubPHr664GsSHKf7k8vyCA8CQouYtHY"} 
authority="12HUsEmzc2oLHDnrhTANQGZUQaw23syjnyC6PWmydmooFqw8" peer_id=PeerId("12D3KooWKpHeZpZR5KigC87dbuABD5hQMjmQ78xCcSYpZ8Nu2DAE") addresses="Parity Polkadot/v0.9.41-e203bfb396e (teufel-00-02)" version={"/ip4/152.228.223.158/tcp/30111/p2p/12D3KooWKpHeZpZR5KigC87dbuABD5hQMjmQ78xCcSYpZ8Nu2DAE", "/ip6/2001:41d0:303:f29e::/tcp/30111/p2p/12D3KooWKpHeZpZR5KigC87dbuABD5hQMjmQ78xCcSYpZ8Nu2DAE"} 
authority="12NCas2SyxUAZbBk4Ro4HUHudnn469XTupBSjJJ9N56E8BvD" peer_id=PeerId("12D3KooWK9ZyBGUJuzHaKmkbKbYrg6oqK41nhBUSFAGnHmfgi9kz") addresses="Parity Polkadot/v0.9.41-e203bfb396e (teufel-00-01)" version={"/ip4/51.178.74.120/tcp/30111/p2p/12D3KooWK9ZyBGUJuzHaKmkbKbYrg6oqK41nhBUSFAGnHmfgi9kz"} 
authority="12sCPzrgTQwbmDXyFDW2LqUsRguzpr2nzpdm2KU5gEaUTPSi" peer_id=PeerId("12D3KooWMxvAqUDhvyZm4zn3tFHk58eMKgeTNhhSrMLu9icZCpD1") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (JjyYfep1)" version={"/ip6/2001:41d0:700:721d::/tcp/30333/p2p/12D3KooWMxvAqUDhvyZm4zn3tFHk58eMKgeTNhhSrMLu9icZCpD1", "/ip4/162.19.234.29/tcp/30333/p2p/12D3KooWMxvAqUDhvyZm4zn3tFHk58eMKgeTNhhSrMLu9icZCpD1"} 
authority="13Bcmdmn3whidasRqMj7LbQXgP8ReUWWVMg53xPr83TfUL6Z" peer_id=PeerId("12D3KooWL8eWuSUwngZcz82qk6bmdw6yak9ZLCyY34fjaddnP44W") addresses="Parity Polkadot/v0.9.41-e203bfb396e (teufel-03-09)" version={"/ip4/51.89.7.22/tcp/30333/p2p/12D3KooWL8eWuSUwngZcz82qk6bmdw6yak9ZLCyY34fjaddnP44W", "/ip6/2001:41d0:700:2c16::/tcp/30333/p2p/12D3KooWL8eWuSUwngZcz82qk6bmdw6yak9ZLCyY34fjaddnP44W"} 
authority="13BxUF8jUcmBgqJa4j1RHdwax1FYPRrTju48Ag5BtEhkWHM5" peer_id=PeerId("12D3KooWK4goNMpYkvgSVv7ibs7MBFtW69a3UQCuBrVqcB1LNsyr") addresses="Parity Polkadot/v0.9.41-e203bfb396e (teufel-02-04)" version={"/ip4/54.38.44.133/tcp/30111/p2p/12D3KooWK4goNMpYkvgSVv7ibs7MBFtW69a3UQCuBrVqcB1LNsyr"} 
authority="14KfTnpc2iyTHJjjc1PyWvRVT1vLj1Ck46uheT1MoDu1SQiv" peer_id=PeerId("12D3KooWCt9P98Ef8DsSYfUxJsTTbzmEkwhrXyUuR7xAZbW4XXVZ") addresses="Parity Polkadot/v0.9.41-e203bfb396e (teufel-00-03)" version={"/ip4/146.59.47.240/tcp/30111/p2p/12D3KooWCt9P98Ef8DsSYfUxJsTTbzmEkwhrXyUuR7xAZbW4XXVZ"} 
authority="14XtaZUR6Nyy2aSFNNWE2dM7nRR2J1deJyJ4Tysdi1zfmgvh" peer_id=PeerId("12D3KooWR4tTL3epEWTyvLdfjuidYiBHR5jKiEvaygJknqDR5F5w") addresses="Parity Polkadot/v1.3.0-7c9fd83805c (mysterious_mongo)" version={"/ip4/138.59.133.242/tcp/30333/p2p/12D3KooWR4tTL3epEWTyvLdfjuidYiBHR5jKiEvaygJknqDR5F5w", "/ip4/162.19.57.17/tcp/30333/p2p/12D3KooWR4tTL3epEWTyvLdfjuidYiBHR5jKiEvaygJknqDR5F5w"} 
authority="15KChULvwa8wChG9wLwFQTvkXoh9GkgwakMY2ZKuRP4m5HJo" peer_id=PeerId("12D3KooWH58YJMSFzhRMpizqyDNZpfS59uyjMfyeWwCCdCLi2r8U") addresses="Parity Polkadot/v0.9.41-e203bfb396e (teufel-02-06)" version={"/ip4/54.38.44.133/tcp/30222/p2p/12D3KooWH58YJMSFzhRMpizqyDNZpfS59uyjMfyeWwCCdCLi2r8U", "/ip6/2001:41d0:203:2985::/tcp/30222/p2p/12D3KooWH58YJMSFzhRMpizqyDNZpfS59uyjMfyeWwCCdCLi2r8U"} 
authority="15PiSyKWRpjcgtdHZKdu7cpiSM7StseXAbizqkibgCrtRNPp" peer_id=PeerId("12D3KooWCr5qt3B8Sa8rQZHVnxtCoZymA4NpnUcwLPknb8DxBysB") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (FwId20nR)" version={"/ip6/2001:41d0:700:7355::/tcp/30333/p2p/12D3KooWCr5qt3B8Sa8rQZHVnxtCoZymA4NpnUcwLPknb8DxBysB", "/ip4/162.19.235.85/tcp/30333/p2p/12D3KooWCr5qt3B8Sa8rQZHVnxtCoZymA4NpnUcwLPknb8DxBysB"} 
authority="15uRgi6g9J2Na5eFX5VWenowAadT9jJ2K64pmURQwoFbw6Bu" peer_id=PeerId("12D3KooWDhH8U7TJmYH84v4u54iFax2xs2672QdgF5FVtrfu9Ryj") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (PMHsevNy)" version={"/ip6/2001:41d0:306:348d::/tcp/30333/p2p/12D3KooWDhH8U7TJmYH84v4u54iFax2xs2672QdgF5FVtrfu9Ryj", "/ip4/37.59.23.141/tcp/30333/p2p/12D3KooWDhH8U7TJmYH84v4u54iFax2xs2672QdgF5FVtrfu9Ryj"} 
authority="15wXM2xDxnzL7XNp4p5dWfe1koYyEQ51xXV8fwqYiJzvJdA5" peer_id=PeerId("12D3KooWFrqf19z5P7MJ3TMKyXocL5GyZN53syLRZNGFfoaiHijn") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (p05pcqWs)" version={"/ip6/2001:41d0:800:38e3::/tcp/30333/p2p/12D3KooWFrqf19z5P7MJ3TMKyXocL5GyZN53syLRZNGFfoaiHijn", "/ip4/198.244.176.227/tcp/30333/p2p/12D3KooWFrqf19z5P7MJ3TMKyXocL5GyZN53syLRZNGFfoaiHijn"} 
authority="16G2Y2MiEkhASib1ogrTfdCxkFyG2bXMTuXoYXu3nHCYepvK" peer_id=PeerId("12D3KooWMAaJzcZiqDnV7QM7eCZYYbvWgzh28hSGAnhnDsQC6TVr") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (fabulous_ape)" version={"/ip4/51.222.74.49/tcp/30333/p2p/12D3KooWMAaJzcZiqDnV7QM7eCZYYbvWgzh28hSGAnhnDsQC6TVr"} 
authority="16MAyoqDY3Au3i3cr8kJ5js8xywYbJACqSgxQu2rmMSqYHFo" peer_id=PeerId("12D3KooWLXG6aZ5NMLSXpY3jMKRYzitmnZRw4U31vp5sW78KEgFC") addresses="Parity Polkadot/v1.0.0-1ed6e2e50a4 (lN93pbg5)" version={"/ip4/37.59.23.143/tcp/30333/p2p/12D3KooWLXG6aZ5NMLSXpY3jMKRYzitmnZRw4U31vp5sW78KEgFC", "/ip4/138.59.133.242/tcp/30333/p2p/12D3KooWLXG6aZ5NMLSXpY3jMKRYzitmnZRw4U31vp5sW78KEgFC", "/ip6/2001:41d0:306:348f::/tcp/30333/p2p/12D3KooWLXG6aZ5NMLSXpY3jMKRYzitmnZRw4U31vp5sW78KEgFC"

@Polkadot-Forum
Copy link

This issue has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/async-backing-development-updates/6176/8

@Polkadot-Forum
Copy link

This issue has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/polkadot-digest-12-apr-2024/7404/1

@alexggh
Copy link
Contributor Author

alexggh commented Apr 23, 2024

Kusama async backing enabled with the enactment of https://kusama.subsquare.io/fellowship/referenda/70.
Triaging the dashboard doesn't show any issues, parachains are producing blocks at the same rate:

Screenshot 2024-04-23 at 07 46 26

The only side effect of enablement is that system parachains will see occasionally the following error:
no space left for the block in the unincluded segment

The error is harmless it is just the collator trying to build a block sooner than its parent being included and the parachain runtime rightly rejecting it. The error will go away once all system parachains get migrated to async backing with: polkadot-fellows/runtimes#266

Full stack

2024-04-23 07:44:48.445	             at ./rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys/unix/thread.rs:108:17
2024-04-23 07:44:48.445	  14: std::sys::unix::thread::Thread::new::thread_start
2024-04-23 07:44:48.445	             at ./rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/alloc/src/boxed.rs:2007:9
2024-04-23 07:44:48.445	  13: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
2024-04-23 07:44:48.445	             at ./rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/alloc/src/boxed.rs:2007:9
2024-04-23 07:44:48.445	  12: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
2024-04-23 07:44:48.445	  11: core::ops::function::FnOnce::call_once{{vtable.shim}}
2024-04-23 07:44:48.445	  10: std::sys_common::backtrace::__rust_begin_short_backtrace
2024-04-23 07:44:48.445	   9: tokio::runtime::task::raw::poll
2024-04-23 07:44:48.445	   8: tokio::runtime::scheduler::multi_thread::worker::run
2024-04-23 07:44:48.445	   7: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
2024-04-23 07:44:48.445	   6: tokio::runtime::task::raw::poll
2024-04-23 07:44:48.445	   5: <tracing_futures::Instrumented<T> as core::future::future::Future>::poll
2024-04-23 07:44:48.445	   4: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll
2024-04-23 07:44:48.445	   3: polkadot_parachain::service::start_asset_hub_node::{{closure}}::{{closure}}::{{closure}}
2024-04-23 07:44:48.445	   2: cumulus_client_consensus_aura::collators::basic::run::{{closure}}
2024-04-23 07:44:48.445	   1: <cumulus_client_consensus_proposer::Proposer<B,T> as cumulus_client_consensus_proposer::ProposerInterface<B>>::propose::{{closure}}
2024-04-23 07:44:48.445	   0: anyhow::error::<impl anyhow::Error>::new
2024-04-23 07:44:48.445	Stack backtrace:
2024-04-23 07:44:48.445	
2024-04-23 07:44:48.445	           5: 0x3345ba - <unknown>!BlockBuilder_finalize_block
2024-04-23 07:44:48.445	           4: 0x2d5705 - <unknown>!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::idle_and_finalize_hook::haa3fec3c6eaf9e9b
2024-04-23 07:44:48.445	           3: 0x2df180 - <unknown>!<cumulus_pallet_parachain_system::pallet::Pallet<T> as frame_support::traits::hooks::Hooks<<<<T as frame_system::pallet::Config>::Block as sp_runtime::traits::HeaderProvider>::HeaderT as sp_runtime::traits::Header>::Number>>::on_finalize::h5da12b36fe71b6b4
2024-04-23 07:44:48.445	           2: 0x10440 - <unknown>!core::option::expect_failed::h8c6285275ef25794
2024-04-23 07:44:48.445	           1: 0x20fa - <unknown>!core::panicking::panic_fmt::h0d63004982a68afd
2024-04-23 07:44:48.445	           0: 0x3ddc87 - <unknown>!rust_begin_unwind
2024-04-23 07:44:48.445	       error while executing at wasm backtrace:
2024-04-23 07:44:48.445	       WASM backtrace:
2024-04-23 07:44:48.445	    1: Execution failed: Execution aborted due to trap: wasm trap: wasm `unreachable` instruction executed
2024-04-23 07:44:48.445	           5: 0x3345ba - <unknown>!BlockBuilder_finalize_block
2024-04-23 07:44:48.445	           4: 0x2d5705 - <unknown>!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::idle_and_finalize_hook::haa3fec3c6eaf9e9b
2024-04-23 07:44:48.444	           3: 0x2df180 - <unknown>!<cumulus_pallet_parachain_system::pallet::Pallet<T> as frame_support::traits::hooks::Hooks<<<<T as frame_system::pallet::Config>::Block as sp_runtime::traits::HeaderProvider>::HeaderT as sp_runtime::traits::Header>::Number>>::on_finalize::h5da12b36fe71b6b4
2024-04-23 07:44:48.444	           2: 0x10440 - <unknown>!core::option::expect_failed::h8c6285275ef25794
2024-04-23 07:44:48.444	           1: 0x20fa - <unknown>!core::panicking::panic_fmt::h0d63004982a68afd
2024-04-23 07:44:48.444	           0: 0x3ddc87 - <unknown>!rust_begin_unwind
2024-04-23 07:44:48.444	       error while executing at wasm backtrace:
2024-04-23 07:44:48.444	       WASM backtrace:
2024-04-23 07:44:48.444	    0: Error at calling runtime api: Execution failed: Execution aborted due to trap: wasm trap: wasm `unreachable` instruction executed
2024-04-23 07:44:48.444	Caused by:
2024-04-23 07:44:48.444	
2024-04-23 07:44:48.444	2024-04-23 04:44:48 [Parachain] err=Error { inner: Proposing
2024-04-23 07:44:48.444	set_validation_data inherent needs to be present in every block!    
2024-04-23 07:44:48.444	2024-04-23 04:44:48 [Parachain] panicked at /home/builder/cargo/registry/src/index.crates.io-6f17d22bba15001f/cumulus-pallet-parachain-system-0.8.1/src/lib.rs:265:18:
2024-04-23 07:44:48.444	    7: 0x334223 - <unknown>!BlockBuilder_apply_extrinsic. Dropping.    
2024-04-23 07:44:48.444	    6: 0x2d4c66 - <unknown>!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::apply_extrinsic::h1bf27714e70715ae
2024-04-23 07:44:48.444	    5: 0x30f32 - <unknown>!<asset_hub_kusama_runtime::RuntimeCall as sp_runtime::traits::Dispatchable>::dispatch::h03a6f447d0a4c78a
2024-04-23 07:44:48.444	    4: 0x335d4 - <unknown>!<asset_hub_kusama_runtime::RuntimeCall as frame_support::traits::dispatch::UnfilteredDispatchable>::dispatch_bypass_filter::h8d01a3f74b8dd600
2024-04-23 07:44:48.444	    3: 0x3a27f - <unknown>!<cumulus_pallet_parachain_system::pallet::Call<T> as frame_support::traits::dispatch::UnfilteredDispatchable>::dispatch_bypass_filter::{{closure}}::hd9ed944fef5b0ce3
2024-04-23 07:44:48.444	    2: 0xc108a - <unknown>!cumulus_pallet_parachain_system::pallet::Pallet<T>::set_validation_data::{{closure}}::hcb921fabd3befceb
2024-04-23 07:44:48.444	    1: 0x20fa - <unknown>!core::panicking::panic_fmt::h0d63004982a68afd
2024-04-23 07:44:48.444	    0: 0x3ddc87 - <unknown>!rust_begin_unwind
2024-04-23 07:44:48.444	error while executing at wasm backtrace:
2024-04-23 07:44:48.444	WASM backtrace:
2024-04-23 07:44:48.444	2024-04-23 04:44:48 [Parachain] ❗️ Inherent extrinsic returned unexpected error: Error at calling runtime api: Execution failed: Execution aborted due to trap: wasm trap: wasm `unreachable` instruction executed
2024-04-23 07:44:48.444	2024-04-23 04:44:48 [Parachain] 1 storage transactions are left open by the runtime. Those will be rolled back.    
2024-04-23 07:44:48.444	2024-04-23 04:44:48 [Parachain] 1 storage transactions are left open by the runtime. Those will be rolled back.    
2024-04-23 07:44:48.436	no space left for the block in the unincluded segment   

@s0me0ne-unkn0wn
Copy link
Contributor

The error will go away once all system parachains get migrated to async backing with: polkadot-fellows/runtimes#266

Indeed, it will go away sooner when collators upgrade to include #3630

@alexggh
Copy link
Contributor Author

alexggh commented May 9, 2024

Async backing has been enabled with https://polkadot.subsquare.io/referenda/688 on polkadot, closing the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Completed
Development

No branches or pull requests

4 participants