fix: sealing pipeline: Fix PC1 retry loop

[magik6k]

Related Issues

• Any issue which refers PC1 loops

Proposed Changes

Instead of retrying PC1 forever, only retry PC1 up to 3 times

Additional Info

This fixes the issue where PC1 will be retried e.g. in cases where for whatever reason unsealed data doesn't match what should have come from deals:

	seal pre commit(1) failed: storage call error 0: presealing sector 6705 (/data/4/store/unsealed/s-t02620-6705): pieces and comm_d do not match [name: okonomiyaki]: presealing sector 6705 (/data/4/store/unsealed/s-t02620-6705): pieces and comm_d do not match
7928.	2023-07-18 21:52:31 +0200 CEST:	[event;sealing.SectorRetrySealPreCommit1]	{"User":{}}
7929.	2023-07-18 21:54:09 +0200 CEST:	[event;sealing.SectorSealPreCommit1Failed]	{"User":{}}
	seal pre commit(1) failed: storage call error 0: presealing sector 6705 (/data/4/store/unsealed/s-t02620-6705): pieces and comm_d do not match [name: okonomiyaki]: presealing sector 6705 (/data/4/store/unsealed/s-t02620-6705): pieces and comm_d do not match
7930.	2023-07-18 21:55:09 +0200 CEST:	[event;sealing.SectorRetrySealPreCommit1]	{"User":{}}
7931.	2023-07-18 21:56:45 +0200 CEST:	[event;sealing.SectorSealPreCommit1Failed]	{"User":{}}
	seal pre commit(1) failed: storage call error 0: presealing sector 6705 (/data/4/store/unsealed/s-t02620-6705): pieces and comm_d do not match [name: okonomiyaki]: presealing sector 6705 (/data/4/store/unsealed/s-t02620-6705): pieces and comm_d do not match
7932.	2023-07-18 21:57:45 +0200 CEST:	[event;sealing.SectorRetrySealPreCommit1]	{"User":{}}
7933.	2023-07-18 21:59:24 +0200 CEST:	[event;sealing.SectorSealPreCommit1Failed]	{"User":{}}
	seal pre commit(1) failed: storage call error 0: presealing sector 6705 (/data/4/store/unsealed/s-t02620-6705): pieces and comm_d do not match [name: okonomiyaki]: presealing sector 6705 (/data/4/store/unsealed/s-t02620-6705): pieces and comm_d do not match

Checklist

Before you mark the PR ready for review, please make sure that:

• Commits have a clear commit message.
• PR title is in the form of of <PR type>: <area>: <change being made>
  • example: fix: mempool: Introduce a cache for valid signatures
  • PR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, test
  • area, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps
• New features have usage guidelines and / or documentation updates in
  • Lotus Documentation
  • Discussion Tutorials
• Tests exist for new functionality or change in behavior
• CI is green

[magik6k]

Verified that it works on my setup:

[before restart, lots of infinite retries]
3988.	2023-07-18 23:11:29 +0200 CEST:	[event;sealing.SectorRetrySealPreCommit1]	{"User":{}}
3989.	2023-07-18 23:12:56 +0200 CEST:	[event;sealing.SectorSealPreCommit1Failed]	{"User":{}}
	seal pre commit(1) failed: storage call error 0: presealing sector 6705 (/data/4/store/unsealed/s-t02620-6705): pieces and comm_d do not match [name: okonomiyaki]: presealing sector 6705 (/data/4/store/unsealed/s-t02620-6705): pieces and comm_d do not match
3990.	2023-07-18 23:13:56 +0200 CEST:	[event;sealing.SectorRetrySealPreCommit1]	{"User":{}}
3991.	2023-07-18 23:15:36 +0200 CEST:	[event;sealing.SectorSealPreCommit1Failed]	{"User":{}}
	seal pre commit(1) failed: storage call error 0: presealing sector 6705 (/data/4/store/unsealed/s-t02620-6705): pieces and comm_d do not match [name: okonomiyaki]: presealing sector 6705 (/data/4/store/unsealed/s-t02620-6705): pieces and comm_d do not match
3992.	2023-07-18 23:16:36 +0200 CEST:	[event;sealing.SectorRetrySealPreCommit1]	{"User":{}}
3993.	2023-07-18 23:24:06 +0200 CEST:	[event;sealing.SectorSealPreCommit1Failed]	{"User":{}}
	seal pre commit(1) failed: storage call error 101: worker [name: okonomiyaki] restarted
3994.	2023-07-18 23:25:06 +0200 CEST:	[event;sealing.SectorRetrySealPreCommit1]	{"User":{}}
3995.	2023-07-18 23:29:07 +0200 CEST:	[event;sealing.SectorSealPreCommit1Failed]	{"User":{}}
	seal pre commit(1) failed: storage call error 0: presealing sector 6705 (/data/4/store/unsealed/s-t02620-6705): pieces and comm_d do not match [name: okonomiyaki]: presealing sector 6705 (/data/4/store/unsealed/s-t02620-6705): pieces and comm_d do not match
3996.	2023-07-18 23:30:07 +0200 CEST:	[event;sealing.SectorRetrySealPreCommit1]	{"User":{}}
3997.	2023-07-18 23:31:44 +0200 CEST:	[event;sealing.SectorSealPreCommit1Failed]	{"User":{}}
	seal pre commit(1) failed: storage call error 0: presealing sector 6705 (/data/4/store/unsealed/s-t02620-6705): pieces and comm_d do not match [name: okonomiyaki]: presealing sector 6705 (/data/4/store/unsealed/s-t02620-6705): pieces and comm_d do not match
3998.	2023-07-18 23:32:44 +0200 CEST:	[event;sealing.SectorRetrySealPreCommit1]	{"User":{}}
3999.	2023-07-18 23:34:22 +0200 CEST:	[event;sealing.SectorSealPreCommit1Failed]	{"User":{}}
	seal pre commit(1) failed: storage call error 0: presealing sector 6705 (/data/4/store/unsealed/s-t02620-6705): pieces and comm_d do not match [name: okonomiyaki]: presealing sector 6705 (/data/4/store/unsealed/s-t02620-6705): pieces and comm_d do not match
4000.	2023-07-18 23:34:22 +0200 CEST:	[event;sealing.SectorRemove]	{"User":{}}
4001.	2023-07-18 23:34:37 +0200 CEST:	[event;sealing.SectorRemoved]	{"User":{}}

[magik6k]

Also added a more correct retry "soft" errors, like when the worker that was running the work restarts.

[shrenujbansal]

LGTM except a couple of nits!