Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: use sqlite/msgindex.db as epoch=>tipsetkey index to speed up eth_getBlockByNumber queries #10922

Closed

Conversation

i-norden
Copy link
Contributor

@i-norden i-norden commented May 25, 2023

Related Issues

Builds on-top of #10632
#10663
vulcanize/filecoin-indexing#20

Proposed Changes

in sqlite/msgindex.db we have a table which maps message_cids to epochs and tipset cids for the tipset in which these messages appear. We could also leverage this table as an index for looking up the tipset corresponding to a specific epoch when proceeding down the eth_getBlockByNumber search path described in vulcanize/filecoin-indexing#20 (comment) (assuming my understanding there is correct).

Additional Info

Opening this as a draft hoping to spur some discussion/feedback, I am not entirely confident in my line of thinking for this. Additionally, if the reasoning is sound, I would appreciate maintainer's opinion on how to best access this index from the ChainStore/ChainIndex (i.e. https://github.com/filecoin-project/lotus/pull/10922/files#diff-6032ffd931dc723fcaa5d9b7926dae0b7fa6bea74763d8e0a0693d2b45992976R164).

Checklist

Before you mark the PR ready for review, please make sure that:

  • Commits have a clear commit message.
  • PR title is in the form of of <PR type>: <area>: <change being made>
    • example: fix: mempool: Introduce a cache for valid signatures
    • PR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, test
    • area, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps
  • New features have usage guidelines and / or documentation updates in
  • Tests exist for new functionality or change in behavior
  • CI is green

This commit updates the GetMsgInfo to include the execution tipset of
the message (if it exists). This allows us to call GetTipSetByCid
instead of GetTipsetByHeight which should be considerably faster and
help speed up both StateSearcMessage and StateWaitForMessage calls.
@fridrik01
Copy link
Contributor

@i-norden is this not a duplicate of #10632?

@i-norden
Copy link
Contributor Author

i-norden commented May 26, 2023

@fridrik01 there is duplicate code here, I didn't see that PR- will rebase this on-top of that! The difference here is we also expose a GetTipsetByEpoch method on-top of the newly created tipset_epochs index to go from epoch => tipsetkey directly and then use that method in ChainIndex.GetTipsetByHeight. This is for enhancing the performance of queries such as eth_getBlockByNumber where we don't start with a message_cid and so we can't resolve a choice between GetTipSetByCid and GetTipsetByHeight by looking up MsgInfo by message_cid and then by the returned epoch + 1.

@i-norden i-norden force-pushed the ian/epoch_to_tipsetkey_index branch from 792ea07 to 0c47a8e Compare May 26, 2023 14:12
@i-norden
Copy link
Contributor Author

i-norden commented Jun 12, 2023

Hey @fridrik01 wanted to check if my response makes sense, and if so if this would be a desirable update. I believe such a GetTipsetByEpoch method could also replace the tipset skiplist as it is used in the eth_getTransactionReceipt code path as well.

@Stebalien Stebalien requested a review from fridrik01 June 26, 2023 16:12
@fridrik01
Copy link
Contributor

I don't think this is required as the skip list is very fast, I tried it on a FEVM archival node and IIRC it was in the order of 10ms so this won't help much. See also #10528 which was not merged due to the complexity not being worth it

@i-norden
Copy link
Contributor Author

Thanks @fridrik01, that makes sense.

@i-norden i-norden closed this Jun 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants