Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: route commands to correct persistence service #9435

Merged
merged 3 commits into from
Jul 12, 2024

Conversation

Rjected
Copy link
Member

@Rjected Rjected commented Jul 10, 2024

This refactors the persistence related services, renaming the Persistence service to the database service. This is because it mainly writes to the database. This also introduces a new PersistenceHandle type, which sends to the static file or database service, depending on the command. This type should be used for consumers of the db / static file services.

TODO:

  • helpers for initializing persistence / static file services, since they require handles to each other

@Rjected Rjected added C-enhancement New feature or request A-db Related to the database A-engine Related to the engine implementation labels Jul 10, 2024
Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

like the direction

@fgimenez for vis

@Rjected Rjected force-pushed the dan/use-higher-level-sender branch 2 times, most recently from 9f2ae54 to 7083c5e Compare July 11, 2024 21:30
@Rjected
Copy link
Member Author

Rjected commented Jul 11, 2024

This is RFR now, merge conflicts resolved with recent tests, but the PersistenceHandle API is basically the same, so no huge changes from the consuming side

@Rjected Rjected marked this pull request as ready for review July 11, 2024 21:33
@Rjected Rjected force-pushed the dan/use-higher-level-sender branch from 9a5f979 to bc38020 Compare July 11, 2024 22:46
// Write state and changesets to the database.
// Must be written after blocks because of the receipt lookup.
let execution_outcome = block.execution_outcome().clone();
execution_outcome.write_to_storage(&provider_rw, None, OriginalValuesKnown::No)?;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw, I find this api a bit horrible cc @joshieDo

Copy link
Collaborator

@joshieDo joshieDo Jul 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah i agree, should be trait/type impl

noted

Comment on lines +118 to +123
let static_file_service =
StaticFileService::new(provider_factory, static_file_service_rx, database_handle);
std::thread::Builder::new()
.name("Static File Service".to_string())
.spawn(|| static_file_service.run())
.unwrap();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, why are those two separate threads?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mainly for organization, but they could be combined

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going to leave combining them into a followup, should not be hard

{
/// This is the main loop, that will listen to database events and perform the requested
/// database actions
pub fn run(mut self) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need a loop here? according to spawn_services this run method is going to be run on a separate thread, not polled?

Copy link
Member

@fgimenez fgimenez Jul 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, this will be handled by the while and sender is not expected to be dropped, nvm

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this has also made me realize that the current run loop implementation in EngineApiTreeHandlerImpl was wrong, fixed it and updated the test here 0b100ee

Rjected and others added 3 commits July 12, 2024 16:15
feat: add spawning for the persistence task

rm duplicate key

fix compilation

fix compilation again

fix docs

fix test receipts for single default block
Co-authored-by: Federico Gimenez <[email protected]>
Co-authored-by: Matthias Seitz <[email protected]>
@Rjected Rjected force-pushed the dan/use-higher-level-sender branch from 177ff25 to f0e2538 Compare July 12, 2024 21:51
@Rjected
Copy link
Member Author

Rjected commented Jul 12, 2024

going to merge this, and start working on some tests for existing code + scope out remaining work

@Rjected Rjected added this pull request to the merge queue Jul 12, 2024
Merged via the queue into main with commit 39c5382 Jul 12, 2024
32 checks passed
@Rjected Rjected deleted the dan/use-higher-level-sender branch July 12, 2024 22:29
@gakonst
Copy link
Member

gakonst commented Jul 12, 2024

remaining work includes the "on reorg truncate static files" right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-db Related to the database A-engine Related to the engine implementation C-enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants