Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add perfect hashing function to nippy-jar #4601

Merged
merged 8 commits into from
Sep 20, 2023

Conversation

joshieDo
Copy link
Collaborator

@joshieDo joshieDo commented Sep 14, 2023

Adds two implementations of perfect hashing functions.

In computer science, a perfect hash function h for a set S is a hash function that maps distinct elements in S to a set of m integers, with no collisions. In mathematical terms, it is an injective function.

This is particularly useful for when querying transactions or blocks using their hashes. Instead of storing them, and iterating over each value when querying for a specific hash, we can use this function to give us an integer. Then, use that to find the offset of the provided value in the file

Example: If giving the transaction hash 0xdeadbeef to the function would provide us the integer 5, we could use this information to look-up in a separate list, the offset where the associated transaction is located.

@joshieDo joshieDo added C-enhancement New feature or request A-static-files Related to static files labels Sep 14, 2023
@joshieDo joshieDo requested a review from rakita as a code owner September 14, 2023 15:51
@joshieDo joshieDo mentioned this pull request Sep 14, 2023
@joshieDo joshieDo marked this pull request as draft September 14, 2023 15:55
@joshieDo joshieDo marked this pull request as ready for review September 14, 2023 15:56
@codecov
Copy link

codecov bot commented Sep 14, 2023

Codecov Report

Merging #4601 (45e3b05) into joshie/nippy-jar (d8cfa06) will increase coverage by 0.12%.
The diff coverage is 77.31%.

Impacted file tree graph

Files Changed Coverage Δ
crates/storage/nippy-jar/src/phf/fmph.rs 59.32% <59.32%> (ø)
crates/storage/nippy-jar/src/phf/go_fmph.rs 60.00% <60.00%> (ø)
crates/storage/nippy-jar/src/phf/mod.rs 93.33% <93.33%> (ø)
crates/storage/nippy-jar/src/compression/mod.rs 38.23% <100.00%> (+1.87%) ⬆️
crates/storage/nippy-jar/src/filter/cuckoo.rs 85.36% <100.00%> (ø)
crates/storage/nippy-jar/src/filter/mod.rs 83.33% <100.00%> (+1.51%) ⬆️
crates/storage/nippy-jar/src/lib.rs 92.78% <100.00%> (+1.91%) ⬆️

... and 18 files with indirect coverage changes

Flag Coverage Δ
integration-tests 16.50% <0.00%> (-0.04%) ⬇️
unit-tests 64.20% <77.31%> (+0.13%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
reth binary 30.85% <ø> (ø)
blockchain tree 83.58% <ø> (+0.74%) ⬆️
pipeline 90.53% <ø> (ø)
storage (db) 75.63% <77.31%> (+0.05%) ⬆️
trie 94.88% <ø> (+0.03%) ⬆️
txpool 49.74% <ø> (+0.47%) ⬆️
networking 77.32% <ø> (+0.10%) ⬆️
rpc 57.36% <ø> (+<0.01%) ⬆️
consensus 63.40% <ø> (+0.43%) ⬆️
revm 31.56% <ø> (ø)
payload builder 6.12% <ø> (ø)
primitives 86.52% <ø> (+0.06%) ⬆️

@joshieDo joshieDo requested a review from gakonst as a code owner September 14, 2023 16:34
@joshieDo joshieDo requested a review from shekhirin September 20, 2023 09:44
@joshieDo joshieDo merged commit 7e15b2a into joshie/nippy-jar Sep 20, 2023
@joshieDo joshieDo deleted the joshie/nippy-phf branch September 20, 2023 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-static-files Related to static files C-enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants