Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(store): SS Raw Benchmarks #17504

Merged
merged 35 commits into from
Aug 29, 2023
Merged

Conversation

alexanderbez
Copy link
Contributor

@alexanderbez alexanderbez commented Aug 22, 2023

Description

Raw benchmarks, i.e. benchmarks of raw API calls against each SS backend engine.

$ nix develop
<nix-shell> $ go test -tags rocksdb -benchmem -benchtime=5s -run=^$ -bench ^Benchmark ./store/storage/ -v

https://pastebin.com/raw/q5LhTcRy


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title
  • added ! to the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • followed the guidelines for building modules
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • included comments for documenting Go code
  • updated the relevant documentation or specification
  • reviewed "Files changed" and left comments if necessary
  • run make lint and make test
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed ! in the type prefix if API or client breaking change
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic
  • reviewed API design and naming
  • reviewed documentation is accurate
  • reviewed tests and test coverage
  • manually tested (if applicable)

@alexanderbez alexanderbez changed the title [DNM]: SS Raw Benchmarks test(store): [DNM] SS Raw Benchmarks Aug 22, 2023
@yihuang
Copy link
Collaborator

yihuang commented Aug 23, 2023

BTW, the esasist way for me to setup rocksdb to run the test is:

$ nix develop
<nix-shell> $ go test -tags rocksdb -benchmem -benchtime=5s -run=^$ -bench ^Benchmark ./store/storage/ -v

@alexanderbez
Copy link
Contributor Author

So neat @yihuang! I really need to spend more time playing with nix and getting to know it. Thanks for sharing. I'll use this going forward as it's much easier!

@alexanderbez alexanderbez changed the title test(store): [DNM] SS Raw Benchmarks test(store): SS Raw Benchmarks Aug 23, 2023
Base automatically changed from bez/btree-ss-backend to feature/store-v2 August 23, 2023 17:58
return pebbledb.New(dataDir)
},
"btree(sqlite)": func(dataDir string) (store.VersionedDatabase, error) {
"btree_sqlite": func(dataDir string) (store.VersionedDatabase, error) {
Copy link
Collaborator

@yihuang yihuang Aug 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we keep the btree prefix to the sqlite, but no lsm prefix to the other dbs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My intention was to indicate that it's not a direct BTree implementation, but rather an existing storage engine that utilizes BTrees under the hood.

I can add the type to all of them if you'd like.

Copy link
Collaborator

@yihuang yihuang Aug 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean we are just benchmarking sqlite, not btree, and we can't simply benchmark btree without referring to a specific implementation, and there are so many design decisions a db implementation need to make.

@alexanderbez alexanderbez marked this pull request as ready for review August 24, 2023 03:17
@alexanderbez alexanderbez requested a review from a team as a code owner August 24, 2023 03:17
Copy link
Contributor

@cool-develope cool-develope left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just curious about the result

@alexanderbez alexanderbez merged commit a7e7671 into feature/store-v2 Aug 29, 2023
@alexanderbez alexanderbez deleted the bez/ss-raw-benchmarks branch August 29, 2023 02:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants