Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Serialize bytes-based stored structs faster #179

Merged
merged 3 commits into from
Jan 19, 2022
Merged

feat: Serialize bytes-based stored structs faster #179

merged 3 commits into from
Jan 19, 2022

Conversation

huitseeker
Copy link
Contributor

Without specialization, Rust forces Serde to treat &[u8] just like any other slice and Vec just like any other vector (and iterates through elements). Serde can do better.

This uses [serde_with]'s features to opt into serialization optimizations originally brought about by serde_bytes on these bytes-based types we serialize often. Most specifically it uses serde_with's serde_as attribute.

See https://github.com/jonasbb/serde_with/pull/277/files

Perf: cargo test cert -- --nocapture in fastx_types
With this PR:

Write Cert: 61 microsec

Without:

Write Cert: 100 microsec

Perf: cargo run --release --bin bench -- --db-cpus=2 --tcp-connections=10
With this PR:

    Total time: 2586051us, items: 40000, tx/sec: 15467.599053537613

Without:

   Total time: 2759989us, items: 40000, tx/sec: 14492.811384393199

@gdanezis
Copy link
Collaborator

Oh I like that. Can we use this for the content of objects? This is likely to be the biggest byte string of them all.

@huitseeker
Copy link
Contributor Author

With:

Write Cert: 52 microsec
Write Order: 223 microsec
Write Vote: 229 microsec

Without:

Write Cert: 93 microsec
Write Vote: 264 microsec
Write Order: 274 microsec

@sblackshear
Copy link
Collaborator

Both the object trick and the #'s associated with it LGTM! Thanks.

Without specialization, Rust forces Serde to treat &[u8] just like any other slice and Vec<u8> just like any other vector.

This uses `[serde_with]`'s features to opt into serialization optimizations originally brought about by [`serde_bytes`](https://crates.io/crates/serde_bytes) on these bytes-based types we serialize often.
See https://github.com/jonasbb/serde_with/pull/277/files
@huitseeker huitseeker merged commit 0d83a72 into MystenLabs:main Jan 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants