Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allocate rx buffer #1768

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 1 addition & 5 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 0 additions & 6 deletions DEFAULT_CONFIG.json5
Original file line number Diff line number Diff line change
Expand Up @@ -514,12 +514,6 @@
},
/// Configure the zenoh RX parameters of a link
rx: {
/// Receiving buffer size in bytes for each link
/// The default the rx_buffer_size value is the same as the default batch size: 65535.
/// For very high throughput scenarios, the rx_buffer_size can be increased to accommodate
/// more in-flight data. This is particularly relevant when dealing with large messages.
/// E.g. for 16MiB rx_buffer_size set the value to: 16777216.
buffer_size: 65535,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we keep the config and mark it not available instead of removing it?
I'm not sure how much we want to maintain compatibility.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't know honestly. But this config is misleading, it's not a question of buffer size but of a number of batches, and the only case where more than one batch will be used is when a payload is cloned in a subscriber (e.g. a channel subscriber), so it's not the case for router for example.
So user may be tempted to increase the "buffer size" for a router with big traffic, but it will just allocate more unused memory.

/// Maximum size of the defragmentation buffer at receiver end.
/// Fragmented messages that are larger than the configured size will be dropped.
/// The default value is 1GiB. This would work in most scenarios.
Expand Down
6 changes: 6 additions & 0 deletions commons/zenoh-buffers/src/zslice.rs
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,12 @@ impl ZSlice {
self.len() == 0
}

#[doc(hidden)]
#[inline]
pub fn capacity(&self) -> usize {
self.buf.as_slice().len()
}

#[inline]
#[must_use]
pub fn as_slice(&self) -> &[u8] {
Expand Down
1 change: 0 additions & 1 deletion commons/zenoh-config/src/defaults.rs
Original file line number Diff line number Diff line change
Expand Up @@ -315,7 +315,6 @@ impl Default for BatchingConf {
impl Default for LinkRxConf {
fn default() -> Self {
Self {
buffer_size: BatchSize::MAX as usize,
max_message_size: 2_usize.pow(30),
}
}
Expand Down
6 changes: 0 additions & 6 deletions commons/zenoh-config/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -519,12 +519,6 @@ validated_struct::validator! {
threads: usize,
},
pub rx: LinkRxConf {
/// Receiving buffer size in bytes for each link
/// The default the rx_buffer_size value is the same as the default batch size: 65535.
/// For very high throughput scenarios, the rx_buffer_size can be increased to accommodate
/// more in-flight data. This is particularly relevant when dealing with large messages.
/// E.g. for 16MiB rx_buffer_size set the value to: 16777216.
buffer_size: usize,
/// Maximum size of the defragmentation buffer at receiver end (default: 1GiB).
/// Fragmented messages that are larger than the configured size will be dropped.
max_message_size: usize,
Expand Down
10 changes: 3 additions & 7 deletions commons/zenoh-sync/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ version = { workspace = true }
repository = { workspace = true }
homepage = { workspace = true }
authors = [
"kydos <[email protected]>",
"Luca Cominardi <[email protected]>",
"Pierre Avital <[email protected]>",
"kydos <[email protected]>",
"Luca Cominardi <[email protected]>",
"Pierre Avital <[email protected]>",
]
edition = { workspace = true }
license = { workspace = true }
Expand All @@ -32,10 +32,6 @@ description = "Internal crate for zenoh."
event-listener = { workspace = true }
futures = { workspace = true }
tokio = { workspace = true, features = ["sync"] }
zenoh-buffers = { workspace = true }
zenoh-collections = { workspace = true, features = ["default"] }
zenoh-core = { workspace = true }

[dev-dependencies]
tokio = { workspace = true, features = ["macros", "sync", "rt-multi-thread", "time"] }
zenoh-result = { workspace = true }
82 changes: 0 additions & 82 deletions commons/zenoh-sync/src/fifo_queue.rs

This file was deleted.

12 changes: 0 additions & 12 deletions commons/zenoh-sync/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,18 +28,6 @@ use futures::FutureExt;
pub mod event;
pub use event::*;

pub mod fifo_queue;
pub use fifo_queue::*;

pub mod lifo_queue;
pub use lifo_queue::*;

pub mod object_pool;
pub use object_pool::*;

pub mod mvar;
pub use mvar::*;

pub mod condition;
pub use condition::*;

Expand Down
81 changes: 0 additions & 81 deletions commons/zenoh-sync/src/lifo_queue.rs

This file was deleted.

Loading