Reduce TX/RX buffers allocation

[Mallets]

This PR:

1. Lazily allocates the TX batches upon link creation;
2. Adopts a less-aggressive policy for allocating excessive memory for RX buffers.

This allows to reduce the number of allocated batches that are not utilised.
In general it should improve the overall memory consumption when many links are established.

Validation

Target

AMD Ryzen 7 5800X 8-Core Processor
32GB of RAM

Config

Queue configuration (queue.conf.json5)

{
  transport: {
    link: {
      tx: {
        queue: {
          size: {
            control: 1,
            real_time: 1,
            interactive_high: 1,
            interactive_low: 1,
            data_high: 2,
            data: 4,
            data_low: 4,
            background: 4,
          },
          allocation: {
	    // mode: "init",
            mode: "lazy",
          },
        },
      },
    },
  },
}

Test

Run zenohd:

./target/release/zenohd --no-multicast-scouting -l tcp/localhost:7447 -c queue.conf.json5

Run 100 z_sub:

for i in `seq 1 100`; do (./target/release/examples/z_sub --no-multicast-scouting -l tcp/localhost:0 -e tcp/localhost:7447 -c queue.conf.json5 &); done

Run 100 z_pub:

for i in `seq 1 100`; do (./target/release/examples/z_pub --no-multicast-scouting -l tcp/localhost:0 -e tcp/localhost:7447 -c queue.conf.json5 &); done

Results

Baseline memory usage: 3.45G.
Memory mode:

• Init: 14.9G/31.3G → 11.45G in total (56.97M per process)
• Lazy: 10.2G/31.3G → 6.75G in total (33.58M per process)

In case of z_sub / z_pub the gain is 4.7G less memory.

[github-actions]

PR missing one of the required labels: {'new feature', 'documentation', 'enhancement', 'bug', 'dependencies', 'internal', 'breaking-change'}

[wyfo]

In pipeline.rs of main branch, if we insert the following line between lines 268 and 269:

None => break WBatch::new_ephemeral(self.batch_config),

the throughput is then increased by 1Mmsg/s for 8B messages on my computer (MacBook Air M3). The effect of this line is that each batch is freshly allocated instead of going through the recycling pipeline.
It tells us two things:

• the current default batch count (4) may not be optimized to reach the best throughput
• amortized allocations may not be so costly

If we then modify the code to limit the number of concurrent allocated batches to the one of the config, still keeping the allocation on demand behavior, we obtain this time a small throughput reduction, around 100kmsg/s or less, so 1.6% of 6.3Mmsg/s. This should be the cost of allocating on demand, so not very high regarding throughput. On the other hand, it gives a fully elastic memory consumption, that could decrease the total memory footprint.

This allocation on demand behavior could even be customized to start with a smaller batch, for example 1kB, and only increase the size if in case of demanding throughput/big payloads.

[wyfo]

Is there a result that shows that the first allocation has a sensible effect, which would justify the need of this configuration addition?

[Mallets]

I agree that 4 may not be ideal for the default number of batches.
I'm also thinking that reducing the default number of batches to 2 with a lazily memory allocation would make the best default. However, I'd prefer leaving changing the default parameters to a separate PR.

Regarding the sensible effects on the first allocation, I didn't observer any.
But for the sake of not introducing changes to the default behaviour, I added the configuration support.

I can do a new PR for changing the defaults on top of this one for better traceability.

[wyfo]

If there is no sensible effect, I would be against adding more stuff to the configuration.

[Mallets]

I see your point, but the lack of sensible effect is the effect we measured on our machines.
It doesn't mean no sensible effect will be the norm to every user on every target platform.

[Mallets]

#1751 changes the default configuration and targets this PR branch.

[wyfo]

You decide then.