deduplication causes more utilization instead of less?

[agowa]

Hi,

I've been running bees on a 12TB BTRFS volume and instead of freeing up disk space it slowly was eating away the free space (before about 200GB). The DB size is 16 G, there was nothing else accessing the drives while bees was running and after it ate away 160GBs and I was down to only 39GB of free space I "CTRL+C"-ed bees to avoid it from running out of space and potentially corrupting data.

So what am I doing wrong here?
How much space should I expect to be taken for e.g. "First run may require temporary disk space for extent reorganization" on a 12TB volume? Everything still normal??

What I did so far (nothing special, but for completeness):

Before I ran bees I create a subvolume on the to be deduplicated volume for the .beeshome directory and pointed the BEESHOME environment variable towards it. After that I created a file within /etc/bees/ with an UUID= and DB_SIZE= line within it. As the BTRFS volume is on a LVM I also created a symlink within /dev/disk/by-uuid/$partitionUUID towards the correct /dev/mapper/* device. Than finally I ran bees using beesd $UUID

My beesstats.txt looks like this:

Now:     2025-01-07-08-04-55
Uptime:  132042 seconds
Version: UNKNOWN
Kernel:  Linux 6.6.52-0-lts x86_64 #1-Alpine SMP PREEMPT_DYNAMIC 2024-09-19 08:02:59

Hash table page occupancy histogram (1073741458/1073741824 cells occupied, 99%)
                                                                 4194304 pages
                                                               # 2097152
                                                               # 1048576
                                                               # 524288
                                                               # 262144
                                                               # 131072
                                                               # 65536
                                                               # 32768
                                                               # 16384
                                                               # 8192
                                                               # 4096
                                                               # 2048
                                                               # 1024
                                                               # 512
                                                               # 256
                                                               # 128
                                                               # 64
                                                               # 32
                                                               # 16
                                                               # 8
                                                              ## 4
                                                              ## 2
                       #                  #                   ## 1
0%      |      25%      |      50%      |      75%      |   100% page fill
compressed 26557126 (2%)
uncompressed 1047184332 (97%) unaligned_eof 236002 (0%) toxic 13481 (0%)

bees -h shows:

bees version UNKNOWN
2025-01-07 10:20:31 18443.18443<7> bees: Masking signals
2025-01-07 10:20:31 18443.18443<7> bees: context constructed
2025-01-07 10:20:31 18443.18443<7> bees: Parsing option 'h'
Usage: bees [options] fs-root-path
...
2025-01-07 10:20:31 18443.18443<5> bees: Exiting with status 1 (failure)

apk reports the installed version of bees as: 0.10-r2

[Zygo]

Try the current master branch. v0.11 will have significant improvements against this kind of expansion. See #296 for details.