bootstrap-tools in libidn2's closure

[misuzu]

Describe the bug

libidn2 package has the bootstrap-tools package in it's closure for some reason.

$ nix why-depends /nix/store/c56lvnhbyb9pf5p7c7qfj7ki8b3r0dqv-netdata-1.32.1 /nix/store/p4s4jf7aq6v6z9iazll1aiqwb34aqxq9-bootstrap-tools
/nix/store/c56lvnhbyb9pf5p7c7qfj7ki8b3r0dqv-netdata-1.32.1
└───/nix/store/rdsmqcan6ws916mkqcqx92qdj0514680-libidn2-2.3.2-dev
    └───/nix/store/82jnccxb8djlhgzwb87d3jk01awaw9kb-libidn2-2.3.2-bin
        └───/nix/store/p4s4jf7aq6v6z9iazll1aiqwb34aqxq9-bootstrap-tools

$ nix path-info -shr /nix/store/c56lvnhbyb9pf5p7c7qfj7ki8b3r0dqv-netdata-1.32.1 | LC_ALL=C sort -k2 -hr | head
/nix/store/p4s4jf7aq6v6z9iazll1aiqwb34aqxq9-bootstrap-tools                      126.5M
/nix/store/hpppiam2slc5x9jbf7g67051bygf0bzi-perl-5.34.1                           52.9M
/nix/store/c56lvnhbyb9pf5p7c7qfj7ki8b3r0dqv-netdata-1.32.1                        31.1M
/nix/store/lyl6nysc3i3aqhj6shizjgj0ibnf1pvg-glibc-2.34-210                        29.2M
/nix/store/c2785a9h9138impj7wjk16g6md0wg2p1-netdata-go.d.plugin-0.31.0            27.6M
/nix/store/j3jkxwyidvqyzm6pfsscgga9rzqqh9yk-freeipmi-1.6.9                        13.8M
/nix/store/d86ic2p5ym09zrh8h06px677bifah21s-file-5.41                              7.2M
/nix/store/g9x3cm8wdmpm86pyiff6arx8skmdwkn7-gcc-11.3.0-lib                         6.1M
/nix/store/8jj82lv4a1wd2vlvf0ppk3vxnqg619j8-util-linux-minimal-2.37.4-bin          6.1M
/nix/store/sysqj1krpfqw1mjph9x7i5sniw07rn24-shadow-4.11.1                          4.0M

Expected behavior

No bootstrap-tools in system closure unless really needed.

Additional context

Something happened with libidn2 between 437e1f6 and 500a8ff.

https://hydra.nixos.org/build/111559400#tabs-details
https://hydra.nixos.org/build/112044947#tabs-details

Notify maintainers

@fpletz

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

> nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.41, NixOS, 22.11 (Raccoon), 22.11.20220526.83658b2`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.8.1`
 - nixpkgs: `/nix/store/ny7x54058ki7k1d3ny0jx5h6kw926mxv-source`

[trofi]

A bit of debugging:

depgraph (via $ nix-store --query --graph $(nix-build -A libidn2) | dot -Tsvg > /tmp/depgraph.svg):

Confirm that libidn2.bin is linked against bootstrap glibc:

$ nix path-info -rsh $(nix-build -A libidn2.bin) |& fgrep bootstrap
these 2 derivations will be built:
  /nix/store/xvrq6jbjaxs7flgaa9xhw2iyxpwa9rkw-libunistring-1.0.drv
  /nix/store/r50vgyqgwpa5xmhl9f00fq3ysa5cd067-libidn2-2.3.2.drv
/nix/store/i3ibpx67yncp4w4mpkf5pwvjjsd0aqln-bootstrap-tools                      126.5M
/nix/store/39k40hf9z4wr5wac5xbnznza1ym2f8kz-bootstrap-stage0-glibc-bootstrap     608.0

We explicitly use bootstrap glibc (via interpreter):

$ LANG=C readelf -a /nix/store/v7zppsi2kmj2zzrjr1mjaqvl2p7sv1jh-libidn2-2.3.2-bin/bin/idn2 |& fgrep bootstrap
      [Requesting program interpreter: /nix/store/39k40hf9z4wr5wac5xbnznza1ym2f8kz-bootstrap-stage0-glibc-bootstrap/lib/ld-linux-x86-64.so.2]
 0x000000000000001d (RUNPATH)            Library runpath: [/nix/store/gymz5nlyi567m9rbi6gmqc19glgvldsw-libidn2-2.3.2/lib:/nix/store/7mm9v9csgjhhkrblgv180ns024997sg7-libunistring-1.0/lib:/nix/store/gymz5nlyi567m9rbi6gmqc19glgvldsw-libidn2-2.3.2/lib64:/nix/store/39k40hf9z4wr5wac5xbnznza1ym2f8kz-bootstrap-stage0-glibc-bootstrap/lib:/nix/store/i3ibpx67yncp4w4mpkf5pwvjjsd0aqln-bootstrap-tools/lib]

Looking at our bootstrap steps we carefully thread libidn2 built at stage2 through stage3, stage4 and final as it's a low level dependency of glibc. We carefully mangle references in $out output (libraries), but not the $bin one.

netdata happens to retain libidn2.bin reference.

[trofi]

Also netdata itself retains too much of it's build closure (including likely redundant libidn.dev, libidn.out) as it persists ./configure call as is:

$ strings /nix/store/vh5xn98669bndmximy8lgq5cvzr2c53y-netdata-1.34.1/bin/netdata | fgrep libidn
 '--disable-dependency-tracking' '--prefix=/nix/store/vh5xn98669bndmximy8lgq5cvzr2c53y-netdata-1.34.1' '--localstatedir=/var' '--sysconfdir=/etc' '--disable-ebpf' '--enable-cloud' '--with-aclk-ng' 'CC=gcc' 'CXX=g++' 'PKG_CONFIG=pkg-config' 'PKG_CONFIG_PATH=/nix/store/qzv0bjnwh6pfx8k9balps2zagjhccf5p-curl-7.83.1-dev/lib/pkgconfig:/nix/store/0j3088m6kpp24sb9wv4391xkn7xmlnf9-brotli-1.0.9-dev/lib/pkgconfig:/nix/store/hpmr9yb2xpfax0iras79r23hkwx1qghq-libkrb5-1.19.3-dev/lib/pkgconfig:/nix/store/dk2qbvhx9jac56f7jg3xv9rm3m22kdvf-nghttp2-1.47.0-dev/lib/pkgconfig:/nix/store/a8vs4ynvd3bkra7m59drlbzdc3ip77i9-libidn2-2.3.2-dev/lib/pkgconfig:/nix/store/ai66a14mnz4df6ryw9spic9lvj1rvgdq-openssl-1.1.1o-dev/lib/pkgconfig:/nix/store/x6p8mankj1bywgvy19bgah00l64pg1mv-libssh2-1.10.0-dev/lib/pkgconfig:/nix/store/36xafmi2kf4rq0k24j272mwc83n26qhb-zlib-1.2.12-dev/lib/pkgconfig:/nix/store/mwqx74axg4za4kkyb8ycillkrkx0rp8k-zstd-1.5.2-dev/lib/pkgconfig:/nix/store/yp3l775jh4xnx9zn7jn7r4ilh46dp81i-protobuf-3.19.4/lib/pkgconfig:/nix/store/w4l0id9d71k0sw5miaha9vm8b5f0zhhl-libcap-2.63-dev/lib/pkgconfig:/nix/store/0mr9zy57zm01cpbqlacw5d56alhx2n1h-attr-2.5.1-dev/lib/pkgconfig:/nix/store/9kcvm3lcmjvmdylch1jg4fxng3x6lf3y-util-linux-minimal-2.37.4-dev/lib/pkgconfig:/nix/store/6p1zsvjy4vgm6m5lg8n8dqk0cfnsygvg-libuv-1.44.1/lib/pkgconfig:/nix/store/qbs73p01hv6k6w8xjcrhdrf49bwwc36s-lz4-1.9.3-dev/lib/pkgconfig:/nix/store/lpw8njxp6wj35qkan5ik7hh6jp5fs1ms-freeipmi-1.6.9/lib/pkgconfig:/nix/store/0j6dxrwdgj125xfah1qfncpbzbrcvpas-libmnl-1.0.5/lib/pkgconfig:/nix/store/5ly7si6ad4m5yg9f724yviwzqqwycqpx-libnetfilter_acct-1.0.3/lib/pkgconfig:/nix/store/7sq1shl9vx7n19skhivk9gpadv1brzd8-json-c-0.15-dev/lib/pkgconfig'

[trofi]

Proposed closure shrinking for netdata as #175719.

It does not fix underlying libidn2.bin reference. But at least it does not pull in too much.

[vcunat]

Something happened with libidn2

glibc got dependency on libidn2, instead of their not-very-good bundled code. That was in 2aea16c.

I propose yet another hack in PR #175785

[misuzu]

I guess this is fixed now.