Update to hashbrown 0.11 (MSRV 1.49)

[cuviper]

No description provided.

[cuviper]

I've left this as a draft because I'm not keen on such a recent MSRV. Technically, hashbrown's change only really needed 1.38, but they chose to require 1.49 for future flexibility. And that's fine for their semver break, but awkward for us if we try to update behind the scenes in a minor release.

[cuviper]

MSRV aside, it looks like performance has generally improved:

$ cargo benchcmp hashbrown-0.9 hashbrown-0.11 --threshold 2
 name                                         hashbrown-0.9 ns/iter  hashbrown-0.11 ns/iter  diff ns/iter   diff %  speedup
 few_retain_indexmap_100_000                  466,828                451,769                      -15,059   -3.23%   x 1.03
 grow_fnv_hashmap_100_000                     2,045,277              2,003,353                    -41,924   -2.05%   x 1.02
 grow_fnv_indexmap_100_000                    3,273,339              2,404,784                   -868,555  -26.53%   x 1.36
 half_retain_hashmap_100_000                  758,590                739,954                      -18,636   -2.46%   x 1.03
 indexmap_clone_for_sort_s                    337,654                374,439                       36,785   10.89%   x 0.90
 indexmap_merge_shuffle                       403,669                357,382                      -46,287  -11.47%   x 1.13
 indexmap_merge_simple                        334,023                287,750                      -46,273  -13.85%   x 1.16
 indexmap_simple_sort_u32                     617,842                645,138                       27,296    4.42%   x 0.96
 insert_hashmap_100_000                       1,909,170              1,868,587                    -40,583   -2.13%   x 1.02
 insert_hashmap_string_10_000                 869,639                788,938                      -80,701   -9.28%   x 1.10
 insert_hashmap_string_10_000                 781,690                849,330                       67,640    8.65%   x 0.92
 insert_hashmap_string_oneshot_10_000         732,745                796,495                       63,750    8.70%   x 0.92
 insert_indexmap_100_000                      2,233,657              2,139,867                    -93,790   -4.20%   x 1.04
 insert_indexmap_10_000                       197,870                191,055                       -6,815   -3.44%   x 1.04
 insert_indexmap_150                          3,051                  2,923                           -128   -4.20%   x 1.04
 insert_indexmap_int_bigvalue_10_000          259,843                252,176                       -7,667   -2.95%   x 1.03
 insert_indexmap_string_10_000                778,259                711,972                      -66,287   -8.52%   x 1.09
 insert_indexmap_string_10_000                700,151                805,230                      105,079   15.01%   x 0.87
 iter_black_box_indexmap_10_000               4,041                  3,016                         -1,025  -25.37%   x 1.34
 lookup_hashmap_10_000_exist                  71,975                 70,509                        -1,466   -2.04%   x 1.02
 lookup_hashmap_10_000_noexist                68,321                 66,301                        -2,020   -2.96%   x 1.03
 lookup_indexmap_100_000_inorder_multi        88,308                 85,774                        -2,534   -2.87%   x 1.03
 lookup_indexmap_100_000_multi                120,509                117,904                       -2,605   -2.16%   x 1.02
 lookup_indexmap_100_000_single               24                     23                                -1   -4.17%   x 1.04
 lookup_indexmap_10_000_exist                 89,724                 87,527                        -2,197   -2.45%   x 1.03
 lookup_indexmap_10_000_exist_string_oneshot  143,628                140,333                       -3,295   -2.29%   x 1.02
 lookup_indexmap_10_000_noexist               71,752                 70,296                        -1,456   -2.03%   x 1.02
 new_hashmap                                  4                      5                                  1   25.00%   x 0.80
 shift_remove_indexmap_100_000_few            10,942,364             10,567,369                  -374,995   -3.43%   x 1.04
 shift_remove_indexmap_2_000_full             2,617,014              2,471,701                   -145,313   -5.55%   x 1.06
 with_capacity_10e5_hashmap                   164                    168                                4    2.44%   x 0.98

insert_indexmap_string_10_000 is a weird case because we have identical benchmarks in bench.rs and in faststring.rs, but their performance went different directions. I guess this is pretty sensitive to codegen-units, inlining, etc.

[bluss]

Oh that's a big improvement. And this is the hashbrown with the compile-time improvements too, I assume (The rust-lang/rust/pull/77566 issue)

[bluss]

Yes I agree with you about waiting with the update.

Since rustc uses us, I guess they might want that we update, though. With that thought, we can't wait too long.

[cuviper]

Since rustc uses us, I guess they might want that we update, though. With that thought, we can't wait too long.

Maybe so, but rustc is not super aggressive with its own dependency updates. AFAICS indexmap is the only direct dependent on hashbrown in the compiler, and that doesn't share the sysroot build of hashbrown used by std, even when the semantic versions match. (Although, I wonder if it could somehow?)

[klensy]

Rustc switched to hashbrown 0.11 some time ago, btw.

[cuviper]

FWIW, my own primary concern for MSRV is for RHEL, but we just released RHEL 8.4.0 with Rust 1.49!

So I would be OK with bumping this now, if there are no other objections.

[bluss]

I wish we could be less agressive about raising the MSRV. Our problem is that hashbrown is a core dependency, and they have already bumped.

[cuviper]

I wish we could be less agressive about raising the MSRV.

Is there perhaps a time-based policy you'd be comfortable with? e.g. For Rayon we chose to stay at least 1 year old for MSRV, and we haven't heard any complaints since we started that a couple years ago.

Rust 1.49 was released on New Year's Eve, 2020-12-31, so it's now 5 months old.

[xMAC94x]

one alternative if you want want to raise MSRV is to allow both versions of hashbrown via hashbrown = { version = ">=0.9, <0.12" } if they compile fine.

[bluss]

Yes that's a good idea - can the distributions work with that? Cargo isn't "nice" about combining dependency ranges, so for most users that change would be exactly the same as the change in the PR as it is at present. But if distributions (like debian stable) can work with it, it is worthwhile to use a range like that.

[bluss]

@cuviper We could just go with having desired-required time limits. Let's say 1 year is desired, but we still go with this change now, because we just have to follow hashbrown (our major upstream), so we put a "required" limit on that, which is less than 1 year, for example a few releases.

[bluss]

Let's merge this now, current Rust version is 1.53.

[cuviper]

OK, I'll merge and publish it!

[bluss]

Thanks for doing it!