docker: allow grouping derivations into explicit layers

[Sohalt]

Motivation for this change

Sometimes the heuristic used by streamLayeredImage to distribute the store paths to individual layers based on usage is not ideal. E.g. when using graalvm17-ce that's a 1.7G, but it easily gets thrown into the last layer with a lot of other derivations. The last layer thereby grows very large and since it changes often it needs to be re-downloaded or pushed.

This change allows more control over how layers should be assembled. You can specify an optional explicitLayers attribute, containing a list of lists of derivations, corresponding to a list of layers containing one or more derivations.

E.g. expliticLayers = [[hello] [fortune cowsay]]specifies that, apart from the usual heuristic-based grouping, hello should be put into one separate layer and fortune and cowsay should be put into another separate layer.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 22.05 Release Notes (or backporting 21.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

[Sohalt]

I'll add some documentation and tests, if there is no objection to the general idea.

[roberth]

The general idea is ok, but would be subsumed by the more general pr #122608

To summarize other solutions:

1. current solution: prioritize most-referenced store paths; does a bad job because it ignores size
2. pkgs.dockertools.buildLayeredImage: customisable layering strategy #122608: allows all solutions to exist behind one interface
3. idea: optimize for minimal layer difference (in bytes) given any one-byte change in output. Seems like the right function to optimize for when the goal is to optimize redeployment instead. Effect on cross-image sharing is unknown, but seems to be of lesser importance.
4. idea: just put the largest store paths in their own layers. This was the reason why reversing solution 1 has led to an improvement for some

I don't think we should merge changes to the layering interface until we have a general solution. An implementation of such a general interface has been implemented in #122608 but needs to be cleaned up.