Nix terminates with uncaught exception of type nix::SysError: error: getting status of /etc/ssl/certs/ca-certificates.crt on Darwin with sandbox enabled

[al3xtjames]

Describe the bug

After running into #7273 while running darwin-rebuild from nix-darwin, I tried deleting /nix/store/.links and rebuilding. darwin-rebuild then failed with the following error:

libc++abi: terminating with uncaught exception of type nix::SysError: error: getting status of /etc/ssl/certs/ca-certificates.crt: Operation not permitted
/private/tmp/nix-build-options-db.xml.drv-0/.attr-0l2nkwhif96f51f4amnlf414lhl4rv9vh8iffyp431v6s28gsr90: line 10: 75243 Abort trap: 6           nix-instantiate --store dummy:// --eval --xml --strict --expr '{file}: builtins.fromJSON (builtins.readFile file)' --argstr file /nix/store/9pmqhvmmmjphfs0k3n51fdmxrrcb39mh-options.json > options.xml

Steps To Reproduce

1. Enable the sandbox (I set nix.settings.sandbox = true in my flake, which sets sandbox = true in /etc/nix/nix.conf).
2. Somehow reproduce "error: cannot link '/nix/store/.tmp-link' to '/nix/store/.links/...': File exists" #7273
3. sudo rm -rf /nix/store/.links
4. darwin-rebuild --flake .#

I'm not sure how to easily reproduce this since I only saw it after encountering #7273.

Expected behavior

darwin-rebuild should have succeeded.

nix-env --version output

nix-env (Nix) 2.13.3

Additional context

I see that Nix adds the cert bundle to the chroot on Linux here:

nix/src/libstore/build/local-derivation-goal.cc

Lines 1780 to 1781 in bf7dc3c

	if (settings.caFile != "")
	dirsInChroot.try_emplace("/etc/ssl/certs/ca-certificates.crt", settings.caFile, true);

On Darwin, should Nix add it to the sandbox profile? Maybe like this (untested):

diff --git a/src/libstore/build/local-derivation-goal.cc b/src/libstore/build/local-derivation-goal.cc
index b0289ac75..8b679c12f 100644
--- a/src/libstore/build/local-derivation-goal.cc
+++ b/src/libstore/build/local-derivation-goal.cc
@@ -2014,11 +2014,15 @@ void LocalDerivationGoal::runChild()
                     #include "sandbox-defaults.sb"
                     ;
 
-                if (!derivationType.isSandboxed())
+                if (!derivationType.isSandboxed()) {
                     sandboxProfile +=
                         #include "sandbox-network.sb"
                         ;
 
+                    if (settings.caFile != "")
+                        sandboxProfile += fmt("(allow file-read*\n\t(literal \"%s\")\n", settings.caFile);
+                }
+
                 /* Add the output paths we'll use at build-time to the chroot */
                 sandboxProfile += "(allow file-read* file-write* process-exec\n";
                 for (auto & [_, path] : scratchOutputs)

Priorities

Add 👍 to issues you find important.

[purcell]

Yup, I'm seeing this issue too. I have the sandbox enabled and also auto-optimise-store (as in #7273). I also tried adding /etc/sss/certs to extra-sandbox-paths via nix-darwin, but this made no difference.

[purcell]

Still getting this with Nix 2.15.1, fwiw, since that version apparently made it into nixpkgs.

[purcell]

P.S. @al3xtjames I suspect you might need an extra closing paren in the string written by fmt in the diff snippet you suggested above.

[purcell]

Is it possible this is the same issue that was fixed by @thufschmitt in #8240? If so, I think it's in Nix 2.16, which hasn't hit nixpkgs yet. (Or, at least, isn't the default Nix version there.)

[purcell]

@al3xtjames Fwiw, I seem to have worked around this issue locally by disabling documentation.enable, then updating nixpkgs (unstable), setting nix.package = pkgs.nixVersions.nix_2_16; and rebuilding. (All with sandbox left enabled.)

[amarshall]

I am seeing this issue on Nix 2.16.1, so that PR does not fix this.

[purcell]

@amarshall Huh, it's certainly hard to trigger consistently, so I'm not too surprised.

[amarshall]

FWIW, while Nix on the host is 2.16.1, the Nix in the drv itself is 2.15.1. However, I can’t figure out where the source for this drv is… Here’s the nix show derivation for it. I tried grepping across all flake inputs + Nix src for the text in the buildCommand to no avail.

{
  "/nix/store/3d23aqkp753x9wm1a2k8bk98rrzmqgs7-options-db.xml.drv": {
    "args": [
      "-e",
      "/nix/store/6xg259477c90a229xwmb53pdfkn6ig3g-default-builder.sh"
    ],
    "builder": "/nix/store/y0sbqh18cqlha5inki7aj6bcncx3fy85-bash-5.2-p15/bin/bash",
    "env": {
      "__darwinAllowLocalNetworking": "",
      "__impureHostDeps": "/bin/sh /usr/lib/libSystem.B.dylib /usr/lib/system/libunc.dylib /dev/zero /dev/random /dev/urandom /bin/sh",
      "__propagatedImpureHostDeps": "",
      "__propagatedSandboxProfile": "",
      "__sandboxProfile": "",
      "__structuredAttrs": "",
      "buildCommand": "# Make sure we don't try to access paths outside the sandbox.\n# This should be redundant with --store dummy:// if not for Nix bugs...\nexport NIX_STORE_DIR=$TMPDIR/store\nexport NIX_STATE_DIR=$TMPDIR/state\nnix-instantiate \\\n  --store dummy:// \\\n  --eval --xml --strict \\\n  --expr '{file}: builtins.fromJSON (builtins.readFile file)' \\\n  --argstr file /nix/store/166miqzhkia3h3cv3c70lvlm66mlfk51-options.json \\\n  > options.xml\n\nmkdir -p $out/nmd-result\n\nxsltproc \\\n  --stringparam elementId 'home-manager-options' \\\n  --stringparam optionIdPrefix 'opt' \\\n  --nonet \\\n  -o $out/nmd-result/home-manager-options.xml \\\n  /nix/store/41vh9fr1znwk5vjjprk046s6g4f3jc2m-options-to-docbook.xsl options.xml\n",
      "buildInputs": "",
      "builder": "/nix/store/y0sbqh18cqlha5inki7aj6bcncx3fy85-bash-5.2-p15/bin/bash",
      "cmakeFlags": "",
      "configureFlags": "",
      "depsBuildBuild": "",
      "depsBuildBuildPropagated": "",
      "depsBuildTarget": "",
      "depsBuildTargetPropagated": "",
      "depsHostHost": "",
      "depsHostHostPropagated": "",
      "depsTargetTarget": "",
      "depsTargetTargetPropagated": "",
      "doCheck": "",
      "doInstallCheck": "",
      "enableParallelBuilding": "1",
      "enableParallelChecking": "1",
      "enableParallelInstalling": "1",
      "mesonFlags": "",
      "name": "options-db.xml",
      "nativeBuildInputs": "/nix/store/07ml5pxrzbch3ihlmdyhgfcs0dpib10w-nix-2.15.1-dev /nix/store/pndc3wg6idzzlqipgxmpd0zql2fd2d2k-libxslt-1.1.37-bin",
      "out": "/nix/store/5fhcafdkhqwclqgm23vjfnrwrh6vjjr9-options-db.xml",
      "outputs": "out",
      "passAsFile": "buildCommand",
      "patches": "",
      "propagatedBuildInputs": "",
      "propagatedNativeBuildInputs": "",
      "stdenv": "/nix/store/fdwwz5dah6ik07psv3jz1agb2i2aic5z-stdenv-darwin",
      "strictDeps": "",
      "system": "x86_64-darwin"
    },
    "inputDrvs": {
      "/nix/store/4irgdm88jilqxhdflnx99mhk6zp94hci-options.json.drv": [
        "out"
      ],
      "/nix/store/7n4pcnh87whkmsid8zasr6r9kwydph7v-stdenv-darwin.drv": [
        "out"
      ],
      "/nix/store/bzr49ah58krc9zy6lagqdga6ijwdqm91-libxslt-1.1.37.drv": [
        "bin"
      ],
      "/nix/store/daiw8gs04p2sj0caifpj8699bdkc8pfr-nix-2.15.1.drv": [
        "dev"
      ],
      "/nix/store/vzcl2b1fib9628ldmgj7v1q6ggagpn7d-bash-5.2-p15.drv": [
        "out"
      ]
    },
    "inputSrcs": [
      "/nix/store/41vh9fr1znwk5vjjprk046s6g4f3jc2m-options-to-docbook.xsl",
      "/nix/store/6xg259477c90a229xwmb53pdfkn6ig3g-default-builder.sh"
    ],
    "name": "options-db.xml",
    "outputs": {
      "out": {
        "path": "/nix/store/5fhcafdkhqwclqgm23vjfnrwrh6vjjr9-options-db.xml"
      }
    },
    "system": "x86_64-darwin"
  }
}

[amarshall]

Okay, it’s in nmd, included from Home Manager. It looks like nmd is quite out-of-date compared with what is in nixpkgs, even though it purports to be a standalone extraction of it.

[purcell]

Yeah, there's a somewhat related issue in home-manager here which was fixed by bumping nmd: nix-community/home-manager#3612

[utkarshgupta137]

I install all of my packages using home-manager, so darwin-rebuild switch is working for me while home-manager switch wasn't. Only thing that helped was setting manual.manpages.enable to false.
My nix-config: https://github.com/utkarshgupta137/nix-config

[emilazy]

This works for me with pkgs.nix.nixVersions.2_16 (nix (Nix) 2.16.1; test case: nix build home-manager#docs-manpages) but not with pkgs.nix (nix (Nix) 2.15.1). I have no idea what change fixed it and there's a lot of comitts to wade through. Since this breaks home-manager out of the box when using the Darwin sandbox and it'll probably be a while before 2.16 trickles down into pkgs.nix (never mind 23.05's entire support period), it'd be great if someone could track down what the relevant commits were and backport them to 2.15.

[emilazy]

Is it possible this is the same issue that was fixed by @thufschmitt in #8240? If so, I think it's in Nix 2.16, which hasn't hit nixpkgs yet. (Or, at least, isn't the default Nix version there.)

Okay, it is just this :) I wrote this PR off because of the title and missed the additional commits that are clearly related until I searched the git log myself. Opened a backport for 2.15: #8535

[al3xtjames]

Thanks for investigating this! I was able to run darwin-rebuild after adding your backport:

final: prev: {
  nix = prev.nix.overrideAttrs (old: {
    patches = (old.patches or []) ++ [
      (final.fetchpatch {
        url = "https://patch-diff.githubusercontent.com/raw/NixOS/nix/pull/8535.patch";
        hash = "sha256-CTbwtlB/veZWP+xqYSSZeUJiAUhIn3r6QJgZwRx4QRM=";
      })
    ];
  });
}

Using Nix 2.16 also worked:

final: prev: {
  nix = final.nixVersions.nix_2_16;
}

[amarshall]

I believe this is fixed and can be closed.