Can't replicate except with nixos: error: error: unable to open global cache directory '/homeless-shelter/.cache/zig': AccessDenied

Explain
1

I encountered this easily resolvable error but I’m trying to understand what is causing it.

My nix flake was originally running(full code):

        buildPhase = ''
          PACKAGE_DIR=${pkgs.callPackage ./deps.nix {}}
          zig build install --global-cache-dir $(pwd)/.cache --system $PACKAGE_DIR -Dtarget=${target} -Doptimize=ReleaseSafe --color off --prefix $out
        '';

This led to this error:

@nix { "action": "setPhase", "phase": "buildPhase" }
install
+- install zig-status
   +- compile exe zig-status ReleaseSafe x86_64-linux failure
error: error: unable to open global cache directory '/homeless-shelter/.cache/zig': AccessDenied
       If this location is not writable then consider specifying an alternative with the ZIG_GLOBAL_CACHE_DIR environment variable or the --global-cache-dir option.
       
error: process exited with error code 1
failed command: /nix/store/70z6wa6fr4x4fafjj5dy61kk8gakpj0l-zig-0.16.0/bin/zig build-exe -OReleaseSafe -target x86_64-linux -mcpu baseline --dep zig-status -Mroot=/build/source/src/main.zig -target x86_64-linux -mcpu baseline -Mzig-status=/build/source/src/root.zig -lc --cache-dir .zig-cache --global-cache-dir .cache --name zig-status --zig-lib-dir /nix/store/70z6wa6fr4x4fafjj5dy61kk8gakpj0l-zig-0.16.0/lib/ --listen=-

Build Summary: 0/3 steps succeeded (1 failed)
install transitive failure
+- install zig-status transitive failure
   +- compile exe zig-status ReleaseSafe x86_64-linux failure

error: the following build command failed with exit code 1:
.zig-cache/o/e9a4f82a227a1f9d19b56a042847710b/build /nix/store/70z6wa6fr4x4fafjj5dy61kk8gakpj0l-zig-0.16.0/bin/zig /nix/store/70z6wa6fr4x4fafjj5dy61kk8gakpj0l-zig-0.16.0/lib /build/source .zig-cache .cache --seed 0x539d549b -Z15174abf363170f4 install --system -Dtarget=x86_64-linux -Doptimize=ReleaseSafe --color off --prefix /nix/store/vm94bgaixq5j5pbhzi3pqvx8kwa3k3bx-zig-status

Which is especially interesting to me because it initially comes off as for at least some portion of the build the --global-cache-dir is ignored and instead zig attempts to write to the $HOME directory which isn’t writable in the nix flake. The issue is two fold, one being the two zig generated commands appear to contain the global cache arg, and any way I attempt to replicate the error such as HOME=/nonexistent zig build --global-cache-dir $(pwd)/unlocked, there is no error at all. And this is complicated by that the error is fixed instead just defining the global cache environment variable(export ZIG_GLOBAL_CACHE_DIR=$(pwd)/.cache or export HOME=$(pwd)/.cache), so it is something special about the arg, in specifically this environment.

Is this intended or a bug with --global-cache-dir?

2

Not really sure why, but look at the failed command line:

failed command: /nix/store/70z6wa6fr4x4fafjj5dy61kk8gakpj0l-zig-0.16.0/bin/zig build-exe -OReleaseSafe -target x86_64-linux -mcpu baseline --dep zig-status -Mroot=/build/source/src/main.zig -target x86_64-linux -mcpu baseline -Mzig-status=/build/source/src/root.zig -lc --cache-dir .zig-cache --global-cache-dir .cache --name zig-status --zig-lib-dir /nix/store/70z6wa6fr4x4fafjj5dy61kk8gakpj0l-zig-0.16.0/lib/ --listen=-

there’s --global-cache-dir there as well.

nixpkgs typically use the zig.hook in nativeBuildInputs to build zig programs which set the ZIG_GLOBAL_CACHE_DIR, it does seem like there might be a bug regarding the --global-cache-dir switch.

There’s also:
https://github.com/Cloudef/zig2nix
https://github.com/silversquirl/zig-flake
which provide packaging.

Dynamic binaries are also kind of problem with zig 0.16 under nix right now due to https://codeberg.org/ziglang/zig/issues/31760 (fixed in master)

1 Like
3

Yea I noticed that, and it still just confused me.

As for those two others this is off-topic, but I’ve seen your flake, and have used it- but for most projects I don’t because of a few things(not necessarily all well founded but the disaster that is npm has made me paranoid about dependencies):

  • Most of the time I lean towards the more established(as in wider user base dependency) when otherwise largely equivalent for my usecase

  • I lean towards ones with fewer external dependencies, ie flake-utils, vs the “more official” flake-compat used by zig-overlay

  • This is more of hypothetical risk, but if either you or mitchellh were to slip a fake checksum into the repo, you could also be in control of one of the numerous mirrors you use and lead to a fake binary being installed- whereas zig-overlay gets its binaries from ziglang.org , which would be more immediately suspicious if he were to change to a different binary source, and then forge a checksum.

4

That is fine. I think the --global-cache-dir might be a bug anyhow.

1 Like
5

Regarding this though. I don’t understand this comment as all the projects above fetch the zig the same way using the community mirrors and ziglang provided index. Though in addition both zig2nix and zig-flake also check the minisign signature of the binary packages which zig-overlay does not do.

The additional cachix cache for zig2nix is optional and has to be opt-in explicitly.

6

Oh wait, you’re right, I completely missed the mirrors file in zig-overlay and was only looking at sources.json then that does seem like the more secure option.

1 Like
7

I managed to reproduce this bug outside of nix.

This fails:

bwrap --dev-bind / / --unshare-user --uid 256 --gid 256 --bind "$PWD" "$PWD" --clearenv --setenv HOME /homeless-shelter --setenv PATH "$PATH" zig build install --global-cache-dir "$PWD/.cache"

But this works:

bwrap --dev-bind / / --unshare-user --uid 256 --gid 256 --bind "$PWD" "$PWD" --clearenv --setenv HOME /homeless-shelter --setenv PATH "$PATH" --setenv ZIG_GLOBAL_CACHE_DIR "$PWD/.cache" zig build install

The reason is that nix’s sandbox looks like this:

drwxr-xr-x   1 nobody nogroup   4 May  8 01:57 bin
drwx------   1 nixbld nixbld   28 May  8 01:57 build
drwxr-xr-x   1 nobody nogroup 120 May  8 01:57 dev
dr-xr-xr-x   1 nobody nogroup  32 May  8 01:57 etc
drwxr-xr-x   1 nobody nogroup  10 May  8 01:57 nix
dr-xr-xr-x 490 nobody nogroup   0 May  8 01:57 proc
drwxr-xr-x   1 nobody nogroup  12 May  8 01:57 run
drwxrwxrwt   1 nobody nogroup   0 May  8 01:57 tmp

I believe zig at some point tries to stat or do something with / that then fails and reports the misleading error. It is weird that setting ZIG_GLOBAL_CACHE_DIR works though.

1 Like
8

Hmm, I see, good job even reproducing it