Stage4/bin/zig build test and qemu

Help
1

Hi all!

I just built zig from source for the first time and got stuck trying to run the tests you should run before submitting a PR for the first time as specified in the readme.

Building llvm and zig seemingly went well. Running the tests, I see a few failing test-incremental tests which may just be broken on recent master (?), but the main question I had was about all the link_test_cases which are failing for me.

I’m building on x86_64 linux, and it seems like zig build is trying to run all the aarch64-linux variants of the tests as well. I do have qemu installed, but I passed -Dskip-non-native and I also expected qemu not to be used by default since AFAICT -fqemu is disabled by default. Also, passing -fno-qemu doesn’t change anything, it still triese to run the tests.

FWIW the command I run is stage4/bin/zig build test -Dskip-release -Dskip-non-native and the errors are along the lines of

test-link
└─ link_test_cases
   └─ test-elf
      └─ test-elf-tls-gd-aarch64-linux-gnu-Debug-llvm-no-lld
         └─ run exe main2 failure
qemu-aarch64: Could not open '/lib/ld-linux-aarch64.so.1': No such file or directory
error: ========= expected this stdout: =========
       1 2 3 4 5 6
       
       ========= but found: ====================
       
failed command: /home/user/projects/upstream/zig/.zig-cache/o/3e17d9f7850d919ccab167c2793038ff/main2

It seems like skip_non_native is simply not used to prevent building non-native link tests.

Uninstalling qemu on my host will cause zig to no longer fail on trying to run the aarch64-linux link test cases (but of course I’d like to keep it installed).

I tried looking through the zig source code to find what might cause zig build to try to run these tests despite enable_qemu being false by default, but I haven’t found the cause yet.

The tests themselves seem to be set up here.

Does anyone have tips or knows how this is meant to work?

2

Okay, here are my impressions so far.

tldr:

  • the qemu-linux-user on my system will configure binfmt to automatically open aarch64 binaries using qemu, but transparently / “invisibly” to the user
  • when that happens zig doesn’t know qemu was used
  • which means that skip_foreign_checks is ineffective, which means the failing tests are treated as errors instead of being marked as skipped

tldr tldr; if you want zig to be qemu-aware, disable the relevant binfmt arch config on your host system with something like echo 0 | sudo tee /proc/sys/fs/binfmt_misc/qemu-aarch64.

-fqemu / -fno-qemu

  • Run.zig will simply try to execute your binary
  • if that fails due to InvalidExe or FileNotFound, it’ll try to use getExternalExecutor() to figure out whether it can use a tool like qemu to run the binary instead
  • if -fqemu is set, rerun the binary, but with qemu (for example) instead
  • if -fqemu is not set or -fno-qemu is set, show an error message unless skip_foreign_checks is true

Can be confirmed by running the zig init project with something like zig build -fqemu --verbose -Dtarget=aarch64-linux test, which will show the two tests being (attempted to be) executed twice, once directly and once with qemu:

<snip>
./.zig-cache/o/c6e87dd54f4067f71999fd7d7c7d3467/test --cache-dir=./.zig-cache --seed=0xe5974969 --listen=-
./.zig-cache/o/f0b71a1e8891f97d4a8f7c0d8a9d2583/test --cache-dir=./.zig-cache --seed=0xe5974969 --listen=-
qemu-aarch64 ./.zig-cache/o/c6e87dd54f4067f71999fd7d7c7d3467/test --cache-dir=./.zig-cache --seed=0xe5974969 --listen=-
qemu-aarch64 ./.zig-cache/o/f0b71a1e8891f97d4a8f7c0d8a9d2583/test --cache-dir=./.zig-cache --seed=0xe5974969 --listen=-

link_test_cases failing even with fno-qemu

Why do the link_test_cases still run (and fail) even when passing -fno-qemu and with the tests being marked as skip_foreign_checks?

This seems to be due to binfmt. Installing qemu (on my distro) also configures the kernel to automatically execute binaries with a given magic byte sequence with qemu. This makes it feel like you can execute binaries directly without qemu, because qemu is being used behind the scenes. Therefore, with qemu installed, the zig init project can be tested on a non-native arch with -fno-qemu and the same applies to the link_test_cases. When running the above test command you will see only the first two commands, and it will look like qemu is not being used at all. In other words, zig doesn’t know that qemu is being used.

The current aarch64 binfmt config can be inspected with cat /proc/sys/fs/binfmt_misc/qemu-aarch64 and dynamically disabled with echo 0 | sudo tee /proc/sys/fs/binfmt_misc/qemu-aarch64. See the bimfmt docs for more info.

With binfmt disabled for aarch64 zig will get an InvalidExe when trying to call the test executable and will then either warn the user about enabling qemu or passing a matching libc or mark the test as skipped if skip_foreign_checks is set.

However, binfmt is enabled by default for aarch64 if you have the qemu packages installed. So the executable will spawn directly, zig won’t get an InvalidExe and the qemu logic won’t kick in. In my case, since I was missing the aarch64 libc, qemu will immediately exit again the outcome will be treated as a test failure.