std.Build.tmpPath() directory not cleaned up

Help
1

I’m trying to pass a temporary directory to a test. However the temporary subdir created by the build system doesn’t seem to get deleted? Am I doing something wrong?

Minimal reproducible example:

Zig 0.17.0

build.zig 
const std = @import("std");

pub fn build(b: *std.Build) void {
    const exe = b.addExecutable(.{
        .name = "main",
        .root_module = b.createModule(.{
            .root_source_file = b.path("main.zig"),
            .target = b.standardTargetOptions(.{}),
            .optimize = b.standardOptimizeOption(.{}),
        }),
    });

    const options = b.addOptions();
    options.addOptionPathDirectory("tmp_dir", b.tmpPath());
    exe.root_module.addImport("options", options.createModule());

    const run_exe = b.addRunArtifact(exe);
    b.default_step.dependOn(&run_exe.step);
}
main.zig 
const std = @import("std");
const options = @import("options");

pub fn main(init: std.process.Init) !void {
    _ = try std.Io.Dir.createFile(
        .cwd(),
        init.io,
        options.tmp_dir ++ "/delete-me",
        .{},
    );
}

Run

> zig build
> find .zig-cache/tmp
.zig-cache/tmp
.zig-cache/tmp/9451b85aaf0d789b
.zig-cache/tmp/9451b85aaf0d789b/delete-me
2

This may be wrong so take it with a grain of salt. But as far as I understand it the tmp_dir in the build.zig is just for building the code and not running it. Meaning that it may or may not later be deleted when you run the tests.

You can have a temporary directory for tests like this:

test "tmp_dir" {
    var tmp = std.testing.tmpDir(.{});
    defer tmp.cleanup();

    // ... Your code
}

Here is the entire code:

build.zig 
const std = @import("std");

pub fn build(b: *std.Build) void {
    const exe = b.addExecutable(.{
        .name = "main",
        .root_module = b.createModule(.{
            .root_source_file = b.path("main.zig"),
            .target = b.standardTargetOptions(.{}),
            .optimize = b.standardOptimizeOption(.{}),
        }),
    });

    const exe_tests = b.addTest(.{ .root_module = exe.root_module });
    const run_exe_tests = b.addRunArtifact(exe_tests);

    const test_step = b.step("test", "Run tests");
    test_step.dependOn(&run_exe_tests.step);
}
main.zig 
const std = @import("std");
const testing = std.testing;

test "tmp_dir" {
    const io = testing.io;

    var tmp = testing.tmpDir(.{});
    defer tmp.cleanup();

    _ = try tmp.dir.createFile(io, "delete-me", .{});

    // ... Your code
}

Then just run zig build test.

3

I was not aware of std.testing.tmpDir. That sounds more like what I want and now the folder gets cleaned up!

4

Actually, I think @Zemogus what you originally wanted is fundamentally a better approach.

The problem with std.testing.tmpDir(.{}) and any equivalents is that the clean up is cooperative — it’s the testing process that deletes it, before it exits. That is problematic, because the code we are testing often crashes (or hangs, and is then killed by the test-running infrastructure), so no defers are run, and the are leftovers on the file system, which could eventually occupy all of the disk space (don’t ask :smiley: )

It’s more reliable to make the parent process create and dispose of a temporary directory. In some sense, it is just kicking the can down the road (what if the parent dies abruptly), but in practical sense it isn’t: the parent code (build.zig) is stable, it rarely changes and is assumed to be bug free. But the code under test is buggy, that’s why we are testing it in the first place!

Alas, I don’t think you can achieve this in “userspace” in build.zig:

  • As far as I am aware, you can’t create a temporary directory before the step runs, and then clean it up later.
  • You’d want to pass that directory as a runtime argument to the tests, rather than a comptime-one, but this isn’t possible.

I think I do with that build.zig had this sort of facility, but, practically, just use std.testing.tmpDir and maybe drop .zig-cache/tmp once in a while manually.

5

I must consider myself quite lucky then. Because what my test does is essentially:

  • write some files
  • spawn the process to be tested
  • check stdout and stderr

So it’s unlikely that the test code ever crashes.

6

I’m also not aware something like this exists. But one could just have a stable tmp_dir across runs and let the next run clean it up if it already exists. But this is then inherently racy, which could lead to other problems.

One could pass it via some sidechannel like a file but this also bad because of various obvious reasons.

7

Have we forgotten that Zig lets you write your own test runner?