How to tell Zig not to rerun a certain step?

[hachanuy]

I am building a Zig wrapper for libzmq. In my build.zig, I have

const cmake_build = b.addSystemCommand(&.{
    cmake,
    "--build",
    upstream.path("build").getPath(b),
});
cmake_build.step.dependOn(&cmake_configure.step);

cmake_build will produce a static library named libzmq.a. I can already link and use this library. However, every time I run zig build, zig will rerun cmake_build step again (it’s instant but it’s annoying). Is there a way for me to tell zig that it does not have to rerun that step if libzmq.a is already produced?

[tensorush]

One way is to check if the path exists, smth like this:

const build_abs_path = upstream.path("build").getPath(b);

std.fs.accessAbsolute(b.pathJoin(&.{ build_abs_path, "libzmq.a" }), .{}) catch {
    const cmake_build = b.addSystemCommand(&.{
        cmake,
        "--build",
        build_abs_path,
    });
    cmake_build.step.dependOn(&cmake_configure.step);
}

[hachanuy]

Thank you very much, I didn’t think that the dependency can be declared dynamically like that (no idea why I thought so).

[castholm]

@hachanuy

I am building a Zig wrapper for libzmq. In my build.zig, I have

In addition to plain string arguments, run steps (std.Build.Step.Run) can take special arguments representing input/output paths via std.Build.Step.Run.addFileArg and related methods. These methods return a lazy path which you can subsequently pass around to other build steps.

Run steps that only take string arguments are considered impure and will be re-executed every time they are invoked. However, if a run step takes an input/output path argument, it is considered pure, will participate in the cache system and will only be rerun when any of its inputs change.

So you likely want to rewrite your build script as such:

const cmake_build = b.addSystemCommand(&.{
    cmake,
    "--build",
});
const build_dir = cmake_build.addOutputDirectoryArg("build");

build_dir now points to a path in .zig-cache, and any steps that needs to access anything in that directory should do so via the build_dir lazy path variable. By doing so, the build system will automatically set up step dependencies and ensure the CMake command is rerun when it needs to.

(Edit: I realize that this is the build phase of CMake and not the configure step, so this directory is an input arg and not an output arg. But for the configure phase, the same principles apply.)

Never use getPath/2/3 from your build function. These APIs are intended to be used internally by the build system after it has verified that all the correct prerequisite steps have been run. Using it from your own code is almost always a bug.

@tensorush

One way is to check if the path exists, smth like this:

This is the wrong fix. You should not use any of the std.fs APIs that accesses the file system from your build script either. Your build function is supposed to be (mostly) pure function that declares a graph of build step dependencies without interacting with the host system.

@castholm

Don’t run imperative code that accesses the file system or the network directly from your build.zig. In the future, the build system will be sandboxed, and your build function will be prohibited from accessing system resources, so this is not future proof, and even now it’s a poor idea since the code will not take advantage of the build system cache and might be inadvertently invoked by innocuous things like zig build --help or tooling like ZLS. If you need to e.g. fetch something from the Internet, you should use a run step as described above. The only thing your build function should do is declaratively construct a (serializable) graph of build steps to be processed by a build runner.

[hachanuy]

Thank you for the write up. I see a glimpse of how this should be done, but I’m not sure how I should declare the path to libzmq.a to make zig aware of it.

[castholm]

I posted a possible solution in your other thread.

How should i generate CC and CXX in build.zig

As a follow-up to my post in your other thread (TL;DR: don’t use getPath), a more proper solution would look something like this: const cmake = b.findProgram(&.{"cmake"}, &.{}) catch @panic("CMake not found"); const triple = target.result.linuxTriple(b.allocator) catch @panic("OOM"); const cmake_configure = b.addSystemCommand(&.{ cmake, "-G=Ninja", "-B", }); const build_dir = cmake_configure.addOutputDirectoryArg("build"); cmake_configure.addArg("-S"); cmake_configure.addDirectoryA…

[hachanuy]

This feels like postception to me , but this is awesome knowledge. I will try out the things you suggested and write up a polished solution.

[hachanuy]

Following your advice, I now have the following build script

const upstream = b.dependency("upstream", .{});

const triple = target.result.linuxTriple(b.allocator) catch @panic("OOM");
const cc = b.fmt(
    "-DCMAKE_C_COMPILER='{s};cc;--target={s}'",
    .{ b.graph.zig_exe, triple },
);
const cxx = b.fmt(
    "-DCMAKE_CXX_COMPILER='{s};c++;--target={s}'",
    .{ b.graph.zig_exe, triple },
);
const build_type = if (optimize == .Debug) "-DCMAKE_BUILD_TYPE=Debug" else "-DCMAKE_BUILD_TYPE=Release";
const cmake = b.findProgram(&.{"cmake"}, &.{}) catch @panic("CMake not found");

const cmake_configure = b.addSystemCommand(&.{
    cmake,
    "-G=Ninja",
    cc,
    cxx,
    "-DWITH_TLS=OFF",
    "-DWITH_PERF_TOOL=OFF",
    "-DZMQ_BUILD_TESTS=OFF",
    "-DENABLE_CPACK=OFF",
    "-DENABLE_DRAFTS=ON",
    "-Wno-dev",
    build_type,
    "-S",
});
cmake_configure.addDirectoryArg(upstream.path(""));
cmake_configure.addArg("-B");
const build_dir = cmake_configure.addOutputDirectoryArg("build");

const cmake_build = b.addSystemCommand(&.{
    cmake,
    "--build",
});
cmake_build.addDirectoryArg(build_dir);

const library = b.addTranslateC(.{
    .root_source_file = upstream.path("include/zmq.h"),
    .target = target,
    .optimize = optimize,
});
library.step.dependOn(&cmake_build.step);

const module = b.addModule("libzmq", .{
    .root_source_file = library.getOutput(),
    .target = target,
    .optimize = optimize,
    .link_libc = true,
    .link_libcpp = true,
});
module.addObjectFile(build_dir.path(b, "lib/libzmq.a"));

Another project can use the libzmq module fine, but the cmake_build step is still always rerun. I guess it’s because there is no way to specify its output files.