Deduplicating identical build steps

matklad · April 20, 2026, 1:14pm

Help me find a nice pattern for the following CI setup. For the sake of an example, suppose I am
building a binary with unit tests and integration tests.

So, I might have something like

fn build_binary(b: *Build, target: ResolvedTarget, optimize: OptimizeMode) *Step.Compile {
    const root_module = b.addModule("binary", .{
        .root_source_file .b.path("./src/main.zig"),
        ...
    });
    ...
}

fn build_unit_tests(b: *Build, target: ResolvedTarget, optimize: OptimizeMode) *Step.Compile { ... }

fn build_integration_tests(b: *Build, target: ResolvedTarget, optimize: OptimizeMode) *Step.Compile {
    const binary = build_binary(b, target, optimize);

    const options = b.addOptions();
    options.addOptionPath("binary", binary.getEmittedBin());

    const root_module = b.addModule("test", .{
        .root_source_file .b.path("./src/test_integration.zig"),
        ...
    });
}

Note how integration tests need the binary build.

And then I wire things up in main:

const target = b.standardTargetOptions(.{});
const optimize = b.standardOptimizeOption(.{});

b.step("compile", "").dependOn(&build_binary(b, target, optimize).step);

b.step("test:unit", "").dependOn(&b.addRunArtifact(
    build_unit_tests(b, target, optimize),
).step);

b.step("test:integration", "").dependOn(&b.addRunArtifact(
    build_integration_tests(b, target, optimize),
).step);

Note how in the above there are essentially two identical Step.Compile that build the binary. I
think this is ok: if I run zig build compile and then zig build test:integration, the binary
will be built only once. Although the steps are distinct in build.zig process memory, they are equal
as values, have the same dependency information, and re-use each other caches.

Now, assume I also want to also have a CI step, which compiles my binary for a set of supported
targets, and runs the tests in debug and release:

const ci = b.step("CI", "");
for (supported_targets) |target| {
    ci.dependOn(&build_binary(b, target, .Debug).step);
}
for (.{.Debug, .ReleaseSafe}) |optimize| {
    ci.dependOn(&build_unit_tests(b, b.graph.host, optimize).step);
    ci.dependOn(&build_integration_tests(b, b.graph.host, optimize).step);
}

Now this becomes more problematic: a single invocation of zig build ci will attempt to build exe
twice, once in the target loop for cross-compilation, and once in the optimize loop for integration
tests. Again, this probably won’t lead to recompiling the code twice, but I think at least “is
this fresh” checks will be executed twice? And I think if I have some “has_side_effects = true”
dependencies (e.g, embedding current time or git commit as build metadata), those will also be
executed twice, and could actually force extra compilation?

And, with two loops, it’s not entirely trivial to re-use the build_binary step programmatically,
as I need to match the right target and optimization level.

It looks like I need some sort of step interning here, where I cache the steps created so far,

fn build_binary(b: *Build, target: ResolvedTarget, optimize: OptimizeMode) *Step.Compile {
    const Cache = struct {
        var global: std.AutoArrayHashMapUnmanaged(
            struct { ResolvedTarget, OptimizeMode },
            *Step.Compile,
        ) = .{};
    };

    const gop = Cache.global.getOrPut(b.allocator, .{ target, optimize }) catch @panic("OOM");
    if (gop.found_existing) return gop.value_ptr.*;
}

but this feels somewhat complicated, especially if I need to do this for many different kinds of
steps, and want to manage the cache in a less add hoc manner than via a local static variable
(admitedly, local statics feel ok for build.zig).

Is this the best pattern to “deduplicate” the build graph?

ScottRedig · April 20, 2026, 1:42pm

I believe you can do the following:

make the build binary its own named step.
use dependencyFromBuildZig with @This() to get the build file as its own dependency, passing in the target and optimize values.
use that dependency to getArtifact to get the compile step out of the self dependency.

I think this would work for you. (I normally test before answering, but I’m away from my computer and I’m being lazy.)

matklad · April 20, 2026, 1:52pm

Wow, I didn’t realize that dependencyFromBuildZig is a thing, this is a neat thing, thanks!

github.com/ziglang/zig

std.Build: add `dependencyFromBuildZig` (#18908)

master ← castholm:dependencyFromBuildZig

opened 11:34PM - 12 Feb 24 UTC

castholm

+26 -0

Given a struct that corresponds to the build.zig of a dependency, `b.dependencyF…romBuildZig` returns that same dependency. In other words, if you have already a `@import`ed a depdency's build.zig, you can use this function to obtain the corresponding `Dependency`: ```zig // in consumer build.zig const foo_dep = b.dependencyFromBuildZig(@import("foo"), .{}); ``` --- This function is also important for packages that expose functions from their build.zig files that need to use their corresponding `Dependency` (for example, for package-relative paths, or for running system commands and returning the output as lazy paths). This would be accomplished through: ```zig // in dependency build.zig pub fn getImportantFile(b: *std.Build) std.Build.LazyPath { const this_dep = b.dependencyFromBuildZig(@This(), .{}); return this_dep.path("file.txt"); } // in consumer build.zig const file = @import("foo").getImportantFile(b); ``` Currently, such use cases need to either 1\. take a `Dependency` as argument ```zig // in consumer build.zig const mach_core = @import("mach_core"); const mach_core_dep = b.dependency("mach_core", .{ ... }); const app = try mach_core.App.init(b, mach_core_dep, .{ ... }); ``` which is prone to user error, 2\. call `b.dependency` with a hard-coded name ```zig // in dependency build.zig pub fn addWebsite(b: *std.Build, ...) !void { const zine_dep = project.dependency("zine", .{ ... }); } // in consumer build.zig const zine = @import("zine"); try zine.addWebsite(b, .{ ... }); ``` which only works if the consumer has added that package as a dependency under that exact name (i.e. it must be `.zine` in the build.zig.zon, not `.zine_ssg` or some other name the consumer may have chosen), or 3\. implement the same logic as `dependencyFromBuildZig`, which requires the user to know how the undocumented `@import("root").dependencies` works or that it's even a thing. --- There's currently no way to test this in the compiler without setting up a build.zig.zon and some local packages, but I have tested this function with a few packages of my own locally and can report that it works exactly as advertised.

LucasSantos91 · April 20, 2026, 3:21pm

Why don’t you just pass the *Step.Compile you already created, instead of making new ones?

fn build_unit_tests(b: *Build, binary: *Step.Compile) *Step.Compile { ... }

fn build_integration_tests(b: *Build, binary: *Step.Compile) *Step.Compile {
    const options = b.addOptions();
    options.addOptionPath("binary", binary.getEmittedBin());

    const root_module = b.addModule("test", .{
        .root_source_file .b.path("./src/test_integration.zig"),
        .target = binary.root_module.resolved_target,
        .optimize = binary.root_module.optimize
        ...
    });
}

fn build_ci_all_targets(b: *Build, binaries: []const *Step.Compile) *Step.Compile{
    const ci = b.step("CI", "");
    for |binaries| |binary| {
        ci.dependOn(&binary.step);
    }
    return ci;
}

matklad · April 20, 2026, 3:30pm

In

for (supported_targets) |target| {
    ci.dependOn(&build_binary(b, target, .Debug).step);
}
for (.{.Debug, .ReleaseSafe}) |optimize| {
    ci.dependOn(&build_unit_tests(b, b.graph.host, optimize).step);
    ci.dependOn(&build_integration_tests(b, b.graph.host, optimize).step);
}

I can do

for (supported_targets) |target| {
    ci.dependOn(&build_binary(b, target, .Debug).step);
}
for (.{.Debug, .ReleaseSafe}) |optimize| {
    const binary = build_binary(b, g.graph.host, optimize);
    ci.dependOn(&build_unit_tests(b, b.graph.host, optimize).step);
    ci.dependOn(&build_integration_tests(b, b.graph.host, optimize, binary).step);
}

but this just makes the duplication explicit. To remove the duplication, I also need to somehow match build_binary’s from the first loop with the second loop, but the mapping is non-trivial, as the loops are over independent axes.