Running tests with a matrix of options

Help
, ,
1

I got a project with a compile-time option to choose the implementation of a specific system

The project already got tests to check if everything is working as expected but to test both implementations I am currently manually toggling the backend i.e.
zig build -Dbackend=a test and zig build -Dbackend=b test, is there a way to do this automatically? The backend is bassed to the module via Step.Options

2

In your build.zig, something like this should work, just looping over all options & creating Option and building the test:

const test_all_step = b.step("testall", "Run test matrix");

for ([_][]const u8{ "foo", "bar" }) |backend| {
    const main_module = b.createModule(.{
        .root_source_file = b.path("src/main.zig"),
        .target = target,
        .optimize = optimize,
    });
    const options = b.addOptions();
    options.addOption([]const u8, "backend", backend);
    main_module.addOptions("options", options);

    const test_main = b.addTest(.{ .root_module = main_module });
    const test_exe = b.addRunArtifact(test_main);

    test_all_step.dependOn(&test_exe.step);
}
1 Like