How do I get zig build to Run all the tests?

Help
1

Today I ran into an issue where I had issues trying to get code to be tested. I’m trying to understand how I’m “supposed” to do this.

Project Structure

src/
  |- main.zig
  |- root.zig
  |- sqlite/
      |- varint.zig

I have sqlite setup where it is essentially a sub-module of the root.zig module. I have varint expose public structs with public functions, and a test is established in the module:

varint.zig (Yes none of this should compile)

/// A signed twos-complement variable integer, BigEndian
const std = @import("std");
const testing = std.testing;

pub const VarInt = struct {
    const MAX_SIZE = 9;

    value: i64,

    /// Represent the varint in it's binary form
    pub fn serialize(self: VarInt) [VarInt.MAX_SIZE]u8 {
        const is_negative = self.value < 0;
        _ = is_negative;
    }

    /// Read in the binary representation of a varint
    fn deserialize() VarInt {}
};

test "serialize" {
    const v = VarInt{ .value = 0 };
    try testing.expectEqual([9]u8{ 0, 0, 1, 0, 0, 0, 0, 0, 0, 0 }, v.serialize());
}

Now when i run my tests I don’t get a compile error. I realize it was because my sub-module wasn’t referenced. But when i added const varint = @import("sqlite/varint.zig") to my root.zig folder, it still didn’t run the tests. I had to make it public. But I don’t want varint to be public outside of my project. I just want it to be used within the project.

I got around this by doing the following in the root.zig

test {
    const x = VarInt{1};
    _ = x;
}

My Question

I Understand that the test runner/compiler don’t want to compile things that are unused. That makes sense to me. However sometimes I’m writing code that
I haven’t integrated yet into my project, and still want to run those tests. When I program I usually just have a watchexec process running that runs tests on save, but it isn’t helpful if what I’m working on isn’t being built/tested.

So My question is: Is there something i’m missing or not groking, to get all my test decls to be picked up and ran?

2

I think the way to do this is using std.testing.refAllDecls(). Basically as long as you reference varint and any other module you want tested in your root.zig, you can use that function so that the semantic analyzer sees them:

// root.zig
const varint = @import("...");

test {
    std.testing.refAllDecls(@This());
}

I believe this only works for public declarations, and I read elsewhere that this is basically a temporary hack anyway, so there will likely be a cleaner way in the future.

2 Likes
3

@n0s4 's advice is spot on. I’d like to add more details on refAllDecls and some tips for running tests.

Your understanding is correct. In order to compile something you have to use it.

That means:

  • @import the module to the root testing zig file, because the tester knows only about root zig testing file
    and
  • use the imported symbol, in order to actual use the tests of the imported module
const varint = @import("sqlite/varint.zig");
test {
    _ = varint;
}

Because every module requires its import and dummy use of the imported symbol, the comptime function refAllDecls is a helper that generates the dummy references for all declarations.

const varint = @import("sqlite/varint.zig");
test {
    std.testing.refAllDecls(@This);
}

When running tests, each test source is cached, on the next run if the tests are the same they don’t run.
My recommendation is to use a mode that prints a summary about the tests that actually run (new or changed tests in addition to the failed tests):

zig build test --summary new

The documentation of the --summary flag is:

  --summary [mode]             Control the printing of the build summary
    all                        Print the build summary in its entirety
    new                        Omit cached steps
    failures                   (Default) Only print failed steps
    none                       Do not print the build summary
7 Likes
4

I think the way to do this is using std.testing.refAllDecls() . Basically as long as you reference varint and any other module you want tested in your root.zig , you can use that function so that the semantic analyzer sees them:

I had been using that, though, as you note, it only seems to be discovering public attributes, though that might be because i wasn’t using them.

My concern with this is, as you state, refAllDecls has been declared to be a hack. I wasn’t sure if this “hack” had had a candidate for replacement yet. Looks like not yet.

When running tests, each test source is cached, on the next run if the tests are the same they don’t run.
My recommendation is to use a mode that prints a summary about the tests that actually run (new or changed tests in addition to the failed tests):

I didn’t know about the new option. I had been using all but sometimes it’s hard to know if the file i’m actually working on is being tested. New might be able to fix that.

2 Likes