Testing all tests

Help
1

I want to transform my tests, which I do in debugmode @runtime, to zig tests.

So in my build.zig i added this.

    // Tests
    const unit_tests = b.addTest(.{
        .root_module = exe.root_module,
    });

    const run_unit_tests = b.addRunArtifact(unit_tests);

    const test_step = b.step("test", "Run unit tests");
    test_step.dependOn(&run_unit_tests.step);

Then I added one test inside main.zig.

Then run

zig test src/main.zig

That works.

But how can I run all tests in all files at once?

2

you should run zig build test to use the test step you defined.

zig test is outside the build system, or rather, it is what the build system eventually calls.

reference those files in a test that is run, this is just zigs lazy evaluation again.

1 Like
3

So I should add a root.zig file, referencing all other files that contain tests?
then run zig test src/root.zig?

(Sorry if I ask stupid things, I never dived into build.zig things..)

4

you can just do that in your already existing module.

5

but how? add things in build.zig?

6

Add

test {
   _ = @import("foo.zig");
   _ = @import("bar.zig");
}

To your exe root source file.

2 Likes
7

I generally do

comptime {
    if (builtin.is_test) {
        _ = @import("foo.zig");
        // etc
    }
}

so that it doesn’t count as a test

2 Likes
8

ok that i will manage.

Now i have a huge problem.

My tests need a global initialization and finalization.
When running one test everything is fine.
When running more tests during finalization - a global hashmap is freed - it crashes.
(I am setting my global allocator to the testing allocator during initialization).

thread 10080 panic: incorrect alignment
C:\zig\zig-x86_64-windows-0.15.2\lib\std\hash_map.zig:784:44: 0x7ff617aae7f9 in header (test_zcu.obj)
            return @ptrCast(@as([*]Header, @ptrCast(@alignCast(self.metadata.?))) - 1);

Each test starts with

    try lib.initialize();
    defer lib.finalize(); // freeing my global hashmap
9

I think you have two options either initialize (re-)creates the global hashmap or finalize doesn’t free it.

Basically finalize shouldn’t free stuff that initialize hasn’t initialized.

If this hashmap is something that always exists then it might make sense to “leak” it and just let it be cleaned up by the os on program exit, but I don’t know enough about your program to be sure.

1 Like
10

Somehow the global stays in the air over several tests. Now multiple tests don’t crash anymore.
But not sure what it really going on.

pub fn finalize() void {
    layout_map.deinit(ctx.gpa);
    layout_map = .empty; // This solves testing problems for now.
}
11

Looks like you are basically resetting it to its initial state.
Does this work if you put the layout_map = .empty; inside of initialize instead?

12

It is declared as a global.

var layout_map: std.AutoHashMapUnmanaged(LayoutKey, Layout) = .empty;
13

Sure but after your first test ran it gets deinit-ed in finalize so it makes sense to re-initialize it explicitly in initialize so that it is set to empty by the second test.

14

I will do that as well!

15

You only need it either in finalize or initialize, but I think initialize makes more sense to a reader.

16

But it’s only for the sake of testing this problem pops up…
Normally it’s an exe just executed once and then the global declaration as .empty should be enough.

It’s just like var x: i32 = 21 on startup…

17

I had a similar problem trying to use a global arena to use for all tests, which made tests crash (arena initialized with std.testing.allocator). Initializing the arena with DebugAllocator stopped the crashes, so maybe it has to do with the testing allocator not being a good fit for something that should survive single tests. (I don’t use a global arena anymore because I prefer to stick with std.testing.allocator).

Anyway in your case I’d try something like

// in main.zig or whatever your tests use as entry point
comptime {
    if (builtin.is_test) {
        // initialize globals here, but not with testing allocator
        _ = @import("foo.zig"); // and other modules with tests
        // deinitialize globals
    }
}
2 Likes
18

The std.testing.allocator is a DebugAllocator itself. But indeed it gets de-/initialized between each test:
https://codeberg.org/ziglang/zig/src/branch/master/lib/compiler/test_runner.zig#L286

1 Like
19

You can also use

std.testing.refAllDecls(@This());

rather than calling them out one by one.

1 Like