What is the best way to output extra information in tests?
For example: if I loop through 100 test cases, I need info on where the failure was, if any.
“test failed” is not enough then.
Just write an extra helper function which calls std.texting.expectXXX which does a std.debug.print()?
1 Like
try std.testing.***() are printing using std.debug.print() so it seems like a good choice.
Hmm, doesn’t it print a stack trace on the console for the failed test? For instance in VSCode I get this:
I can then Cmd/Ctrl-click on the logged location above the try expect(false) and VSCode takes me directly to that place in the code. IMHO that’s good enough?
Wait there’s more, the Zig VSCode plugin even has test integration (I just discovered this now). Looks like I’m not setting up the tests correctly though because when running them in ‘single-file-mode’ they’re all failing hmm…):
PS: ok, obvious why it fails when running zig test directly on the file, because I’m importing modules that are only setup in the build system. Would be good to have a similar test discovery and filter which test should run in the build system I guess…
Command failed: /Users/floh/.zvm/bin/zig test --test-filter init AY38912 /Users/floh/projects/chipz/tests/ay3891.test.zig
tests/ay3891.test.zig:3:24: error: no module named 'chipz' available within module 'test'
const ay3891 = @import("chipz").chips.ay3891;
1 Like
Ok… This is a bit crazy but working as I expected.
test "100" {
var idx: usize = 0;
for (0..100) |i| {
if (i < 42) idx = i;
std.testing.expectEqual(idx, i) catch {
std.debug.print("omg we had an error at case {}\n", .{i});
try std.testing.expect(false);
};
}
}
Never heard of that one…
1 Like
If you’re after extra debug prints on test failures, I think you can use this trick:
test "100" {
var idx: usize = 0;
for (0..100) |i| {
if (i < 42) idx = i;
errdefer std.debug.print("omg we had an error at case {}\n", .{i});
try std.testing.expectEqual(idx, i);
}
}
You can modify the configuration of the ziglang plugin to change its test execution behavior, no longer using zig test but zig build test.
{
"zig.testArgs": [
"build",
"test",
"-Dtest_filter=${filter}"
]
}
test_filter needs to be configured in build.zig:
const test_filters: []const []const u8 = b.option(
[]const []const u8,
"test_filter",
"Skip tests that do not match any of the specified filters",
) orelse &.{};
const test_targets = [_]std.Target.Query{
.{}, // native
};
const test_step = b.step("test", "Run unit tests");
for (test_targets) |test_target| {
const unit_tests = b.addTest(.{
.root_module = createTestModule(test_target),
.filters = test_filters,
});
const run_unit_tests = b.addRunArtifact(unit_tests);
test_step.dependOn(&run_unit_tests.step);
}
2 Likes
Thanks all! I have enough material again to dive in.
BTW: is errdefer not on the deathlist?
errdefer with capturing of error e.g. errdefer |err| is on deathlist. Regular errdefer is staying.
errdefer |err| currently allows you to prevent errors from actually been returned. For example this code will compile and panic at runtime:
const std = @import("std");
pub fn main() void {
errdefer |e| {
std.debug.panic("{t}", .{e});
}
return error.Error;
}
This type of behavior is on deathlist
3 Likes
I do that but instead of expect(false) I use return error.TestUnexpectedError. Any error will work.
1 Like
Oh of course! Smarter 
1 Like
Are you sure? I would be surprised to see panicking in an errdefer become impossible. It’s not related to errdefer |err| for one thing, you can just errdefer @panic("yikes");.
noreturn casts to anything, including void, so yeah. Losing that would be odd.
1 Like
wow I didn’t know errdefer @panic(") also worked. I assumed it only worked with capturing version.
I don’t think this is related. return’s is also returning noreturn so why would this not work?
pub fn main() void {
errdefer {
return;
}
return error.Error;
}
It seems to just be special cased
return; is of type void, not type noreturn.