How do I know which files, or which tests, `zig build test` covers?

[mark-rushakoff]

I have a default build.zig file from zig init.

My main.zig file has a couple local imports, like const foo = @import("./foo.zig") and foo.zig contains another relative import.

In my main.zig I have:

test {
    std.testing.refAllDeclsRecursive(@This());
}

I thought this would run every test in main.zig, and every test in every file I transitively import from main.zig.

But the output from zig build test --summary all is:

Build Summary: 5/5 steps succeeded; 7/7 tests passed
test success
├─ run test 1 passed 2ms MaxRSS:2M
│  └─ zig test Debug native cached 55ms MaxRSS:58M
└─ run test 6 passed 3ms MaxRSS:4M
   └─ zig test Debug native cached 55ms MaxRSS:58M

This doesn’t look right to me at first glance. I have exactly one other test in main.zig, so maybe that’s the 1 passed test, and then 6 passed is the collection of all the transitive imports’ tests? I could temporarily inject failures into the other tests I think it is supposed to run, or I could add new tests in the other files to see if the count goes up, but neither of those seem like the right solution to understand what exactly is being run in tests.

It could also be that I am misunderstanding std.testing.refAllDeclsRecursive, but if so I would still want to confirm that by seeing what tests are actually run in this setup.

Is there another option I can pass to zig build test to print out which files it is testing, or the fully qualified names of the tests it runs? I am not seeing any relevant flags in zig build test --help.

I’m running 0.14.0-dev.2097+d30e28754.

[cztomsik]

IMHO the test runner is one of the current pain-points in Zig. I hope it gets fixed/improved one day but you can also use your own test-runner.

@karlseguin shared one which you can use for reference

https://gist.github.com/karlseguin/c6bea5b35e4e8d26af6f81c22cb5d76b

[kj4tmp]

related:

Getting `zig build test` to find all the tests in my module

Edit: Changed title from “Should I put std.testing.refAllDecls(@This()); in all my files?” Ziggits docs suggest using test { std.testing.refAllDeclsRecursive(@This()); } in my root.zig But this is problematic when I declare a top-level declaration like so: pub const SyncM = packed struct(u64) { physical_start_address: u16, length: u16, /// control register control: esc.SyncManagerControlRegister, status: esc.SyncManagerActivateRegister, enable_sync_manager: Ena…

Improving Test Runner API

I think testing in zig should be easier. The issues I have could be summarized as: The test runner is just as lazy as the compiler. See How do I get zig build to Run all the tests? This means I have to know the intricacies of conditional compilation / lazy compilation to even begin to reason about which tests are “detected”. The observabilty of tests is poor. Even with --summary all and --summary new it is still difficult for me to know what tests have ran. This has resulted in me consciousl…

How do I get zig build to Run all the tests?

I think the way to do this is using std.testing.refAllDecls() . Basically as long as you reference varint and any other module you want tested in your root.zig , you can use that function so that the semantic analyzer sees them: I had been using that, though, as you note, it only seems to be discovering public attributes, though that might be because i wasn’t using them. My concern with this is, as you state, refAllDecls has been declared to be a hack. I wasn’t sure if this “hack” had had a …