Custom test runner for freestanding Cortex-M4 system

Help
, ,
1

I need to perform some platform specific unit tests and my first thought was to write a custom test runner that would run on my freestanding platform. The basic idea is to use the build system to create a test program, install it in an appropriate place and from there I can download and run it.

I started with test_runner.zig from the compiler and chopped away. This is the code I obtained:

//! Adapted from Zig compiler default test runner for unit tests.
//! Striped down to freestanding, microcontroller size.
comptime {
    _ = @import("start_main"); // platform specific start up
}
const builtin = @import("builtin");
const std = @import("std");
const SvcService = @import("SvcService"); // platform specific to obtain console I/O
pub const std_options = .{
    .logFn = SvcService.logToConsole,
};
pub fn main() void {
    @disableInstrumentation();
    SvcService.log_console.open(921_600, true, .terminal) catch return;
    const writer = SvcService.log_console.writer();
    const test_fn_list = builtin.test_functions;

    var ok_count: usize = 0;
    var skip_count: usize = 0;
    var fail_count: usize = 0;

    for (test_fn_list, 0..) |test_fn, i| {
        writer.print("{d}/{d} {s}...", .{ i + 1, test_fn_list.len, test_fn.name }) catch return;
        if (test_fn.func()) |_| {
            ok_count += 1;
            writer.print("OK\n", .{}) catch return;
        } else |err| switch (err) {
            error.SkipZigTest => {
                skip_count += 1;
                writer.print("SKIP\n", .{}) catch return;
            },
            else => {
                fail_count += 1;
                writer.print("FAIL ({s})\n", .{@errorName(err)}) catch return;
            },
        }
    }
    if (ok_count == test_fn_list.len) {
        writer.print("All {d} tests passed.\n", .{ok_count}) catch return;
    } else {
        writer.print("{d} passed; {d} skipped; {d} failed.\n", .{ ok_count, skip_count, fail_count }) catch return;
    }
    return;
}

Alas, I cannot build this test runner. I get the following errors:

$ zig build reltuple-test -freference-trace
reltuple-test
└─ install reltuple_unit_test
   └─ zig test reltuple_unit_test Debug thumb-freestanding-eabihf 3 errors
/home/andrewm/opt/ziglang/zig-linux-x86_64-0.14.0-dev.1952+9f84f7f92/lib/std/Thread.zig:556:9: error: Unsupported operating system freestanding
        @compileError("Unsupported operating system " ++ @tagName(native_os));
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/andrewm/opt/ziglang/zig-linux-x86_64-0.14.0-dev.1952+9f84f7f92/lib/std/posix.zig:107:33: error: struct 'posix.system__struct_1350' has no member named 'STDERR_FILENO'
pub const STDERR_FILENO = system.STDERR_FILENO;
                          ~~~~~~^~~~~~~~~~~~~~
/home/andrewm/opt/ziglang/zig-linux-x86_64-0.14.0-dev.1952+9f84f7f92/lib/std/posix.zig:48:13: note: struct declared here
    else => struct {
            ^~~~~~
/home/andrewm/opt/ziglang/zig-linux-x86_64-0.14.0-dev.1952+9f84f7f92/lib/std/posix.zig:1255:26: error: struct 'posix.system__struct_1350' has no member named 'write'
        const rc = system.write(fd, bytes.ptr, @min(bytes.len, max_count));
                   ~~~~~~^~~~~~
/home/andrewm/opt/ziglang/zig-linux-x86_64-0.14.0-dev.1952+9f84f7f92/lib/std/posix.zig:48:13: note: struct declared here
    else => struct {
            ^~~~~~

Clearly, some POSIX references are being mixed in and I’m at a loss for determining where. No reference trace is printed.

My question is, has anyone else done anything in this area ? Has anyone built a custom test runner for a freestanding system?

2

You need a custom entry point.
See: Testing for freestanding - #8 by dimdin

2 Likes
3

Further digging and head banging did turn up where the posix I/O references are coming from. In testing.zig in the standard library the print function is directed at standard error. Although I have plenty of I/O ability on my freestanding system, it does not look like stderr. I did make a number of attempts to fake things since you can supply a root declaration of the os.io.getStdErrorHandle function, but there lies a tangled rabbit hole that ultimately wants to lock standard error with a mutex during the printing along with declaring the type of fd_t as void if you don’t smell like the right OS environment.

Needless to say, we “freestanding” folks accept our status and find other solutions. In this case, I took a copy of testing.zig from the standard library and excised the std.debug.print dependency along with some other stderr hard coding and was able to build and run the same tests on my target platform as I am able to run on the desktop.

I’m happy about the outcome, despite the required hackery. I did not want to create a different testing mechanism for the target when there is a good one available. I will now be able to run most of my unit tests on both the target and desktop and for those tests that require target specific features, I now have a path to do that testing on the target.

1 Like