*If you use the default test runner, if you write your own test runner you can make up your own rules.
2 Likes
Wow, my failure to communicate seems to be at an all time low.
Ok, will try it this way. (Not that there’s anything that remains to be solved, but clearly confusion persists….)
- I want to write a lib that does stuff.
- I want good test blocks, naturally. I write several lines that test the set-up/initialization; standard… great.
- One function in the lib takes an Io.Writer, and writes to it.
- I add another test (or just make an earlier test longer) that demonstrates and tests the lead-up to calling this function.
- As I’m writing this function, incrementally, because it’s a bit complicated, I’d like to see its output, which will get a little closer to “complete” as I work out the function’s innards.
- I write a little. I run
zig build test. I notice the progress written to stdout/terminal. I write a little more. Ibuild test. I fix bugs. Rinse, lather, repeat. - I always use std.testing.allocator to expose any leaks right away; this is (only) appropriate within a test block, afaik.
- Finally, the function is complete - the output is looking like I want.
- I no longer need the Writer to write to the terminal - that was useful then, but now I just want to replace that with a
try testing.expectEqualSlices()that asserts the expected outcome. This makes everything clear to a reader, as documentation, it holds still so that if I change something that inadvertently messes up the output, then my test fails, etc.
So, in a way, yes, I’m describing “prototyping code”, and yes, that can be done from main(), but I don’t want to have to copy my code back and forth from main() to a test block when I make the transition to “move on”, which I do constantly, incrementally, bit by bit. And I don’t want to have to fundamentally change the code (too much). The test block is sufficient for me to see my progress work, and then it can become a regular test. Is that a bad way of doing things? But, perhaps this clarifies why I “over-simplified” in my OP, and just said the basics of what I wanted to do: to send a Writer to a function and let the write()ing spit to the console for my viewing. Perhaps my procedure is rather abnormal, and that’s why people are scratching their heads, but I don’t know much better, and it seems to be working quite well as I inch along my development, leaving nice little test blocks in my wake. It worked well, that is, until I bumped into this Writer (-to-stdout) “problem”, because I wasn’t aware that you couldn’t do that (unless, as @Sze mentions, you roll your own test runner), and I hadn’t thought sufficiently about how else to write() … er, “indirectly” to the terminal. Now that I have that nipped, I feel pretty good… except that I feel, now, maybe, that I’m doing it all wrong in general….?
You don’t need to write to terminal to test the output. One way would be to check the length of a buffer like i’m doing here:
test "Time struct formatting" {
var arena_allocator = heap.ArenaAllocator.init(heap.page_allocator);
defer arena_allocator.deinit();
const alloc = arena_allocator.allocator();
// Setup IO (Required for clock.now)
var io_init = std.Io.Threaded.init(alloc, .{ .environ = .empty });
const io = io_init.io();
// Create the Time instance
const tnow = Time.create(io);
// Test 'now()'
const now = tnow.fmt(.now);
const buf_now = try std.fmt.allocPrint(alloc, "{f}", .{now});
defer alloc.free(buf_now);
std.debug.assert(buf_now.len == 23);
// Test 'syslog()'
const syslog = tnow.fmt(.syslog);
const buf_syslog = try std.fmt.allocPrint(alloc, "{f}", .{syslog});
defer alloc.free(buf_syslog);
std.debug.assert(buf_syslog.len == 15);
}
You can switch to testing allocator and io if needed, I did not care for that. You can use std.mem.eql for example to compare strings if you want to compare the exact output if it’s known. In my case since I’m working here on the timestamp, I cannot compare exact output but I could add a code that compares the format. In case you are interested in seeing more of the library, how I’m doing formatting and other time calc stuff you can see it here:
https://codeberg.org/Jmarkovic/sortcp/src/branch/0.16/src/timestamp.zig
Since I was not able to find how to test formatting the way I think as a format is, I’ll show some options for my own code:
/// Only call in tests
fn testFormat(expected: []const u8, actual: []const u8) !void {
// Confirm length correctness
try std.testing.expectEqual(expected.len, actual.len);
// Pattern examples:
// dddd-dd-dd dd:dd:dd.ddd
// Ccc dd:dd:dd.ddd
for (actual, expected) |char, spec| {
switch (spec) {
'C' => try std.testing.expect(std.ascii.isUpper(char)),
'd' => try std.testing.expect(std.ascii.isDigit(char)),
'c' => try std.testing.expect(std.ascii.isLower(char)),
else => try std.testing.expectEqual(spec, char),
}
}
}
/// Only call in tests
fn testNowFormat(actual: []const u8) !void {
try std.testing.expectEqual(@as(usize, 23), actual.len);
// Pattern: dddd-dd-dd dd:dd:dd.ddd
for (actual, "dddd-dd-dd dd:dd:dd.ddd") |char, spec| {
switch (spec) {
'd' => try std.testing.expect(std.ascii.isDigit(char)),
else => try std.testing.expectEqual(spec, char),
}
}
}
/// Only call in tests
fn testSyslogFormat(actual: []const u8) !void {
try std.testing.expectEqual(@as(usize, 15), actual.len);
// Validate Month (First 3 chars)
const month = actual[0..3];
var month_found = false;
for (MONTH_NAMES) |m| {
if (std.mem.eql(u8, m, month)) {
month_found = true;
break;
}
}
try std.testing.expect(month_found);
// Validate remaining structure: " dd dd:dd:dd"
const remainder = actual[3..];
for (remainder, " dd dd:dd:dd") |char, spec| {
switch (spec) {
'd' => try std.testing.expect(std.ascii.isDigit(char)),
else => try std.testing.expectEqual(spec, char),
}
}
}
I quite prefer the output of testing.expectEqualSlices(), for test blocks in particular. Have you seen how nicely it formats output to help you zero in on any differences?
In my case, the result is hundreds of characters long. Again, to be clear, I do not (and do not promote) some kind of “combo” use of debug.print() and testing.expect*() at the same time. I’m talking about using debug.print() to help me tweak my input data to make the output represent a fullest-sweep of the code under scrutiny. Once satisfied, I’m talking about replacing the debug.print() with a testing.expect*() for long-term re-testing, when I’ve moved on to other code, and just want to be alerted if changes elsewhere accidentally cause a change in these hundreds of characters in this test. But I don’t really have that nice long test comparison-string until I’ve … “prototyped” (as one put it) completely, and changed my mind a few times, upon seeing output, and making the test input more rigorous, to create still more nuanced output… until I’m happy, and “done”, and no longer in need of debug.print().
*as an aside on mem.eql()… I’m not sure why it’s not mem.equal(), while mem.allEqual() exists. Or why not mem.allEql()? But that’s beside the point. I’m just normally a stickler for full words; I have no problem with mem, but eql? Oh well.
I gave std.mem.eql() just as an example. At that time I did not know what testing equivalent is. And testing equivalent is the function you chose which I as well thing it’s better to use. I’m just a bit torn between writing tests as real code implications or using testing functions. I assume testing functions have stricter/extensive or better explanatory checks, but I’d like to actually see how a real code implication works as a test.
That was the reason why i was not using testing io/alloc in my first example. I could as well just use a mockup timestamp to test all this, but I decided to actually call and use a system clock for getting and comparing a timestamp.
1 Like