Inspired by @matklad’s blog post on snapshot testing for the masses, I decided to fork the TigerBeetle library to add a couple features. This lead down a small rabbit hole, where I completed a port of diff-match-patch, and wrote a regex library. All of which took quite a bit longer than the original task, which was quite simple. Anyway, Yaks shaved, here’s Oh Snap!
Oh Snap! Easy Snapshot Testing for Zig
It’s hard to know if a program, or part of one, actually works. But it’s easy to know if it doesn’t: if there isn’t a test for some part of the program, then that part doesn’t work.
Snapshot testing is a great way to get fast coverage for data invariants in a program or library. The article I just linked to goes into great detail about the advantages of snapshot testing, and you should read it.
Using ohsnap
The interface will be familiar if you read the linked blog post, which, really, you should.
One difference between ohsnap and the original, is that ohsnap includes pretty, a clever pretty-printer for arbitrary data structures. So you don’t need to write a custom .format method to use ohsnap, although if you have one, you can use that instead. Or both. Belt and suspenders kinda thing.
Writing a snap is simple, to get started, do something like this:
const OhSnap = @import("ohsnap");
test "snap something" {
const oh = OhSnap{};
// You can configure `pretty` by using `var oh` and changing settings
// in `oh.pretty_options`.
const snap_me = someFn();
try oh.snap(@src(),
\\
,
).expectEqual(snap_me);
}
Note that the call to @src() has to be directly above the string, and the string has to be multi-line style, with the double backslashes: \\. Both this:
try op.snap(@src(),
\\ etc
,).expectEqual(snap_me);
And this:
try op.snap(
@src(),
\\ etc
,).expectEqual(snap_me);
Will work just fine.
This test will fail, because the snapshot generated by pretty won’t be equal to the empty string. ohsnap will diff that empty string with what it gets out of snap_me, and print what it got in all-green, because that’s what happens when you diff an empty string against a string which isn’t empty.
If you like what you see, updating is simple. Change the file to the following:
const OhSnap = @import("ohsnap");
test "snap something" {
const oh = OhSnap{};
// You can configure `pretty` by using `var oh` and changing settings
// in `oh.pretty_options`.
const snap_me = someFn();
try oh.snap(@src(),
\\<!update>
,
).expectEqual(snap_me);
}
The snaptest will see the <!update>, which must be the beginning of the string, and replace it in your file with the output of the pretty printing. Easy!
If your data structure has a .format method, and you’d prefer to use that as a basis, just call oh.snapfmt in the same way.
If, down the road, the snapshot doesn’t compare to the expected string, ohsnap will use diffz, a Zig port of diff-match-patch, to produce a terminal-colored character-level diff of the expected string with the actual string, making it easy to see exactly what’s changed. These changes are either a bug, or a new feature. If it’s the former, fix it, if it’s the latter, just add <!update> to the head of the string again, and ohsnap will oblige.
Pattern-Matching Snapshots
This is fine and dandy, if the data structure, exactly as it prints, will always be the same on every test run. But what if that’s only true of some of the data?
Consider this example. We have a struct which looks like this:
const StampedStruct = struct {
message: []const u8,
tag: u64,
timestamp: isize,
pub fn init(msg: []const u8, tag: u64) StampedStruct {
return StampedStruct{
.message = msg,
.tag = tag,
.timestamp = std.time.timestamp(),
};
}
};
Which we want to snapshot test, like this:
test "snap with timestamp" {
const oh = OhSnap{};
const with_stamp = StampedStruct.init(
"frobnicate the turbo-encabulator",
17337,
);
try oh.snap(
@src(),
\\ohsnap.StampedStruct
\\ .message: []const u8
\\ "frobnicate the turbo-encabulator"
\\ .tag: u64 = 17337
\\ .timestamp: isize = 1721501316
,
).expectEqual(with_stamp);
}
But of course, the next time we run the test, the timestamp will be different, so the test will fail. We care about the message and the tag, we care that there is a timestamp, but we don’t care what the timestamp is, because we know it will be changing.
For cases like this, ohsnap includes mvzr, the Minimum Viable Zig Regex library, which I wrote specifically for this purpose.
Simply replace the timestamp like so:
try oh.snap(
@src(),
\\ohsnap.StampedStruct
\\ .message: []const u8
\\ "frobnicate the turbo-encabulator"
\\ .tag: u64 = 17337
\\ .timestamp: isize = <^\d+$>
,
).expectEqual(with_stamp);
Through the magic of diffing, ohsnap will identify the part of the new string which matches <^\d+$>, and try to match the regular expression against that part of the string. Since this matches, the test now passes.
Note that the regex must be in the form <^.+?$> (the exact regex we use is <\^.+?\$>, in fact), the ^ and $ are essential and are load-bearing parts of the expression. This prevents partial matches, as well as making the regex portions of a snapshot test easier for ohsnap to find. Note that because this is a multi-line string, you don’t have to do double-backslashes: its <^\d+$>, not <^\\d+$>. To be very clear, the < and > demarcate the regex, they aren’t part of it.
Let’s say you make a change:
const with_stamp = StampedStruct.init(
"thoroughly frobnicate the encabulator",
17337,
);
The test will now fail: the word “thoroughly” will be highlighted in green, turbo- will be marked in red, and the timestamp will be cyan, indicating that the regex is still matching the pattern string. If a change in the test data means that the regex no longer matches, then the part of the test string which should match is highlighted in magenta.
Since this was an intentional change, we need to update the snap:
try oh.snap(
@src(),
\\<!update>
\\ohsnap.StampedStruct
\\ .message: []const u8
\\ "frobnicate the turbo-encabulator"
\\ .tag: u64 = 17337
\\ .timestamp: isize = <^\d+$>
,
).expectEqual(with_stamp);
Once again, through the magic of diffing, ohsnap will locate the regexen in the old string, and patch them over the new one.
try oh.snap(
@src(),
\\ohsnap.StampedStruct
\\ .message: []const u8
\\ "thoroughly frobnicate the encabulator"
\\ .tag: u64 = 17337
\\ .timestamp: isize = <^\d+$>
,
).expectEqual(with_stamp);
Voila!
Usage note: in some cases, the changes to the new string will displace the regex, you can tell because some part of the regex itself will be exposed in red. When that happens, the update may not apply correctly either: the regex will always be moved to the new string intact, but it may or may not be in the correct place (usually, not). This can generally be fixed by making changes to the expected-value string until whatever part of the regex was sticking out of the diff is no longer exposed. Sometimes running <!update> twice will fix it as well.
That’s It!
One of the great advantages of snapshot testing is that it’s easy, so ohsnap, like the library it’s based upon, is intentionally quite simple. Simple, yet versatile, the latter to a large degree is owed to pretty, which can handle anything I’ve thrown at it, types, unions, you name it.
It’s a new library, but I expect the core interface to remain stable. It’s meant to do one thing, well, and otherwise stay out of the way. I’m willing to consider suggestions for ways to make ohsnap better at what it already does, however.
That said, the regex library mvzr is pretty new, and so is the added code in diffz, so version-bumps to fix any bugs in those can be expected over time. The build system doesn’t currently do update checks, so you’ll need to check for updates manually, for now.
I hope you enjoy it! Test early, test often, and do it the easy way.