Fuzz testing example?

[kj4tmp]

I want to fuzz test a serialization library I wrote. Here is my simplest example:

test "fuzz ints" {
    const expected: u64 = 123; // <- put random number here
    const encoded: []const u8 = encodeBounded(expected, .{}).slice();
    try std.testing.expectEqual(expected, decode(@TypeOf(expected), encoded, .{}));
}

I would like to test 1000 random iterations per build.

[squeek502]

For the integrated fuzz testing (which is still a work-in-progress), there is an example within zig init if you’re using the master version of Zig:

zig init
zig build test --fuzz

The test code currently looks like this:

test "fuzz example" {
    const global = struct {
        fn testOne(input: []const u8) anyerror!void {
            // Try passing `--fuzz` to `zig build test` and see if it manages to fail this test case!
            try std.testing.expect(!std.mem.eql(u8, "canyoufindme", input));
        }
    };
    try std.testing.fuzz(global.testOne, .{});
}

Note that this currently doesn’t have a sophisticated way of generating inputs, though (relevant issue is enhance the fuzzing algorithm to be competitive with other mainstream fuzzers · Issue #20804 · ziglang/zig · GitHub).

If you want something more sophisticated, you can use AFL++, see either:

• GitHub - kristoff-it/zig-afl-kit: Convenience functions for easy integration with AFL++ for both Zig and C/C++ programmers!, or
• Fuzzing Zig Code Using AFL++ - ryanliptak.com (I wrote this article and it’s the approach I still use personally, but I am behind the times and zig-afl-kit is probably the better way to go; the code in the article needs an update as well, see here for a more up-to-date example of what’s detailed in the article)

Finally, if you don’t care about coverage-guided fuzzing, you can just write some code to generate inputs and use a loop. You can then also use build options to control the number of iterations if you’d like. See here for an example of that sort of thing (note the use of std.testing.random_seed and that iterations comes from the fuzzy-iterations option in the build.zig)

[andrewrk]

That reminds me I have some local work that pushes fuzzing into MVP status, I just need a couple of bug fixes before it’s mergeable. I’ll decide whether to try to land that for 0.14.0 after I finish the active branch I’m working on.

[gonzo]

@andrewrk this would be such a great topic for a… stream! Were you able to fix your connectivity issues? Just imagine what a great end-of-year present this could be for the zig community!

[andrewrk]

Just had another wifi driver crash this morning
Still haven’t followed up with either running a cable through my hallway or hiring an electrician. Soon…