ZlowCheck, a model testing library to integrate with Zig’s fuzzer

Showcase
1

Dear Ziggiters,

Here’s a new library for you. It’s the first one I’ve ever implemented, though I promise I tested it as much as I could. The initial idea came from matklad’s article

Pre-pre-pre-alpha. Using this means participating in its development.

Zlowcheck does model testing and property-based testing the slow-food way: gustoso !

Docs
Devlog1: how the PRNG module works
Devlog2: how the model tester works

ZlowCheck implements a finite-entropy PRNG, which is meant to be used by its data generators:

// from the README:
const FinitePrng = @import("finite_prng");
// Create a fixed byte array as the entropy source
const bytes = [_]u8{ 0x12, 0x34, 0x56, 0x78, 0x9A, 0xBC, 0xDE, 0xF0 };
// Initialize the PRNG with the byte array
var prng = FinitePrng.init(&bytes);
// Get a random generator from the PRNG
var random = prng.random();

// Generate random values of different types
const int_value = try random.int(i32);

When the PRNG runs out of entropy to consume, it just throws an error.
Thus, the try in try random.int().

const int_value = try random.int(i32);

You can do basic property testing with it:

const sortedProperty = zlowcheck.property(
  []i32,
  i32SliceGenerator,
  predicateFn
);

const result = try zlowcheck.assert(
  sortedProperty, 
  .{}, // config
  std.testing.allocator
);

And you can do model testing with it:

const result = try assertStateful(
    ModelType,
    SystemType,
    &commands_list,
    &model,
    &system,
    .{}, // config
);

The overarching goal:
Using zig’s fuzzer to produce the byte slices that feed the PRNG:


fn runStatefulTest(_: void, fuzzer_input: []const u8) !void {
  var prng = FinitePrng.init(fuzzer_input);
  var random = prng.random();
  // run a bunch of model tests here
}

test "fuzz stateful testing" {
    try std.testing.fuzz({}, runStatefulTest, .{});
}

This would make for a nice bug-detection setup - for apps that use inputs other than strings: the prng acts as a de-serializer for the fuzzer’s input bytes.

There’s a bunch of mini-technical problems I ran across while putting the lib together, and would be happy to discuss them.

10 Likes
2

Thanks for sharing. I wrote a functional unit-test extractor for legacy fortran code and thought the next step would be to create framework to define property tests for them, thinking it’d be a nice way to incorporate zig at work.

3

Nice!
A word of warning: I’m still in the process of testing those of the generators that produce allocations (slice generators and array generators), so I can’t recommend them too warmly atm. Scalar value generators are fine.

You also have the option of implementing your own generators. Should be relatively easy, provided they conform to the interface - which is just a struct carrying a genFn, and a shrinFn.

1 Like