Fuzz testing

[Sze]

@jmctagger

EDIT: I see the @disableInstrumentation() at the heads of many fuzz functions like eosWeightedSimple() - might be significant. I just haven’t pieced it all together.

Documentation for the @disableInstrumentation() builtin

I was reading the std.debug module and found the @disableInstrumentation() builtin, but I haven’t found any documentation that explains how it is supposed to be used.

Fuzz testing is basically generating random inputs (sometimes guided) trying to find ways to trigger failures, so it makes sense that inputs that don’t trigger errors are just counted as one more run that didn’t trigger any error (so no reason to output anything for that, because potentially you might run the fuzzer for a long time searching for a failure, so you wouldn’t be interested in non-interesting successful cases).

Also take a look at the web interface for the fuzzer (--fuzz without limit):

@sno2

$ zig 0.16.0 build test --fuzz
info(web_server): web interface listening at http://[::1]:46785/
info(web_server): hint: pass '--webui=[::1]:46785' to use the same port next time

Which then can provide a more interactive experience, presenting the statistics and showing green/red dots for the files indicating what has been explored by the fuzzer.

I am not super familiar with the fuzzer, but it seemed like you might have fallen into a routine always passing a limit to --fuzz and thus maybe being unaware of the webui, so I wanted to mention it, in case that’s true.