Basic fuzzing

In the sense that you are generating unknown values, yes; however, not quite in the sense of how they are generated.

If you are asking if you should break the generation of complex types into smaller reusable functions, that is generally a good idea; and yes, they would use Smith as it is the the root source of getting values from the fuzzer. Take as an example the compiler’s own AstSmith which generates Zig ASTs (abstract syntax trees, essentially Zig code). It is currently used to fuzz test AST-parsing and zig fmt.

Not quite, Smith is not doing any value generation. Most of the time, it calls into the fuzzer (example) to generate values, though it is also able to read a pregenerated input (Smith.in) for rerunning inputs. Hashes are only used as a way to give the fuzzer context on the source of different values so it can more effectively mutate them. They don’t get included in the inputs, even which the fuzzer stores on disk for if it gets restarted, in any way. The implementation of the fuzzer logic and mutations can be found here.

Doesn’t the Smith need to understand higher level constructs to do mutations?

By default, the fuzzer already has several details on the construction of the input, from which value requests are end-of-streams, bytes, etc., from hashes, and from which blocks the input hits. However, the fuzzer expects that you give some effort in guiding it to construct interesting inputs, for example by using weights. The more you guide the fuzzer, the more efficient your fuzzing campaign (test) will be. For example,

an efficient input generator for such a case would request a bool which decides if the checksum will be populated with the correct value for the fuzzer. It would weigh at runtime the values of other fields based off what makes sense for the already generated dependencies.

No matter the fuzzer implementation, a test where more inputs lead to interesting behavior is going to be more efficient than the contrary, which is why Zig fuzzing leans into the smith-based approach.

When a fuzz test crashes the input gets saved to .zig-cache/f/crash (this path is also printed when the test crashes). You can copy the file into the module with you fuzz test and use the .corpus option with the input included using @embedFile (more detailed post). The corpus is automatically run with the test even in non-fuzzing mode.

7 Likes