Marionette: deterministic simulation testing for Zig, built on std.Io

Showcase
,
1

Marionette

Marionette is a deterministic simulation testing library for Zig code written against std.Io.

Repo: https://github.com/sb2bg/marionette
Docs: https://sb2bg.github.io/marionette/

Existing code using std.Io can run under Marionette today with no changes.

In tests, your code keeps accepting std.Io, but the backend becomes Marionette’s single-threaded deterministic simulator. Clock, randomness, disk, tasks, futex waits, and a growing std.Io.net subset all run from one seed, so every failure replays identically. Long term the goal is to be the deterministic std.Io for Zig, since the new Io interface makes FoundationDB style simulation testing something a library can offer instead of something you architect your whole system around.

We already run real unmodified third-party code:

  • xit-vcs/xitdb runs through the simulated std.Io.File backend with crash and torn-write faults, plus a fuzzer that shrinks failures to a maintainer-readable operation sequence.
  • g41797/mailbox runs on the cooperative scheduler, exercising timed receive, send/wake, and same-deadline timeout ordering.
  • lalinsky/dusty’s real Server.listen accept loop runs as a simulated task over simulated std.Io.net streams, including cooperative-cancellation shutdown with error.Canceled landing in its accept park.

None of these were patched because std.Io allows us to change backends with zero code changes. It also runs an unmodified std.http.Client against a simulated server (http://localhost:<port>/ included, with lookup resolving address literals deterministically), with identical replay.

Test code gets a second surface called control for injecting faults:

try control.disk.crash();
try control.disk.corruptSector(path, offset);
try control.network.partition(&side_a, &side_b);
try control.network.setLossiness(.{ .drop_rate = .percent(20) });
try control.network.heal();

Beyond disk and network there’s deterministic allocation-fault injection (Env.allocator() with seeded fail-after and quota faults via control.allocation) and FoundationDB-style env.buggify hooks. Every run emits a structured trace, and failed runs print it with the seed.

Has it found anything? Yes. Running a small external KV store through an exact model oracle turned up three real bugs, two of them with no fault injection at all. It has found a B+tree leaf insert that overwrote an existing entry’s payload (shrunk to a four-put repro), delete rebalancing that broke separator invariants and made unrelated keys unreachable, and recovery that trusted a corrupt root page.

Details are in FOUND_BUGS.md, which also records the negative and boundary results. Marionette is alpha with no API stability before 1.0, and it requires Zig 0.16.

Boundaries: this is cooperative std.Io concurrency, not preemptive thread or memory-model testing. The simulated std.Io surface is a subset of the full interface, and code that steps outside it won’t run under simulation yet. Windows x86_64 fibers are deliberately disabled until the Win64 entry ABI has execution coverage.

zig fetch --save https://github.com/sb2bg/marionette/archive/refs/tags/v0.5.0.tar.gz

The examples/ directory is the best starting point. If you maintain a library that takes std.Io and you want it run under simulation, I’d love to hear from you. The external validation targets (xitdb, mailbox, dusty, ochi) have been the best forcing function for the simulator so far, and I’m looking for the next one.

25 Likes
2

I have checked the docs a little, and to me it seems very limited because it creates its own Io. I can not use it to test my single threaded application that relies on evented Io like one provided by std.Io.Evented or zio.Runtime. To me it would seem much more useful if it could wrap provided Io. Or am I misunderstanding, or not seeing something?

EDIT: I guess that would be difficult without actually getting the simulator tied to the specific implementation? Since for example read operation has to yield back to the runtime, which Marionette can not do without actually running the read operation? I don’t have that good understanding of these things… But I know zio has a yield function that I assume does that. Maybe marionette could somehow generically use that.

1 Like
3

It is not meant to wrap an existing backend. For deterministic simulation, Marionette has to be the backend, or else it wouldn’t be (guaranteed to be) deterministic. It owns the clock, scheduler, task wakeups, disk, network, randomness, and fault injection. If it wrapped a real evented runtime underneath, the real runtime would still be making scheduling and readiness order decisions, which would leak nondeterminism.

With the new Io interface, the idiomatic Zig pattern is to accept std.Io, which makes swapping in Marionette extremely non-invasive (at least for libraries/projects that only use the subset that Marionette implements currently).

So the intended pattern is application/library code accepts std.Io, and in normal runs you pass std.Io.Evented/Threaded/zio/etc., while in simulation tests you pass Marionette’s deterministic Io instead.

If an application directly depends on zio-specific runtime APIs, or goes below std.Io into a particular event loop, then Marionette cannot simulate that without either an adapter/facade for that runtime or some refactoring toward std.Io. That is a limitation but it’s different from “Marionette should wrap an existing Io.” Replacing the backend is what gives the replay guarantee.

On your edit: that is the hard part. A generic yield hook alone is not enough, because Marionette also needs to control readiness, wakeups, timers, ordering, faults, and simulated time. If a runtime exposed enough hooks for Marionette to drive those decisions, then an adapter could be possible. But that would be a runtime specific integration and not a generic wrapper around arbitrary Io.

3 Likes
4

Well, I mean I don’t exactly depend on specific Io, but I know it runs in green threads, and I want to test that it works well when run with green threads where individual fibers are interleaved. It feels like Marionette should be able to do that while keeping determinism, since it is still single threaded program. Maybe Marionette could have option to use green threads to simulate evented Io?

EDIT: I am checking the codebase. I think I am just not understanding this correctly, maybe Marionette already does what I need/want. I will have to try it out later.

1 Like
5

I understand now. Marionette has what you’re describing, but through Marionette’s own scheduler rather than by reusing another runtime’s scheduler.

In simulation, std.Io.async/std.Io.concurrent spawn scheduler owned cooperative tasks. Those tasks are fibers on one OS thread. When multiple tasks are runnable, Marionette chooses the next ready task from the simulation seed. Blocking operations like await, futex waits, sleeps, and supported io waits park the current task, and wakeups/timers put tasks back into the ready set. Owning the Io backend is what lets Marionette control those interleavings.

I should probably make this clearer in the docs/post. Thanks for bringing it up :slight_smile:

2 Likes
6

I’ve not read the sources yet, but I want to understand, you fully simulate even things like disk?

So if I create a file, write something, then open it, it works? How does it work?

And for networking, I guess that means only internal networks within the apps works? Or can you reach the internet from the app?

1 Like
7

Hi Lukáš, big fan of zio! Speaking of reading the source, zio helped me fix a nasty bug when working on my green thread scheduler.

To answer, simulated, yes, fully, deliberately no. I think these questions should be known without having to read the code. It seems my documentation is maybe not the best, and prevents this, so I’ll take that as a push to improve them.

Here is the breakdown!

Disk

Marionette implements std.Io (just like zio, as you know), so unmodified code doing createFile → write → openFile → read just sort of works. File contents and the directory namespace live in an in-memory, sector based simulated disk. It covers create/open/stat, positional and streaming read/write, setLength, sync, delete, rename, and file locks. Anything not routed through simulator state (symlinks, mmap, …) fails closed.

The disk tracks synced vs. pending writes separately, and on a simulated crash, pending writes can be dropped, torn at sector granularity, or reordered. This makes it seeded, traced, and replayable. That’s how it catches “you forgot to fsync” and “recovery trusts bytes it shouldn’t” bugs.

Network

Internal only, also as a feature. Real network, obviously, is a major nondeterminism source, so it fails closed too. The std.Io.net backend (listen/connect/accept/read/write) runs over simulator owned packet queues with seeded latency, drops, partitions, and node kill/restart. Concretely for dusty, your client and server ran as tasks inside one simulation, which is how I could kill the “server,” count the client’s dials exactly, and replay it from a seed.

Scheduling

io.concurrent, mutexes, events, sleeps, and cancellation all run on a cooperative scheduler with seeded, replay visible interleaving, and time is virtual, sleep(100ms) just advances the simulated clock. That’s what surfaced the shutdown drain bug, the busy-spin froze virtual time at 100% CPU, which is an obvious bug in a simulation but could read as shutdown being slow under real threads.

Marionette also has other simulation primitives such as a seeded PRNG, an allocation authority that wraps the harness allocator with deterministic failure injection, etc., and I am happy to go into those too, but I wanted to keep this to the big three.

TLDR: Your code just takes std.Io and never knows Marionette exists. And the zio error injection idea is complementary rather than competing, it tests the real backend’s error paths, simulation tests the logic above the io boundary (both dusty findings were in that second category and neither involved an I/O fault).

7 Likes
8

That’s very interesting, thank you. I’ll definitely give it a try. It also makes me want to extend std.Io, forcing you to simulate a bit more. Currently I’m using my zio APIs in my database, because std.Io is very limited for my needs, but I see a lot of value in such simulation, assuming it works well, might be the motivating factor for me to start using std.Io directly. I just wish the Zig PR queue would move a bit faster…

1 Like
9

I was reviewing your context switching, as I was recently debugging a related issue. You need to add xmm# and ymm# register aliases to the list for x86_64, because LLVM will not mark them as clobbered despite you having zmm# there, if the built targets a CPU that only has the smaller registers. Note that only the x86_64 LLVM backend has this issue, the aliases work fine in Zig’s own backend, an also work correctly in the LLVM ARM backend. Also, x18 on AArch64 is only a system register on Windows and macOS, not on other systems and I know for a fact that LLVM will use it as a general purpose register in release builds on Linux, so you need to mark it as cloberred. This might save you hard-to-debug issues, I spent long hours staring at assembly in debugger over these. :slight_smile:

2 Likes
10

Thank you! I had a similar painstaking experience.

I thought we had covered this because Marionette already carries a corrected copy of std.Io.fiber.contextSwitch, but those fixes addressed different problems :laughing:. Specifically,

  1. the message register being listed as input, output, and clobber
  2. the missing AArch64 nzcv clobber
  3. the unsafe inlining of the returns twice style assembly.

I copied the remainder of std’s register list, so Marionette inherited the zmm only coverage and skipped x18.

One small nuance I found while checking is that LLVM reserves X18 by default on Android, Darwin, Fuchsia, Windows, and OHOS, so I gated it using that full set rather than only Windows/macOS.

I’ve added all xmmN/ymmN clobbers and the conditional x18 clobber locally. Thank you for reviewing this and saving me another register allocation debugging session :smiley:

Also - extending your std.Io would definitely be helpful! Having libraries as validation to determine what to simulate has been the most productive. If you ever find something that can’t be simulated, raise an issue and I’ll get on it :wink:

2 Likes
11

Btw, regarding the inline marker for the context switch, it’s actually essential for performance to keep it as inline, so that the compiler sees the register allocations in the calling function and spills only what’s needed there, without preamble and possibly other unnecessarily saved registers. In your simulation, it probably doesn’t matter, but it does matter in general.