For the past few months, I’ve been working full-time on a long-standing frustration of mine: I want a performance/behavior profiler natively designed around non-linear execution flows.
By non-linear, I mean any software that has to detach itself from the strict LIFO order of native threads and callstacks: asynchrony/concurrency, events loops, queues, M:N scheduling, coroutines/fibers/etc, any form of distributed compute, any form of batching… Basically, anything multiplexing compute, I/O, data, or all of the above.
Another way to think about this is in terms of execution spans. In traditional LIFO systems, child spans start and end neatly within the scope of their parents. In heavily multiplexed systems, spans may…:
- Outlive their parents, or any of their ancestors
- Share zero temporal overlap with the context that spawned them
- Arrive out of order
- End up seemingly orphaned
- Unintentionally stall or never finish at all
- Seemingly finish before they even started (yes)
- Any combination of the above
- Etc
Inspecting these systems through the lens of a native thread is usually not useful at best, and often actively harmful at worst (i.e. sends you on wild goose chases). What matters is the logical execution flow, not the physical one.
Right now, we have tools like perf and Tracy on one end of the spectrum (legitimately incredible for LIFO, but fall short on multiplexed flows) and high-level observability protocols like OpenTelemetry on the other (great for logical workflows, but designed for enterprise observability rather than profiling, meaning: heavy sampling, bloated SDKs, slow UIs, and generally a focus on aggregated patterns rather than raw data).
I want Tracy-level UX and performance, but built specifically for non-LIFO paradigms.
Crucially, It must also be great at detecting things that are not executing, or stopped doing so halfway through: IME, things that should be running and silently aren’t (a coroutine lost at sea, a queue that doesn’t get popped, purely userspace deadlocks and livelocks, etc) often turn out to be more damaging than active bottlenecks in these kinds of systems.
Here’s a screenshot from an early prototype, where Balamb is observing its own ingestion pipeline:
(This is Dear Imgui, although I do want to give DVUI another try when I get a minute – it seems to have made a lot of progress since I last looked.)
After months of exploring the problem space, wrestling with the core data model, trying many different things for the main data structures, and endlessly suffering through UI/UX design, I believe I finally have an architecture I’m happy with on all fronts. Now I just have to build it all
.
Building on Zig has been incredible. Kudos to the team.