Zprof - Cross-allocator profiler that minimizes overhead: tracks allocations, detects memory leaks and logs memory changes

https://github.com/ANDRVV/zprof

8 Likes

Cool project! Why capture logical requested sizes and not real allocator footprint? If i’m reading it right… alloc it records n, and on free it records buf.len, and on resize/remap it only records the diff between old and new logical lengths?

So if the underlying allocator rounds up, keeps slack, fragments, or burns pages internally, zprof doesn’t see any of that. Its “live bytes” means “bytes the program asked for and hasn’t freed,” not “actual memory cost” right? Or did I miss something?

1 Like

Yes, it tracks only logical memory footprint.

1 Like

Cool project, I was seriously considering trolling through the archives to find the late LoggingAllocator to do something like this.

Maybe worth mentioning in case someone expects something else, but a good choice in my view. It’s the program being profiled, not the allocator, at least usually. The things @Slice mentioned matter, in a sense, but they aren’t actionable unless the point is to choose an allocator for the program, or benchmark a few to make comparisons: and it usually isn’t.

2 Likes

Appreciate the clarification. I mostly chimed in because i’ve been hacking around with a similar concept. Cool project!

1 Like

What other allocation values ​​do you recommend I track?

2 Likes

Only one I can think of is using @returnAddress to try and make a bag of callsites and how often they get hit. Except that probably won’t work because of the VTable: you’ll just see Allocator calling your impl. But there’s probably a dead-slow version which would work, which you could hack together from some techniques in std.debug and Panic.

That allows for casting a wide net, just run the program and find where all the hits are coming from. I think what you have right now is providing most of the use at much less cost (runtime penalty and implementation burden), since the tracker can be hooked in to specific parts of the program already.

Y’know, maybe add the ability to name them? That way if more than one is in play, the reports can be told apart. I’d say that disposes of most of the argument for examining the stack, at no cost worth mentioning.

1 Like

Depends what you want to profile but the core is usually stock and flow: live/peak bytes, plus byte rates over time. After that, maybe alloc/free counts and per-scope labels. Aka BYTES semantics. You’re on a good scent I like the library

1 Like

Depends on whether you want a debugging profiler, a benchmark profiler or an allocator-internals profiler. For a generic wrapper ’d track these first tom start for real then build from there

  • live requested bytes

  • peak live requested bytes

  • total allocated bytes

  • total freed bytes

  • outstanding allocation count (which then could prob be the basis for a unch of other metrics)

1 Like