Hi folks, figured I’d share something I’ve been using for a while to gather some light brainstorming. Long post below!
Often when working with external C libraries (e.g. raylib) you’re required to call functions that allocate resources opaquely. This is pretty normal in other languages, but in Zig it’s considered bad manners. This means these resources aren’t tracked in the same way the lifetimes of pure-zig allocations are tracked, and we lose all the lovely benefits Zig grants us by doing so.
I also like to keep my “platform” code walled off and this presented another challenge when needing to store resouces in the higher up “generic” layer - in Zig it typically meant storing *anyopaque, which isn’t type safe and is somewhat prone to mistakes. It’s especially confusing when the platform type is itself a pointer, making the *anyopaque actually a **T, which is little messy for my liking.
In my personal projects this discrepancy at the “platform” layer bothered me and I came up with a few core requirements for some kind of “fix”:
- Detect leaks of external resources, especially in tests
- Track and free resources arena-style when needed
- Allow type-safe transfer of external resources through code unaware of the external library
I spent some time trying to hack my way into doing this with just a std.mem.Allocator but kept coming up short - how would it know how to free the resource? How would I implement an arena? This was not something that could magically “just work”.
The solution I eventually settled on is Annexer. An annexer is very similar to std.mem.Allocator, but rather than only providing memory it also takes ownership of resources and knows how to free them.
Just like a zig allocator it has alloc and free which simply allocate space for the resource, but it also adds annex and liberate. These allow the caller take ownership of and free a resource. annex must be told how to free the resource, whereas a zig allocator’s alloc/create needs no extra knowledge.
With some safety checks and boilerplate removed the functions look like so:
/// Reserves memory to store type T
/// Calls to `free` will safely free this memory
pub fn alloc(self: Annexer, T: type) error{OutOfMemory}!*T {
const ptr = self.rawAlloc(@sizeOf(T), .of(T), @returnAddress()) orelse return error.OutOfMemory;
return @ptrCast(@alignCast(ptr));
}
/// "Annex" given data, storing it for later liberation
/// Note that this copies the resource value so you must use the ptr in the annex to do any operations that might mutate it
/// For this reason, prefer to use `create`
pub fn annex(self: Annexer, T: type, value: anytype, ptr: *T, comptime liberateFn: *const fn (T) void) Annex(T) {
...
self.rawAnnex(std.mem.asBytes(ptr), .of(T), std.mem.asBytes(value_ptr), TypeErased.liberate);
return .{ .ptr = ptr };
}
/// Liberates the resource using the liberateFn set in `annex`
/// Safe to call if `annex` has not been called
pub fn liberate(self: Annexer, T: type, ptr: *T) void {
self.rawLiberate(std.mem.asBytes(ptr), .of(T));
}
/// Frees the memory of the annex
/// Unsafe to call if `annex` was used without a call to `liberate`
pub fn free(self: Annexer, T: type, ptr: *T) void {
self.rawFree(std.mem.asBytes(ptr), .of(T), @returnAddress());
}
Just like std.mem.Allocator an Annexer also provides “helper” functions for better safety and predictability creating concrete types: create and destroy. create especially follows a “reserve first” approach for error safety, as well as ensuring the returned value from createFn is copied into the annex and discarded before accidental use - a common mistake that could occur when using the raw annex function.
/// Reserve and annex the return value from a function in a safe way, correctly using errordefer
pub fn create(self: Annexer, T: type, comptime createFn: anytype, args: anytype, comptime liberateFn: *const fn (T) void) !Annex(T) {
...
// Safely allocate, create the resource and then annex it's value
// The key here is to ensure we reserve the memory first, prepare the free, and then create the object
// This follows proper "Reserve First" principles as seen in https://matklad.github.io/2025/08/16/reserve-first.html
const ptr = try self.alloc(T);
errdefer self.rawFree(std.mem.asBytes(ptr), .of(T), @returnAddress());
const val = try @call(.auto, createFn, args);
return self.annex(T, val, ptr, liberateFn);
}
/// Liberate the resource and free the annex memory
/// Providing the type here at the call site ensures mistakes are not made, e.g. when the resource is itself a pointer
pub fn destroy(self: Annexer, T: type, ptr: *T) void {
self.rawLiberate(std.mem.asBytes(ptr), .of(T));
self.rawFree(std.mem.asBytes(ptr), .of(T), @returnAddress());
}
One thing that makes it clear how an annexer is still very close to an allocator is it’s vtable, which only adds two extra functions on top of std.mem.Allocator’s.
ptr: *anyopaque,
vtable: *const struct {
/// Copy the memory provided into the reserved buffer, in doing so set the
/// function to free the resource, used when free is called.
/// If `annex` is not used, liberate must still be safe to call
annex: *const fn (self: *anyopaque, memory: []u8, alignment: std.mem.Alignment, data: []const u8, liberateFn: LiberateFn) void,
/// Liberate the resource held in the annex using the chosen liberateFn
liberate: *const fn (self: *anyopaque, memory: []u8, alignment: std.mem.Alignment) void,
/// Store the vtable for a zig allocator using the same backing memory
/// The alloc/free provided here are used to allocate and free the annex
allocator: std.mem.Allocator.VTable,
},
For convenience Annexer also provides access to a std.mem.Allocator. This largely came about about because it’s common to be making “real” allocations alongside creating resources, and having fn(allocator: std.mem.Allocator, annexer: Annexer) everywhere was getting quite tedious. That this allocator will have identical lifecycle semantics to the annexer - if the annexer was created as an arena, then this allocator will behave the same too.
/// Return a zig allocator to use for regular allocations
pub fn allocator(self: *const Annexer) std.mem.Allocator {
return .{ .ptr = self.ptr, .vtable = &self.vtable.allocator };
}
One type seen above is an Annex(T). This is a small wrapper providing type safe access and a init/deinit lifecycle that allows for simplified usage. The “annexed” resource (i.e. T) may itself be a pointer, so an Annex(T) removes the **anyopaque type confusion for the user.
/// An Annex represents a references to an external object, ID, handle or pointer
pub fn Annex(T: type) type {
return struct {
const Self = @This();
ptr: *T,
fn noLiberate(_: T) void {}
pub fn init(a: Annexer, comptime annexFn: anytype, args: anytype, comptime liberateFn: ?*const fn (val: T) void) !Self {
return a.create(T, annexFn, args, if (liberateFn) |f| f else noLiberate);
}
pub fn deinit(self: Self, a: Annexer) void {
a.destroy(T, self.ptr);
}
pub fn value(self: Self) T {
return self.ptr.*;
}
...
}
}
That final bullet point requirement was to pass the type through type-unaware code safely, this is achieved by borrowing some runtime type safety other’s have come up with. Annex(T) has one extra function that returns an OpaqueAnnex which can be passed around as needed, although so far this need has been rare.
/// An OpaqueAnnex represents a type-erased reference to a external object/id/handle/ptr
/// This type can be stored above the platform layer and used generically between different implementations
/// When the type is known, prefer Annex(T)
pub const OpaqueAnnex = struct {
memory: []u8,
typeid: if (std.debug.runtime_safety) TypeId else void,
/// Zig native "type ID" in userspace
/// See https://github.com/ziglang/zig/issues/19858#issuecomment-2369861301
const TypeId = *const struct {
_: u8,
};
/// Generate a typeID at comptime
fn typeId(comptime T: type) TypeId {
return &struct {
comptime {
_ = T;
}
var id: @typeInfo(TypeId).pointer.child = undefined;
}.id;
}
pub fn typed(self: OpaqueAnnex, comptime T: type) Annex(T) {
// Protect against misuse, ensure we only resolve to the correct type
if (std.debug.runtime_safety)
std.debug.assert(typeId(T) == self.typeid);
return .{ .ptr = @ptrCast(@alignCast(self.memory.ptr)) };
}
};
pub fn Annex(T: type) type {
return struct {
...
pub fn toOpaque(self: Self) OpaqueAnnex {
return .{
.memory = std.mem.asBytes(self.ptr),
.typeid = if (std.debug.runtime_safety) OpaqueAnnex.typeId(T) else {},
};
}
};
}
Put together this allows me to write the code below when interacting with external libraries. This example uses the excellent raylib-zig that already wraps the external library with error handling and zig lifecycles.
const raylib = @import("raylib");
pub const Shader = struct {
annex: Annex(raylib.Shader),
pub fn init(vs: [:0]const u8, fs: [:0]const u8) !Shader {
return .{
.annex = try .init(annexer, raylib.loadShader, .{ vs, fs }, raylib.unloadShader),
};
}
pub fn deinit(self: *const Shader, annexer: Annexer) void {
self.annex.deinit(annexer);
}
...
}
The key thing to note here is that for the caller this Shader class largely behaves exactly like any other zig class, only needing an Annexer rather than an Allocator. In tests, a failure to call deinit will result in triggered leak detection, and in normal usage an arena-style Annexer could be passed in and deinit can be completely skipped.
Put all this together and it allows me to write concise and zig-like code, freed from (most) of the shackles of using untracked external resources.
// Set up a general annexer, similar to the Zig GeneralPurposeAllocator
var gpannexer: GeneralPurposeAnnexer = .init(std.testing.allocator);
const annexer = gpannexer.annexer();
const allocator = gpannexer.allocator();
// Perform regular work
const shader: Shader = try .init("foo.vs", "foo.fs", annexer);
defer shader.deint(annexer);
const mem = allocator.create(u128);
defer allocator.destroy(mem);
// Set up an arena annexer, similar to the zig ArenaAllocator
var arenannexer: ArenaAnnexer = .init(annexer);
defer arenannexer.deinit();
const arena_annexer = arenannexer.annexer();
const arena = arenannexer.allocator();
// No need to clean up
const shader2: Shader = try .init("foo.vs", "foo.fs", annexer);
const mem2 = allocator.create(u128);
The full implementation of Annexer, GeneralPurposeAnnexer and ArenaAnnexer along with much more thorough example/testing code can be found here. They could all use a little more refinement but for my needs they’re in a good place already - the memory and performance overhead is small enough that the benefits of found leaks and cleaner code has far outweighed any potential losses.
What do you think? Is this overkill? Are there other known solutions that meet my needs? Personally I hugely enjoyed working on this, especially discovering how elegant std.mem.Allocator already is and how possible it was to build on it’s foundation. If someone points out a far, far simpler solution then if nothing else I learned a lot about allocations and memory lifecycles!
