Generic Managed Implementation

Currently the managed variations of ArrayList are marked as deprecated and I think that’s the right choice, but providing a way to opt-in to higher level ways of managing memory I think is valuable to have.

The dream would be:

// AFAIK this can't be done
test "struct managed memory" {
    // we opt-in to the choice of having the struct keep track of it's allocator
    var arr: Managed(std.array_list.Aligned(u8, null)) = .initCapacity(std.testing.allocator, 10);
    defer arr.deinit();

    try arr.append(10);
}

I’ve been able to create a working proof of concept that gets as far as the following:

// sc == Static call
// ic == Instance call
// see full implementation below for details
test "struct managed memory" {
    var arr: Managed(std.array_list.Aligned(u8, null)) = .sc("initCapacity", .{std.testing.allocator, 10});
    defer arr.ic("deinit", .{});

    try arr.ic("append", .{10});
    try arr.ic("append", .{20});
    try arr.ic("append", .{30});
}
Full implementation
// zig 0.15.2

const std = @import("std");

pub fn Managed(comptime T: type) type {
    const ti = @typeInfo(T);
    if (ti != .@"struct") @compileError("Can only manage structs.");

    const M = struct {
        allocator: std.mem.Allocator,
        managed: T,

        /// `init` methods will need to return the type `T`, wrapped in the managed struct `M`
        fn willMakeManagedInstance(comptime name: []const u8) type {
            if (errorlessType(fnReturnType(T, name)) == T) return @This();
            return void;
        }

        /// Static call
        pub fn sc(comptime name: []const u8, args: anytype) willMakeManagedInstance(name) {
            const method = @field(T, name);
            const method_ti = fnTypeInfo(T, name);
            if (method_ti.params.len > 0 and method_ti.params[0].type == T) @compileError("Use ic instead.");
            if (errorlessType(fnReturnType(T, name)) == T) {
                return @This(){
                    // WARN: just assumes the first arg is the allocator, should actually check
                    .allocator = args[0],
                    // WARN: just a hacky fix to not address errors for now
                    .managed = @call(.auto, method, args) catch @panic("oom"),
                };
            }
        }

        // Instance call
        pub fn ic(self: *@This(), comptime name: []const u8, args: anytype) fnReturnType(T, name) {
            const method = @field(T, name);

            // TODO: should probably check if the param of the method being called actually needs an allocator

            return @call(.auto, method, .{&self.managed, self.allocator} ++ args);
        }
    };

    return M;
}

// Example

// test "struct managed memory" {
//     var arr: Managed(std.array_list.Aligned(u8, null)) = .initCapacity(std.testing.allocator, 10);
//     defer arr.deinit();
// 
//     try arr.append(10);
// }

test "struct managed memory" {
    const ManagedArrayList = Managed(std.array_list.Aligned(u8, null));
    var arr: ManagedArrayList = .sc("initCapacity", .{std.testing.allocator, 10});
    defer arr.ic("deinit", .{});

    // ideally it would be nice to create an actual function in some way so that you could just
    // do `try arr.append(10)`.
    try arr.ic("append", .{10});
    try arr.ic("append", .{20});
    try arr.ic("append", .{30});
}

// Type Helper functions

/// Get function type info of named function.
fn fnTypeInfo(comptime T: type, comptime name: []const u8) std.builtin.Type.Fn {
    const field = @field(T, name);
    const field_ti = @typeInfo(@TypeOf(field));
    if (field_ti != .@"fn") @compileError("Named field is not a function.");

    return field_ti.@"fn";
}

/// Get return type of named function.
fn fnReturnType(comptime T: type, comptime name: []const u8) type {
    const method = fnTypeInfo(T, name);
    const method_rt = method.return_type.?;

    return method_rt;
}

fn errorlessType(comptime T: type) type {
    const ti = @typeInfo(T);
    if (ti == .error_union) {
        // We don't care about the error part in this example.
        return ti.error_union.payload;
    }
    return T;
}

This was just an afternoon experiment so am sure could improve it a bit further but the current comptime limitations I think would prevent my ideal case. I’m using array lists as the example but the extended goal is that this is generic to anything that does allocations in it’s functions.

For some context, I’m coming from a background where I’ve done a lot of tutoring and teaching programming across a lot of languages and skill levels, so accessibility and ease of use is something I’m always thinking about even when working on my own projects.
I like zig a lot and would be cool if I could have it in my cards of choice when someone asks me about learning to make something.

I had this idea already, I decided against it for the following.

The only reason to have managed containers is to associate an allocator with a collection at runtime.
All other reasons are not important to zig or even go against the zig zen.

Most of your Managed implementation is overcomplicated by trying to pass the allocator implicitly. One of the main reasons managed containers have been deprecated is a preference for the allocation clarity you get from unmanaged containers.

There is also the complication of which argument is the allocator, with the upcoming Io interface you will find that you can’t just assume the allocator is the first argument. There are also APIs that may deem other parameters more important than the allocator and put them before it.

You can simplify your current and future implementation of Managed by removing that “feature” and just accessing the unmanaged collection directly. That also retains the allocation clarity zig prefers.

At that point Managed is just a struct with 2 fields and no functions, such a type is trivial to add to any project that needs it, so why should it be in std?

Another big reason against a std.Managed is that when you do need it you likely have multiple containers you want to associate with an allocator, you would not want to store the same allocator multiple times. Such a managed collection is best implemented on a case by case basis.

3 Likes

Thanks for the reply! Some things I’m still trying to wrap my head around:

All other reasons are not important to zig or even go against the zig zen.

I feel like I’m not understanding this. Both in why zig would find certain reasons unimportant and which part of the zen it ties to. Is the intention that the user should always be the one to decide and implement management?

One of the main reasons managed containers have been deprecated is a preference for the allocation clarity you get from unmanaged containers.

I think the allocation clarity of unmanaged containers has some trips that got me looking for a managed implementation. When passing an allocator to a function like append, there’s no guarantee an allocation will occur, just that it might need an allocation. Originally (when unmanaged containers was released) I thought that requirement of an allocator was to signify allocation will happen.

Long lived unmanaged containers felt like you need to write a managed container wrapper anyways unless it’s expected to drill the allocator through a parameter at every function layer. Then if you are passing it through every layer then risk of passing the wrong allocator seems to create a risk of memory leaks?


In higher level projects such as things like small games, simple websites/servers, random CLI tools, the convenience to sacrifice some management and optimisation for something 90% of the way there feels nice to work with. It also makes good examples of how to manage things so when I do need that last 10%, I can use the existing management as basis.

Perhaps I’m also being a bit pedantic in putting the language through the lens of “if someone brand new to programming”. A lot of the people I make things with in the spare time aren’t as technically inclined, so I don’t really get to reach for zig as much since I know there are a lot of questions I’d have to answer with “just because” since I know a full explanation isn’t going to fly for someone like that.

this is getting maybe slightly off-topic, but it is interesting to me that std.Io implementations sometimes need an allocator (but not all do) and wind up being “managed” structs.

1 Like