Stringifying structures containing an allocator

Help
,
1

Hello everyone,

Recently I have run into a problem and I think that I have figured out the reason – I have a structure that holds data, some of which is dynamic.
I have decided to make it the most zig-like and in it’s init
allow for setting the allocator.
Allocating works fine but now I cannot stringify my structure :frowning:

   +- zig build-exe zig Debug native 5 errors
/opt/zig-bin-0.14.0/lib/std/json/stringify.zig:641:52: error: cannot load opaque type 'anyopaque'
                            return self.write(value.*);
                                              ~~~~~^~
/opt/zig-bin-0.14.0/lib/std/json/stringify.zig:641:52: error: values of type 'fn (*anyopaque, usize, mem.Alignment, usize) ?[*]u8' must be comptime-known, but operand value is runtime-known
                            return self.write(value.*);
                                              ~~~~~^~
/opt/zig-bin-0.14.0/lib/std/json/stringify.zig:641:52: note: use '*const fn (*anyopaque, usize, mem.Alignment, usize) ?[*]u8' for a function pointer type
/opt/zig-bin-0.14.0/lib/std/json/stringify.zig:641:52: error: values of type 'fn (*anyopaque, []u8, mem.Alignment, usize, usize) bool' must be comptime-known, but operand value is runtime-known
                            return self.write(value.*);
                                              ~~~~~^~
/opt/zig-bin-0.14.0/lib/std/json/stringify.zig:641:52: note: use '*const fn (*anyopaque, []u8, mem.Alignment, usize, usize) bool' for a function pointer type
/opt/zig-bin-0.14.0/lib/std/json/stringify.zig:641:52: error: values of type 'fn (*anyopaque, []u8, mem.Alignment, usize, usize) ?[*]u8' must be comptime-known, but operand value is runtime-known
                            return self.write(value.*);
                                              ~~~~~^~
/opt/zig-bin-0.14.0/lib/std/json/stringify.zig:641:52: note: use '*const fn (*anyopaque, []u8, mem.Alignment, usize, usize) ?[*]u8' for a function pointer type
/opt/zig-bin-0.14.0/lib/std/json/stringify.zig:641:52: error: values of type 'fn (*anyopaque, []u8, mem.Alignment, usize) void' must be comptime-known, but operand value is runtime-known
                            return self.write(value.*);
                                              ~~~~~^~
/opt/zig-bin-0.14.0/lib/std/json/stringify.zig:641:52: note: use '*const fn (*anyopaque, []u8, mem.Alignment, usize) void' for a function pointer type

I do not need the allocator JSONed, I just need the data.

The only solution I can think of is to not include it a structure by hardcoding it
Do you have any better ideas?

1 Like
2

Hi,
take a look here.
You can always wrap your types in a struct and implement a:

pub fn jsonStringify(self: *@This(), jw: anytype) !void

function yourself to avoid the standard behavior of std.json.serialize on it.

ah and here is a test in the standard library

4 Likes
3

FWIW, it’s generally not recommended to embed the allocator in your struct.

1 Like
4

The better way to do it would be to simply remove the allocator, and transition your implementation to an unmanaged variant, you can still wrap it in a struct that exposes the underlying API while holding an allocator, but that’s up to you

1 Like
5

I do not know anything about the unmanaged variants.
Could you point me to some documentation on that?

I would say that my structure is similar to this Game.

1 Like
6

My bad, so if you look in the std namespace you will see that many data structures exist often in 2 variant a for example std.ArrayList and std.ArrayListUnmanaged the main difference is that one of them doesn’t store the allocator instead each function that requires dynamic memory allocation ask you to pass an allocator. essentially instead of storing the allocator in the type itself you simply pass it the allocator as needed.

7

And more generally, an interesting talk by Andrew on how to program without pointers which makes serializing/deserializing structures easier: Video: Programming without pointers

2 Likes
8

I wouldn’t consider this good general advice actually. Structs cover a huge amount of ground, and sometimes it’s absolutely the correct thing to do to include the Allocator as a field of the struct.

It’s not a great pattern for plain-old-data structs, where there will be a lot of them and the two words the Allocator takes up is a significant fraction of the size of the struct.

But for a “God object” which represents the entire program state and has a bunch of embedded structs and pointers to structs, which is either a singleton or is just normally instantiated once, that’s definitely a good place to put an allocator if the program does any heap allocation at all.

In between it gets murky, so that’s why I’m saying it isn’t good general advice. Structs are just used in too many ways to justify a statement like that.

10 Likes
9

Fair. Better to say that it’s something one should consider thinking twice about, just as the std library is evolving to remove/deemphasize the managed collections.

3 Likes
10

I ran into a similar thing just a few days ago, I wanted to debug a fairly complex data structure, where the root object held the allocator so that it could safely deinit itself.

I cared little for the correct serialization of the allocator, honestly I’m not even sure what that would imply. But I would have preferred anything above it failing.