Here’s a design choice that seriously undermines Zig’s goal to replace C: auto structs/unions/enums/optionals/functions/errors in Zig have no guaranteed representation at all. This would be fine and cool, if there weren’t programs made to be extended at runtime. Because their layout is generally undefined, the compiler can theoretically just randomize the order of fields on each invocation, or encrypt them with a per-ZCU key, meaning that passing them across ABI boundaries is illegal behavior. When trying to write such a program, you will face several challenges that might suck the joy of programming in Zig right out of you:
- No slices
- No errors
- No allocators
- No standard containers
- No auto-callconv functions
- No bit integers (I’m not sure why this is the case, since clang has _BitInt which is ABI-safe, but it is)
To use a real-world example, here’s a generic “options” struct, mapping a string key to a union value:
pub const Options = struct {
pub const Value = union(enum) {
bytes: []u8,
const_bytes: []const u8,
array: []Value,
signed: i64,
unsigned: u64,
float: f64,
pointer: ?*anyopaque,
boolean: bool,
};
map: std.StringHashMapUnmanaged(Value) = .{},
pub fn deinit(o: *Options, allocator: std.mem.Allocator) void {
o.map.deinit(allocator);
}
pub fn set(o: *Options, allocator: std.mem.Allocator, key: []const u8, value: Value) error{OutOfMemory}!void {
return o.map.put(allocator, key, value);
}
pub fn get(o: *Options, key: []const u8) ?Value {
return o.map.get(key);
}
};
So far, so simple. There are several issues if you want to use it across ZCUs:
Optionshas auto layoutValuehas auto layoutValuecontains slicesstd.StringHashMapUnmanaged(Value)has auto layoutsetandgethave auto layoutsetandgethave parameters with no guaranteed representation
There are two main approaches to making this code ABI-safe. The first, easy one is not to bother, and expose an ABI-safe version:
pub const AbiSafeOptions = opaque {
pub const Value = extern struct {
// enum with native backing type
pub const Tag = enum(u8) {
bytes,
const_bytes,
array,
signed,
unsigned,
float,
pointer,
boolean,
};
pub const Payload = extern union {
bytes: extern struct {ptr: [*]u8, len: usize},
const_bytes: extern struct {ptr: [*]const u8, len: usize},
array: extern struct {ptr: [*]Value, len: usize},
signed: i64,
unsigned: u64,
float: f64,
pointer: ?*const anyopaque,
boolean: bool,
};
tag: Tag,
payload: Payload,
};
const Map = std.StringHashMapUnmanaged(Value);
pub fn create() callconv(.c) ?*AbiSafeOptions {
// global allocator
const p = std.heap.smp_allocator.create(Map) catch return null;
p.* = .{};
return @ptrCast(p);
}
pub fn destroy(self: *AbiSafeOptions) callconv(.c) void {
const options: *Map = @alignCast(@ptrCast(self));
options.deinit(std.heap.smp_allocator);
std.heap.smp_allocator.destroy(options);
}
pub fn set(self: *AbiSafeOptions, key_ptr: [*]const u8, key_len: usize, value: Value) callconv(.c) bool {
const options: *Map = @alignCast(@ptrCast(self));
const r = options.put(std.heap.smp_allocator, key_ptr[0..key_len], value);
return if(r) |_| true else false;
}
pub fn get(self: *AbiSafeOptions, key_ptr: [*]const u8, key_len: usize, out_value: *Value) callconv(.c) bool {
const options: *Map = @alignCast(@ptrCast(self));
out_value = options.get(key_ptr[0..key_len]) orelse return false;
return true;
}
};
Let’s be frank: This code is ass. It:
- is much longer than the code it replaces,
- isn’t as safe as the code it replaces,
- forces the use of a specific global allocator,
- forces heap allocation of
Map, since its size is also not part of the ABI, - forces an inconvenient tagged union that can’t be switched on as easily and lacks type checking.
- generally goes against Zig’s goal of being a better C.
This is the easy approach. The hard approach is to take, in this case, std.hash_map.HashMapUnmanaged, std.mem.Allocator, and slices, make them ABI-safe, and use those. Don’t bother.
Of course, some things can be improved with comptime magic:
However, you will quickly run into the fact that decls can’t be reified.
As it stands now, trying to write a runtime-extensible program in Zig is, for the most part, worse than doing it in C. If the representation of slices and auto-layout stuff were defined for a given (compiler version, build options) pair, which is the de-facto behavior of the Zig compiler right now (there are plans to change that), it would be a lot less painful.
. from the anger in your post, it sounds like you must be facing this issue with more frequency than me! do you have a proposed solution (maybe even just “these structs in