If it’s their code, it wouldn’t be a problem since they could just change it.
I think the main reason pub must exists is actually what is implied between the lines. Zig makes a point of being allowed to shuffle fields based on compile logic. That means that locking away the inner content would either create two messy intertwined and overlapping structs (yuck) or limit Zig’s ability to shuffle fields. So if the zig devs “can’t” “hide” stuff by moving it into the please_dont_look_here struct then you need pub.
But as soon as you add pub you’ve opened pandora’s box. Some people will be writing libraries exploiting the file as struct because then they can use pub (and gain private).
Now if I have to choose between being locked out or having full access I prefer full access. But when designing a language there are more options than that.
Being locked out is bad enough in C# where you can still get around any barrier with reflection. Getting locked out in zig is even worse, if it’s as simple as reading a field you could use compile time reflection to get the offset and access it with some pointer magic. Same as C# that would be string based and such very fragile. But if it’s an entire utility struct that’s hidden then it gets far worse. There is no way you’ll be able to instantiate it as the type doesn’t even exist at runtime. Without such an instance, you would have to do insane amounts of reflection to fill out a piece of memory to look like an instance before you could call a method on it. Not saying it can’t be done, just saying that it’s far worse than C#.
In conclusion I will repeat myself. I really wish there was a keyword to bypass the visibility rules, as they always end up biting everyone in the ass.
I don’t think you meant to reply to me, but I will respond anyway 
They wouldn’t be intertwined/overlapping, I think you meant that they would be dependent on each other.
I would not describe that as messy, its fairly normal for any complex data.
That being said, it’s good to avoid over complicating your data, and its also true it would impact zigs ability to reorder the data as it cant ‘inline’ a field for various reasons.
I am not sure what you are trying to say, whether a struct is a file struct or not seems irrelevant to the discussion.
I really don’t think it’s as bad as you say, look at @field(T, "field name") you still get a compile error if you made a typo, it’s no different from foo.bar just ‘uglier’ syntax. AFAIK that’s a lot better than c# (I barely use it, don’t quote me)
I think you have some fundamental misunderstandings about zig, ALL types only exist at compile time, reflection only happens at compile time. You can still instantiate private types just the same as public types, you just can’t access the type declaration, i.e. no foo.PrivateBar. You can still @TypeOf instances, you can still @fieldType types that contain the private type.
The discussion has focused on the namespace-extension aspect of traits (not surprisingly, as this was also the focus of the OP!), and gravitated towards a broader discussion of ‘namespace-subdivision’.
I just wanted to note that there is more to traits than that. One thing they provide is default methods - methods that can be expressed in terms of other, more basic methods, meaning that you only need to provide implementations of basic methods for your type, and you automatically get implementations of the others.
This behavior is very elegantly (IMO) achievable in Zig, as long as you’re willing to give up on the idea of context-dependent namespace extension, and instead accept the ‘trait implementation’ as a simple method struct that provides functions that operate on your data.
Example: one may implement Iterator that accepts some concrete types (container type, value type) and a struct providing an implementation of next for them, and returns a struct providing implementations of lots of extra methods. The use would look like that:
const Impl1OfIterator = Iterator(.{
.container_type = T,
.value_type = V,
.impl = struct {
fn next(t: *T) -> ?V { … }
},
});
const s = Impl1OfIterator.sum(it);
real zig (untested) of your example
pub fn Iterator(Ctx: type, Item: type) type {
return struct {
ctx: Ctx,
// pretty sure comptime is required to make
// this type usable at runtime
// comptime fields act as constants accessible through instances of the type
// otherwise `fn...` is a comptime only type
// as fields that would force this to only work in comptime
// I think this is still wrong though
comptime next: fn(*@This()) ?Item,
comptime sum: fn(*@This()) Item = defaultSum,
fn defaultSum(it: *This()) Item {
var sum: Item = 0;
while (it.next) |item| {
sum += item;
}
return sum;
}
}
}
//...
fn myNext(it: *Iterator([]const u8, u8) ?u8 {
if (it.ctx.len == 0) return null
const item = it.ctx[0];
it.ctx = it.ctx[1..];
return item;
}
//...
var it = Iterator([]const u8, u8){ .ctx = "abcde", .next = myNext};
const sum = it.sum();
that is generic, in addition to ctx and item types, fn bodies are comptime known
for a runtime (vtable) example: std.Io.Reader.Vtable
pub const VTable = struct {
// required
stream: *const fn (r: *Reader, w: *Writer, limit: Limit) StreamError!usize,
// provides defaults
discard: *const fn (r: *Reader, limit: Limit) Error!usize = defaultDiscard,
readVec: *const fn (r: *Reader, data: [][]u8) Error!usize = defaultReadVec,
rebase: *const fn (r: *Reader, capacity: usize) RebaseError!void = defaultRebase,
};
Pretty sure my using of comptime fields is incorrect, I can’t be bothered testing rn and the alternative is more verbose which I also can’t be bothered with rn.
2 Likes
Oh, I wasn’t trying to have the namespace that provides the trait methods also hold the data. The point is just that you can make a function that:
- takes a small ‘method namespace’, a struct type which provides functions that operate on some type of data (but has no fields)
- returns a big ‘method namespace’, which provides the same functions as the small ‘method method’ plus some extra ones that it derives
And I think that my code is real Zig code, if you overlook the elided body of the next function and missing implementation of Iterator struct. At least, it wasn’t meant to be hand-wavy.
You’re right that was a mistake, your post was last and I hit the wrong reply button.
The only way I know to make parts of a struct private is the file struct. That might be a misconception on my part but that hardly matters to the argument.
Oh, I failed to realize you get compile errors, that is nice.
Oh I meant what I wrote… when you have two structs that are clearly co-dependant with their only use being together, the compiler could totally overlap them. Imagine they have same “address” but strange field offsets with gaps… You’d get some fields from one struct, then some fields from the other, some of the first… Yuck! But possible.
Exactly because the type only exist at compile time, I can take ANY piece of memory write a pointer and a size next to each other and claim it’s a slice to that memory and successfully call slice methods on it. But since the type isn’t known at runtime, I have to do it all by hand, which would be a nightmare. Such making zig private worse than C# private, even if you get compile errors on field names.
For iterators I would expect it.next() to work, but that only works if the function has the type it is contained in as the first parameter, but by using *T it isn’t a valid member function.
You can’t define methods for some existing type externally, so I would maybe expect next to use a new type generated by Iterator, but for that the struct would have to parameterized to take a type…
So it does seem hand-wavy to me, either I don’t fully understand what you intended with the code, or the code you imagine doesn’t actually work like that.
You can’t call “methods” when they are declared in some other namespace than that of the receiver/self. So they wouldn’t be methods you would have a namespace of freestanding functions, which can be useful and can be called, just not with method-invocation syntax.
Here are a bunch of ways to “add” methods/functions:
const std = @import("std");
pub const Vec2 = struct {
x: f32,
y: f32,
};
const example = struct {
// freestanding function
pub fn add(a: Vec2, b: Vec2) Vec2 {
return .{ .x = a.x + b.x, .y = a.y + b.y };
}
};
const example2 = struct {
// pointlessly a method on another type
pub fn add(_: @This(), a: Vec2, b: Vec2) Vec2 {
return .{ .x = a.x + b.x, .y = a.y + b.y };
}
};
pub const IVec2 = struct {
x: i32,
y: i32,
// weird zero-sized type using @fieldParentPtr based "methods",
// I guess they are indirect methods
// seems to be the closest thing to having extra namespacing
// for methods
example_methods: example3 = .{},
methods: Example4(@This(), "methods") = .{},
};
const example3 = struct {
// zero sized fieldParentPtr-based "method bundle"
// but this still needs to be added to IVec2 manually
// so this could at most be helpful if someone needs to
// implement a lot of types with similar methods
//
// but it doesn't allow for open extension from the outside like traits do
pub fn add(ptr: *const @This(), b: IVec2) IVec2 {
const a: *const IVec2 = @alignCast(@fieldParentPtr("example_methods", ptr));
return .{ .x = a.x + b.x, .y = a.y + b.y };
}
};
// generic variant of 3
pub fn Example4(comptime T: type, comptime field: []const u8) type {
return struct {
pub fn add(ptr: *const @This(), b: T) T {
const a: *const T = @alignCast(@fieldParentPtr(field, ptr));
return .{ .x = a.x + b.x, .y = a.y + b.y };
}
};
}
const Example5 = struct {
// extension by wrapping
// for types that aren't treated as immutable values this also could be a pointer
// essentially a pointer to the "real" self
a: Vec2,
pub fn add(self: @This(), b: Vec2) Vec2 {
const a = self.a;
return .{ .x = a.x + b.x, .y = a.y + b.y };
}
};
pub fn example5(a: Vec2) Example5 {
return .{ .a = a };
}
pub fn main() !void {
{
const a: Vec2 = .{ .x = 1, .y = 3 };
const b: Vec2 = .{ .x = 2, .y = 2 };
const c1 = example.add(a, b);
// methods.zig:39:6: error: no field or member function named 'add' in 'methods.Vec2'
// a.add(b);
// ~^~~~
// a.add(b); <-- no method invocation syntax possible
const e: example2 = .{};
const c2 = e.add(a, b); // <-- method invocation syntax, but example2 has a method
std.debug.print("c1: {}\n", .{c1});
std.debug.print("c2: {}\n", .{c2});
}
{
const a: IVec2 = .{ .x = 1, .y = 3 };
const b: IVec2 = .{ .x = 2, .y = 2 };
const c3 = a.example_methods.add(b);
const c4 = a.methods.add(b);
std.debug.print("c3: {}\n", .{c3});
std.debug.print("c4: {}\n", .{c4});
}
{
const a: Vec2 = .{ .x = 1, .y = 3 };
const b: Vec2 = .{ .x = 2, .y = 2 };
const c5 = example5(a).add(b);
std.debug.print("c5: {}\n", .{c5});
}
}
I find that example5 can be pretty useful sometimes, if it is a struct that just contains a pointer to the real instance, then it is sort of like a handle to the thing that can provide extra methods that are useful in the context of that pointer.
There are also cases where you can use it less like a handle and more like a full wrapper around the thing, it depends a bit on what you are wrapping.
I think example1 and variations of example5 are pretty normal, the others I haven’t really used in practice.
1 Like
“Namespace of freestanding functions” - yes, that’s exactly what I meant, when I said
“as long as you’re willing to give up on the idea of context-dependent namespace extension, and instead accept the ‘trait implementation’ as a simple method struct that provides functions that operate on your data”.
I thought I was clear but maybe I created confusion with the word ‘method’, which might be taken to strictly mean functions that are dispatched from the data they operate on.
Anyway, the point I’m trying to make is that, even if it’s awkward to do contextual namespace extension in Zig (IMO for good reason), which is what everyone is focusing on, there’s another aspect to traits that is very elegant in Zig.
Going back to my example:
const s = Impl1OfIterator.sum(it)
here we computed the sum of it but we only provided an implementation of next. We leveraged a mechanism for upgrading small bags of freestanding functions into big bags of freestanding functions. Isn’t this ‘automatic upgrade’ part of what traits are for?
1 Like
I’m interested but having trouble filling in the elided pieces, and I can’t tell if this is a pattern I haven’t encountered or not. If you have a chance and are willing, I’d love to see all the pieces together.
nope, file structs work the same way as normal structs.
while that is something a compiler could do, such a thing would mean you could not get a pointer to fields whose type is a compound type.
const A = struct {
field1: u32,
field2: u16,
};
const B = struct {
field1: A,
field2: i32,
};
//if A were inlined into `B`
// you would not be able to do `&b.field1` for a pointer to `A`. at least not reliably
// as it would be spread through `B` in a compiler defined fasion.
Such an optimisation just gets in the way of the programmer, at least that’s the case for a language as low level as zig. I am confident zig will never do this.
the type itself doesn’t exist at runtime, but that doesnt mean you cant use information about types, like size and alignment, at runtime (they are just numbers after all).
The language is also aware of that metadata so you do not have to, and shouldn’t, do it by hand.
const A = struct {
a: u32,
b: i16,
};
pub fn main() !void {
const a: A = .{ .a = std.math.maxInt(u32), .b = 42 };
// will refer to the correct amount of memory
const bytes: []const u8 = @ptrCast(&a);
std.debug.print("{any}", .{bytes});
}
const std = @import("std");
Keep in mind that the language/compiler is still pre 1.0, so things are missing, especially safety checks, but the work is being done to get there.
A relevant accepted proposal add safety checks for pointer casting · Issue #2414 · ziglang/zig · GitHub
That’s just the first I could find, there are plenty more.
This is the sort of thing I had in mind:
const std = @import("std");
const IteratorImplementation = struct {
containerType: type,
valueType: type,
baseImpl: type,
};
fn Iterator(implementation: IteratorImplementation) type {
const containerType = implementation.containerType;
const valueType = implementation.valueType;
const baseImpl = implementation.baseImpl;
return struct {
const next = baseImpl.next;
const sum = if (@hasDecl(baseImpl, "sum")) baseImpl.sum else (struct {
fn impl(it: *containerType) valueType {
var res: valueType = 0;
while (next(it)) |val| {
res += val;
}
return res;
}
}).impl;
};
}
test "Iterator" {
const MyImpl = Iterator(.{
.containerType = []const f64,
.valueType = f64,
.baseImpl = struct {
fn next(it: *[]const f64) ?f64 {
if (it.*.len == 0) {
return null;
} else {
defer it.* = it.*[1..];
return it.*[0];
}
}
},
});
var x: []const f64 = &.{ 3.141, 1.0, -42.0 };
std.debug.print("{}", .{MyImpl.sum(&x)});
}
Thank you!