Union MultiArrayList even worse in safety builds

Brainstorming
,
1

So if I have a tagged union with 64bit fields and u8 tag, total size of the union due to padding will be 128bits. If I need a list of them and I use array then I am wasting 7 bytes per element.

Now MultiArrayList should fix this and it does but only in ReleaseFast builds. Debug and ReleaseSafe builds, that I still care about and that are already struggling now get even worse, since now in MultiArrayList I have a list of u8 tags and then additional list of 128 bits elements where 7 bytes are wasted (since they are stored as bare unions inside which in safety builds still have a hidden tag field).

Am I the only one bothered by this and wishing language had something like raw union where I can say “don’t you worry about safety”? I mean API of MultiArrayList is already taking care of safety aspect in this regard, right?

PS. I know I can hack my way around this but in even unglier ways, like making all field types and their field types extern or packaged but those have limitations. You can’t make all Zig types extern and my biggest problem with using packed structs and unions is that they show up as plain integeres in the debugger now.

2

I believe there have been discussions on the issue tracker or zulip about introducing a builtin like @optimizeFor(mode) where mode is the optimization mode being targeted, so like .Debug, .ReleaseSafe etc (i believe there were discussions of renaming the optimization modes as well, but that’s irrelevant).

The point being that you could use this builtin at function scope instead of “project” scope, so if you wanted more safety for a function you would do something like

fn foo() void {
    @optimizeFor(.safety);
    // ...
}

I don’t remember if struct/union level safety was discussed though.

But regardless, for your use case, generally i avoid using MultiArrayLists on tagged union types, specifically for the reason you mention. Instead, I manually create a Tag enum and a Data bare union, and then use a MultiArrayList(struct { tag: Tag, data: Data }). It removes all safety benefits from using a tagged union though :^).

3

But that is exactly what MultiArrayList does for tagged unions. It internally makes a bare unions with all the same fields and a separate tag field. The issue is that bare unions are still safe (still have a tag field) in safe builds.

1 Like
4

Woops you’re right! Confused myself for a sec.

I went and looked up the discussions around @optimizeFor (and the current @setRuntimeSafety), which happen in this github issue. Doesn’t look like this specific problem has been mentioned yet.

5

Yes. For now those only affect the generated code but not type layout and similar. The issue mentions an idea to also support it for types but nothing further than that.

6

Could this work with a packed union?

7

I believe it would yes, but that forces your union variants to all have the same size, in which case using a MultiArrayList which can be a pretty big constraint.

8

the safety tag of bare unions is actually subject to @setRuntimeSafety, but unfortunately, it does not propagate through function calls.

so you’d have to copy the source of MultiArrayList, and also std.meta.BareUnion, and insert @setRuntimeSafety(false) into your copy of the latter.

But, after doing all that, you will have actually untagged unions even in safe mode

1 Like
9

Wau nice! That actually works. Thank you very much.

I wonder if Zig would consider doing this in std library since, as I said, the API of the MultiArrayList should already take care of the needed safety.

10

It is as simple as list.items(.data)[i] to access the untagged union and do unchecked illegal behaviour with it even in safe modes.

So no, it does not take care of the safety, though there is an argument that you should be aware of this since you’ve already separated the tag by using a multi array list in the first place.

That doesn’t mean std won’t do this, just that there is an argument against it.

2 Likes