Advice needed for replacing `usenamespace` in an existing codebase

Zig’s recent (and sudden) removal of usingnamespace completely destroyed my nD vector types library

The basic setup so far was to have a single comptime type generator which accepts a vector size and component value type, and then depending on these params conditionally includes various features/functions in the returned vector type. This was elegantly solved by splitting out different aspects and (sometimes nested) conditions into functions returning groups of functionality as structs and then merging them all in the returned main type, like so (in principle):

pub fn Vec(comptime N: u32, comptime T: type) type {
    return struct {
        pub usingnamespace Vec2Swizzles(N, T);
        pub usingnamespace Vec3Swizzles(N, T);
        pub usingnamespace Vec4Swizzles(N, T);

        pub usingnamespace VecSizeSpecific(N, T);
        pub usingnamespace VecTypeSpecific(N, T);

        // ...
    };
}

// swizzle functions for 3D, 4D and higher-dim vectors only
pub fn Vec3Swizzles(comptime N: u32, comptime T: type) type {
    const V = @Vector(N, T);
    const V2 = @Vector(2, T);
    const V3 = @Vector(3, T);
    const V4 = @Vector(4, T);
    return if (N < 3) struct {} else struct {
        pub fn z(a: V) T {
            return a[2];
        }
        pub fn xz(a: V) V2 {
            return [_]T{ a[0], a[2] };
        }
        pub fn yz(a: V) V2 {
            return [_]T{ a[1], a[2] };
        }
        pub fn zx(a: V) V2 {
            return [_]T{ a[2], a[0] };
        }
        pub fn zy(a: V) V2 {
            return [_]T{ a[2], a[1] };
        }
        pub fn zz(a: V) V2 {
            return [_]T{ a[2], a[2] };
        }
        pub fn xxz(a: V) V3 {
            return [_]T{ a[0], a[0], a[2] };
        }
        
        // plus hundreds of similar functions...
    };
}

Now I must find a way to rewrite and expose hundreds of swizzle functions (many of them conditional & dependent on different vector sizes) as well as other size-specific and value type-specific functionality, all whilst trying to keep the existing API intact and not to pollute or complicate it with additional sub-namespaces, forced by the removal of this syntax. This readme gives an overview of supported operations for different sizes & component types.

The only way I can see (so far) involves a great amount of code duplication and/or avoidance of comptime type generation. Even though I agree with some of the reasons & explanations given in the release notes, they’re feeling somewhat handpicked and maybe are not considering many other use cases for which this struct-merging feature was actually very helpful…

usingnamespace was such a great tool to split up defintions of complex types over several functions/files and then merge the returned structs via a single comptime wrapper into a single exposed type… Maybe I’m just not privy to better approaches, but right now I feel a great language feature has been lost and this decision might have simplified some Zig internals, but is pushing complexity into userland code bases.

I’d very much appreciate any ideas/approaches how to update/rework/redesign this current architecture whilst keeping duplication and API breakage to a minimum. For example, in this case I’d find it unacceptable having to introduce sub-namespaces, which would then turn callsites from vec3.zyx(a) into vec3.swizzle.zyx(a) (for example). Also sub-namespace do not work for conditional functionality which only exists for certain value types (e.g. integer vectors) or certain sizes (e.g. the cross product is only defined in 3D)…

3 Likes

It’s less elegant, but have you considered code generation?

You could expose the various VecXXXSwizzles as a separate module, the code generator could import that module and use comptime reflection to introspect them and generate the source for the Vec(comptime N: u32, comptime T: type) struct. It just moves usingnamespace into the build system.

I think you would need to refactor a bit and move the conditionals into Vec, so Vec() would look like:

pub fn Vec(comptime N: u32, comptime T: type) type {
    return if (N < 2) struct {
        const foo = Vec2Swizzles(N, T).foo;
        const bar = Vec2Swizzles(N, T).bar;
        // ...
    } else if (N < 3) struct {
        const foo = Vec2Swizzles(N, T).foo;
        const bar = Vec2Swizzles(N, T).bar;
        // ...
        const foo3 = Vec3Swizzles(N, T).foo3;
        const bar3 = Vec3Swizzles(N, T).bar3;
        // ...
1 Like

Thank you! I’ll think about this more and try out how feasible this is, but already can see these two problems with this approach:

  1. There’re hundreds of swizzle functions (esp. for the 4D case) and there’re also overlaps (i.e. the 3D case is a superset of 2D, 4D is a superset of 3D swizzles…). So this kind of conditional will lead to up to 3x duplication. With the older approach this was additive and nicely taken care of automatically…
  2. Adding int/uint & float type specifics to this mix, makes this even more complex (and likely even more duplication)…

In general, outside this specific use case, I’m just wondering if your approach is now the only way left to achieve these kinds of modular/conditional type definitions. Somehow these manual re-exports introduce friction, duplication and maintenance effort and also feel a little non-idiomatic (not your fault!). Just feels like there still must be a better way, but I too can’t see it…

:thinking:

One option you see sometimes is to unconditionally include every name but assign a value of @compileError(“helpful error message”) in the case where you want to ban its use. This will ban its use at compile time.

That allows effectively conditional declaration - but it doesn’t solve for automatic injection.

It’s not ideal, but if the duplication is happening only in generated code, is it really an issue? The other branches of the if aren’t even semantically analyzed by the compiler, so the only part that will slow down is lexing/parsing.

Another option would be to just unconditionally define all methods directly inside of Vec even if they may not make sense for a given N or T. It won’t cause a compilation error unless they’re actually called for an incompatible N or T.

You could even add a if (N < 3) @compileError("Vec.xxz requires dimension N >= 3"); to them, or write a helper method:

fn minDimension(comptime dimension: u32, comptime src: std.builtin.SourceLocation) void {
    if (N < dimension) {
        @compileError(std.fmt.comptimePrint("Vec.{} requires dimension N >= {}", . .{
            src.fn_name,
            dimension,
        }));
    }
}

pub fn xxz(a: V) V3 {
    minDimension(3, @src());
    return [_]T{ a[0], a[0], a[2] };
}

If you called this with N = 2 you’d get a compile error like this:

src.zig:7:9: error: xxz requires dimension N >= 3
        @compileError(std.fmt.comptimePrint("{s} requires dimension N >= {}", .{

It’s much more verbose though.

Did you consider the zlm approach of using a comptime string (like fmt) to generate the vector field extraction? They also have a mixin-reexport in there.

Didn’t see your reply until I hit submitted mine, but looks like we’ve suggested more or less the same thing.

1 Like

Two things:

1: I am going to repost my swizzle code:

fn swizzle(v: anytype, comptime dims: []const u8) @Vector(dims.len, f32) {
    comptime var mask: [dims.len]i32 = undefined;
    inline for (dims, &mask) |dim, *m| {
        comptime std.debug.assert(dim >= '0' and dim <= '9');
        m.* = dim - '0';
    }
    return @shuffle(f32, v, undefined, mask);
}

Usage, eg:

swizzle(p, "0011")

(I use numbers instead of xyz in my code, but switching to letters would be easy.)

2: If you really want all of these functions…just don’t care and put everything in one namespace? I probably wouldn’t even add the check others are proposing, simply rely on lazy compilation. If someone tries to use a method that requires a vector that’s too big, the compile will fail on the indexing.

3 Likes

Hey, that’s a really cool pattern. If you have a family of methods whose implementation is logically a function of their name, you can just dispatch in a comptime string rather than needing to inject a whole name to the namespace. Thanks for sharing!

I recommend to try one of the techniques Matthew outlines in the “Use Cases” heading of Remove `usingnamespace` · Issue #20663 · ziglang/zig · GitHub

2 Likes

I want to point out that the proposal for removing usingnamespace was accepted in March so there was some time to prepare. Zig is still very much in development and it really helps to keep an eye on the Github milestones and such if you want to stay in the loop.

Thank you, everyone for the advice and various possible approaches.

After almost a week of refactoring and experimenting with some of these different approaches, I’ve updated my nD vector library to be compatible with 0.15.1, and at the same time cleaned up some internals.

The solution I settled on is a mix of techniques proposed here by others. I don’t particularly like that the new source code is now more than 2x larger and involves a huge amount of duplication to address the many special cases of supported operations for different vector sizes and types. I might still take another pass to eliminate those by using @compileError() for unsupported cases instead of outright excluding functions from those structs, but that’d be an implementation detail downstream users don’t have to care about. I tried AOT code generation as well, but the special case handling made this feel less maintainable…

The only breaking change is the handling of vector swizzles. I had to remove the hundreds of named swizzle functions and replaced them with a single (comptime optimized) .swizzle(vec, pattern), e.g. .swizzle(vec, "xxyy"). It’s amazing that this swizzle function gets compiled into single WASM i8x16.shuffle ops (per 4 vector components, i.e. swizzling into an 8-dimensional vector would require 2 shuffles):

If you’re interested, the full new code is here:

The readme contains details about the many supported operations (and size/type differences):

Installation instructions in the main repo readme:

Thank you again! :handshake:

4 Likes