Constness of fields and coercion

Continuing the discussion from Arrays vs vectors:

I recently came across the curious case of wanting to apply a newtype pattern to
a slice.

So let’s create a type specifically for lower triangular matrices. Let’s also assume it shall work without allocation, using arrays as backing storage and slices referring to that backing memory.

Given an element type T, we can have two different slices of type T:

  • []T
  • []const T

We need to use the latter when we have some const memory where our data comes from, but the former when we want to support mutation.

Now instead of creating two types (boilerplate :scream:), it would be nice to create a single type constructor:

pub fn Triangular(comptime Linear: type) type {
    return struct {
        linear: Linear,
   }
}

This way, given elements of some type, say i32, we can create two distinct types:

  • Triangular([]i32), which is a mutable lower triangular matrix
  • Triangular([]const i32), which is a constant lower triangular matrix

We know from slices that a []const i32 can coerce to a []i32 (see also Type Coercion: Stricter Qualification in the Zig documentation):

Values which have the same representation at runtime can be cast to increase the strictness of the qualifiers, no matter how nested the qualifiers are:

  • const - non-const to const is allowed

Nowever, this “nesting” doesn’t seem to apply to struct fields, as demonstrated below:

const std = @import("std");
const assert = std.debug.assert;
const print = std.debug.print;

fn triNum(x: usize) usize {
    return x * (x + 1) / 2;
}

fn triIdx(row: usize, col: usize) usize {
    assert(col <= row);
    return triNum(row) + col;
}

pub fn Triangular(comptime Linear: type) type {
    return struct {
        const Element = std.meta.Elem(Linear);
        linear: Linear,
        pub fn assertDim(self: @This(), dim: usize) void {
            assert(self.linear.len == triNum(dim));
        }
        pub fn get(self: @This(), row: usize, col: usize) Element {
            return self.linear[triIdx(row, col)];
        }
        pub fn set(self: @This(), row: usize, col: usize, value: Element) void {
            self.linear[triIdx(row, col)] = value;
        }
        // Do we need the following function? Unfortunately, we do!
        pub fn asConst(self: @This()) Triangular([]const Element) {
            return Triangular([]const Element){ .linear = self.linear };
        }
    };
}

pub fn printTriangularInt(dim: usize, matrix: Triangular([]const i32)) void {
    matrix.assertDim(dim);
    for (0..dim) |row| {
        for (0..row + 1) |col| {
            if (col > 0) print(" ", .{});
            print("{d:3}", .{matrix.get(row, col)});
        }
        print("\n", .{});
    }
}

pub fn main() void {
    // Let's say we have some constant data:
    const const_data: [6]i32 = [_]i32{ 1, 2, 3, 4, 5, 6 };
    // Note that we then need a `[]const` slice here:
    const triangular1 = Triangular([]const i32){ .linear = &const_data };
    // This works fine:
    printTriangularInt(3, triangular1);

    // Now let's have some non-const backing memory, allowing mutation:
    var buffer: [6]i32 = undefined;
    const triangular2 = Triangular([]i32){ .linear = &buffer };
    for (0..3) |row| {
        const i: i32 = @intCast(row);
        for (0..row + 1) |col| {
            const j: i32 = @intCast(col);
            triangular2.set(row, col, 10 * (i + 1) + (j + 1));
        }
    }
    // Now this doesn't work:
    //printTriangularInt(3, triangular2);
    // Instead we need:
    printTriangularInt(3, triangular2.asConst());
}

Question: Should Triangular([]const i32) coerce to Triangular([]i32) automatically, without needing the asConst function?

What you are asking, coercing structs if their members coerce, is fundamentally changing the semantics of types. Such an implicit conversion would undermine type safety in many cases.

Note that the compiler treats structs created by the same function are fundamentally different types. So what applies to structs created by the same function also needs to apply to separate structs.

So for example it would allow coercing a triangular matrix into a dense matrix:

const Triangular = struct {linear: []T};
const Dense = struct {linear: []T}; // Same parameter names → indistinguishable to the compiler

const triangular: Triangular = ...
const dense: Dense = triangular; // This would be legal with your proposed semantics.

I think adding a toConst function is a small price to pay, compared to the side effects of implicit conversion.

No, that’s not what I’m asking and that would indeed violate type identity: Creating a new struct with the same fields is a different type (and must be a different type).

However, when we call the same type constructor twice with the same argument, we don’t get distinct types:

const std = @import("std");
const assert = std.debug.assert;

pub fn Square(comptime Linear: type) type {
    return struct {
        linear: Linear,
    };
}

pub fn Triangular(comptime Linear: type) type {
    return struct {
        linear: Linear,
    };
}

pub fn main() void {
    const T1 = Triangular([]i32);
    const T2 = Triangular([]i32);
    const T3 = Square([]i32);
    assert(T1 == T2);
    assert(T1 != T3);
    const T4 = Triangular([]const i32);
    const T5 = Triangular([]const i32);
    const T6 = Square([]const i32);
    assert(T4 == T5);
    assert(T4 != T6);
}

What I discuss is that a value of type T1 (or T2, which is identical) could coerce to a value of type T4 (identical to T5), and that a value of type T3 could coerce to a value of type T6 in the given example.

I do not want a Square([]i32) coerce to a Triangular([]i32). And I do not want a Square([]i32) coerce to a Triangular([]const i32).


To clarify further, I also do not want these to coerce:

const T7 = struct { member: []u8 };
const T8 = struct { member: []const u8 };

I.e. I only propose those types coercing which have been created using the same type constructor (but with a comptime argument that only differs in const-ness).

Yeah, that’s Zig comptime function caching.
My point is that Triangular([]i32) and Triangular([]const i32) are fundamentally different types, and the compilers doesn’t have any way of knowing that Triangular([]i32) is intended to coerce to Triangular([]const i32) but Square([]i32) is not.

Consider the following:

fn OddMatrix(slice: anytype) type {
    if(isConst(slice)) return Triangular(slice);
    return Square(slice);
}

Now according to your rules OddMatrix([]i32) would coerce to OddMatrix([]const i32) right? They were created with the same function, have coercible fields, likely even the same set of functions, right?
Now, because of Zig’s type caching rules, you’ll see that
OddMatrix([]const i32) == Triangular([]const i32)
OddMatrix([]i32) == Square([]i32)
So, since the compiler cannot (easily) know that such a function exists, it must assume that Square([]i32) is coercible to Triangular([]const i32)

1 Like

Currently yes. But what is also true is that []i32 and []const i32 are not fundamentally different types: []const i32 is a super-type of []i32.

But they have (in theory), because the compiler could determine whether types are created at the same position in the code (where the struct statement is).

I didn’t propose any rule as of yet. But let me try to propose one (just a draft, trying to find a rule that might work):

Two types may coerce if their fields are coercible and they have been created by the same struct statement in the code.

In your example, the inner-most type constructor and the struct statement that created those types are distinct, thus the types should not coerce.

I’m not sure how this could/should/can be implemented. As of now, I just try to make sense of the semantics. To me, a Triangular([]const i32) is (semanitically/obviously) a super-type of Triangular([]i32), even if Zig doesn’t treat it as such. They are related insofar as that the struct keyword of their definition is in the same line number / position in the code.

OddMatrix([]i32) is not a sub-type of OddMatrix([]const i32) because ultimately, a different struct is created (literally the struct keyword used to create the type is in a different line in the code).

Yeah, that could work.

However I think the problem is that this makes coercion rules rather complex. With status quo of you can know if two types are coercible by looking at their name and knowing the coercion rules.
But with your proposal you have to look at the source code and implementation of a type to know if they are coercible.

Furthermore tying it to the struct keyword location in the source code can lead to surprises, what if someone does declare both types of matrices in one function?

pub fn Matrix(Slice: anytype, typ: enum{square, triangular}) type {
    return struct{...};
}

Let me give a (somewhat weird) example to demonstrate this:

const std = @import("std");
const assert = std.debug.assert;

pub fn C1(comptime S: type) type {
    return struct { s: S };
}

pub fn C2(comptime S: type) type {
    return struct { s: S };
}

pub fn weird1(comptime S: type) type {
    return if (@typeInfo(S).pointer.is_const) C1(S) else C2(S);
}

pub fn weird2(comptime S: type) type {
    return if (@typeInfo(S).pointer.is_const) C2(S) else C1(S);
}

pub fn main() void {
    const T1 = weird1([]const u8);
    const T2 = C1([]const u8);
    assert(T1 == T2);
    const T3 = weird2([]u8);
    const T4 = C1([]u8);
    assert(T3 == T4);
    const x: T3 = undefined;
    const y: T1 = x; // This could coerce, but it doesn't.
    _ = y;
}

Compiler error:

coerce.zig:28:19: error: expected type 'coerce.C1([]const u8)', found 'coerce.C1([]u8)'
    const y: T1 = x; // This could coerce, but it doesn't.
                  ^
coerce.zig:5:12: note: struct declared here (2 times)
    return struct { s: S };
           ^~~~~~~~~~~~~~~

I think we already have a surprise:

pub fn Matrix(comptime T: type, comptime form: enum { square, triangular }) type {
    _ = form;
    return struct { linear: []T };
}

pub fn main() void {
    const T1 = Matrix(i32, .square);
    const T2 = Matrix(i32, .triangular);
    @import("std").debug.assert(T1 == T2); // surprise!
}

:tada:

That is indeed a surprise. I guess Zig’s comptime caching somehow ignores unused parameters?

Using it in a function (which you would probably do if you’d write this anyways), does seem to fix it:

    return struct {
        linear: []T,
        fn xy() void {
            _ = form;
        }
    };
1 Like

But say, couldn’t the same mechanism then also prohibit coercion?

To clarify / rephrase the proposed rule:

Two types may coerce if they have been created by the same struct statement in the code and they only differ in const-ness of their fields.

But wouldn’t that make it useless?
E.g. for your triangular matrix you would probably want to have some functions that only work for the mutable matrix, but to do so the function declarations will have to be different, just like in my example.

Container type equivalence is based on the set of AST nodes they capture. The struct/enum/union/opaque declaration needs to “use” the comptime parameter in order to be considered specialized and different from other invocations of the type-returning function.

2 Likes

Yes. In my example the set function only makes sense when there is inner mutability.

Note that in my example in the original post above, I declared a set function. As long as I don’t use it, it is fine, even for Triangular([]const i32). I was a bit surprised by that as well.

So I don’t see a real problem here.

Edit: I think I got a bit confused about covariance and contravariance below. I will have to think this through more carefully, I guess.

Now I wonder if a []const T is captured instead of a []T and only used in places where covariance applies (e.g. type of a field or return type of a function), then the resulting type could be a supertype too.

Or, when a []T is captured (instead of a []const T) and only used as an argument type in some function (we have contravariant behavior there, see Rust subtyping and variance for co- and contravariance), then the resulting type could also be a supertype.

P.S.: I’m actually not sure if it makes sense (in theory (just brainstorming here)) to have subtyping rules that respect anything else than fields. On the other hand, I see the need to create a different type when arguments to the type constructor are “used” somehow by a function. So not sure if there could be any reasonable rule which solves my Triangular issue in a clean, non-contradictory way. (Disregarding of whether it’s too complicated to be implemented, or not.)

Okay, I’ll try to clarify a few things.

First of all, I think it was a bit unfortunate to use the terms super-type, sub-type, co-variance and contra-variance without really explaining them. We don’t use that terminology in Zig, I guess?

Zig, however, allows automatic coercion from some types to other types. For example:

  • []T[]const T
  • u8u16
  • i32i64

So we could say that []const T is a super-type of []T, because we can store any value of type []T in a variable of typ []const T. Also u16 is a super-type of u8. And i64 is a super-type of i32. Coercion is possible where the target type is a super-type of the original type. Let’s try:

pub fn main() void {
    var buf: [3]usize = [3]usize{ 0, 0, 0 };
    const a: []usize = &buf;
    const b: []const usize = a; // this works!
    _ = b;
    const c: u8 = 5;
    const d: u16 = c; // this works!
    _ = d;
    const e: i32 = 1000000;
    const f: i64 = e; //this works!
    _ = f;
}

Interestingly (though maybe not surprisingly), Zig doesn’t always allow coercion if this sub-type/super-type relationship exists nestedly. Consider:

const U8Dispenser = fn () u8;
const OtherU8Dispenser = fn () u8;
const U16Dispenser = fn () u16;

fn u8Dispenser() u8 {
    return 5;
}

fn u16Dispenser() u16 {
    return 1000;
}

pub fn main() void {
    @import("std").debug.assert(U8Dispenser == OtherU8Dispenser);
    const a: U8Dispenser = u8Dispenser;
    const b: OtherU8Dispenser = a; // this works!
    _ = b;
    //const c: U16Dispenser = a; // but this doesn't work!
    //_ = c;
}

I guess it makes sense because both functions may work differently in assembler, so we can’t just cast them.

However, that the following works might be surprising :tada: (but I appreciate that it does work):

const Dispenser = fn () []u8;
const ConstDispenser = fn () []const u8;

var buf: [3]u8 = [3]u8{ 1, 2, 3 };

fn dispenser() []u8 {
    return &buf;
}   

fn constDispenser() []const u8 {
    return &buf;
}   

pub fn main() void {
    const a: Dispenser = dispenser;    
    const b: ConstDispenser = a; // this works!  
    _ = b;
}

Even more surprising might be that the following works (and I think it’s right that it works):

const Consumer = fn (arg: []u8) void;
const ConstConsumer = fn (arg: []const u8) void;

fn consumer(arg: []u8) void {
    for (0..arg.len) |i| arg[i] = i;
}

fn constConsumer(arg: []const u8) void {
    for (0..arg.len) |i| {
        @import("std").debug.print("{}", .{i});
    }
}

pub fn main() void {
    const a: ConstConsumer = constConsumer;
    const b: Consumer = a; // this works!
    _ = b;
}

So we can coerce a Dispenser to a ConstDispenser, but for consumers it works the other way around: a ConstConsumer can be coerced into a Consumer.

(These code examples are all valid Zig as of version 0.15.0-dev.1148+67e6df431.)

Why is it reversed in case of the consumer? That is because a function type acts contra-variant with regard to its argument types. That means the super-type/sub-type relationship changes:

  • []const u8 is a super-type of []u8, but
  • fn (arg: []const u8) void is a sub-type of fn (arg: []u8) void)

In other words, we can store a value of type fn (arg: []const u8) void in a variable or constant of type fn (arg: []u8) void).

Until here, Zig works really nice in my opinion.

Now I know that two distinct structs with equal fields are distinct types, and that is also good and should not be changed.

However, I do believe that it would be possible to have a similar super-type/sub-type relationship with structs that only differ in const-ness of their fields and(!) which share the same “source location”. Citing the 0.12 release notes:

In 0.12.0, this has changed. Namespace types are now deduplicated based on two factors: their source location, and their captures.

Now the issue is that these captures always seem to result in a totally different type, even if the captured type only differs by const-ness and when it’s only used for a field type, where co-variance might be applicable, i.e. where a super-type should/could/would result in the resulting type being a super-type as well.

This isn’t extremely exotic. As demonstrated above, Zig already follows the notion of co-variance in case of function types (example above that used the Dispenser and ConstDispenser types).

I wonder if

  • it would make (theoretically) sense to extend this concept to returned structs,
  • there would be unforseen side-effects, and if
  • it is technically feasible.

Moreover, I wonder how to proceed when types are captured in other places than for field types. Maybe there could exist some co-variance, contra-variance, or invariance rules, but I’m really not sure.

No, Zig doesn’t have subtyping.
Zig only has a small number of explicitly supported type coercions.

Type coercions are only allowed when it is completely unambiguous how to get from one type to another, and the transformation is guaranteed to be safe. There is one exception, which is C Pointers.

Personally I am glad that it is this way, having structs that behave as separate types, except in weird corner cases when fields happen to be named the same and have almost but not quite the same types, is just too complicated and niche to add to the language. And making it dependent on what created the struct type, would make it even more difficult to understand. Also the moment you added that, somebody would want it for structs that were created in different ways too.

Just because some type was created by a shared function doesn’t mean that those types are related or unrelated, it depends on how that generic function works.

It also feels like wanting to bend the language to implement your specific corner case. I don’t think it is good to have clear nominal typing for structs and then mix in a sprinkle of subtyping to confuse everyone.

From my standpoint something like this only makes sense for a language that already has decided to have a type system with monstrous and user extensible complexity, basically when the user can declare or annotate their wanted features for their user defined types, so I don’t think it fits Zig.

3 Likes

I’m not sure about terminology. Wikipedia says:

[…] program elements (typically subroutines or functions), written to operate on elements of the supertype, can also operate on elements of the subtype.

A function that operates on a []const i32 may (effectively) also operate on a []i32. Sure, internally this is implemented by coercion, but isn’t that effectively the same, because Zig performs implicit copies and coercion? (I’m really not sure, and happy to learn the distinction here.)

I basically used these terms to talk about concepts, not about implementation as of yet:

So sure, we can stick with “coercion”. I find the sub-type / super-type model easier to reason about, but maybe it’s factually wrong.

They could still be separate types, just like fn (arg: []const u8) void and fn (arg: []u8) void) from my example above.

To be fair, I don’t think the newtype construct is a “weird corner case”.

As said above, this isn’t about fields being “named” the same. It’s related (but not equal) to type identity that, in Zig, relies on where a struct is defined:

In 0.12.0, this has changed. Namespace types are now deduplicated based on two factors: their source location, and their captures.

Complicated? Maybe. I can imagine this makes the compiler too complex, I don’t really know, and I would totally understand if it’s not supported for this reason.

Niche? Maybe. Like I said, I don’t see the newtype construct (possibly adding also functions) being a corner case.

I don’t want that, and I think it’s unreasonable to “want” that because of it. This has to do with the nature of my proposal (that of course may be misunderstood). In either case, I personally think this is not a good argument when brainstorming about the specific proposal here.

Today’s Zig sometimes makes these types even equal. For a good reason: otherwise Newtype(u8) couldn’t be equal to Newtype(u8).

As said above, I think the newtype pattern isn’t “my” specific corner case. It’s a nice example though, IMHO.

Let’s call it coercion then. Just like what Zig already states in its docs:

Also, compare to

  • ProducerConstProducer
  • ConstConsumerConsumer

from my example above.

These coercions exist, and they are not really exotic, IMO.

Well, maybe it does overall increase complexity too much. I don’t know. But perhaps give it a chance to think about it. I feel like it would integrate well with the remaining type system (disregarding implementation challenges). And it would also be (in my opinion) more consistent with existing Zig rules.

Again: I do not want two different structs to coerce just because the fields coerce. That is not what I wanted to discuss. (And thanks to @IntegratedQuantum for having brought that up, because it’s important to differentiate here.)