Updating comptime formatting that used to work more than a year ago

Hi, coming back to trying zig after more than a year, and there are so many changes.

I had this piece of code that would have a helper function to do comptime formatting over some other formatting function. this used to work

fn foo(writer: anytype, bar: []const []const u8) !void {
    // some formatting
    try std.fmt.format(writer, "this {s}", .{bar[0]});
}

pub fn buffComptimeFmt(
    comptime buff_len: usize,
    comptime fmt_func: anytype,
    comptime fmt_func_args: anytype,
) []const u8 {
    comptime {
        // Creates the buffer and allocator
        var buffer: [buff_len]u8 = undefined;
        var stream = std.Io.fixedBufferStream(&buffer);
        @call(
            .compile_time,
            fmt_func,
            // Pass the buffer writer as argument together with needed func arguments
            .{stream.writer()} ++ fmt_func_args,
        ) catch {
            @compileError("No space left in buffer");
        };
        // Return from comptime to runtime
        const written = stream.getWritten();
        const final = written[0..written.len].*;
        return &final;
    }
}

Searching and searching i found what i would imagine would be the substitute with

var stream: std.Io.Writer = .fixed(&buffer);

and just pass stream instead of stream.writer() to @call, but this then fails with

src/lib/utils.zig:81:23: error: unable to evaluate comptime expression
.{stream} ++ fmt_func_args,
^~~~~~
src/lib/utils.zig:81:15: note: operation is runtime due to this operand
.{stream} ++ fmt_func_args,
^~~~~~

so how to make it comptime again?

also its probable the method stream.getWritten()has probably changed names too, but i havent looked into that yet and i will find it when the compilation stops failing with this and inmediatly fails there lol

I don’t think you can use io function at comptime, because IO is for runtime code. You will find similar functionality in std.fmt.comptimePrint I suppose ?

they are not doing io, they are just using a fixed writer.

But yes, std.fmt.comptimePrint probably fits their use case, and is implemented similarly over a fixed writer.

comptime var should fix the error, but you should be passing a ptr.

An important thing to note: the fixed implementation is special for giving an instance of std.Io.Writer directly. Other implementations return their own type that has the interface as a field, and you are required to use it via a pointer to that field due to ptr shenanigans.

well, i dont know if std.fmt.comptimePrint is going to work for this; in the code snippet i shared foo is very simple, but in my real case its much more complex, formatting different strings based on some logic. So what i used to have was a writer and i would write different blocks appending to the buffer array. As far as i can tell comptimePrint i have to give the entire str to format and it returns the whole thing again. I guess i could do multiple comptimePrint and join them together?

thats the first thing that i thought too, but the compiler reminded me that: no, its already inside a comptime block

how so (like, literally what would i have to write)? i dont know if im messing the syntax, but what ive tried hasnt worked either

i dont think i understand any of this. I belive that you are making some reference to how in other writer methods, the first argument is the io(?)

After a lot of poking and prodding, I got it working:

fn format_u32s(writer: *std.Io.Writer, a: u32, b: u32) std.Io.Writer.Error!void{
	try writer.print("{X:0>8} {X:0>8}", .{a, b});
}

/// Function must be inline to propagate comptime-ness to the return value
pub inline fn buffComptimeFmt(
    comptime buff_len: usize,
    comptime fmt_func: anytype,
    comptime fmt_func_args: anytype,
) []const u8 {
	comptime{
		var buffer: [buff_len]u8 = undefined;
		var writer: std.Io.Writer = .fixed(&buffer);
		
		// We gotta reify the tuple
		var types: []const type = &.{*std.Io.Writer};
		for(fmt_func_args) |arg| types = types ++ &[1]type{@TypeOf(arg)};
		var tuple: @Tuple(types) = undefined;
		
		tuple[0] = &writer;
		for(fmt_func_args, 0..) |arg, i| tuple[i + 1] = arg; 
		
		@call(.auto, fmt_func, tuple) catch unreachable;
		
		const final = buffer[0..writer.end].*;
		return &final;
	}
}

test buffComptimeFmt {
	std.debug.print("{s}\n", .{buffComptimeFmt(32, format_u32s, .{0x2009, 0xDEADBEEF})});
}

2 Likes

literally just &writer.

It was 2 important things for when you use writer/readers in the future.

  1. most implementations of the interface will provide their own type, that contains the Io.Writer as a field. Whereas the fixed implementation will provide an Io.Writer directly.
  2. for most implementations you interact with the interface through a field, and it must be through that field. If you copy that field into a variable then use it you will cause undefined behaviour!!
// most implementation will name the `Io.Writer` field "interface" or "writer"
var writer = file.writer(io, buf);
const good = &writer.interface;

var bad = writer.interface;
var also_bad = file.writer(io, buf).interface;
// using it causes undefined behaviour
try bad.writeAll("foo");

// I recomend not make a variable for the interface at all
// ofc you need one for the implementation, you'd do:
try writer.interface.writeAll("foo");
// it is more verbose but it avoids a big mistake that is only a single character away

io is not relevant to what I am talking about, but implementations that do I/O will require an io instance.

then i tried right and it fails.

and i think @tholmes answer solves precisely that error, and it works. Thanks. although its quite hard to understand whats happening in the reify tuple section. let me try:

  • create a var array of types, with the pointer to a tuple with the reference from std.Io.Writer
  • loop through the format arguments and append to types the type of the argument
  • use the tuple of types to create a tuple with specifically the types
  • to that tuple add the writer and the arguments

why is all this needed (obviously, it failed without it, but why did it fail)?
Is there a particular reason for doing catch unreachable instead of catching the buffer being too small?

also, the method that i used to have already felt quite hacky and unelegant. Now more so.
Is there some other way that this could be done?

but thanks a lot for the help already :smiley:

this things have obviously a lot of thought behind, and the devs decided that this is the way that it should be done, but it feels so Convoluted and like you mention even prone to errors for something thats so common use
its like if fr adding instead of being able to just use + you had to use some super verbose methods and… i think im lacking a good analogy here, but its the opposite of ergonomic i guess i could say.

a little overengineered, just const args = .{&writer} ++ fmt_func_args should work. Though I haven’t tested so could be wrong.

I think this is just lack of understanding about it. Not to say it couldn’t be improved further!

I would like to go on about it, but there are plenty of posts on here about it, many of them I have answered. So you shouldn’t have to look far; but if you have trouble finding them, or find them lacking, feel free to ask (probably best as a new post).

doesnt work

src/lib/utils.zig:108:33: error: unable to evaluate comptime expression
        const args = .{&writer} ++ fmt_func_args;
                     ~~~~~~~~~~~^~~~~~~~~~~~~~~~
src/lib/utils.zig:108:24: note: operation is runtime due to this operand
        const args = .{&writer} ++ fmt_func_args;
                       ^~~~~~~
src/lib/utils.zig:96:5: note: 'comptime' keyword forces comptime evaluation
    comptime {
    ^~~~~~~~

whish it did tho

My guess is that the reason why .{&writer} ++ fmt_func_args is treated as a runtime expression is cuz it’s using the address of a comptime variable, and it’s being very slightly overly restrictive in order to prevent us from sneaking a reference to a comptime variable out of block/function scope.
E.g. if we had a slice of pointers at comptime, some of them can be valid (pointers to comptime-known constants) and others can be invalid (pointers to comptime-known variables), and so it’s easier for the compiler to just forbid slice concatenation with pointers to comptime-known variables, rather than looping over the slice to validate it later.

2 Likes