Container-level comptime vars

[biosbob]

a slightly different question from an earlier post, now that i’ve learned a few more things…

consider the following code, which i’ve already compiled for my bare-metal MCU and have inspect the generated .asm file:

const std = @import("std");

fn REG(adr: u32) *volatile u32 {
    const reg: *volatile u32 = @ptrFromInt(adr);
    return reg;
}

fn done() void {
    var dummy: u32 = 0xCAFE;
    const vp: *volatile u32 = &dummy;
    while (vp.* != 0) {}
}

const Counter = struct {
    _val: u8 = 0,
    fn inc(self: *Counter) void {
        comptime self._val += 1;
    }
    fn get(self: *Counter) u8 {
        return self._val;
    }
};

fn mkCounter() Counter {
    comptime var c: Counter = .{};
    return c;
}

export fn main() void {
    comptime var ctr = mkCounter();
    comptime {
        ctr.inc();
        ctr.inc();
        std.debug.assert(ctr.get() == 2);
    }
    REG(12345678).* = ctr.get();
    done();
}

as you can see, i’ve created a Counter instance which i’ve manipulated at comptime using the (comptime-only) inc method… i then call get when writing the counter’s value to some (phony) memory-mapped address…

[[ as an aside, i can in fact write the _val field at runtime (by design in zig); disciplined use of the Counter methods preserves the semantics i’m looking for ]]

what i want to do, however, is effectively have my comptime var ctr exist at container-level – where in fact zig prevents such declarations…

as an example, i’d like MyMod.zig to expose a single Counter instance to any of its clients; these clients can then call the inc method as often as they wish at comptime… at runtime, however, any client (as well as MyMod itself) can read the finally tally through the get method…

thoughts on how to realize this in zig ???

[biosbob]

i’ve also been reading about “comptime closures” here, which seems somewhat relevant… is there some current docs that address this topic, presumably after some of the dust has settled ???

[dimdin]

const std = @import("std");
const assert = std.debug.assert;

const Counter = struct {
    _val: u8 = 0,
    fn inc(self: *Counter) void {
        comptime self._val += 1;
    }
    fn get(self: *Counter) u8 {
        return self._val;
    }
};


pub const MyMod = struct {
    pub fn ctr() *Counter {
        comptime var _ctr = Counter{};
        return &_ctr;
    }
};

pub fn main() void {
    comptime {
        assert(MyMod.ctr().get() == 0);
        MyMod.ctr().inc();
        MyMod.ctr().inc();
        assert(MyMod.ctr().get() == 2);
        MyMod.ctr().inc();
        assert(MyMod.ctr().get() == 3);
    }
}

EDIT:

@biosbob

as an example, i’d like MyMod.zig to expose a single Counter instance to any of its clients; these clients can then call the inc method as often as they wish at comptime… at runtime, however, any client (as well as MyMod itself) can read the finally tally through the get method…

It is not correct, it works only for compile time.

[Sze]

My suggestion is to unlearn the pre-processor macro magic tricks from C and similarly the template meta-programming and design pattern C++isms from C++.

Instead of dragging the cruft from C and C++, into another language, try to think from first principles and build your program from the pieces that Zig gives you, instead of re-inventing things like Singletons and similar bad patterns.

Personally I would urge you to get away from magic counters at build time and instead build / compose / declare things manually, it may require a bit more typing, but in the end it results in more readable and straightforward code, that you can easily reason about locally, without having to understand global side effects, of how some counter is manipulated, when it gets frozen etc.

Container level variables have static lifetime and are order-independent and lazily analyzed.

Zig is a lot more flexible than C or C++, so I don’t think you need to do these counter shenanigans.

If you have a bunch of things that should get an id then just accumulate all of those together with for example tuples, by combining tuples together with ++ and then when you have all of them use that to declare a static array that is initialized with that tuple. And then the counter just becomes the length of the resulting array and type-ids become the index in the array, if the array collects some kind of type information.

If you really want to do the counter manipulation things (not recommended), there is this topic: C/C++ macro challenge #1: BOOST_PP_COUNTER

[mlugg]

This code will be broken by my recent PR. Zig will not support global comptime state for several reasons:

• it makes incremental compilation virtually impossible
• it makes analysis order – an implementation detail – observable

Pointers to comptime vars will also be disallowed from becoming runtime-known to avoid issues related to mutating comptime vars at runtime.

What you (OP) want here is an explicit non-goal I’m afraid, and this part of the language design is almost guaranteed to not change too significantly.

[AndrewCodeDev]

Hey @mlugg, can you clarify something for me? When you say pointer to comptime var I’m curious about how this effects []const u8 at comptime and comptime string variables in general as we’re talking about a slice (just a pointer and a length).

[AndrewCodeDev]

For instance, currently, I could do something like return a []const u8 from a comptime var buffer of u8, so I’m curious if that’s what you’re referring to.

[mlugg]

Firstly, I should note that this is specifically about comptime-mutable memory - i.e. comptime var, not comptime-known consts. But yes, as of that change, one admittedly slightly irritating thing is constructing strings with type []const u8 at comptime. This code will no longer work:

pub fn main() void {
    var runtime_var: []const u8 = undefined;
    runtime_var = comptime getString();
}
fn getString() []const u8 {
    return comptime str: {
        var str: [5]u8 = undefined;
        str = "hello".*;
        break :str &str;
    };
}

This fails because the slice returned at comptime from getString references str, which is a comptime var. Of course, in this situation it could just be const, but in many cases you’ll want var to construct the string in a loop or something. The solution to this is to copy your finalized data into a const, e.g.:

fn getString() []const u8 {
    return comptime str: {
        var str: [5]u8 = undefined;
        str = "hello".*;
        const final = str;
        break :str &final;
    }
}

You can see in the PR that I’ve had to make this change in a couple places in std. I will freely admit that this is a little annoying, but it serves to resolve some serious soundness issues with comptime var – if a pointer to one is permitted to become runtime-known like today, there are some major footguns:

• A load from such a pointer has different behavior if performed at runtime vs comptime. A comptime load will get the current value of the comptime var at the time of analysis, while a runtime load will get its “final” value which is lowered into the binary.
• If any reference to such memory becomes runtime-known with a mutable type, there is a trivial issue: you cannot legally mutate that pointer, but the type system cannot enforce this. We could theoretically resolve this by const-ifying all such pointers, but this is a bit tricky implementation-wise and could lead to some incredibly unintuitive compile errors, so it makes sense to stop you from doing it at all.

If you have any specific code example which you’d like to ask about, feel free – I can tell you whether it will still work, and if not, point to how to could be made to compile.

[AndrewCodeDev]

Great - thanks for the example too. Yes, I was was referring to the string construction in a loop like doing comptime parsing. It looks like the current workaround isn’t terribly painful though, so I’ll try that first and see where it gets me

[biosbob]

let’s move away from counters and talk about a “real-world” problem that i need to solve on a resource-constrained MCU – an optimal FFT module used in a deeply-embedded system…

to minimize runtime overhead, let’s assume the size of the FFT (eg, 128 points) is known at build-time along with the appropriate scaling factor to use when performing a Q15 fixed-point multiply… knowing this apriori, i can effectively synthesize an optimial 3/4 sine-wave plus a table of twiddle factors…

in the world i’m coming from, the FFT is a “lower-level” module (similar to a zig container) which might be imported by any number of “higher-level” clients that ultimately want to call fft.exec() at run-time… in general, however, these multiple clients may have different needs in terms of FFT size and scaling factor – hence the FFT module also exposes some build-time properties used by different clients…

in the world that i’m coming from, build-time is also a very distinct phase-of-translation that has a clearly-defined beginning and end point; and during build-time, modules can interact with one another through public functions, properties, and so forth…

once build-time completes, each module has an opportunity to reflect on its current state and in general initialize run-time constants and variables accordingly… once the FFT module knows the final concensus on size and scaling factor, it can then (and only then) produce my optimal sine and twiddle tables…

one major question in my mind is whether my build-time is actually “more general” than comptime within zig… should i really think about my FFT module as having a build-time aspect that literally runs at “build-time” as part of a native program – output of which is then fed to the FFT module upstream from its cross-compilation for my resource-constrained MCU target???

said another way, once i collect a hierarchy of modules imported from some main, i could translate this into a “build-time” program that executes natively; output from this program are initial values (and perhaps some synthesized code fragments) that are then compiled into what ultimately becomes the “run-time” program…

question: suppose i want to compile a single .zig source file twice – once for my native host and once for my target MCU… is there a syntactically clean way in .zig to “conditionally-compile” certain functions and declarations – an #ifdef in effect??? if so, i have another approach to explore

[Sze]

What you describe sounds to me, like you possibly could use a combination of custom build options and custom compile steps, have one compile step that compiles and runs the application in “build-time” parameter-configuration-collection-mode where the output is these size and scaling factors then you would use the output of that execution to parameterize your application to put it into “runtime-mode” where the size and scaling factors are filled out with build options.

It seems to me it would be possible to do this, but I have to admit, that I haven’t dared to use the build system in such an advanced way yet.

But in general build options are the way to replace #ifdef.
Also take a look at Build system tricks and Zig Build System ⚡ Zig Programming Language