Hello, anytype is a bit of a wild card. I would love to know why do you need it.
You could for example make the null safety check before calling the function, and then expect only non-nullable type (can be checked at compile time using @TypeInfo() and @compileError() for better error message, at top of the function)
If you trully need nullable anytype, then you can again do some compile time evaulation on the type of the argument you receive and work based on that. You can just literally do a switch on the @TypeInfo() and do the null safety check only if it is a nullable type (unused compile time logic, which type inspection / tinckering is, gets compiled away when the function is generated with specific type).
Overall there is no best way to do this, depends on your use-case / situation.
Type management / generation is possibly the most complex feature in this otherwise pretty simplistic language (language is simple, low level software can be hard).
Mabye its my personal preference, but i find this as an ugly trick.
In this case it might not have any runtime cost, if optimizer does its thing correctly, but can be possibly missused and have “runtime casting” (which can turn of some optimizations due to the size difference) and null checking penality for ensured non-null values.
Good points, I honestly wasn’t even thinking about optimization. I generally agree with the principle that, all else being equal, you should prefer semantically-guaranteed optimizations with comptime rather than relying on the compiler.
As an exercise for me, and also to learn more about Zig, I wanted to try writing a small wrapper for libpq. This isn’t intended as a to-be-used library, but more as an experiment for myself to see how certain things could be implemented when using existing C infrastructure.
First of all, I noticed that the Zig error types do not carry payload. As far as I understand, I need a separate mechnism to do this sort of specific error reporting. I’m not sure what’s the best approach here, but I attempted something like this:
If the caller doesn’t care about the error message, it’s possible to call Db.connect(…, {}). But when the error message is of interest, the caller may pass some handler that can print or copy the error message. I assumed that I could avoid any extra allocation for the error string if I do it this way, because the error string lives until I call PQfinish.
I think I don’t really need a nullable anytype as nullable is for runtime “nullability”. At compile-time, I guess the right way would be to pass a value of type void (i.e. {}) if I want to indicate that I don’t want diagnostics. But correct me please if this is a bad approach. Maybe there is also an entirely different way of (optionally) reporting the error string on connection failure.
Interesting, I wasn’t aware that null without any specific type information has its own type @TypeOf(null).
Rethinking about this, I don’t think it’s necessary to support .optional. But this makes me wonder, should I use {} (of type void) or null (of type @TypeOf(null)) to indicate that I don’t want diagnostics? Or my own special type like NoDiagnosticsPleaseDiscardAllErrorStrings?
To get back to my simple (dumbed down) example of an optional writer:
Which of these variants seem best, when I only want “compile-time optionality”?
Thanks! I’m very happy to have found Zig and I’m eager to take a closer look at it and to understand all its concepts. Its simplicity reminds me a bit of Lua, but with a lot of stuff happening at compile-time and the efficiency of C.
You are correct that Zig errors do not carry any payload, it was determined that its a bit too much of the foot gun, since in case of some errors the payload may need to be deallocated and this might not happen.
I cant give you the best possible way to do this, but i would personally go for something like this (if you take single handler function only):
Keep in mind this implementation creates unique function for each handler provided for her (but that is the same for your previous version aswell). There is really no reason this might be a problem but if needed, can be solved by runtime function pointer.
Also since you are integrating with C, you might have different type of msg: []const u8.
So the main difference between them is that @typeInfo(...) gives us .is_tuple = true for the anonymous struct literal and false for the other. The former is anonymous and shares a type identity with all zero element anonymous struct literals, while the latter is named and has its own identity. (however the anonymous literal coerces to DoNotWrite when it is assigned)
Considering that you already use anytype to specialize the function at comptime, my preferred way to solve this would be to just use a dummy implementation which is used unconditionally and does nothing (and thus the compiler should be able to figure out that these calls are no-ops, because the interface’s type/implementation is known at comptime):
An additional benefit is that you can use the DummyWriter for testing that your code works with the DummyWriter too and once you do that, you can use that to enforce that the writer only uses a subset of the methods that are usually present in a writer (which can make it easier to write custom writers (which can be beneficial if you use many different writers for different purposes)).
I guess the downside is that you would have to declare a lot of dummy functions if you use the full interface, but the approach makes up for it by not requiring any special casing in the code that uses the interface (so in this example the foo function) and the calling code often also can be written in generic ways, where whatever writer is given is just passed along.
I assume there is no runtime overhead if I pass null with type @TypeOf(null) here?
I further assume the approach with ?fn has the downside that I do not have a context passed to the function, i.e. it is all static? I can’t pass something that would resemble a “closure like” thing that carries context. For that, I think I would need to provide an additional opaque pointer as a first argument to the callback, which seems to make the whole thing more complex and involve pointer casting. That’s why I thought anytype is a cleaner approach. Am I right?
Oh, interesting. I did not realize.
That seems to be a quite clean approach, but is a bit more verbose on the caller side than if I can just write {}, for example. I assume there is no commonly used “default argument value at comptime” pattern?
Side question: If I provide a dummy writer, would it be more idiomatic to provide a type that has to be instantiated with DummyWriter{}, or should I do that already in my module and then provide a lowercase dummyWriter singleton value (of type DummyWriter)?
Another question: Disregarding whether to use anytype, specific types, (optional) function pointers, etc. here, is it a reasonable approach to accept an optional message handler as an argument to handle the error payload in case of a failed database connect with libpq?
Or would it be better to have some sort of “Connection Context” implemented in Zig that holds the inner PGconn C pointer, has a function to retrieve the error, and which has to be explicitly "deinit"ed to discard the error if not needed anymore?
Thanks for all this input. My questions are really mostly of hypothetical nature to help me understand common idioms better. I thought implementing a small library interfacing another C library would be a good exercise to face a variety of challenges as a newcomer to the language.
yup, dont need the comptime since fn types are already comptime only, runtime functions are *const fn...
you can implement context via opaque pointers or you could: fn doStuff(ctx: anytype: ?fn(@TypeOf(ctx), []const u8) void) void
one pattern that is used is to just have a default_... somewhere that the caller can use, but i think an optional is more clear and prevents overlooking of such a thing.
arbitrary and doesnt matter, but i dont think you should optimise the number of buttons the users of your library are pressing, at least not if its so small.
The reason i said that the optimiser needs to do its thing correctly, is because you essentially end up with if () that performs a null check on non-null type that was casted to a nullable type, and you need optimiser to notice that (which it should).
I am not trying to act, as i understand the optimisation of everything, but how would it be free in situations like this? (you can avoid them, but as i said it can be possibly miss used)
You have another function that behaves differently based on the type passed into it, you casted to a optional value so you will use the optional version of that function. Resulting in 2 possible degradations, most optional values are larger than the counterpart so when you call the function itself, you have to “runtime cast it” to a larger space, resulting of possibility it not fitting into a register. And then the function proceeds to make null checks when they are not needed.
Similar behaviour like in the above but you made a local variable / possibly even constant of the nullable type. Again forcing the program to go thru nullable way might introduce overhead and you rely on the optimiser, to realize it cant be null.
To the question of “idiomatic” way, from what i have seen the common way to handle similar situations is to strip the nullability as soon as possible and have generic non-null behavior used instead, like @Justus2308 showed.
That seems a bit dangerous to me, doing all the type casting, I guess.
Interesting! But then I have two arguments, which makes the function signature of doStuff a bit more complex. I feel like using a single anytype argument to carry both context and function seems to be more straight forward.
Passing msghdlDiscard disables message handling, and MsghdlStderr provides some easy to use implementation.
And no runtime overhead, which I like! (Avoiding runtime overhead in Rust has sometimes been a pain, even if the gained/missed advantage was small in practice.)
But feel free to criticize the above draft if it doesn’t feel idiomatic. I’m not sure how else to provide access to the error message returned by PQerrorMessage without using an allocator (which is not desireable, I guess?).
I would also like to ask in a more general way: What is the usual pattern where in other languages you would accept a closure? Is it right/correct/helpful/idiomatic to accept a value of anytype, like I did above? I feel like this is a good way to represent code+context. Or should I work with opaque pointers or explicit context type arguments?
Are there some examples that are worth looking at, where problems that usually demand closures are resolved?
Many thanks already for all your input so far.
P.S.: I got the feeling that my attempt isn’t really idiomatic as my connect function contains control-flow logic (inversion of control). In a language without closures, this doesn’t seem to be a good approach. Likely, it’s better to expose some sort of connection state to the user of the library, which the user has to deinit explicitly. I’ll think more about that.
zig std follows the pattern I described, which IMO is better than a single anytype parameter, as it better communicates to the caller what should be provided and less often requires a wrapper around existing functions that already implement what you want.
zig std does also use the pattern of a context type that contains relevant functions, which you use depends on the situation, std does the latter when generating types to have type safety for the context.
you did what I would recommend, and what zig std does when using libc
Okay, so bottomline is, if I have a single “callback”, then use two arguments. One context argument of type anytype and one function that takes a context as first argument.
I feel like this is effectively equivalent to using a single anytype handler, but a different flavor. Not really having practical experience with the language yet, I feel like a single argument is semantically more correct, but maybe using two arguments causes practically less friction. I will keep these two approaches in mind and see how each of them works for me.
I guess as soon as there is more than one method, it can make sense to summarize all the methods in a single anytype.
The function you were replying to, which you described as a “dirty trick”, does none of these things. It is purely a comptime mechanism to obtain a coercion which Zig will normally perform, but which anytype blocks.
It’s in fact quite common, ‘idiomatic’ even, for a parameter to a function to be nullable. Sometimes this is because what it receives is also nullable, sometimes it’s to provide some data which is semantically optional, and the parameter is filled either by providing a T, or null itself. Sometimes both of these things.
The issue is that the use of anytype prevents the coercion from happening. If you fill an anytype with a T, it’s just a T, so it can’t be treated like a ?T.
The dirty unidiomatic trick is a clever way of dealing with this.
Sure, a null check isn’t free. But nor is it expensive. Actual optimization is achieved through measurement, not vague policies about avoiding one branch condition on the way into a function, on the mere supposition that performance will suffer for it.
Of course in this case, the optimizer would be clever to merge separate anytype calls with a ?T and a null into a single function to begin with. Not because of the T, but because of the null, which is not obviously a null-T.
I’d like to point out that the DummyWriter strategy is only likely to be cost-free if the string to be written is computed in a way that the compiler can optimize away. If producing the string requires allocation, then it’s basically guaranteed that the compiler won’t optimize the computation away (except for maybe when you’re using a stack allocator). Even if your error string generation has no side effects (like changing allocator state or doing syscalls), you’d still be putting a lot of faith in the compiler.
From my view, the opposite is true. The easiest thing to do is to simply make a single anytype parameter, but that is much less expressive than the ctx+function technique, whose optional nature can be inferred from the types alone.
In my opinion, anytype becomes less nice the more unique constraints the type has to fulfill. Going from “this type needs to have method x” to “this type needs to have method x, or be an optional of a type that has method x, or be the null type” makes things a little less elegant.
In the case of the ctx+function strategy, anytype is actually used in the most semantically clear way, because it literally can be ‘any type’.
This is my rundown of what solutions I’d prefer under different conditions:
If any minuscule runtime cost needed to be eliminated and code size was of no concern, I’d use the comptime switch based solution.
If I was okay with the cost of a null check, but was not okay with any error generation or writing overhead, and wanted to minimize code size, I’d pass a comptime argument for the writer type and make the writer argument be an optional of the writer type.
If I didn’t care about the cost of producing the error message strings and wanted the API to be simple, I’d use the DummyWriter solution.
If I wanted to eliminate all runtime costs but still wanted provide an expressive API that’s consistent with other zig code, I’d use the ctx+function solution.
If I didn’t care about the cost of a few extra function pointer calls, I’d use a vtable-based writer interface and just make the argument be an optional. Then, you could pass the argument into a function that returns either a dummy implementation of the interface if the argument is null, or the unwrapped interface if not. Note that, while anytype interfaces are often objectively more performant than their vtable equivalents, it’s often the case that the overhead is nothing compared to everything else you are doing. The main scenario where vtable-based writers fail miserably is when you need to incrementally write something a few bytes at a time.
That is an interesting statement. Without having a lot of experience in the language, I wonder if I would agree at this point (and this might be crucial to understand the philosophy behind Zig and/or compile-time duck-typing).
Maybe I’m overthinking, but I would argue as follows:
Zig allows polymorphic functions, which undergo monomorphization (Rust terminology) at compile-time (which is akin to “template instantiation” in C++, I guess?).
Zig deliberately chose to omit type classes for polymorphism, but instead follows duck typing in a static typing context (at compile-time).
So I would conclude that whenever you want to accept a generic type that has some constraints, e.g.
can be added
can be .start()ed
can be printed
can be converted to a binary format (serialized)
can be used to store or print a certain debug message
etc.
etc., then you would use anytype in Zig, and you would document which constraints are to be fulfilled (outside of the program code).
Now following your statement, this could be considered an anti-pattern in Zig (at least if the number of constraints becomes too big?). This makes me wonder: Is compile-time duck typing a paradigm / common idiom in Zig? Or an anti-pattern? Or both? (I’m asking seriously.)
Thanks a lot for all these different options and explaining them to me. It looks like there are a lot of different paths to choose from, depending on the specific goals when being polymorphic.
(I’m asking seriously.)