According to the standard library documentation, the current Io interface contains the following:
file system
networking
processes
time and sleeping
randomness
async, await, concurrent, and cancel
concurrent queues
wait groups and select
mutexes, futexes, events, and conditions
memory mapped files
A lot of this stuff is completely unavailable in certain environments such as in embedded systems, so attempting to use the Io interface yields a lot of wasted memory, which can be very important in memory constrained environments such as embedded systems.
Perhaps it’s time for Zig to consider an smaller Io interface (Io-core) in the standard library that still supports async and concurrency, but doesn’t have stuff like filesystems which require an entire operating system.
Getting rid of the os-dependent filesystem stuff alone from the Io interface already gets rid of 54 function pointers, which is more than half of the function pointers in the current Io interface. That and removing whatever other os-dependent stuff can result in the Io-core. Ideally Io-core would support anything that could be done with Rust core or Rust core+alloc.
Just wait a few releases. Restricted function pointers will essentially turn this into static dispatch, so code can be eliminated from the binary. And the cost of indirect calls is gone.
Io.Operation where some operations are grouped together in one function (à la “ioctl” )
Restricted function pointer which is complicated name to detect that there is only one Io implem in your program, so all Io function call are static dispatched (not sure how it removes the vtable?)
Zig’s goal is optimal code generation across all environments, including embedded. Solutions which split the std lib (like rust’s no_std) are sub optimal. The compiler knows where the dead code is, and it’s reasonable to expect a compiler to be able to delete it. We should be more imaginative and not be beholden to status quo of embedded being annoying AF to work on.
It’s just an abstraction for using the most important part of the std library, IO. You could almost consider it the OS interface. Since IO in the OS can be done in two ways these days, sync or async, an abstraction is justified. It is large because IO is a big topic. I’m sure it is annoying for those who don’t need or want the ability to write code that works for both sync and sync, but solving this problem is part of Zig’s charter and has been for a long time.
That IO is the OS interface is a big problem for using the IO interface in environments without an OS, since the IO then contains a whole bunch of unnecessary information about OS operations such as filesystem operations, that ends up wasting memory and code. Hence this discussion about a smaller IO abstraction that doesn’t contain any code relevant to an OS.
No, it’s fine; you weren’t the one who started the comments about general IO interfaces. The person who you replied to called the current Io interface a “God object”.
The smaller Io interface proposed here, stripped of OS relevant function pointers, would also be a “God object” in the OS-less contexts where the smaller Io interface would be relevant, and there likely will be plenty of people who would hate the smaller Io interface as well for being a “God object”.
I have read the proposal, but admit that the details are a bit over my head.
I can see how RFP eliminates the dead code from the output, but I’m not sure if the vtable itself also shrinks to fit .. or .. if going static dispatch effectively eliminates the vtable overhead altogether by inlining everything.
The vtable can’t shrink this way, but it becomes just an array of ints, not backed by the actual implementation. The bloat comes from the functions that are included in binary because they are used in the vtable. And that is no longer needed.
RFP will essentially turn a function pointer into u32 index into the table of allowed implementations and then do something like this at the call site:
In the current implementation, it’s actually done differently, it delegates the optimization to LLVM and just gives it additional metadata. I’m not sure if that’s actually better than emitting a switch like this from zig’s codegen, especially for small number of implementations like Io has, but on a high-level, it’s functionally the same.
If you only have one allowed implementation, it turns the function calls into a static dispatch, no runtime checking at all. If there are multiple implementations in your binary, it still does dynamic dispatch, but because all variants are known, it can actually inline or do similar optimizations.
But you can already do that today? Just create one IO vtable in your program with undefined for the API doesn’t have and you’re good.
IO is the new POSIX.
It’s mind wrapping because it reverse how we think about OS. But it also amazing eg I implemented in 1 hour an IO for a closed source OS and could reuse previous code.
