Maybe this helps your endeavour, but hard to say from my perspective: glibc (and its thread implementation) doesn’t use the fork system call these days anymore for its implementation for fork, but clone(2) for the implementation of fork, vfork and pthread_create (and similar stuff if it exists).
More news 
I finally succeeded in creating a vulkan instance on stock chimera linux:
(hey @geemili 
)
I’ll soon put together the famous vulkan triangle example.
It turns out that this distribution uses a heavily patched musl, which, among other things, replaces the internal allocator with mimalloc. The way it is integrated makes it really hard, if not impossible, to reliably initialize it from the outside. So I decided to create a bridge between the libc malloc functions and a zig allocator, in the same way I did for the pthread functions (I already wanted to do that anyway). Now every call to malloc, calloc, realloc, etc. ends up being handled by zig.
So as of today, these systems are managed by zig:
- dl
- threads
- allocations
I also removed the need to use a patched musl build. Now, while the libc shared library is being loaded, it is inspected to discover important non exported symbols (like __libc for musl). To do this, I inspect the machine code of chosen small functions that are known to reference these symbols.
Another thing I had to do was come up with a strategy to replace internal calls to malloc and calloc inside musl, because the usage of macros like #define malloc __libc_malloc ends up creating calls that don’t go through a relocation, making them immune to the previous redirection strategy. Using inspection and some reserved space in memory just after the loaded library image, I now basically create a custom PLT and patch the calls in place in the binary.
The locations and patterns used when inspecting the binary are chosen to theoretically work regardless of the optimization level.
Compatibility across different linux distributions and different libc flavors should now be much higher. But code efficiency and quality still need a lot of improvements.
I originally wanted to rant against libc and its privileged treatment on linux, but since it’s mostly personal opinion, I’ll refrain.
12 Likes
This is going to be one of the most exciting vulkan triangles for sure. I think your work has overlap with zig’s own libc implementation efforts.
4 Likes
Vulkan triangle from a static executable without libc: achieved!
This is a direct copy-paste of the vulkan-zig triangle example, where uses of glfw to load vulkan and create/manage the window have been replaced by calls to the dynamic loader and direct x11 window handling. Source is here.
Tested the same executable (!) on multiple ubuntu machines (one is kinda old), on a manjaro one, and surprisingly it also works on the chimera virtual machine.
Since implementing some thread safety was required, and was done in a not fully correct way to speed things up (maybe I was starting to feel vulkan fatigue…), I’m starting to think that the great rewrite is coming.
I also tested that loading the ubuntu bundled glibc versions since 2.23 (april 2016) works. I discovered some nasty bugs while going back in time. To my knowledge, distributions don’t patch glibc too much, so I think this is a good methodology.
I plan to do the same for musl, using chimera and void. I already know that musl pre-1.2.1 will have problems (the __libc struct wasn’t the same before that).
I should take a step back from this project for a couple of months, since my real life work is slowly getting deprioritized, and projecting the current trend leads to bankruptcy 
17 Likes
I have a lot of paid work ahead of me, so I guess it’s time for the latest news of the year
I will move forward with a full rewrite of the library.
I have also settled on a project that will make use of it: a real desktop application framework for linux. Put another way: A GLFW replacement fully written in zig, statically linked, libc not included, that (I hope) will work everywhere.
Here is the big picture (open to suggestions if I miss something):
Window, keyboard/mouse/touch input: Window interface
- WaylandVulkan11Window => Window and keyboard/mouse/touch input: libwayland-client.so / libdecor.so, GPU: Vulkan 1.1
- X11Vulkan11Window => Window and keyboard/mouse/touch input: Xlib.so, GPU: Vulkan 1.1
- WaylandGL46Window => Window and keyboard/mouse/touch input: libwayland-client.so / libdecor.so, GPU: OpenGL 4.6
- X11GL46Window => Window and keyboard/mouse/touch input: Xlib.so, GPU: OpenGL 4.6
- WaylandGL33Window => Window and keyboard/mouse/touch input: libwayland-client.so / libdecor.so, GPU: OpenGL 3.3
- X11GL33Window => Window and keyboard/mouse/touch input: Xlib.so, GPU: OpenGL 3.3
Multiple windows supported. All of the same kind (I think?).
Wayland vs X11: default will depend on environment variables, user overridable.
Vulkan 1.1 vs Opengl 4.6 vs OpenGL 3.3: will test availability in this order for default, user overridable. Differences from GLFW are that native graphics APIs will be abstracted, and a GUI toolkit will be included.
Gamepad, drawing tablets: Gamepad and Tablet interfaces
- EvdevGamepad, EvdevTablet => libevdev.so
- JoystickGamepad => custom wrapper over direct
/dev/input/js*reads
Mulitple peripherals supported.
Will discover all peripherals using every available method.
Sound: SoundManager interface
- PipewireSoundManager => libpipewire-0.3.so.0 (or whatever the version will be)
- PulseAudioSoundManager => libpulse.so
- AlsaSoundManager => libasound.so
- OpenALSoundManager => libopenal.so
Mulitple sound manager supported. At most one per kind.
Will test availability in this order for default, user overridable.
I suspect it will easily get me to 2027
As a side note: I have learned a lot about the roots of my discomfort when thinking about the coherence of the linux ecosystem (this sentence is a very polite version of what I really think about the libc’s hegemony), but at least it is a good source of fun challenges.
11 Likes
I don’t think you need to try openal at all, Pipewire and Alsa is honestly enough, and you can even give them different api as pipewire allows much more advanced usage, like supporting video streams as well, jack might be another interesting api for audio production use though (pipewire supports it too).
I have native zig client implementation for pipewire here too https://github.com/Cloudef/pipewrangler but it can only query the graph atm..
Also, if you want to get something useful quickly, as making app framework like this is a lot of work (but very useful!), you could also start by writing wrapper over SDL where your dynamic lib loader loads it! That means static binary that loads SDL and works everywhere!
I agree, openal without at least alsa would be a very strange setup.
Maybe I’m missing something, but it seems it imply asking users to install the SDL shared library, which doesn’t align with the philosophy of the project
1 Like
Maybe I’m missing something, but it seems it imply asking users to install the SDL shared library, which doesn’t align with the philosophy of the project
You could bundle the .so with the project, or `@embedFile` and extract it. Since your translation layer supports both musl and glibc, even if this SDL was compiled for musl, it loading system vulkan implementation that is based on glibc should still work right?
I have a pure Zig implementation of the wayland protocol, and it even has some support for interacting with libwayland-client (it doesn’t work with the pure Zig API, unfortunately).
I also have evdev support in seizer, specifically for reading input from controllers. It supports reading SDL style controller databases, though I wouldn’t be surprised if it has bugs. I’ve only used it for seizer-solitaire, and then that was mostly on a ARM-based retro-gaming handheld. I’m pretty sure only me and my brother have really played it.
To be honest I would like to stop using evdev directly if the gamepad-v1 protocol ever gets implemented, as it allows the compositor to properly direct focus. Although it would probably be smarter to leave it in as a backup.
seizer is my project to handle windowing and input, more narrowly focused on supporting Wayland and doing software rendering. Right now it doesn’t work on the latest stable Zig, and updating it to work feels like it would take a major re-write of the library. I’ve been putting off updating it until std.Io is ready.
All this is to say, if you’ve got any questions, or if you want to use any of my libraries, feel free .
2 Likes
I think there is a misconception about how this library works. It is intended to be a dynamic loader that tries to comply with the ELF specification and its various “extensions,” and that knows what special operations are performed by existing dynamic linkers to correctly initialize libc, nothing more. If a libSDL*.so was compiled against glibc, then somewhere in the dependency chain, symbols coming from glibc’s libc.so.6 will be required, with specific versions, and those might not be provided by musl. Even the absence of the libc.so.6 file on musl-based systems will be a problem.
When you say “translation layer,” I guess you’re thinking about the fact that calls to malloc, for instance, are forwarded to calls to allocator.alloc in zig, but this isn’t really translation. This forwarding strategy is only needed because it interacts somehow with dynamically loaded libraries, threads, and thread-local storage (I want to be able to start threads in zig without making it impossible for a loaded library to still work correctly).
I plan to write an extensive section in the README about these details to avoid such confusion.
Don’t worry, I’m not in a rush to see the first window pop up anymore. This vulkan triangle was satisfying enough
Besides that, I don’t want to distribute binaries compiled from code I didn’t write. Using the “already distributed” binaries on a user’s computer (by loading them) is less problematic from my point of view.
Seizer is already cloned locally, and it has already started to be used as a trusted source of information
I wasn’t aware of the gamepad-v1 protocol, thanks for pointing it out. After reading the linked merge request, I won’t rely on it being merged soon. Regarding gamepads and tablets, there is also /dev/hidraw/*, which I somehow missed. I’ve added it to my roadmap: I will try to implement something like (or make a direct port of) hidapi. As I understand it, some controllers and tablets expose more complete functionality via these devices than via the input subsystem.
Could you elaborate on this? I’m afraid I don’t fully get what the problem is. I thought a wayland client could always know if one of its windows is focused or not.
And in case anyone is curious about what’s new regarding this project:
I added WaylandSoftwareWindow and X11SoftwareWindow to the list, which, as the name suggests, will use software rendering. I also think I want a “canvas-like” API over 2D layers that can be added to windows.
I decided to follow the pattern used by the Writer interface for Window and such. I have great faith in restricted function types to avoid paying the full cost of using vtables and function pointers.
6 Likes
Focus in Wayland is tied to the input device, e.g. the mouse or keyboard. On something like the RG35XXH, there is no mouse, keyboard, or touchscreen:
Now you could present a virtual keyboard (in fact they might, I don’t remember at this point), but now the app has to use keyboard events to determine if the gamepad events should be used. This requires custom support from the application.
Further, on a desktop computer you might not want gamepad focus to follow the keyboard. You might want to tab over to a web browser while retaining control over a game.
I think SteamOS gets around this issue by creating a virtual controller device.
1 Like
Hmm… I see.
I think it will be less of a problem on desktop (which will be my primary target) where “normal navigation” is done through keyboard and mouse events that Wayland will route correctly. I’ll keep the virtual device concept in mind though.