Not necessarily Zig-related, but I wanted to ask about this here since I gravitate towards the community’s general view of what good software is.
I have recently fallen down the Plan 9 rabbit-hole and am now looking for more material and discussions regarding OS Design.
The scope is broad and includes things like:
Basically, start from the ideas in AmigaOS 2.0 and modernize it (the newer AmigaOS versions are mostly just maintenance instead of proper modernization).
Most notably:
E.g. get back to that idea that the computer is a ‘workbench of tools’ instead of a vehicle to sell shit to users. The old UNIX philosophy did that for the command line, AmigaOS started to transfer that idea of pluggable tools to UI applications but was killed too early.
I don’t care too much how the internals are implemented as long as the design is somewhat sane and accessible. Both Linux and Windows are ‘good enough’ under the hood. The problem is the user-facing side, all the UI/UX research between the 70s and late 90s has been ignored in the last 15 years and it’s bad enough that at least programmers stayed on or went back to a pure terminal environment, not much different from 50 years ago.
It’s especially a shame for desktop Linux that it copies the ‘all-in-one’ UI application model of the commercial world without questioning its usefulness. E.g. Linux doesn’t need Photoshop/Word/Excel/… clones, but much smaller and more specialized UI applications that can work together and extend each other.
When I first found out about DAWs and audio software, one thing that stuck with me was the plugin system used for effects, instruments, etc, and how you could use the same plugins in different software.
I thought that it would be cool if that was expanded to other UI stuff, like being able to use the Inkscape’s vector drawing tools in Krita or Kdenlive as a plugin.
I knew about AmigaOS, but not that it made an effort in this direction, will check it out.
AmigaOS 2.0 had this idea implemented as ‘data types’, e.g. system-wide DLLs for file loading and saving. If one image-processing-tool or paint-tool installed a data type for PNG files, all other applications which deal with images suddenly also knew how to load and save PNG files (same for all other file formats).
The other nice convention on the Amiga was that most file formats followed the IFF standard (Interchange File Format - Wikipedia), this allowed applications to “sort of” parse unknown file formats and skip over data chunks they are not interested in.
It works in the DAW/music production world because some standard emerged and it was economically convenient for both the plugin vendors and the DAWs vendors. Steinberg didn’t had to devellop VSTs with all the cost it entails, and people could sell VSTs for money.
Nowadays, companies make billions selling walled garden and golden cages, so they have interest in not maintaining an ecosystem of tools that you could keep using with another competitor software.
While the lack of standards and OS interfaces is undoubtedly an issue, they have been develloped by companies when they decided to go with that strategy.
Yeah, I assume these kind of efforts won’t originate from the more closed-off/centralizing companies; the fact that it happened in the music industry seems to have been mostly accidental than anything.
Unfortunately, this same kind of “convenience accident” hasn’t happened in the image/visual side of things, which is a shame considering the parallels between the two that would enable a similar approach; a deliberate effort would be required to get the ball rolling for the visual stuff.
Thankfully, the audio ecosystem has more sensible options for those that don’t want to buy into the whole walled garden thing, with stuff like: CLAP | Ardour | Zrythm | Calf | Guitarix | OpenUtau | etc.
Which serves as an example to other companies/industries why they shouldn’t go that route, because they would lose their control and therefore would “lose” profit.
That’s true, but because this business model conflicts with users’ interests, sooner or later it will erode and crumble — resulting in a total loss for them, unlike companies that steadily established themselves within more sustainable open ecosystems.
I found the DLL-like DataTypes a bit clumsy. The Arexx thingie was interesting though, maybe it could be modernly implemented by using more general IPC/9P/plumbing + shell-scripting.
My idea is actually to have specialized, self-contained “canvas” programs that know how to open/display/save particular data types:
Raster Canvas
Vector Canvas
3D Canvas
Video Canvas
Audio Canvas
Spreadsheet Canvas
Support for especific file formats would be optional and selectable via compilation options. It may be better to not subdivide the “canvases” so much, for the sake of interoperability — being able to use vector and raster together, for example.
Each canvas would also work as a “plugin host”, so that you can build your custom workspaces and presets (3D modelling/Image editing/Vector drawing) on top of the same base “visual canvas” program by employing especialized modular toolbox plugins:
Paint/Draw (brushes, fill, erase ..)
Path/Vector (pen/bezier, node editing, shape primitives ..)
Navigation (pan, rotate, zoom ..)
Layer Management (blending modes, visibility ..)
Manipulation (crop, invert, paste ..)
Image & Audio filters/effects
… and so on
At a bare minimum a Zig OS would have syscalls with unambiguously fully-documented error codes.
The word “canvas” reminds me of something I was thinking about today, where it could be cool to have some kind of system where different processes are allowed to read/write each other’s screen buffers, so that the screen buffer becomes an implicit “channel” through which data is transmitted, sometimes in a user-friendly fashion
I don’t think it’s necessarily a very “Zig” idea though
I guess what I meant by “Zig of” was mostly just vibes/general philosophy, not that it has to be an explicit parallel. I’m actually interested in seeing the more “out there” ideas, even if they’re not strictly “ziggy”.
Capabilities.
Just like Zig explicitly passes Allocator and Io, a Zig-of-OS should explicitly pass around your authorities.
From a technical pov, that’s a cool idea.
But one should always think about what happens if the project is actually a success?
In this case, that would be a security nightmare, as history has shown ( I’m thinking about browsers). On Android, apps need a special permission to overlap other apps’ output.
“Capability can be shared/duplicated/forwarded” is also a capability.
Because of this you can apply it at a fine-grained level at the OS, app, process, thread, etc.
I think a ton of problems of modern software operation and development would disappear if we could rethink packages, versioning, (and maybe even linking) from the ground up.
I think Nix / NixOS got a lot of things right but still isn’t quite there. I also see what the Unison programming language is doing, and think there’s a lot of interesting ideas there.
When we have version 1.2.3 and 1.2.4 of a library, where the majority of the code is the same but maybe a few leaf functions are different, I think it’d be amazing to be able to dynamically link them both at the same time in some merged way. Two programs running at the same time, one that requires 1.2.3 and the other that requires 1.2.4, would get the code paths that they need executed without having to load the common surface area in memory twice.
Perhaps this.
So in zigland this would be:
@import(kernel)
if we could rethink packages, versioning, (and maybe even linking) from the ground up.
The simple and elegant solution is static linking for all user code, and only use dynamic linking as the interface to the operating system (instead of syscalls), and for plugin-systems (for plugins WASM/WASI is the better technology though). E.g. every application and tool should be a single, self-contained executable, maybe even including all application data (those data sections would only be mapped into memory when needed, kinda like a memory-backed virtual filesystem).
The biggest problem of dynamic linking isn’t versioning, but that a DLL represents an optimization barrier. First, dead-code-elimination doesn’t work for DLLs. It may well be that a single shared instance of a DLL loaded into memory takes up more memory than as if each process would statically link the library, since unused code is either not linked in the first place, or removed during LTO. And second, DLL functions cannot be inlined and further reduced and optimized. E.g. it may be that a large function body may be reduced to almost nothing after inlining.
Linux distro package maintainers hate this simple trick though 
(since when a security issue is found in a library, all packages that use a specific library must be rebuilt instead of only one package).
PS: A ‘new way’ of linking could be some sort of ‘shared static’ linking. E.g. an executable could have specific pieces of code and read-only data in separate executable sections, identified by a hash-key. The executable loader would keep track of those sections, and if other executables have sections with the same checksum, they could be shared in memory (the exe loader must not just trust the checksum of course, but make sure that the sections are byte-by-byte identical).
This would fix “DLL version hell”, since each executable is still fully self-contained, but the operating system will still be able to eliminate multiple copies of the same code and data in memory. The downside is that such executable sections would still be an optimization barrier. And if the executable only places actually used library code into such sections, then it’s much less likely that those sections can be shared with other processes.
…yeah, the more I think about this weird static/dynamic hybrid the less sense it makes. Static linking is actually fine
Another problem is that once you have both threadlocals and dynamic libraries you invite loads of agony such that you might rip out your hair yourself before starting.
In isolation both are fine. For threadlocal you “just” need a hook when a new thread is created and then allocate the needed memory for the threadlocals. For dynamic libraries you just load the library.
But when you combine those, the dynamic loader/linker and the shared library invariably get entagled. Then you have to decide whether the loader stops the world when a new library is opened to create all the necessary tls stuff or whether to do it lazily which has it’s own complications. Just look at glibc and ld.so.
Therefore I would also like to just have static libraries. It does make sense to have something like the vDSO for some syscalls or data to speed things up. But maybe just having a readonly page the kernel writes, like vvar, for things like time is enough.
Statically linked executables take inevitably more space on the disk. Storage is less of a concern these days in consumer laptops but not every platform can afford this much storage. Recompiling everything also takes a lot of ressources and means all apps have to be downloaded again which takes bandwith. How much perf would we gain on which types of applications ?