I have a recurring problem: every once in a while, I want to contribute to Zig, but then end up spending most of my attention quant just trying to get a version of compiler that can compile master Zig.
I am looking for a magic command that makes the following work:
$ git clone https://github.com/ziglang/zig && cd zig
$ <magic-command-for-dummies>
$ stage4/bin/zig build test -Denable-llvm
I don’t care if <magic-command-for-dummies> takes three hours, I am fine leaving that running in the background while I do something else.
I care very much that I don’t have to understand <magic-command-for-dummies>, and that I don’t have to supply any kind of inputs to it.
Reading that wiki page, option A is definitely not for me: I don’t want to learn which version of LLVM I need, and how to install that using my distro package manager.
Option B is sort-of closer. I still need to add a series of commands, and I need to tweak them for my use-case, but at least it should take care of dependencies.
git clone https://github.com/ziglang/zig-bootstrap --depth 1 && cd zig-bootstrap. That --depth 1 feels important in this case, I wish I could copy-paste that rather than typing git clone without that, only to realize a minute later than that’s going to be long and restarting with --depth=1
This again reuires some non-trivial amount of attention, as I need to set those two paths manually, and not to mix up the two
and that finally culminates in
Could not start dynamically linked executable: ./zig
NixOS cannot run dynamically linked executables intended for generic
linux environments out of the box. For more information, see:
https://nix.dev/permalink/stub-ld
That’s understandable: I use NixOS, so I brought this curse upon myself. But also, for “normal” zig, I can just download that and it works, because it is fully statically liked?
And at this point I decided that I’d rather write a ziggit post hoping that someone will provide a guaranteed-to-work build command, rather than continue to investigate this further
I think that won’t work for -Denable-llvm case? I don’t think ziglang/zig itself vendors LLVM which can be compiled from source, it expects LLVM to be pre-compiled?
But actually, for the change I have in mind I don’t need to build the compiler itself, because I only want to add a test to standard library!
Building LLVM/Clang/LLD from source only takes about an hour and a few GiB, and it works every time. On the other hand, I can’t control what shenanigans Linux distributions get up to with how they organize all the files.
This is the workflow I use because it works everywhere, on every OS, every distro, every time. 1 hour to build LLVM, 10 minutes to rebuild with cmake after pulling, 1 minute for compiler rebuilds.
Someone please correct me if I’m wrong, but I believe this is because at this point you are rebuilding Zig entirely using a Zig binary you built for “stage3”. Rebuilding Zig with Zig if you will. To build “stage3”, however, you’re using CMake to drive LLVM/Clang to compile Zig sources into a binary. This is why, if the versions of Zig are close enough, you can skip the LLVM compilation/stage 3 portion and just use a pre-built Zig binary as “stage3” to build “stage4”.
Very recently went through this myself, and the two links Andrew posted are definitely the magic combo.
Ok, what happens here is that stage4 “inherits” LLVM from stage3. Specifically, there’s a code in build.zig which tries to find LLVM next to the Zig compiler used to drive build.zig:
That config.h file contains the path to LLVM we used to build stage3:
Here’s my version where I make a couple changes, first is I would separate out building llvm from building zig, since you should only need to build llvm once for every version and you can re-use that for multiple zig repositories. Second I’d avoid changing directories as I find it nice to have commands I can copy/paste and run at any point without having to change directories. At least for LLVM, after that I’d change directories to the zig repo and the rest of the commands would be from that directory since that’s where I’d be living for the rest of development.
If someone creates the images, I can write a little JS tool that further automate the process (install Docker, pull the right image, etc.). I got some code lying around already. Then going from zero to building would simply require typing in npx zig-compiler-dev.
I think I can swing this, I just built both from source so it’s fresh in my memory. May be a couple days since it will take a LONG time to build these images and there will likely be some debug cycles
Great! On my poor dual-core notebook it took 3 hours to build, LOL. The image is a whopping 14G. I think if you combine the git clone, cmake, and install commands into one command and add rm -rf /llvm-project-18, that would reduce the image size by 8.4G. We shouldn’t need the LLVM project directory after we install its libraries, right?
I’m sorry for being a downer, but I really wish that people would learn to build from source on their actual host machine rather than introducing a dependency on Fedora and Docker. It’s a high cost to pay just to avoid learning some practically useful stuff.
I put a lot of work into the build-from-source process to make it as painless and standard as possible, so it pains me when people turn to Docker.
At least, after #16270 is done, I can make it truly painless since there will be no dependencies other than a C compiler. I mean, it will be exactly the process detailed in the README except all those listed downsides will be eliminated.
I am wondering if, even without fully eliminating LLVM, it would be a terrible idea to teach the build.zig in the Zig repo to do git clone LLVM && cmake && ninja dance…
The main stumbling block for me I think is that “downloading & installing LLVM dependency” is out of scope for the Zig build process, so you need to teach a human to do that, and humans are lazy learners (well, at least this particular human). I generally see “download&build dependencies” in scope for each project, because it’s the downstream who should control the versions of dependencies (with an optional override for further downstream packager or integrator).
I guess the long-term plan here is to wrapping llvm/clang into a Zig package, so that the relevant cmake logic is encoded in some build.zig, which lives outside of the main repository?
