Zig the Build System - Compiling LLVM with Zig

Hello Everyone!

I’ve been working on writing a language (called Conch) over the past few months (still very much in development, not the point of this post), and I’ve recently started the process of linking LLVM against it. The LLVM build implementation, which is the main topic of this post, can be found here. The project itself is written in C++, but I’m using Zig as the build system since I have a good amount of experience with the language. I switched from CMake a couple months back and haven’t looked back. The build system is incredible, even if I have to hand roll something every now and then (like manually parsing cdb fragments to make a compile commands db for clangd).

As I said, I’m now working on linking LLVM against Conch. Something I dislike about modern development is dynamic linking, but I think static linking can be just as unwieldy and unmaintainable if developers have to manually download a metric ton of dependencies manually. Package managers help with this, but I’ve found the most consistent solution is to build from source. With these opinionated (and maybe not widely agreed with) ideas firmly rooted, I took to compiling LLVM from source. While I’m slowly making my way through the libraries that I need to, it’s been super rewarding seeing compiled artifacts from scratch that can link against a test C++ program. I just crossed over the first hurdle that is compiling LLVMCore, but there’s so so so much more to do. It’s also great being able to cross compile easily since I have Zig backing the build.

Again, there’s still a lot to configure compilation wise for LLVM before it can function as a true backend, but I’m happy with the progress I’ve made so far and just wanted to share with the community. If you’re interested in the rest of the build, you can check it out here. It’s got some cool stuff, like compile command db generation, cross-platform packaging, and even rules for a custom test runner/allocator for C++.

You might take a look at the rest of the repo and wonder why I’m thinking about LLVM so early (i.e. the parser is still being tested). The main reason for this is that I’m extremely busy at the moment with other obligations, so I’m just going through the mindless task of scanning though the llvm-project source tree now instead of when I actually have the bandwidth to do more ‘meaningful’ work.

While it might be inappropriate to ask for stars on a C++ repo, a considerable amount of the project so far consists of Zig code for the build system, so if you like what I’m doing, a star would be greatly appreciated!

Cheers!

21 Likes

Wow, big project. Good luck!

2 Likes

Thanks Andrew, I really appreciate the support. This wouldn’t be possible without the Zig build system! Any chance a ‘native’ compile_commands.json emitter would be added to the build system?

I still need to review https://github.com/ziglang/zig/pull/18391

5 Likes

Cool, didn’t know that was already on your radar! Implementing it myself was my gateway into defining custom steps, so I’m somewhat glad this hasn’t been merged in yet. Very excited to see how Zig continues to evolve!

3 Likes

Glad to hear that PR isn’t dead, quite a few have been looking for that feature.

1 Like

Just a quick update here. I’ve made considerable progress, with LLVM’s example (kaleidoscope) now fully building, linking, and producing workable object files on my machine. About to merge this monstrosity into main before taking a step back into conch’s frontend tests.

1 Like

If I understand what your’re currently do is building a build.zig that can configure and build all of llvm/clang? Naturally this would be for your hobby project, could it be used outside of it to build them?

I ask because it might make bootstrapping llvm/clang easier in LFS environment where the extracted official zig compiler just runs.

Yeah, since it’s for a hobby project, I can’t promise that I’ll be building the entirety of LLVM (just the parts that I need at the moment). The way I have it formatted right now doesn’t uses a somewhat nonstandard modular build system, using the main build.zig to create an LLVM builder/compiler (found in LLVM.zig which I linked in my original post). It should be trivial to change this if you were looking for a more extensible solution in LFS environments, though. In a perfect world, all you need is an internet connection (to download the dependencies) and the zig compiler to build all of Clang and LLVM.

The great thing is zig master added a zig-pkg. Where it can prefect all the deps. Making it possible to vender tarballs that are completely buildable offline even with deps.

I’ll sign up to receive notification to track your progress.

Oh yeah I remember reading about that back when Andrew wrote that devlog at the start of February. Very welcome addition and a huge step in the right direction, though I’m currently using 0.15.2 for this work. Starting either tonight (time permitting) or tomorrow evening, I plan on tackling a large chunk of Clang. Thankfully, starting with LLVM itself means I got through most of the big pain points already (i.e. TableGen, config headers, etc.). Thanks for the support!

1 Like

Quick update on this PR. I had to ease off development for a couple days to prep for an exam, but I’ll be doing Clang today and tomorrow before merging, including clang format. Once this is done, the entire development environment for conch will be self contained, from format utilities to packaging releases. I’ll likely revisit this to add more of LLVM as this pet project matures.

I’ll be hanging out in this folder for quite some time. There original LLVM.zig has been renamed to LLVMBuilder.zig, in case anyone is wondering why the link in the initial post no longer works.

Cheers!

3 Likes