Medium to large-scale projects

I think I see the root of surprise here.

The underlying fact is that Zig doesn’t have separate compilation.

When you build a C++ project, you generally invoke your compiler multiple times. First you compiler a parser, then storage engine, etc, each as a separate clang ... command line, each producing an object file. Finally, there’s a “root” invocation that links all object files together into the final executable with a linker (though, typically the linker still invoked via clang ... command line).

This is not the case in Zig. A Zig project is always (hand waving away special cases) built via exactly one compiler invocation: zig build-exe <all your codebase>.

The “single build.zig” is downstream of that — there has to be a place that assembles that single command line, which is in contrast to C++, where you can separately compile object files, and, at the top level, you don’t necessary have to encode the knowledge how those files came to be. Though, that ultimate cause is of course mediated by decades of path-dependence as well, see Recursive Make Considered Harmful [1998,2006] | Lobsters for recent discussion of the historical context.

9 Likes

OK, I guess that’s what @alanza was trying to explain to me earlier.

However, in a large project, there are multiple executables and libraries being built. For example, in Postgres, it isn’t just the DBMS server executable. There’s also the libpq and libecpg libraries, the psql client and dozens of other client and server executables. I assume that to build something like that, Zig still has to process each compilation “unit” (executable or library) separately.

Furthermore, in something more monolithic like the Linux kernel, aside from drivers, it seems improbable that one would want to process, say, memory management and file system modules, together, in a single pass (and IMHO this has nothing to do with path-dependence, but with the “nature of the beasts”). Even in the case of Postgres, what advantage would be gained from compiling the parser together with the access methods and/or with the executor or optimizer?

1 Like

I think you’ll ultimately be happier with one build.zig and some function-and-import oriented way of taming the complexity, yes.

Same basic thing as make (I’ve used cmake but not enough to know how it handles this): build.zig wants total awareness of the dependency graph, and a single spot to do all the out-of-sight operations (.zig-cache etc.), it’s going to be better to give it what it expects.

I totally get not wanting to have thousands of lines in one file which reach out and touch a big collection of modular components! It’s just a question of how to get away from that.

Ghostty uses a specific /build/ directory, I think having build.zig-esque files in various subdirectory roots also has a lot to recommend it (I would probably do the latter FWIW). Just not literally build.zigs, the name is hard-coded to be special.

1 Like

With zig build step-a step-b step-c you can specify the set of top level steps you want to run and they will run in parallel if they don’t depend on another. I think the benefit is that you don’t have multiple separate build processes running in parallel and instead have one build process that coordinates everything.

Especially when incremental compilation and hot reloading eventually is figured out, I think that this basis for organizing things would be much better for providing a workflow for the developer where things get interactively recompiled without problems. To do that without a single build-process the different build-processes would basically have to discover each other and start to coordinate with another, which seems much more complicated then just having a single build process that coordinates everything.

From a usage standpoint you could have zig build cli server library or a step that just depends on all of them zig build all. Internally, sure zig would look at the associated code and not at files that aren’t actually relevant to the currently processed build step, but I am unsure why that matters.

It could still be beneficial to process all of that in the same build process, things that are used in multiple build steps, like for example generated modules or static assets can be shared between multiple build steps, you also potentially avoid paying more startup, initialization, synchronization and finalization/shutdown costs.

2 Likes