Suggested project layout for multiple entry point for Zig 0.12?

Help
1

Build system question about what’s idiomatic/the happy path.

At TigerBeetle, we have a bunch of sourcecode under the ./src folder. There multiple entry points (pub fn mains) among the source files — there’s:

  • src/tigerbeetle/main.zig for the main binary
  • ./src/unit_tests.zig for unit-tests,
  • ./src/fuzz_tests.zig for fuzznig,
  • ./src/scripts/main.zig for various CI helpers (implemented in Zig of course)
  • and also a bunch of completely wiled things here and there like src/clients/c/tb_client_header.zig which I am not even fully aware about myself

The way this works under Zig 0.11 is that, for all those entry points, we set .main_pkg_path = "./src", and then happily @import("../stdx.zig") or what not.

My understanding is that under Zig 0.12 main_pkg_path is gone: if ./foo/bar.zig is the root file of the module, then all code has to reside under ./foo subfolder.

What’s the best source code layout in this world? One thing we can do (and probably will do at least just for migration) is to say that non-entry-point stuff in src is a separate module, and then in various files with main do

const baz = @import("tigerbetle").bar.baz;

instead of the current

const baz = @import("../bar/baz.zig");

But that doesn’t seem like a proper long-term solution:

  • philosophically, we don’t really have good library level interfaces yet, and would rather treat everything like “one pile of code”, rather than introducing libary/binary internal boundaries.
  • practically, it would be a bit confusing if some code would be able to import file paths, and other code would have to use dotted access.
  • additionally, for dotted access to work we’d have to manually add pub const reexports everywhere, which also feels like needless work in this context, where we dont’ really care about API boundaries.

So, what’s the happy path here? What’s the right mental model to use in this case?

1 Like
2

For the reference, I think this is the commit that removed this feature from the build system:

3
  • philosophically, we don’t really have good library level interfaces yet, and would rather treat everything like “one pile of code”, rather than introducing libary/binary internal boundaries.

I’d also like to know others thoughts on this. My current understanding is you have 2 options, either bifurcate your universe between the root module and a library module like you suggested, meaning you’ll have to import via file path or dotted accessors depending on which context you’re in, or, have one universe by putting all your root module files directly in the root src directory and use file paths for everything. aka src/main_unit_tests.zig src/main_fuzz_tests.zig, src/main_ci.zig.

2 Likes
4

I’ll preface this by saying that I haven’t personally worked on any projects even a fraction of the scale of TigerBeetle, so this is mostly me just theorycrafting and thinking out loud.

That said, if your project has multiple entry points, and you (at this current state) prefer to treat it as a pile of code so that you can move fast and not need to waste time worrying code organization or API boundaries, then the easiest solution is clearly to move all entry point source files as close to the project root as necessary for them to be able to reference all the files they need.

But long term, I get the feeling that the more “philosophically correct” project structure would be to break out all reusable files into one or more modules. Even though the “file exists in multiple modules” compile error is only relevant in the context of a single artifact (you are free to reference the same source file from multiple artifacts), I think it makes the most sense structure your project in such a way that each source file is only referenced by one module/artifact in your build.zig script.

In other words, two different artifacts/modules should not both reference the same file via relative path @import, even if they are built separately and never link with each other. The two exceptions to this rule would be unit test artifacts (std.Build.addTest), since those only test test declarations from the root module and not ones in imported modules, and the build.zig script itself, in case it needs to reference things like constants or enums.

So using a project like TigerBeetle as a concrete example, I’d probably start off by breaking out all non-entry point source files into a tigerbeetle_core module, structuring it as a tree of declarations just like how std is structured. All other modules and artifacts (the main program, fuzzers, CI scripts, etc.) that wish to import any files belonging to this module should only do so via declarations re-exported by this module (i.e. @import("tigerbeetle_core").flags instead of @import("flags.zig")).

The main tigerbeetle executable, which is mainly just responsible for parsing CLI arguments and kickstarting all the necessary processes, would in turn import and delegate the hard work to tigerbeetle_core.

src/unit_tests.zig appears to only consist of a single comptime block referencing various files with tests, so for unit tests I would actually delete this file and move this block to whichever file is the root source file for tigerbeetle_core, so that a tigerbeetle_core_unit_tests artifact can use that same source file as its entry point (looping back to one of the exceptions mentioned earlier).

3 Likes