One Zig folder to store them all?

This is a crazy idea, and I am not a priori claiming that it is useful, but I keep coming back to it every once in a while, so I figured I might as well write it down, if only to get rid of an intrusive thought!

Problem

Today, Zig typically needs a bunch of stuff in the root of the project to work:

/build.zig
/build.zig.zon
/zig-out
/zig-cache

This isn’t a big problem at all, but it is still inconvenient for those who like to keep the root tidy. Plus, there’s usually a number of related files — zls config, a script to download a zig compiler or a configuration for zig version manager.

Solution

So, what if we do the following?

/zig/build.zig
/zig/build.zig.zon
/zig/out
/zig/.cache
/zig/zls.json
/zig/.gitignore

And if, four example, you have a script to fetch a zig compiler, and you prefer to not install zig globally and use it per-project, you can do

./zig/download.sh 
./zig/zig build

(any resemblance to TigreBeetle’s build process is purely coincidental)

Any thoughts on how terrible this would be in practice?

I don’t really mind the status quo, but I don’t see anything that jumps out at me as this being terrible. I do know that I intuitively rely on the build script being in the current directory, but that is not really a requirement. (Indeed, in the past i’ve tried to mimick a rust “workspace” idea with multiple build.zig’s in sub directories).

1 Like

I had a problem with one of my dependencies recently after a breaking change to zig master (both me and my dependency were targeting master).

So I was able to fork my dependecy, get aquainted with it’s build system, and submit a patch in about 10 minutes while sitting in an airport.

This is primarily thanks to convention. The convention being that I can run zig build -h in the root directory of my dependency and get going very quickly.

I think it’s very powerful to keep this convention going.

This was the patch: fix default value pointer reflection after changes to std.builtin.Type by kj4tmp · Pull Request #17 · n0s4/flags · GitHub

3 Likes

I think the zig compiler installation can be solved with tooling that reads the minimum_zig_version in build.zig.zon.

This tooling already exists, for example the vscode zig extension installs the compiler for you when you open the root of the project.

I configured my global gitignore to ignore zig-out (and in older versions zig-cache) and with that I am happy with status quo (.zig-cache is automatically hidden on linux), but I don’t really use project specific zig version managers/management or zls configuration, so I can’t really comment on having to deal with those files.

But I am hesitant on the idea because I think having a zig folder you can add stuff to, would encourage to add an ever growing amount of configuration, I think we should go in the opposite direction, getting rid of the zls config, the version manager that downloads stuff.

Maybe the build-system / package manager could improve optional dependencies to the point where projects that want to have a project specific zls and zig version could add zls and zig-version management via the build-system?

I imagine having something like zig build dev-install installing zls and zig using the package-manager, obviously there is a boot-strapping problem if you don’t have a zig version yet that can run the build.zig, but as language changes decrease the compatible version-ranges could increase and the .minimum_zig_version add a field to build manifest file for minimum zig version · Issue #14475 · ziglang/zig · GitHub in the build.zig.zon could be used to get some stable version and use that to run zig fetch (or some custom installer) to get whatever other version you may need (for example if you need some zig version with custom patches).

So I think instead we should figure out if we can get rid of the other config files for zls/etc. by those being absorbed into what can be done by the build-system / package-manager. And finding out if there are any missing features for that to be possible.

To me this sounds like ‘tidy’ just means ‘have as few items in the root directory as possible’. This sounds nice aesthetically but you can’t just get rid of the files you actually need. Stuffing them in a subdirectory doesn’t seem like it accomplishes anything, you end up with a very ‘tidy’ root dir and a very ‘untidy’ subdir.

This also makes it harder for a newcomer to figure out what the project actually contains. I personally hate having to navigate through a whole bunch of directory paths just to figure out where things are, or even what things are there.

Your solution seems to me at first glance to be overcategorization. if it’s a zig project, there’s no need to put all the zig-related stuff in it’s own place.

I understand the appeal, but I don’t think this is worth it.

7 Likes

Also I’d like to point out that IDEs like VSCode have the option to hide specific files and folders.
You could use that to hide all of the generated folders and files like build.zig.zon which you don’t expect to change from your IDE (assuming that’s the reason you want to clean your root).

1 Like

My 2 ct: IMHO it’s really just the output directories which I don’t particularly like (zig-out and .zig-cache), IMHO those should both go into a .zig directory (e.g. .zig/bin,.zig/lib … and also .zig/cache), and that .zig directory should be safe to delete at any time (e.g. no version-controlled files in there).

That way a Zig project only needs one entry in .gitignore

build.zig and build.zig.zon should remain in the root IMHO.

Not sure if this is a good idea but I throw it in here anyway: Maybe build.zig.zon should contain more (optional) metadata (like the location of the root build.zig file, and maybe also the output and cache location - maybe even custom settings for other tools like ZLS), so that build.zig.zon becomes the only file required in the root and all other locations are defined by this one root file. At that point build.zig.zon should probably be renamed to project.zon though (or package.zon but that would be a bit heretic lol)

5 Likes

Hey, this is a great question, very nice idea!

My instinct is that it is nice when a structure is agnostic to the technology used to build the project, and mirror the conventions of the platform it runs on.

So in MacOS/Linux, we might think in terms of:

/src = input
/bin = output
/etc = /bin configuration

Fold into a project - not the best way, not the right way, not the only way, just…a way:

/repo/.gitignore
/repo/readme.md
------input-------------------------------------------------
/repo/src/           #main.zig, build.zig, build.zig.zon, zls.json               
/repo/src/doc        #doc templates  
/repo/src/etc        #configuration templates
------output-----------------------------------------------
/repo/bin            #executable / shared objects / scripts
/repo/bin/etc/       #configuration 
/repo/bin/doc/man    #generated man pages
/repo/bin/doc/www    #generated www content
/repo/tmp/           #compiled objects

The above goal is a convention for people to follow - people that may have to think about different languages, that when pieced together, assemble a complex system. It can be helpful to have a ‘one true ring’ method for finding the ‘thing’ - search in these places that contain the set of all things.

Project structure is a philosophy that has been debated about since person #3 got access to a computer! Two people agree, three people debate :slight_smile:

A given piece of a system might be broken out:

/app-rdbms/       #postgres, sqlite, whatever
/app-rest/        #zig
/app-ui/          #typescript/angular
/app-etl/         #python
/app-container/   #docker  

The overall system might have /app1, /app2, /app3, /app4, etc.

Good post!

For most of it I like the current way.

Like @n0s4 mention, having it inside a specific folder hides what should be visible quickly for any person.

Yes root projects tend to get huge due to the amount of configuration, settings, etc for all kind of things, from .gitignore, to docker-compose.yml, etc, but I dont think that hidding zig configurations would bring any benefits.

So, you cd /zig and ls and then you see

/build.zig
/build.zig.zon
/out
/.cache
/zls.json
/.gitignore

Here’s my reaction:

You didn’t change anything, you just added a directory up front. Now your project contains a ‘zig’ directory that contains everything that the top level directory contained before. When you work with the code, you’ll be in ‘/zig’ and everything will be like it is now. What do you think it buys you? I didn’t see what you think that helps. Nothing stops you from going arbitrarily deep with your code setup today, or (close to) arbitrarily shallow. So if you feel better with it, what stops you just doing that? The ‘/zig’ directory will be just as “un-tidy” as ‘/’ today. Also, when I’d see a project with a ‘/zig’ directory, I’d expect a ‘/other-option’ and ‘/this-framework’ and ‘/that-lang’ there as well.

So, the critical question that remains now is: what stops you from doing that, if you don’t do it already, and if you do it already, what are you suggesting really? that we all should be doing it? that the tooling should enforce your taste preference? that … what exactly?

If your RFC is really just “how terrible would this be in practice” - you didn’t change anything. Why would it be terrible? You moved all the mess into a beg that is now as messy inside. If it rocks your cradle, go for it and enjoy! Just keep in mind to enter one directory extra deep so that zig build works, but if it makes you happy, why would that be terrible for you? Go make software you can love <3

I really like @floooh 's suggestion. It’d be great to have all the “temp stuff” in a subfolder called .zig. Now there’s two of these at the project top-level.