# Zig build : How to handle an unknown amount of files in a folder

**URL:** <https://ziggit.dev/t/zig-build-how-to-handle-an-unknown-amount-of-files-in-a-folder/8543>\
**Category:** Help\
**Tags:** build-system\
**Created:** [February 13, 2025, 5:59pm UTC](https://ziggit.dev/t/zig-build-how-to-handle-an-unknown-amount-of-files-in-a-folder/8543 "2025-02-13T17:59:47Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![Valden](https://ziggit.dev/user_avatar/ziggit.dev/valden/32/4572_2.png) [@Valden](https://ziggit.dev/u/Valden)\
**Post date:** [February 13, 2025, 5:59pm UTC](https://ziggit.dev/t/zig-build-how-to-handle-an-unknown-amount-of-files-in-a-folder/8543/1 "2025-02-13T17:59:47Z")

</div>

Hello everyone,

I’m currently building a game using zig, and I would like to use the zig build system as an asset pipeline for this game. Such asset pipeline would require for example taking all the files with a given extension in a folder (in my case aseprite files), run a command on them (to export them to one or multiple .png + .json files), put them in another folder, and run another command on that folder to pack all the .png in a nice sprite sheet that would get installed along the game.

I want to use the zig build system to harness its built in caching system and `--watch` feature for live asset rebuild/reloading in the engine (I got it to work for my shaders already), but I don’t know how if the build system can handle “all the files in a folder” without writing the path of all the files manually in the build.zig file (which would be kinda painfull for my artist friend). I tried poking around the build.zig api but I didn’t find anything resembling what I want for that system.

The only zig project that I know that handle a “folder of files” might be [Zine](https://github.com/kristoff-it/zine) but I don’t know if they support the same feature set that I want.

Alternatively, would it be possible for a zig program to use the same api as the zig build system to know which files have changed, where to cache them so I could build an asset pipeline myself, and add that program as a build step ?

Thanks !

---

<div class="post-metadata">

**Author:** ![LucasSantos91](https://ziggit.dev/user_avatar/ziggit.dev/lucassantos91/32/440_2.png) [@LucasSantos91](https://ziggit.dev/u/LucasSantos91)\
**Post date:** [February 13, 2025, 7:14pm UTC](https://ziggit.dev/t/zig-build-how-to-handle-an-unknown-amount-of-files-in-a-folder/8543/2 "2025-02-13T19:14:40Z")

</div>

I think this should work:  
In your `build.zig`, iterate over your resources folder and call `addOutputFileArg` for each file to get a lazy path. Use this lazy path as the `root_source_file` to `createModule`.  
Create a writer from a `std.ArrayList`. For each file on the resources folder, append the name, such that, when you end the iterations, the ArrayList will contain something like this:

```zig
const sprite1 = @embed("sprite1");
const sprite2 = @embed("sprite2");
...

```

Pass this data to a `WriteFile` step, to get a lazy path. Make this step depend on all the “run command” steps.  
Create a module passing the lazy path from `WriteFile` as the `root_source_file` and list all the other modules as imports.

Now in your consumer module, you can access them like this:

```zig
const resources = @import("resources");
const sprite1 = resources.sprite1;

```

---

<div class="post-metadata">

**Author:** ![vulpesx](https://ziggit.dev/user_avatar/ziggit.dev/vulpesx/32/3989_2.png) [@vulpesx](https://ziggit.dev/u/vulpesx)\
**Post date:** [February 14, 2025, 6:35am UTC](https://ziggit.dev/t/zig-build-how-to-handle-an-unknown-amount-of-files-in-a-folder/8543/3 "2025-02-14T06:35:12Z")

</div>

Use `b.build_root.handle` to get a `std.fs.Dir` to the root of your project, open the directory of your resources via this will let you use the `iterate()` function to get a directory iterator.  
`b.addSystemCommand` is your friend

> [@LucasSantos91](#):
>
> In your `build.zig`, iterate over your resources folder and call `addOutputFileArg` for each file to get a lazy path

use `addInputFileArg` too, otherwise zig won’t rebuild when the inputs change.

> note both of those functions are on the step returned  
> By `b.addSystemCommand`

use `b.addOptions()` to get a `*Options`, this lets you declaratively make a module, much nicer and more useful than a `WriteFile`  
works like this `options.addOption(T, "option_name", value)`  
then add it to your root module with `addOptions("import_name", options)`  
and you will be able to `@import("import_name").option_name` to your heart’s content :3

---

<div class="post-metadata">

**Author:** ![Valden](https://ziggit.dev/user_avatar/ziggit.dev/valden/32/4572_2.png) [@Valden](https://ziggit.dev/u/Valden)\
**Post date:** [February 14, 2025, 7:36pm UTC](https://ziggit.dev/t/zig-build-how-to-handle-an-unknown-amount-of-files-in-a-folder/8543/4 "2025-02-14T19:36:29Z")

</div>

Thanks for the tips !

One issue I see is that by listing the file in the setup phase of the build.zig, newly added files wont be picked up if the build is in `--watch` mode, which is one of the use case I would like to support.

I’ll experiment with that and come back with the results

---

<div class="post-metadata">

**Author:** ![castholm](https://ziggit.dev/user_avatar/ziggit.dev/castholm/32/1330_2.png) [@castholm](https://ziggit.dev/u/castholm)\
**Post date:** [February 14, 2025, 9:37pm UTC](https://ziggit.dev/t/zig-build-how-to-handle-an-unknown-amount-of-files-in-a-folder/8543/5 "2025-02-14T21:37:50Z")

</div>

I would not recommend iterating directories in your `build.zig` script, in part because as you noticed it won’t play well with `--watch` but also in part because it’s considered poor form to access or query the host system using `std.process` or `std.fs` APIs from your declarative `build.zig`.

What I would personally try is to write a tiny Zig CLI program in a separate file that takes paths to input files/directories as well as an output directory as command-line arguments, iterates through the file system for input files and uses `std.process.Child` to pass them forward to the command you need to invoke such that they resulting files are outputted to the specified output directory.

The [Running the Project’s Tools](https://ziglang.org/learn/build-system/#project-tools) section of the build system docs has an example of compiling and invoking a program in this way. If you run your compiled utility program using `b.addRunArtifact()`, pass input paths using `run_step.addFileArg()` and `run_step.addDirectoryArg()` and obtain a handle to the output directory using `run_step.addOutputDirectoryArg()`, the build system _should_ cache everything and detect changes.

Note the use of _should_, because unfortunately it doesn’t look like `std.Build.Step.Run` currently properly caches and watches directory input arguments, so what I suggested might not work after all. It might work if you manually also call `run_step.step.addDirectoryWatchInput()` for each input directory argument, but I have not actually tried this in practice.

---

<div class="post-metadata">

**Author:** ![Valden](https://ziggit.dev/user_avatar/ziggit.dev/valden/32/4572_2.png) [@Valden](https://ziggit.dev/u/Valden)\
**Post date:** [February 16, 2025, 8:46am UTC](https://ziggit.dev/t/zig-build-how-to-handle-an-unknown-amount-of-files-in-a-folder/8543/6 "2025-02-16T08:46:26Z")

</div>

Oh I see, however don’t running a sub-program that only takes a directory with `run_step.addDirectoryArg()` mean that the command will get called every time the directory changes, and thus it would rebuild all my assets instead of only the one that have actually changed ?

---

<div class="post-metadata">

**Author:** ![vulpesx](https://ziggit.dev/user_avatar/ziggit.dev/vulpesx/32/3989_2.png) [@vulpesx](https://ziggit.dev/u/vulpesx)\
**Post date:** [February 16, 2025, 9:34am UTC](https://ziggit.dev/t/zig-build-how-to-handle-an-unknown-amount-of-files-in-a-folder/8543/7 "2025-02-16T09:34:17Z")

</div>

yup, as long as you make sure to use the file arg input/output functions you should be fine. Not sure why they are complaining about `std.process` I explicitly stated to use the build systems system command utilities.  
the only reason i said to use `std.fs` is because the build system doesnt provide a way that is as fine grained, as you said it would be annoying if you rebuilt every asset when only one changed

---

<div class="post-metadata">

**Author:** ![castholm](https://ziggit.dev/user_avatar/ziggit.dev/castholm/32/1330_2.png) [@castholm](https://ziggit.dev/u/castholm)\
**Post date:** [February 16, 2025, 3:57pm UTC](https://ziggit.dev/t/zig-build-how-to-handle-an-unknown-amount-of-files-in-a-folder/8543/8 "2025-02-16T15:57:14Z")

</div>

> [@Valden](#):
>
> One issue I see is that by listing the file in the setup phase of the build.zig, newly added files wont be picked up if the build is in `--watch` mode, which is one of the use case I would like to support.

Unfortunately, this is where you have to make a choice between explicitly specifying each file individually, which is fast and only rebuilds what is needed but may result in added files not being detected and removed files causing errors, or watching an entire directory tree, which should properly handle added/removed files but will rebuild everything even if just a single file in that directory tree changes (which will also quickly bloat the cache).

I don’t know if the lack of support for globbing in the build system is an intentional design decision or something that is planned to be solved for in the future, but it should be noted that several prominent modern build systems with a heavy emphasis on performance, [for example Meson](https://mesonbuild.com/FAQ.html#why-cant-i-specify-target-files-with-a-wildcard), don’t support globbing because scanning a directory tree for changes is slow and unreliable.

I don’t know what your workflow is, but if adding or removing assets is a thing that happens infrequently, I would personally probably bite the bullet and list each file individually. It’s the fastest and most cache-friendly option. On 0.14.0-dev, with support for `@import`ing ZON files, you could even have this “asset manifest” in a different file from the build.zig, if having them be separate makes things cleaner and easier to work with.

> [@vulpesx](#):
>
> Not sure why they are complaining about `std.process` I explicitly stated to use the build systems system command utilities.  
> the only reason i said to use `std.fs` is because the build system doesnt provide a way that is as fine grained, as you said it would be annoying if you rebuilt every asset when only one changed

I used `std.process` as an example because I’ve seen people do things like running `git rev-parse` inline for version numbers. The main reason I discourage against using APIs that access the host is because there are plans to [run build.zig scripts in a WASM sandbox](https://github.com/ziglang/zig/issues/14286) as well as [separate the “configure” and “build” phases into two distinct processes](https://github.com/ziglang/zig/issues/20981) by serializing the build graph, so it’s probably unwise to rely on methods that will almost certainly stop working in the future. Your build.zig should ideally be as “pure” as possible.

---

<div class="post-metadata">

**Author:** ![kristoff](https://ziggit.dev/user_avatar/ziggit.dev/kristoff/32/9_2.png) [@kristoff](https://ziggit.dev/u/kristoff)\
**Post date:** [February 16, 2025, 5:00pm UTC](https://ziggit.dev/t/zig-build-how-to-handle-an-unknown-amount-of-files-in-a-folder/8543/9 "2025-02-16T17:00:38Z")

</div>

> [@castholm](#):
>
> Unfortunately, this is where you have to make a choice between explicitly specifying each file individually, which is fast and only rebuilds what is needed but may result in added files not being detected and removed files causing errors, or watching an entire directory tree, which should properly handle added/removed files but will rebuild everything even if just a single file in that directory tree changes (which will also quickly bloat the cache).

The two things are not necessarily mutually exclusive. I think it would make sense to have the build system let you add entries to the list of watched directories.

That said it’s true that this is currently not supported AFAIK.

---

<div class="post-metadata">

**Author:** ![Sze](https://ziggit.dev/user_avatar/ziggit.dev/sze/32/496_2.png) [@Sze](https://ziggit.dev/u/Sze)\
**Post date:** [February 16, 2025, 5:42pm UTC](https://ziggit.dev/t/zig-build-how-to-handle-an-unknown-amount-of-files-in-a-folder/8543/10 "2025-02-16T17:42:01Z")

</div>

What if we create a zig program that used the existing cache functionality to create a manifest for one specific directory (I think the manifest is basically an index that also contains a hash of all the hashes?).

Wouldn’t that file then only change if any of the contents of that directory change?

And then a dependent build step could read that manifest file and create an embedding of all the files.

Just a thought, maybe somebody can try it out, or point out why it may not work.

* * *

Maybe [std.Build.Step.writeManifestAndWatch](https://ziglang.org/documentation/master/std/#std.Build.Step.writeManifestAndWatch) or [std.Build.Cache.Manifest.addListOfFiles](https://ziglang.org/documentation/master/std/#std.Build.Cache.Manifest.addListOfFiles) could be used? I currently haven’t looked into the implementation enough to fully understand how it works or is meant to be used…

But when I look at manifest in the docs it seems to me as if manifests could be used to solve the multiple files syncronization problem, similar to how it already seems to be used to manage packages? But my understanding is lacking and it would be good to have the perspective of somebody who has worked on the build system internals.

---

<div class="post-metadata">

**Author:** ![hrgdavor](https://ziggit.dev/user_avatar/ziggit.dev/hrgdavor/32/6756_2.png) [@hrgdavor](https://ziggit.dev/u/hrgdavor)\
**Post date:** [May 22, 2026, 7:42am UTC](https://ziggit.dev/t/zig-build-how-to-handle-an-unknown-amount-of-files-in-a-folder/8543/11 "2026-05-22T07:42:32Z")

</div>

I have just started using zig in watch mode, and immediately I am concerned. For a fresh project made by `zig init` it reports watching 77 folders (I am on windows 11)

---

<div class="post-metadata">

**Author:** ![nurulhudaapon](https://ziggit.dev/user_avatar/ziggit.dev/nurulhudaapon/32/5148_2.png) [@nurulhudaapon](https://ziggit.dev/u/nurulhudaapon)\
**Post date:** [July 25, 2026, 4:14am UTC](https://ziggit.dev/t/zig-build-how-to-handle-an-unknown-amount-of-files-in-a-folder/8543/12 "2026-07-25T04:14:32Z")

</div>

Had anyone a better suggestion regarding this? (caching directory of files and still being granular on --watch). I think if there were ways to get the list of changed file from --watch something like `run_step.addChangedFileArgs()` that would have been cool.

---

<div class="post-metadata">

**Author:** ![truly-not-taken](https://ziggit.dev/user_avatar/ziggit.dev/truly-not-taken/32/6205_2.png) [@truly-not-taken](https://ziggit.dev/u/truly-not-taken)\
**Post date:** [July 25, 2026, 10:40am UTC](https://ziggit.dev/t/zig-build-how-to-handle-an-unknown-amount-of-files-in-a-folder/8543/13 "2026-07-25T10:40:54Z")

</div>

Did you account for the standard library? You are allowed to modify it. Build system recompiles your program when any of the imported files is modified.

---

<div class="post-metadata">

**Author:** ![hrgdavor](https://ziggit.dev/user_avatar/ziggit.dev/hrgdavor/32/6756_2.png) [@hrgdavor](https://ziggit.dev/u/hrgdavor)\
**Post date:** [August 4, 2026, 8:40am UTC](https://ziggit.dev/t/zig-build-how-to-handle-an-unknown-amount-of-files-in-a-folder/8543/14 "2026-08-04T08:40:42Z")

</div>

If standard lib was also being watched, it would warrant a lighter watch mode in future. For zig core devs it may be more useful default, but option to watch only app code may be beneficial in the future

---

<div class="post-metadata">

**Author:** ![vulpesx](https://ziggit.dev/user_avatar/ziggit.dev/vulpesx/32/3989_2.png) [@vulpesx](https://ziggit.dev/u/vulpesx)\
**Post date:** [August 4, 2026, 11:39am UTC](https://ziggit.dev/t/zig-build-how-to-handle-an-unknown-amount-of-files-in-a-folder/8543/15 "2026-08-04T11:39:22Z")

</div>

Why is it an issue beyond an unexpectedly large number?

In theory, watching more files/folders only expands functionality, without impact on build times, unless you modify std.

---

<div class="post-metadata">

**Author:** ![vincentd](https://ziggit.dev/user_avatar/ziggit.dev/vincentd/32/9870_2.png) [@vincentd](https://ziggit.dev/u/vincentd)\
**Post date:** [August 4, 2026, 12:27pm UTC](https://ziggit.dev/t/zig-build-how-to-handle-an-unknown-amount-of-files-in-a-folder/8543/16 "2026-08-04T12:27:25Z")

</div>

The OS will have a hard limit on how many files/folders can be watched simultaneously. For linux it’'s called `fs.inotify.max_user_watches` and usually defaults to 8192 - for all user process _combined_. I know because Syncthing blows through it really quickly and gives the cryptic `Too many open files` message. You can increase the limit, but each one still uses ~1kB of memory for the kernel to keep track of things.

77 is no big deal though, for a program you’'re only likely to run one or two of at a time.