Best way to release packages for libraries?

@Gonzo brought up a question that I’m searching for an answer to:

I’m not very experienced in automating this process and I haven’t looked into doing this with the Zig build system. Any direction here would be greatly appreciated.

From both the author’s point of view and for the person trying to integrate it, what’s the best way to do that with Zig to date?

Technically it already can be used like it is, by adding fluent as a dependency to your application project (via zig fetch --save=fluent archive.tar.gz details below) and then using something like this:

const mod_fluent = b.addModule("fluent", .{
    .root_source_file = b.dependency("fluent", .{}).path("fluent.zig"),
});
exe.root_module.addImport("fluent", mod_fluent);

However I think the nicer way (for more complex libraries) may be to provide a build.zig that declares a module, steps below.


Library

I think using zig init and removing the application specific parts, then adding the fluent.zig file as a module in the build.zig via:

_ = b.addModule("fluent", .{
    .root_source_file = .{ .path = "fluent.zig" },
});

User application

Then the user can use zig init to initialize a application project, use

zig fetch --save=fluent <commit-specific-archive-download-link>.tar.gz

to add the dependency.

And add

exe.root_module.addImport("fluent", b.dependency("fluent", .{}).module("fluent"));

to the build.zig after the exe declaration to make the module importable in the application.


Commit specific archive download link

To get the <commit-specific-archive-download-link> click on Commits on github, then Browse repository at this point, then Code and copy the Download ZIP link, then change the ending from .zip to .tar.gz

Why commit specific?

If you use an url for a branch the url+hash pair that is stored in the build.zig.zon gets broken as soon as a new commit gets pushed to the library repo, this is bad because then you are forced to update your dependency when ever the library author pushes a new change.

So with the status-quo build system, branch urls are undesireable, because their content isn’t stable and only the latest commit works, if you use it anyway, you might be forced to update the hash of the dependency, in the middle of fixing some other bug in your application.

When you use a commit specific url for the archive download, you avoid that problem, because the commit never changes, the content never changes, which means that the hash will continue to match the content, thus the library author can push new commits without users being affected.

They can update when they want to, instead of one day getting an error that the hash doesn’t match anymore.


This is only based on my current knowledge, there may be ways to make this even simpler / better.


Here is a fluentdirect-example repository that uses the fluent repo without any changes to it, the commits are separated into simple logical steps.

4 Likes

This is pretty close to what I was imagining. It looks like it mostly falls on the user at this point from what you’re presenting here.

2 Likes

I would love a way of doing exactly this, but just with the build.zig.zon file. I would think this is a common requirement: enumerate and use external dependencies (without much further ado).

1 Like

One slightly different take on that idea might be to craft a generic build.zig, that parses the build.zig.zon in the same folder and adds every dependency via a predictable scheme as importable modules.

That way you could just copy that build.zig where you want it and then use it.
Similarly it could take the name for the exe also from the build.zig.zon.

One other possibility would be to write a script that uses build.zig.zon and then calls zig fetch and then uses zig build-exe with passing parameters corresponding to the things in the build.zig.zon, but all of that seems brittle and more complicated to me.

I think if you want to have a whole bunch of small tools with a few dependencies, maybe just put all of them in one project and have one shared build.zig and build.zig.zon for them, and have the build.zig, create separate executables for all the different mini applications.

Basically I think trying to use less files isn’t really paying of, you still would need separate folders for the application.zig+build.zig.zon pair, so adding a third file doesn’t really cost anything.

2 Likes

Could you, in the app’s build.zig, parse build.zig.zon, loop through the dependencies and call addModule on all of them with the appropriate name?

This could be put in a function in some external build-utils.zig and reused.

Oops, from @Sze:

craft a generic build.zig, that parses the build.zig.zon in the same folder and adds every dependency via a predictable scheme as importable modules.

Yes exactly. I am surprise the stdlib does not provide this already.

I’ve been wondering the same.

The package manager is coupled with the build system on a fundamental level.

The presence of dependencies in a build.zig.zon causes the compiler to generate Zig source code containing hashes and metadata about the dependency graph, which is subsequently read by the build runner alongside your root build.zig script in order to figure out how to locate and resolve all dependencies.

You also need to actually invoke a zig build process for the dependency’s build.zig script, passing along any custom -D options you need, and run its build function to even figure out which artifacts and modules are exposed by it, otherwise all you can do with the package is resolve paths to plain files.

If your project needs to manage and download dependencies, it has outgrown the scope of zig build-exe and you should really upgrade it to make full use of zig build.

1 Like