Will the `.version` field in `build.zig.zon` always be semver?

ajoino · November 9, 2024, 8:32pm

In the default build.zig.zon file you got these lines regarding the package version:

    // This is a [Semantic Version](https://semver.org/).
    // In a future version of Zig it will be used for package deduplication.
    .version = "0.0.0",

I think it’s strange that the package manager would force developers to follow a particular versioning scheme. To me it doesn’t matter if you use semver, something looking like semver, calver, or some other fancy scheme the developer wants to use. So I’m wondering what the devs are thinking regarding the version scheme, is it just a coincidence that semver is the only supported option or is it a deliberate choice that is likely to stay that way.

mlugg · November 9, 2024, 8:47pm

This is deliberate and will almost certainly remain. This is currently vaporware, but in the future, the compiler will be able to detect when multiple versions of one package exist in a compilation (typically through transitive dependencies, e.g. you depend on A and B, and A depends on v1.0.0 of C, while B depends on v1.1.0 of C). The key thing is that it can then use Semantic Versioning to figure out what it should do; if the major version is different a compile error must be raised since the APIs will conflict, but if only the minor/patch differ then the compiler can choose to replace the older (lower version) dependency with the new one.

dimdin · November 9, 2024, 8:54pm

Use of semantic versioning is common for package managers.
For example:

javascript npm: About semantic versioning | npm Docs
rust cargo: The Manifest Format - The Cargo Book
go modules: Go Modules Reference - The Go Programming Language

ajoino · November 9, 2024, 9:22pm

@mlugg I can see the theoretical benefits of semver, but they require the developers to be disciplined, and sometimes they have to ask fairly difficult questions like if a bugfix, which changes behaviour, is just a bugfix or if it requires a major bump because someone relied on that particular behaviour.

I disagree that the compiler should check for compatible versions. Just because package C updated to v2.0, it doesn’t mean that all APIs have changed. If I try to compile a project using packages A and B and the compilation failed because the metadata of package C changed while nothing I’m using changed, I would think that is a compiler bug. If the compiler is not smart enough to figure out what APIs have changed between versions and check if the ones I’m using are affected, it should not do this check at all IMO. ^[1]

As a corollary, since we cannot be sure what APIs have changed between version bumps, I think any X.Y.Z…-alfalfa versioning scheme is as good as semver, and especially so if the developers don’t fully follow the semver standard.

I do understand the goal is to make it easier for the developer to find possibly hard-to-find bugs (which is great!), but as you can tell I think it will have too many false positives.

@dimdin I know that semver is a popular format but I still think forcing people to use it the wrong way to about this as I explained above.

You can probably tell I also dislike packages upper-bounding the versions of dependencies. They cannot know what APIs will change so even there it’s best to just lower bound it. ↩︎

mperillo · November 20, 2024, 9:18am

However there are some projects where it make sense to use the date instead of the version.

What about adding a prefix before the version?
As an example:

.version = "v0.1.0"
.version = "d2024-11-20"

castholm · November 20, 2024, 12:50pm

Relevant accepted proposal:

github.com/ziglang/zig

Version selection system for the official package manager

opened 11:17PM - 17 Mar 21 UTC

kristoff-it

proposal accepted

[Accepted Proposal](https://github.com/ziglang/zig/issues/8284#issuecomment-1000…565560) ------------ # Premise ## Go's minimal version selection Some reading about Minimal Version selection (including some criticism of it): https://research.swtch.com/vgo (contains multiple articles, all relevant) https://peter.bourgon.org/blog/2020/09/14/siv-is-unsound.html My takeaway: it's weird and clunky, but like many things in Go, while you might disagree with the solution, it points at some very real truths: - allowing multiple versions of the same package requires both defining a cut point between what's compatible or not and it also requires allowing the user to have away of referring to those versions, and so Go simplifies everything around semantic versioning and forcing pkg maintainers to come up with what's effectively a new name for v2+ releases - there is a coordination problem between parties that are not in direct communication and so, if the pkg manager doesn't take that into consideration, the end user of a package could find themselves in a deadlock because of noone's fault in particular - if you always select the latest compatible version of a pkg you inevitably end up breaking peoples builds in a surprising way and so then you need lockfiles etc. MVS solves all that but it removes some inherent "fluidity" from the system that according to some does help new releases trickle down faster, and it's also a model in open conflict with distro maintainers. ## Distro maintainers Recent discussions both in favor and against what distro maintainers do: https://blogs.gentoo.org/mgorny/2021/02/19/the-modern-packagers-security-nightmare/ https://utcc.utoronto.ca/~cks/space/blog/tech/BundlingHelpsSoftwareAuthors My takeaway: distro maintainers want to manage applications in a way that makes developers sometimes uncomfortable and there is a tension between the system package manager and the package manager that the language provides (npm, cargo, ...). Distro maintainers expect software to be high quality and stable enough to be malleable to their operations, but software not always is and when things break everyone gets angry. Users want their systems to be both stable and up to date when it comes to security. Distro maintainers want to fulfill that wish, and also themselves wish it would take little effort to upgrade a dependency for all packages that depend on it. ## Zig's goals relevant to package management - Robust and maintainable software - Code reuse - Friendly to distro maintainers My takeaway: Obviously there is tension between all these points. Hard for things to be robust if you have a ton of dependencies, but **anything non-trivial will require a stupid amount of duplicated effort if you decide to avoid depending on other people's code**. MVS, or even better/worse vendoring everything, helps with stability but **then the entire development model goes against what distro maintainers expect** to be able to do: upgrade your deps when needed (because of security reasons or whatever). On top of that there is a fact to consider that is somewhat unique to Zig: there is a very high bar to clear for a Zig library to be considered high quality: it needs to work on a vast array of platforms, be compatible with various memory allocation schemes, and if it deals with I/O it also needs to account somehow for blocking and non blocking mode (and actually the same is true for anything that touches upon concurrency, non just I/O). That's a pretty big design space to cover and definitely not something that your average lib on GH usually has to deal with. # The proposal 1. **For packages < 1.0**, the package manager applies MVS, prioritizing stability and reproducibility of builds over everything else. 2. **Packages >= 1.0 can only depend on other >= 1.0 packages.** 3. **For packages >= 1.0**, the package manager selects the highest-available compatible version. In all these cases care should be taken to find a balance in terms of what minor/patch version restrictions a package can declare. This is an incomplete idea that should be fleshed out by looking at the small details, but, as a starting point, it aims to model the duality of software development: software is unstable and really offering a 1.0 experience in terms of stability and retro-compatibility requires a drastic shift of mindset. ### "But won't this mean that basically nobody is going to bump up major version to 1 ever?" Maybe, but if nobody wants to put in that kind of effort then wouldn't that be a lie anyway? ### "What is the point of forcing v1+ packages to only depend on other v1+ packages?" That's the most important part of the design. In a general sense, how can something be considered stable if it depends on unstable code? But more specifically, it's what makes it possible to have "version fluidity" among your dependencies. If we accept that v1+ software should be amenable to what distro maintainers do, you must preserve this property across the entire dependency chain. The nice thing is that is also makes sense in terms of other Zig goals, namely robustness and maintainability. In other words, stability is a transitive property in my opinion. ### "What if somebody erroneously introduces incompatibilities in a v1.x release?" Obviously, something will eventually go wrong, and we should have some kind of escape hatch. This is why there should be a way of specifying "upper" bounds when it comes to minor/patch versions, but it should not something that one routinely does. Go for example only allows specifying restrictions at the topmost level of the dependency chain (so when you are building the final application, basically), maybe that's something that we should take inspiration from for our v1+ model even though we don't use MVS there. I do believe that allowing all kinds of weird restrictions everywhere is bad in practice and fundamentally at odds with this model. ### "That's a ridicolously high bar to clear, how am I even supposed to make changes to my v1+ package without risking destabilizing everyone that depends on my package?" 1. Keep the scope and API surface of your package small. 2. Find out who your most important dependents are and either setup a CI that runs their tests against your changes or coordinate with them so that they do it. You need a big, serious, testing routine anyway to test different platforms etc. 3. Get to a point where you can consider your package "done" and only support bug fixes at that point. Alternatively: 1. Stay at v0.

Here’s what’s accepted. It’s slightly different than what is in the original proposal above:

Always MVS. This means that the package manager will only fetch dependencies that are explicitly listed (with their checksums), but it will pick the maximum version among those options.

Packages with versions >= 1.0.0 may not depend on packages < 1.0.0.

Incompatible major versions between the same packages within a dependency tree will be an error by the package manager, or it will just pick the latest version, even if it is incompatible, and rely on the package author to make any breaking changes compilation errors rather than runtime bugs.

“MVS” refers to Go’s Minimal Version Selection, described in this article. For this algorithm to work out in practice, it requires the use of a semver-like versioning scheme where minor version bumps are backward compatible with older versions of the same package. If a package makes a backward-incompatible change to its public API, it must increment its major version. If a package introduces a breaking change without bumping the major version, it should be considered a bug just like any regular bug introduced by incorrectly written code.

Don’t put too much aesthetic value into versioning. It is okay for the “informational” version (used for promotion/marketing) of your project to be “v2” or “2024-12” while its package version is 7.3.26 (due to it’s development history and breaking changes).

mnemnion · November 20, 2024, 3:02pm

It would be relatively straightforward to add a date as an additional field:

.released = "2024-11-11",

I don’t think Zig should have two ways to write a version, let alone more than that. SemVer isn’t perfect, and it isn’t a law of nature, but the advantages of having one concept of versioning across everything outweighs any minor shortcomings it has.

ajoino · November 20, 2024, 4:22pm

My issue is still that semver just doesn’t make sense to me on the package level. I think it makes sense for specific public interfaces, like function fooBar is version 2.0.0 which introduces a breaking change compared to fooBar 1.3.7. I do realize that it is functionally impossible to version all public APIs, but that is the only place a strictly enforced Semver system makes sense to me.

It’s not about aesthetic values, I don’t give two hoots what the versioning looks like for other’s packages, and I expect the same of the package manager. I also don’t want artificial breakage because something in my dependency chain was updated but the thing in that package that I depend on wasn’t. It’s not a problem right now, maybe it won’t be a problem at all, but my experience in the Python ecosystem packages who set upper bounds tend to be the worst to deal with.

Semver is just as good as any other versioning system, it’s just a tag where bigger number means newer. My problem is when it turns into an enforcement.