ZON support merged

weskoerber · February 3, 2025, 8:53pm

ZON

ziglang:master ← MasonRemaley:zon

opened 01:14AM - 12 Jun 24 UTC

# ZON This PR implements ZON, or Zig Object Notation (think JSON, but with Zig …syntax instead of Javascript.) In particular, it implements: * Runtime parsing of ZON data * Runtime serialization of ZON data * Compile time importing of ZON data A lot of files are added since I added a lot of test cases, the bulk of the work is in `src/zon.zig` and `lib/std/zon`. If there's anything I can do to make this easier to review, let me know. Tests cover all new features. ## Runtime The runtime code can be found at `lib/std/zon`. ### Parsing `lib/std/zon/parse.zig` Detailed doc comments are provided in the source. At a high level, this module provides the following functions and some configuration options: * `std.zon.parse.fromSlice` * `std.zon.parse.fromZoir` * `std.zon.parse.fromZoirNode` * `std.zon.parse.free` Most people will just want `std.zon.parse.fromSlice` and maybe `std.zon.parse.free`, but the more fine grained functions are available for those who need them. ### Stringify `lib/std/zon/stringify.zig` Detailed doc comments are provided in the source. At a high level, this module provides the following functions and some configuration options: * `std.zon.stringify.serialize` * `std.zon.stringify.serializeMaxDepth` * `std.zon.stringify.serializeArbitraryDepth` * `std.zon.stringify.serializer` Most users will just need `serialize`, or one of its variants. However, `std.zon.stringify.serializer` returns a more fine grained interface that is used in the implementation of the other functions. It may be used directly if you need to serialize something piece by piece--perhaps to apply different configuration to different parts of the value, or because the value does not actually exist laid out in memory in the same form in which you want to stringify it. ## Compile Time See `src/zon.zig`. This PR implements importing ZON at compile time: ```zig const Foo = @import("foo.zon"); ``` ## Things that may change later... ### Untagged nonexhaustive enum values not yet supported Zig does not currently have a way to represent an untagged enum value as a literal, and as such there's no way to represent one in ZON. We could resolve this by adding a syntax for untagged enum literals to Zig, or by allowing ZON to coerce integers to untagged enum literals (provided the destination type is known.) Personally I prefer the former solution. * [ ] After this PR is merged I'll file an issue to track this if still relevant. ### Allowing eliding outer braces may make sense There's case to be made to allow eliding braces on an outer struct in ZON, like this: ```zig .x = 10, .y = 20, ``` * [ ] I'll file an issue with some motivating use cases after this is merged, easy change if we decide to make it ### Divergences from Zig Zig has no NaN or inf literals right now. Zon does, since otherwise it would have no way to represent these values. IMO we should add the same literals to Zig, but there are good arguments not to as well, so I'm not trying to address that here. * [ ] After this PR is merged I'll file an issue to track this. ## See Also * #15552 * #17731 * #14531 * #14532 * #14534 * #5039 * #14532 * [x] #20275 (fixed, can probably parse directly as dest types now) * #20277 * #20880 * #20881

ZON support was merged today

I’ve been keeping my eye on this PR for quite some time. Big thanks to MasonRemaley for implementing, and everyone else in the PR who contributed!

andrewrk · February 3, 2025, 10:11pm

If you’re wondering what’s the point of this, there are a few talking points here which I’ll make sure land in the release notes, but I’ll leave you with this one:

You can directly @import .zon files, which skips parts of the compiler pipeline, potentially reducing compilation times, especially for large data sets.

For example, x86_64: use ZON for encodings by mlugg · Pull Request #22736 · ziglang/zig · GitHub

It remains to be seen whether this is true in reality, we’ll see what @mlugg reports after finishing that follow-up caching work he mentions in the linked PR.

It’s also going to be handy to @import("build.zig.zon") from build.zig scripts.

mlugg · February 3, 2025, 10:49pm

FWIW, my local testing without the caching in place shows negligible performance difference building the compiler with encodings.zig vs encodings.zon; it might just be that this part of the compiler is faster than we give it credit for! But I’ll update the PR with the data I get once I sort out caching. (I’m currently a few more yaks deep, but we’ll get there.)

cryptocode · February 3, 2025, 11:16pm

I’d like to replace my makeshift build.zig.zon parser with the new zon support.

What’s the suitable type for this?

The PR indicates this should be possible:

const zon = @import("build.zig.zon");

which yields error: '@import' of ZON must have a known result type

I got it to work without dependencies in the zon file, but what would the type be for the dependencies field?

const zon: struct {
    name: []const u8,
    version: []const u8,
    paths: []const []const u8,
    dependencies: ???
} = @import("build.zig.zon");

my initial attempts with tuples have failed. Suggestions welcome!

When using @import(“my.zon”), I noticed is that changes to the zon file (one I have in src/) does not cause recompilation, so the cached result is used. Turning the zon file into a module fixes that - this is perhaps by design?

mlugg · February 3, 2025, 11:35pm

Hm, that’s actually a good point – build.zig.zon has a slightly weird schema which you can’t actually model in Zig’s type system. In theory I guess it should be allowed to @import ZON without a result type (and hence infer the type like a Zig literal would) but that’s a little tricky to hook up properly with incremental etc.

I’ll try to get this implemented for 0.14.0, but no promises…

MasonRemaley · February 4, 2025, 6:04am

When using @import(“my.zon”), I noticed is that changes to the zon file (one I have in src/) does not cause recompilation, so the cached result is used. Turning the zon file into a module fixes that - this is perhaps by design?

This sounds like a bug, I’ll look into it.

[EDIT] I have a PR that fixes the caching bug here, but it probably needs review from someone familiar with how caching works to make sure it’s correct.

floooh · February 4, 2025, 9:05am

You can directly @import .zon files

Oh nice, I wanted this for my Z80 CPU emulator to describe the instruction decoding and wondered why I can’t simply import a .zon file (eventually I went for a regular Zig file with functions with declarative-looking bodies)