ZON Extensions

Brainstorming
1

Hello and welcome to my shrine to feature creep.

Imagine if you could do this in a ZON file:
Builtins in ZON:

.{
    .birds = @import("birds.zon"),
    .apples = @import("apples.zon"),
}

or this:
Functions in ZON:

.{
    .username = "kj4tmp",
    .joined = date("Jan 2, 1970"),
}

Which could be parsed with an API like this:

pub const User = struct {
    username: []const u8,
    joined: Date,
};

pub const Date = struct {
    year: i64,
    month: u8,
    day: u8,
};

fn dateExtension(date_str: []const u8) Date {
    _ = date_str;
    @panic("not implemented");
}

pub fn main() !void {
    const user_zon: []const u8 = try readFile("user.zon");
    const user: User = try zon.parseFromSliceWithExtensions(
        User,
        user_zon,
        .{
            .function_extensions = .{
                .date = dateExtension,
            },
        },
    );
}

The official ZON specification could define the behavior of builtins, and the functions could be how specific applications add extensions.

Some questions:

  1. Is this better than YAML? Trick question, anything is better than YAML. (I’m looking at you, bamboo-specs, github actions, etc. and your !include directives)
  2. Do you see ZON as a good option for configuration files?
2

When writing the code generator tool for my Z80 emulator I was also a bit surprised about the differences between Zig and Zon, I always thought of Zon to be a subset of Zig (e.g. renaming a .zon file to .zig would allow me to use additional features not allowed in Zon, like using function calls like in your example - but I didn’t get very far with that.

I ended up with ‘declarative looking code’ like this:

…plus a hardwired switch like this:

…I would have preferred a pure data format like this (but .zon instead of .yaml):

(note the ‘cond’ field which are evaluated via Python’s eval() - those basically replace the switch in the Zig version)

1 Like
3

I have also missed functions in zon sometimes.
I especially miss the expressiveness of function declarations. For example in one case I have this hierarchical definition, where functions would allow me to make a much simpler and easier to read expression:

.restriction = .{
	.id = .not,
	.child = .{
		.id = .encased,
		.tag = .precious,
		.amount = 4,
	},
},
vs
.restriction = .not(.encased(4, .precious)),
// Reads much nicer: not encased in 4 precious

I think you would want decl literals though instead of arbitrary global functions.

The one big counter-argument to all of this is however simplicity. I think a big strength of zon is that it is as easy to parse as json, functions would make it more complicated. And you don’t strictly need functions, you can express all of these functionalities in current zon, it is up to you to interpret certain things as function calls during parsing.
In this particular example you could actually just get away with .joined = "Jan 2, 1970", you just need to interpret strings in date fields as calls to the date functions during parsing.

5 Likes
4

This reminds me of:

Which would allow you to write:

.username = "kj4tmp",
.joined = @date("Jan 2, 1970"),

I am not so sure whether I like the idea of having includes in data files, seems like you can easily get back to a mess of files with that again.

But technically you could already write @import("apples.zon"), however I don’t know whether that would make a lot of sense with ziggy, because there it would just be a custom string literal, so you would have to interpret the meaning of it on the read side manually (or add it as a hack to ziggy, which probably wouldn’t fit its more strict/rigorous structuring, e.g. ziggy schema seems stricter than json schema (which is a good thing, but wouldn’t make it easy to hack in includes in a general way)).

However if you don’t actually need general includes, just sometimes have to refer to some external file in some places, then you could do that in Ziggy by using a union where one element type is some concrete type and another is a reference type that just describes the path or id to the external file.

3 Likes
5

The curse of declarative config files… the day always comes when it wants to become a real programming language. :slight_smile:

3 Likes
6

I cant believe I didnt think of decl literals, thats definitely better than global functions!

7

I disagree. Zon isn’t a configuration language, it’s a serialization format of a Zig literal.

That is, Zon is both machine readable and writable. A critical use case is a program reading a Zon file, making a change, and then saving back to that Zon file. Such a program making a change would either need to track additional state around where functions or references are used (defeating the simplicity of working with simple Zig values), or it would clobber such hand crafted statements while editing other information.

For example, you could imagine a video game level editor leveraging Zon. It would load a level.zon, the user could make edits, then save the file. On compilation of the game the level.zon is then @import’d , and baked directly into the code. Such a system has many advantages:

  • The translation into code objects is trivial.
  • Diffs in change lots have a hope of being human readable, and may be mergable. (unlike binary formats)
  • Changes to the format can be handled through algorithmic editing, or hand editing like you’d do for any other literal.

If your data is going to be only edited by hand, and you want additional functionality, then you actually just want a Zig file. If you then want to pass this as a serialized format to a running program as a configuration format: Have your declaration exist inside of a Zig program which emits that declaration as a Zon file, and use that as a build step. You can use the build system to make this very ergonomic and scalable. Zon may not even be the ideal output fromat, depending on constraints such as size, de-serialization speed, and human readability.

You could write your own configuration language or use an existing one if the ergonomics of a configuration language are what you want. However, be warned: You start with literal values. Then users ask for functions. “Oh how about importing.” “Hey can I please have loops and conditionals?” “I really have a strong use case for templating.” All configuration languages converge to programming languages, but worse. Just use a programming language. Please, never waste my time with formatting functions to do indenting to get a yaml file to template properly.

Also keep in mind that you have the full power of Zig at hand. The values [de]serialized by Zon can be generics, where comptime turns easy to edit/serialize values into other values. Eg, (untested, rough code sketch)

fn MyConfigType(comptime baked: bool) type {
  return struct  {
    start_date: if (baked) Date else []const u8,
    end_date: if (baked) Date else []const u8,
  };
}

fn bake(InType: type, OutType: type, v: InType ) OutType {
  if (InType == OutType) return v;
  // if struct, recursively call bake on each field.
  if (InType == []const u8 and OutType == Date) {
    return Date.parse(v) catch unreachable;
  }
  // Other translation cases
  unreachable;
}

const my_baked_value = bake(MyConfigType(false), MyConfigType(true), @import("my_config.zon"));

Not that I would necessarily suggest the above for something as simple as dates, but I think that technique in the broad sense does have some uses where an editor and your application code want slightly different types.

10 Likes
8

Maybe off topic. Last year I discovered NestedText, a simple format with only maps, list and strings (not even numbers, booleans, etc.). It doesn’t want to become a programming language.

1 Like