What embedded language would you implement in Zig projects?

I’ve been looking to work on top of an existing static site generator written in Zig. My idea is to implement custom filters as seen in Go’s text/template library, Jinja2 for Python and Tera for Rust. You could write these in Zig, but you can also allow others to add their own filters in another scripting language at runtime, provided they have bindings like Lua does.

Currently, I am lost with this in regards to Zig. I’ve seen a few bindings for other languages like Lua, Python, Ruby and even PHP in Zig, but I don’t think I’ve seen a project use any of them. So I want to ask what you would use instead, as you would likely know better than I do. I’d be open for arguments against as well.

Thanks as always! :smiley:

1 Like

For a variety of reasons, text-based filter systems like Terra can be really annoying since HTML is not a string, it is structured data.

If you are familiar with jsx (or web development with clojure/a lisp), having easy access to the structure of the data is really great. I’ve messed around a little bit with embedding janet inside zig for this purpose, and I found it to be quite fun and flexible with minimal work.

I’ve also used some fun comptime shenanigans to generate xml directly from normal zig structs:

// Self-contained zig file which generates SVG diagrams explaining isometric
// perspective.
// USAGE:
// zig run isometric.zig -- output_file.svg
// LICENSE: MPL2.0

const std = @import("std");
const Io = std.Io;

const indent_space_count = 2;
const viewbox = [4]f32{ -75.0, -75.0, 150.0, 150.0 };
const grid = .{
    .line_count = 8,
    .extents = .{ .min = -50.0, .max = 50.0 },
};

pub fn main() !void {
    var arena = std.heap.ArenaAllocator.init(std.heap.smp_allocator);
    const allocator = arena.allocator();

    const file_path = parse_args: {
        var args = std.process.args();

        // First arg is path to executable
        std.debug.assert(args.skip());

        break :parse_args args.next() orelse {
            std.log.err("Must provide an output file name", .{});
            std.process.exit(1);
        };
    };

    std.debug.assert(file_path.len > 0);

    const file = try std.fs.cwd().createFile(file_path, .{});

    var file_buffer: [4096]u8 = .{0} ** 4096;
    var file_writer = file.writer(&file_buffer);
    defer file_writer.interface.flush() catch {};
    _ = try file_writer.interface.write(
        \\<?xml version="1.0" encoding="UTF-8" standalone="no"?>
        \\<!-- isometric perspective -->
    );

    var svg: Xml = .{ .writer = &file_writer.interface };
    try svg.open("svg", .{
        .xmlns = "http://www.w3.org/2000/svg",
        .@"xmlns:svg" = "http://www.w3.org/2000/svg",
        .viewBox = viewbox,
    });
    defer svg.close("svg") catch {};

    {
        try svg.open("defs", .{});
        defer svg.close("defs") catch {};

        var grid_path =
            try Io.Writer.Allocating.initCapacity(allocator, 4096);
        defer grid_path.deinit();

        const width: f32 = grid.extents.max - grid.extents.min;
        for (0..grid.line_count + 1) |line| {
            const percent: f32 = @as(f32, @floatFromInt(line)) /
                @as(f32, @floatFromInt(grid.line_count));
            const line_pos = percent * width + grid.extents.min;
            _ = try grid_path.writer
                .print(
                "M {1},{0} h {2} M {0},{1} v {2}",
                .{ line_pos, grid.extents.min, width },
            );
        }

        try svg.voidTag("path", .{
            .id = "grid",
            .d = grid_path.written(),
            .stroke = "var(--grid-color, black)",
            .style = "stroke-width:1px",
            .@"vector-effect" = "non-scaling-stroke",
        });

        try svg.voidTag("path", .{
            .id = "unit-cube",
            .d = "M 0,0 l 0,1 1,0 Z",
            .stroke = "var(--cube-color, black)",
            .style = "stroke-width:1px",
            .@"vector-effect" = "non-scaling-stroke",
        });

        try svg.voidTag("circle", .{
            .id = "point",
            .fill = "var(--point-color, red)",
            .r = 4,
            .cx = 25 + 12.5,
            .cy = 25,
        });

        {
            try svg.open("marker", .{
                .id = "arrowhead",
                .orient = "auto",
                .makerUnits = "strokeWidth",
                .refX = 0,
                .refY = 0,
                .markerWidth = 4,
                .markerHeight = 6,
                .viewBox = [4]f32{ -3, -3, 4, 6 },
            });
            defer svg.close("marker") catch {};

            try svg.voidTag("path", .{
                .stroke = "context-stroke",
                .fill = "none",
                .d = "M-3 -3 L 0 0 L -3 3",
            });
        }

        {
            try svg.open("g", .{ .id = "basis" });
            defer svg.close("g") catch {};

            try svg.voidTag("line", .{
                .stroke = "var(--x-basis-color, red)",
                .style = "stroke-width:2px",
                .@"vector-effect" = "non-scaling-stroke",
                .@"marker-end" = "url(#arrowhead)",
                .x1 = 0,
                .y1 = 0,
                .x2 = 45,
                .y2 = 0,
            });

            try svg.voidTag("line", .{
                .stroke = "var(--y-basis-color, green)",
                .style = "stroke-width:2px",
                .@"vector-effect" = "non-scaling-stroke",
                .@"marker-end" = "url(#arrowhead)",
                .x1 = 0,
                .y1 = 0,
                .x2 = 0,
                .y2 = 45,
            });
        }
    }
}

const Xml = struct {
    writer: *Io.Writer,
    indent: u16 = 0,

    fn open(
        xml: *Xml,
        name: [:0]const u8,
        attr: anytype,
    ) std.Io.Writer.Error!void {
        _ = try xml.writer.writeByte('\n');
        _ = try xml.writer.splatByte(' ', xml.indent * indent_space_count);
        _ = try xml.writer.writeByte('<');
        _ = try xml.writer.write(name);
        try xml.attributes(attr);
        _ = try xml.writer.writeByte('>');

        xml.indent += 1;
    }

    fn voidTag(
        xml: *Xml,
        name: [:0]const u8,
        attr: anytype,
    ) std.Io.Writer.Error!void {
        _ = try xml.writer.writeByte('\n');
        _ = try xml.writer.splatByte(' ', xml.indent * indent_space_count);
        _ = try xml.writer.writeByte('<');
        _ = try xml.writer.write(name);
        try xml.attributes(attr);
        _ = try xml.writer.write(" />");
    }

    fn attributes(
        xml: *Xml,
        attr: anytype,
    ) std.Io.Writer.Error!void {
        const attr_info: std.builtin.Type = @typeInfo(@TypeOf(attr));

        inline for (attr_info.@"struct".fields) |field| {
            try xml.writer.print(" {s}=\"", .{field.name});

            const attr_value = @field(attr, field.name);
            // If the value is a string slice, print it
            if (comptime isString(field.type)) {
                _ = try xml.writer.print("{s}", .{attr_value});
            } else switch (@typeInfo(field.type)) {
                // TODO: wanted sub-struct to be CSS, but comptime'd too close
                // to the sun
                .@"struct" => @compileError("No struct attributes"),
                .void => {},
                // Assume u8 is string, otherwise print values
                .array => try xml.writeArray(@field(attr, field.name)),
                .pointer => |p| switch (p.size) {
                    .slice => _ = try xml.writeArray(attr_value),
                    else => switch (@typeInfo(p.child)) {
                        .array => try xml.writeArray(
                            attr_value[0..],
                        ),
                        else => _ = try xml.writer.print("{any}", .{
                            @field(attr, field.name),
                        }),
                    },
                },
                else => {
                    _ = try xml.writer.print("{}", .{@field(attr, field.name)});
                },
            }

            _ = try xml.writer.writeByte('"');
        }
    }

    fn writeArray(xml: *Xml, array: anytype) Io.Writer.Error!void {
        for (array, 0..) |item, i| {
            if (i > 0) _ = try xml.writer.writeByte(' ');
            _ = try xml.writer.print("{any}", .{item});
        }
    }

    fn close(xml: *Xml, name: [:0]const u8) std.Io.Writer.Error!void {
        // Must open before close to prevent underflow
        std.debug.assert(xml.indent > 0);
        xml.indent -= 1;
        _ = try xml.writer.writeByte('\n');
        _ = try xml.writer.splatByte(' ', xml.indent * indent_space_count);
        _ = try xml.writer.print("</{s}>", .{name});
    }
};

fn isString(t: anytype) bool {
    switch (@typeInfo(t)) {
        .pointer => |info| switch (info.size) {
            .slice => return info.child == u8,
            .one => switch (@typeInfo(info.child)) {
                .array => |array| return array.child == u8,
                else => return false,
            },
            else => return false,
        },
        .array => |array| return array.child == u8,
        else => return false,
    }
}

test isString {
    try std.testing.expect(isString(@TypeOf("a string literal")));
    try std.testing.expect(isString([:0]const u8));
    try std.testing.expect(isString([]const u8));
    try std.testing.expect(isString([]u8));
    try std.testing.expect(isString(*const [4]u8));
    try std.testing.expect(isString(*const [4:0]u8));
    try std.testing.expect(isString(*[8:0]u8));
}

It could be fun to extend this into full HTML to avoid really needing to use a template language at all and use normal zig, kinda like a “zsx”, but maybe not a great fit for a SSG.

If you are set on filter style templates, something like Terra/Jinja2, in my opinion, the filter syntax should probably take some inspiration from functional languages like haskell which is really built around the data in data out paradigm that suits templates well.


As an aside, I think for stuff like the above zig is unironically my favorite scripting language. It compiles so fast it even feels like running a script.

2 Likes

Honestly, I completely get that. I personally would rather use web frameworks like Preact, Vue and Svelte over classic static site generators. My personal website (well, the one I maintain the most) is written in Vue with the Nuxt.js framework, and I would prefer to maintain the Zola page I reserve specifically for public UNIX machines. Honestly, a web framework like Janet and Vue for Zig sounds really interesting, and would be something I’d look into once I can bend spoons with my Zig-fu.

However, I did not specifically choose to make an SSG. Rather, I am contributing to an SSG written in Zig called Makko. There is an issue in that repo for potentially adding Lua scripting support which caught my interest. That does not mean I am disinterested in a JSX-like format for Zig, but I’m just aiming to contribute to a project to learn Zig. :smiley:

2 Likes

An aside (now over email!), I want to expand a bit on Zig as a scripting
language. Zig itself is already fine for implementing custom filters for
SSGs as you have already stated. However, I wanted to tackle adding
these filters at runtime via extensions. While I believe that embedding
languages like Lua would help for the barrier of entry, it may also be
worthwhile to just provide an extension API in Zig, which could be used
by the software had those extensions been compiled into WASM beforehand.
However, that sounds kind of like a mess. I’d be interested in seeing
alternative approaches to loading extensions written in Zig though.

Yes, I misunderstood your original post a bit, my bad! And in my aside, I really meant that it was pretty fun to “script” in zig fun for 1-off stuff like generating those SVG files I needed to generate.

As for embedding languages in zig, I have only worked a bit with jzignet, and it was a pretty nice experience once I wrapped my head around the concepts. I would recommend following the guide from the janet for mortals book if you wanted to go that route, or finding a similar guide for embedding Lua in a C program.

2 Likes

Lua embeds nicely within Zig, by the way!

2 Likes

Another vote for Janet here:

I haven’t really used the language (other than confirm that janet is a single c file that compiles instantly), but, given what I’ve read about the underlying runtime, it’s very well-designed.

I would probably pick Lua if I need a safe choice everyone is using, and Janet if I am aiming at technical perfection.

3 Likes

No worries, I usually fail to get my point across the first time anyways.
Regarding one-off scripts, I specifically remember making a cowsay like program
for a bit in C, and honestly, I regret it because I could have wrote it in Zig
as a learning experience. Maybe next time I need to write a silly program, I
should write it in Zig to make it more worthwhile.

I’ve not heard of Janet until this conversation, and honestly it seems fun to
embed. I’ll write up an issue on that repository proposing that instead of
Lua, and see where that goes. I’ll still probably use Lua in the case that I
cannot though :^).

Which library would you say is best for embedding Lua? I’ve heard of ziglua,
which might be what I’m looking for, but there are also libraries like
zig-luajit which may also be what I am looking for. I think it’s best if I
research and then come to my own conclusion but hearing others’ opinion would
be nice for certainty.

Lua is indeed very accessible. However, sacrificing it may probably lead to a
more quality ecosystem and nicer API, so experimenting with Janet instead of
Lua may be a risk worth taking. I don’t know enough about the Janet runtime,
so I’ll consider it equal to Lua until I can find a good source on it. Maybe
I’m not looking hard enough though :(.

That’s a safe bet if you wanna go with Lua, as it’s used by quite a few projects, including an irc client I use (comlink). neovim has it as a dependency too.

(If you wanna go wild you can also do your own language for the task at hand, like Zine does, also a Zig-based ssg)

1 Like

I see, thanks for sharing! I was thinking that I should probably use ziglua for
embedding Lua in my Zig projects, but I’m glad to be certain now. I honestly
only learned today that Neovim uses ziglua, but alongside that, knowing other
projects use it helps as well.

I’ve written a small wrapper myself with comptime facilities.

I’ve put it here https://codeberg.org/mathieu_cayeux/lua.zig

Note that I update it semi-regularly as it is a dependency on one of my personal projects. It has no additional allocations on the critical paths, and should be small enough to be easily modifiable.

You can just go and copy the needed functions that use comptime for wrapping a function and stuff like that. It is under MIT so feel free to copy anything.

1 Like

I think Zig deserves a native scripting language. I know that doesn’t help you right this instant, but anything which has to go through the C ABI is crippling what Zig is capable of, especially given comptime.

I’ve got notes! But you can’t run notes.

In the meantime, there’s Wren. Of the not-mentioned options it’s the one I want to draw your attention to. It’s… not Lisp-shaped, and it doesn’t do the things which those what don’t like Lua don’t like about Lua (I’m not one of those, but, I understand).

2 Likes

I’m not sure if this counts as the typical “embedding a scripting language”, but here is a little lisp that runs at comptime to synthesize zig code using comptime trickery. Early experimentation phase but hey it’s working :slight_smile:

Disclaimer: AI was used but not harmed in the creation of this.

2 Likes

If you can’t script the deployed binary application it doesn’t count as embedded.

What’s the use case for compile-time lisp?

I’ve been working for the last year and a half on my own statically typed scripting language Ray but it’s not finished yet… But it’s main goal is to be embeddable both in Zig and C programs and it can already be.
The problem is that it lacks documentation and examples, though they are some tests that show how to do it and the specification is mostly up-to-date (in doc/ folder).

When embedded in Zig, you can define your own structures and functions with native Zig types (like the builtin File structure src/core/builtins/file.zig or builtins.zig) and comptime magic will handle the calls.
A little example of embedding + registering custom function + executing code through a string can be found in this tester.

If you’re interested we could discuss more about your use case as it would help me to make it evolove with real word program :slight_smile:

4 Likes

This is very cool. I hope you declare victory and do a showcase soon, this deserves to be better known.

2 Likes

Hey, your Ray language is very close to what I would like my Moin language to be if I just had one year or two of spare time to work on it full-time.

Indeed very cool!

I’ll certainly steal ideas from your implementation. Right now I am at the point where I wonder if I should go down the rabbit hole of adding enforced type hints etc.

Am I right that you started your journey with Lox, more or less?

Can you share a bit more insight into the implementation and the performance and about the comp time FFI magic?

Maybe some of the moderators here can split this thread and create a new one for talking about Ray?

2 Likes

One mod has already encouraged the author to start one. :slight_smile: let’s not jump the gun.

1 Like