What's everybody working on?

Together we serve the users :saluting_face:

and my job security because I’m the only one who knows Zig :laughing:

16 Likes

Looking great!

1 Like

I’m making a platform for games (and subset of graphical apps I guess). While the ecosystem is written mainly in zig, non zig projects can target it.

The idea is very similar to libretro, but major difference from libretro is that I also specify the ABI that the game binary and the frontend that loads your game communicates with. This means that as long as there exists a native frontend that can run your game binary, the operating system does not matter as the game binary itself is a freestanding binary. This also has security implications as the game binary can’t make any syscalls and any calls that depend on underlying platform has to go through the frontend.

The platform is based on extensions similar to vulkan / opengl. The extensions are specified in subset of zig and there’s code generator with backends for different languages. (Currently only zig and js).

I’ve written pure javascript web frontend as the weirdest platform to test this idea and so far I’ve had success porting even more complex codebases like supertux on it. When porting a existing C/C++ projects, I have a libc implemented on top of this platform. For C/C++ projects it’s as if you were programming against posix system.

The main repository is still closed, but I’ll open it once I have first stable extension set that I feel confident with.

Currently there are extensions for vulkan, gles, rasterizing, sandboxed file system access, audio, video timings / display management, threads (very wip), keyboard / mouse input, memory allocation.

The reference native frontend (based on SDL3) makes it so vulkan and gles simply work on any OS without the developer having to think about it (leveraging ANGLE for GLES where it makes sense, and vulkan translation layers similarly).

Example how a GLES program for this platform would look like, using the platform api (extensions) directly:

const std = @import("std");
const sorvi = @import("sorvi");
const gl = @import("gles");
const log = std.log.scoped(.gles_example);

pub const std_options: std.Options = .{
    .logFn = sorvi.defaultLog,
    .queryPageSize = sorvi.queryPageSize,
    .page_size_max = sorvi.page_size_max,
    .log_level = .debug,
};

pub const os = sorvi.os;
pub const panic = std.debug.FullPanic(sorvi.defaultPanic);

comptime {
    sorvi.init(@This(), .{
        .id = "org.sorvi.example.gles",
        .name = "gles-example",
        .version = "0.0.0",
        .core_extensions = &.{ .core_v1, .video_v1, .kbm_v1 },
        .frontend_extensions = &.{ .core_v1, .mem_v1, .gles_v1 },
    });
}

procs: gl.ProcTable = undefined,
shader: u32 = 0,
ns_since_last_update: u64 = 0,

fn compileShader(kind: enum { vertex, fragment }, src: []const u8) !u32 {
    const shader = gl.CreateShader(switch (kind) {
        .vertex => gl.VERTEX_SHADER,
        .fragment => gl.FRAGMENT_SHADER,
    });
    errdefer gl.DeleteShader(shader);

    gl.ShaderSource(shader, 1, &.{src.ptr}, &.{@intCast(src.len)});
    gl.CompileShader(shader);

    var compiled = [_]i32{0};
    gl.GetShaderiv(shader, gl.COMPILE_STATUS, &compiled);
    if (compiled[0] == 0) {
        var buf: [1024:0]u8 = undefined;
        gl.GetShaderInfoLog(shader, buf.len, null, &buf);
        log.err("{}, {s}", .{ shader, buf });
        return error.ShaderCompileError;
    }

    return shader;
}

pub fn init(self: *@This()) !void {
    const ctx = try sorvi.gles_v1.init(.{
        .required_extensions = .empty,
        .context_major_version = 2,
        .context_minor_version = 0,
        .red_size = 0,
        .green_size = 0,
        .blue_size = 0,
        .alpha_size = 0,
        .depth_size = 0,
        .stencil_size = 0,
        .multi_sample_buffers = 0,
        .multi_sample_samples = 0,
        .srgb_capable = .dont_care,
        .context_no_error = false,
        .float_buffers = false,
    });

    const proc: *const fn (name: [*:0]const u8) callconv(sorvi.abi.ccv) gl.PROC = @ptrFromInt(ctx.proc_addr_fn);
    if (!self.procs.init(proc)) return error.GlesInitFailed;
    gl.makeProcTableCurrent(&self.procs);

    log.info("GL_VENDOR: {s}", .{gl.GetString(gl.VENDOR) orelse "unknown"});
    log.info("GL_RENDERER: {s}", .{gl.GetString(gl.RENDERER) orelse "unknown"});

    const vertex_src: []const u8 =
        \\attribute vec4 pos;
        \\void main() {
        \\  gl_Position = pos;
        \\}
        \\
    ;

    const fragment_src: []const u8 =
        \\precision mediump float;
        \\void main() {
        \\  gl_FragColor = vec4(1.0, 1.0, 1.0, 1.0);
        \\}
    ;

    const vertex_shader = try compileShader(.vertex, vertex_src);
    defer gl.DeleteShader(vertex_shader);
    const fragment_shader = try compileShader(.fragment, fragment_src);
    defer gl.DeleteShader(fragment_shader);

    self.shader = gl.CreateProgram();
    gl.AttachShader(self.shader, vertex_shader);
    gl.AttachShader(self.shader, fragment_shader);
    gl.BindAttribLocation(self.shader, 0, "pos");
    gl.LinkProgram(self.shader);

    var linked = [_]i32{0};
    gl.GetProgramiv(self.shader, gl.LINK_STATUS, &linked);
    if (linked[0] == 0) return error.ShaderLinkError;

    const vertices: []const f32 = &.{
        0.0,  0.5,  0.0,
        -0.5, -0.5, 0.0,
        0.5,  -0.5, 0.0,
    };

    var vbo = [_]u32{0};
    gl.GenBuffers(1, &vbo);
    gl.BindBuffer(gl.ARRAY_BUFFER, vbo[0]);
    gl.BufferData(gl.ARRAY_BUFFER, vertices.len * @sizeOf(f32), vertices.ptr, gl.STATIC_DRAW);

    gl.UseProgram(self.shader);
    gl.VertexAttribPointer(0, 3, gl.FLOAT, gl.FALSE, 0, 0);
    gl.EnableVertexAttribArray(0);
}

pub fn deinit(self: *@This()) void {
    gl.makeProcTableCurrent(null);
    sorvi.gles_v1.deinit();
    self.* = undefined;
}

pub fn videoTick(self: *@This(), frame: sorvi.video_v1.frame_t) !u64 {
    const target_rate: u64 = std.time.ns_per_s / 60;
    self.ns_since_last_update += frame.time_ns;
    while (self.ns_since_last_update >= target_rate) {
        self.ns_since_last_update -= target_rate;
    }
    gl.Viewport(0, 0, frame.w, frame.h);
    gl.Clear(gl.COLOR_BUFFER_BIT);
    gl.DrawArrays(gl.TRIANGLES, 0, 3);
    sorvi.gles_v1.swap();
    // render as fast the frontend can
    return 0;
}

pub fn kbmKeyPress(_: *@This(), _: u64, _: sorvi.kbm_v1.absolute_t, mods: sorvi.kbm_v1.modifiers_t, code: sorvi.kbm_v1.scancode_t) !void {
    const fcode: f32 = @floatFromInt(@intFromEnum(code));
    gl.ClearColor(@mod(fcode, 32.0) / 32.0, @mod(fcode, 64.0) / 64.0, @mod(fcode, 128.0) / 128.0, 1);
    log.info("kbm key {} pressed", .{code});
    log.info("kbm mods {}", .{mods});
}

pub fn kbmKeyRelease(_: *@This(), _: u64, _: sorvi.kbm_v1.absolute_t, _: sorvi.kbm_v1.modifiers_t, _: sorvi.kbm_v1.scancode_t) !void {}

pub fn kbmButtonPress(_: *@This(), _: u64, pos: sorvi.kbm_v1.absolute_t, _: sorvi.kbm_v1.modifiers_t, button: sorvi.kbm_v1.button_t) !void {
    log.info("kbm button {} pressed at {}", .{ button, pos });
}

pub fn kbmButtonRelease(_: *@This(), _: u64, _: sorvi.kbm_v1.absolute_t, _: sorvi.kbm_v1.modifiers_t, _: sorvi.kbm_v1.button_t) !void {}

pub fn kbmMouseMotion(_: *@This(), ts: u64, _: sorvi.kbm_v1.absolute_t, _: sorvi.kbm_v1.modifiers_t, rel: sorvi.kbm_v1.relative_t) !void {
    const static = struct {
        var last: u64 = 0;
    };
    if (ts - static.last > 200 * std.time.ns_per_ms) {
        log.info("kbm motion {}", .{rel});
        static.last = ts;
    }
}

pub fn kbmMouseScroll(_: *@This(), _: u64, _: sorvi.kbm_v1.absolute_t, _: sorvi.kbm_v1.modifiers_t, relative: sorvi.kbm_v1.relative_t) !void {
    log.info("kbm wheel {}", .{relative});
}
9 Likes

@Cloudef I confess that I struggle to understand the details of your platform, but your post made me want to talk about my somewhat related hobby project.

When time allows, I work on a linux x86_64 application framework that aims to realize a long-standing goal i have: distributing a graphical application as a single, fully dependency-free executable that works everywhere (on linux x86_64) and makes no compromises in using the user’s hardware. As such, this executable should be static, should not use libc, should not require the user to install any packages, and should still be able to use, for example, a GPU if available (without requiring one), and run without modification on X11 and Wayland. To achieve this, I rely on the custom dynamic library loader I started last year.

The final goal is a framework that provides many common utilities for applications or games, such as:

  • windowing
  • audio
  • gamepads / tablets
  • layout
  • GUI components
  • 2D and 3D graphics API abstractions

I have already achieved support for x11 and wayland, both with software rendering and egl + gles2, and with keyboard (including proper international support) and mouse handling.

Here is a screenshot of the same (static!) executable running in my x11 session and in the weston wayland compositor, using egl + gles2:

The associated code:

const std = @import("std");

pub const debug = struct {
    pub const SelfInfo = dappfw.CustomSelfInfo;
};

const dappfw = @import("dappfw");

pub fn main(init: std.process.Init) !void {
    const allocator = init.gpa;
    const io = init.io;

    try dappfw.init(allocator, io, init.minimal, .{ .app_name = "DAppFW Showcase" });
    defer dappfw.deinit();

    const window = try dappfw.createWindow(.{
        .dm_type = .auto_prefer_wayland,
        .r_type = .auto_prefer_egles2,
        .title = "DAppFW Showcase",
        .width = 800,
        .height = 400,
        .quit_on_close = true,
    });

    dappfw.showWindow(window);

    const root_surface = dappfw.getWindowRootSurface(window);

    while (!dappfw.shouldQuit()) {
        // `true` here means "block until events are available"
        try dappfw.collectEvents(.all, true);

        if (dappfw.keyboardKeyPressed(window, .Q)) {
            break;
        }

        // draw
        try dappfw.clearSurface(root_surface, .black);

        try dappfw.fillCircleOnSurface(
            root_surface,
            dappfw.getSurfaceCenter(root_surface).toFPoint(),
            @floatFromInt(dappfw.getSurfaceMinDim(root_surface) / 12),
            .red,
        );

        try dappfw.drawLineOnSurface(
            root_surface,
            .{ .x = @floatFromInt(dappfw.getSurfaceWidth(root_surface)), .y = 0 },
            .{ .x = 0, .y = @floatFromInt(dappfw.getSurfaceHeight(root_surface)) },
            .white,
        );

        if (dappfw.getWindowPointerPosition(window)) |pos| {
            try dappfw.fillCircleOnSurface(
                root_surface,
                pos.toFPoint(),
                10,
                .blue,
            );
        }

        // present
        try dappfw.flush(.all);
    }
}

Right now, the dynamic library loader is used to load these libraries on demand:

  • libX11.so.6
  • libXext.so.6
  • libXi.so.6
  • libxkbcommon.so.0
  • libGLESv2.so.2
  • libwayland-client.so.0
  • libwayland-egl.so.1
  • libwayland-cursor.so.0
  • libwayland-server.so.0
  • libEGL.so.1

…with many more to come. The only rule for a library to be eligible is its general availability in desktop environments. Other rules I want to enforce are:

  • the same visuals and behavior, regardless of the selected backend
  • full control over the application loop, with no callback registration (I might regret that later…)
  • no bias toward immediate or retained mode for GUI
  • .dependencies = .{} in build.zig.zon
  • zig as the only language allowed to be compiled

The next step is audio. I will not lie, I am already behind where I expected to be at this point in the project.

11 Likes

What I’m doing is not that much different from a wine. The only thing I’m doing really is making smaller and more saner ABI for games / graphical apps. The ABI surface for typical graphical app (especially win32 app) is absurd.

The end goal is to make it so that it’s easy to develop both frontends and the graphical programs the frontends can load. This in contrast makes it easy to load and run graphical programs on weird environments or even a completely new OS. There are other things that also become irrelevant, like how program loads vulkan without depending on the “system” libc, which makes distributing binaries annoying on linux for example, especially on weird distros without FSF hierarchy.

This is the ABI surface for my supertux port for example:

                 U sorvi_audio_v1_cmd
                 U sorvi_audio_v1_deinit
                 U sorvi_audio_v1_init
                 U sorvi_core_v1_exit
                 U sorvi_core_v1_panic
                 U sorvi_core_v1_set_state
                 U sorvi_core_v1_tty_writev
                 U sorvi_ext_query
                 U sorvi_fs_v1_close_dir
                 U sorvi_fs_v1_close_file
                 U sorvi_fs_v1_delete_dir
                 U sorvi_fs_v1_delete_file
                 U sorvi_fs_v1_iterate
                 U sorvi_fs_v1_iterate_finish
                 U sorvi_fs_v1_iterate_next
                 U sorvi_fs_v1_open_dir
                 U sorvi_fs_v1_open_file
                 U sorvi_fs_v1_readv
                 U sorvi_fs_v1_seek
                 U sorvi_fs_v1_stat
                 U sorvi_fs_v1_writev
                 U sorvi_gles_v1_deinit
                 U sorvi_gles_v1_init
                 U sorvi_gles_v1_swap
                 U sorvi_kbm_v1_lock_pointer
                 U sorvi_kbm_v1_unlock_pointer
                 U sorvi_mem_v1_alloc
                 U sorvi_mem_v1_free
                 U sorvi_mem_v1_page_size
                 U sorvi_mem_v1_remap
                 U sorvi_mem_v1_resize
                 U sorvi_raster_v1_damage
                 U sorvi_raster_v1_deinit
                 U sorvi_raster_v1_init
                 U sorvi_thread_v1_detach
                 U sorvi_thread_v1_futex_timed_wait
                 U sorvi_thread_v1_futex_wait
                 U sorvi_thread_v1_futex_wake
                 U sorvi_thread_v1_id
                 U sorvi_thread_v1_join
                 U sorvi_thread_v1_spawn
                 U sorvi_thread_v1_tls_get_address
                 U sorvi_thread_v1_yield
                 U sorvi_time_v1_monotonic
                 U sorvi_time_v1_realtime
                 U sorvi_time_v1_sleep
                 U sorvi_video_v1_configure
                 U sorvi_video_v1_query_display_modes
                 U sorvi_vulkan_v1_deinit
                 U sorvi_vulkan_v1_init

You can play it on the web frontend here: https://cloudef.pw/sorvi/#supertux.sorvi
requires javascript.options.wasm_js_promise_integration enabled on firefox

Zig makes it easy to experiment with this project, because it has such a good freestanding support. Ideally compilers would have target support for this platform though like x86_64-sorvi wasm-sorvi and so on … One compiler related blocker I currently have is that this platform requires emutls support on compiler for thread locals and zig doesn’t have that for zig compilation units currently (well kinda does for some targets, but it is hardcoded to a target).

Here is same binary drawing vulkan triangle running on linux and windows (wine)

5 Likes

I only started learning Zig this year, so I built a tool similar to ls—though it’s probably closer to lsd or eza, since some of the flags are custom.

I mainly wrote it to learn Zig, and also to get a better understanding of file systems and memory allocation. Here’s the repo: GitHub - here-Leslie-Lau/zlist: A modern ls alternative written in Zig. · GitHub

6 Likes

I’m working on a tool that can take a full export of your instagram data and turn it into a static site: https://codeberg.org/edwardloveall/photostatic

It does work, but there are effectively no instructions and the api still has a bit to go. It also requires a zola template that I don’t think I’ve published, so the site doesn’t build.

But other than that, it works! :smiley:

5 Likes

I made a tool to check for broken links in markdown documents: Markdown Toolkit (“mdt”).

Example Usage

demo_dir="$(mktemp -d)"
cd "$demo_dir"
echo "[check this out](https://example.com)" > "good_web.md"
echo "[no good](https://nothing.example.com)" > "bad_web.md"
echo "[that doc has good web links](./good_web.md)" > "good_file.md"
echo "[this doc does not exist](./nothing.md)" > "bad_file.md"
mdt
echo "first time exit code: $?"
rm bad_*
mdt
echo "second time exit code: $?"

Expected output;

bad refs in bad_file.md :: ./nothing.md
bad refs in bad_web.md :: https://nothing.example.com/
first time exit code: 1
second time exit code: 0
6 Likes

I have a toy project or two, for learning zig, but my primary goal is an esp32 application that listens to a student violinist and offers real-time audio feedback. I implemented a non-recursive custom FFT in zig, as a part of this effort, but have a very long way to go, since I’m a slow-poke.

6 Likes

This is fascinating. Are you planning on checking pitch within a margin of error against an encoding of a piece? Will the esp32 provide a metronome to play along with to help track? Sounds like a really fun project!

1 Like

I plan on something similar for my musikus. where I capture the audio input and determine the note. So that instead of typing note names, you can play them on any instrument. But it is a long way to go.

2 Likes

OK, I see. Cool idea!

Interesting question. While the answer is probably yes, ultimately, exactly the opposite is a main interest. Tempo is an essential skill, but with fretless instruments, tone is tough, and a student sometimes needs to dwell on a tone… find it… rely on some feedback (often, a teacher playing along, at the right pitch)… and move on seemlessly, to not lose (too much) track of the melody.

Basically, yes. And playing the proper pitch, as a reference.

1 Like

3 posts were merged into an existing topic: The removal of std.Thread.mutex and std.Thread.futex from the synchronization primitives saddens me

This looks so cool, but if you execute a native binary when you are running a game, I don’t understand how you can make sure the binary can’t execute syscalls native to the host platform and escape the sandbox ?

To be honest, it’s not possible on all operating systems, but on cool ones like linux it’s possible with seccomp-bpf and openbsd won’t let you do syscalls from untrusted memory locations by default(!). For other platforms you can mostly do best effort with what’s available (like restrict file system access), there may also be some slower approaches you could do like launching the frontend inside debugger process that breaks on all syscalls and checks that they are not made from inside the game binary.

2 Likes

Loving Zig 0.16.0 – Phenomenal release! Thank you!

I’ve been hacking on Nexus — a self-hosting parser generator in Zig that turns a .grammar file into a standalone parser.zig (a high performance combined lexer + parser → S-expression generator). No runtime, no support library.

The part I like most: the grammar file is also the AST spec. Actions like (assign 1 3) written next to a rule are the S-expression the parser emits — no visitor classes, no AST type file, no builder boilerplate. Any language becomes a Lisp-shaped tree you can pattern-match, rewrite, or transpile downstream in a few hundred lines.

Bonus nerdy detail: the frontend that reads .grammar files is itself Nexus-generated from nexus.grammar, with a CI fixed-point test to keep the bootstrap honest. Used for Zag, Slash, and MUMPS so far (MUMPS being the stress test — 500+ rules, 800+ states, generates in ~30 ms).

Zig 0.16, MIT: https://github.com/shreeve/nexus

9 Likes

I’ll join team “tricked my company into adopting Zig”. :wink:

I maintain an embedded Linux application. .zig has now replaced every instance of .cmake, .py, .sh in the codebase, and about half of our C code. .zon has replaced every human-maintained config file.

We went from a whole mess of host system dependencies and weird tools to needing just zig, tar, and git to build and deploy in testing. The device went from a whole mess of system dependencies to the Linux kernel, 1 closed-source C library from a hardware vendor, and 3 well-known open source C libraries.

I’m slowly pushing it asymptotically to 100% Zig. It’s a wonderful dev experience.

13 Likes

I can only agree, when I joined, I was given by my boss this project which was basically a 4yo poc that became a product without any consideration about architecture, and infrastructure, the only way to deploy it was to flash the img from an ftp. The code was a mess of python doing some i2c/shm to some C++ doing the heavy lifting, everything was working, but the codebase was so bad to work with (the code was fine to read, but imagine a codebase where from day0 you start piling on tech debt and hacks explained in comments with the failsafe being dedicating a thread in the python code to relaunch the C++ when it crashed), there was no cross-compilation pipeline, so all you could do was ssh into the pi, and compile on device, that meant vim/no lsp/ 6min compilation time per change. Basically veteran mode. I slowly started porting the code to C, piece by piece, but now it’s a build.zig that cross-compile all the C and working on it is as simple as sshfs the pi, and doing zig build -p /home/foo/sysroot/home/pi and pressing a button for the app to reboot on the new version, everything just works flawlessly. And the next step is evidently to get a zig only codebase.

But already going from that awful C++/python mix to simple easy to grasp C code where everything revolves around a big epoll mainloop, is miles better performance wise, and to dev. But getting good stacktraces, memory and IO debugging + Smith fuzzing is how i see myself implementing the most robust, portable and enjoyable codebase :slight_smile:

3 Likes

I’ve been working on a game engine inspired by the cel-shaded style of Wind Waker and similar era games. I’m about 3 years and 15k sLoC deep, with plenty of scope creep left to go :stuck_out_tongue:

Quick demo:

It uses some of the zig-gamedev libraries (of which I’m also contributing back to) for physics and windowing, but otherwise the render engine is completely custom and backend agnostic.

Outside the usual targets, it also runs on iOS/tvOS and web via emscripten.

Loving Zig and very (very!) happy to see a focus on game dev ergonomic improvements in 0.16

12 Likes