Compile time allocator

Help
1

I am trying to parse the data of a json file at compile time. However std json parser requires an allocator, but it doesn’t accept any of the ones I tried at compile time. Is there an allocator in zig that works that comptime?

pub fn initFont(comptime rgbFileData: [ ]const u8,  comptime jsonFileData: [ ]const u8) !FontData {
    const allocator = ????
    
    const jsonResult = try std.json.parseFromSliceLeaky(JsonFontData, allocator, jsonFileData, .{});
    const data: JsonFontData = jsonResult.value;
    
    const fontData = makeFontData(data, rgbFileData); //My function
    return fontData;
}
2

do you mean you got errors about them not being able to be evaluated at comptime? Or something else.

You can pass allocators at comptime, but they need

  1. to be known at comptime
  2. to not use syscalls, ie not use page_allocator under the hood.

The easy solution is to use a comptime std.heap.FixedBufferAllocator.

It is also possible to make an allocator that dynamically allocates at comptime. There was a post about that, but I can’t find it atm.

4 Likes
3

I tried page_allocator which failed immediately and std.heap.FixedBufferAllocator, the fixed buffer allocator got rejected because it did pointer math at compile time.

/home/plebosaur/.local/share/mise/installs/zig/0.15.2/lib/std/mem.zig:4010:18: error: unable to evaluate comptime expression
const addr = @intFromPtrintFromPtr(ptr);
^~~~~~~~~~~~~~~~

I tried writing my own allocator but then I got an un-named error. (error: the following command terminated unexpectedly)
I didn’t press it much further than that, decided to do the call at runtime in build.zig and pass it as an import, which is a very ugly solution that took much more boilerplate, making me make this post to look for an alternative.

4
4 Likes
5

Hi, if you just want to transform some data to json, this is an example on how to do it.

const std = @import("std");
const mem = std.mem;
const heap = std.heap;
const process = std.process;
const log = std.log;
const Io = std.Io;
const json = std.json;
const fs = std.fs;

const example = @embedFile("../contacts.json");

const Contacts = struct {
    name: json.ArrayHashMap(Info),
};

const Info = struct {
    number: []const u8 = "",
    address: []const u8 = "",
};

fn openReadToEndAlloc(allocator: mem.Allocator, io: Io, dir: Io.Dir, file_path: []const u8) ![]const u8 {
    const file = try dir.openFile(io, file_path, .{ .mode = .read_only });
    defer file.close(io);
    const file_stat = try file.stat(io);
    var file_buff: [heap.pageSize()]u8 = undefined;
    var file_reader = file.reader(io, &file_buff);
    const reader = &file_reader.interface;
    const content = try reader.readAlloc(allocator, file_stat.size);
    return content;
}

fn jsonFromFileContentLeaky(comptime T: type, allocator: mem.Allocator, content: []const u8, options: json.ParseOptions) !T {
    return try json.parseFromSliceLeaky(T, allocator, content, options);
}

fn juicyMain(allocator: mem.Allocator, io: Io, argv: [][:0]u8) !void {
    var arena: heap.ArenaAllocator = .init(allocator);
    defer arena.deinit();

    const cwd = Io.Dir.cwd();
    const file_path = if (argv.len >= 2) argv[1] else return error.MissingFilePath;
    const file_content = try openReadToEndAlloc(allocator, io, cwd, file_path);
    defer allocator.free(file_content);
    const file_json = try jsonFromFileContentLeaky(Contacts, arena.allocator(), file_content, .{});

    var stdout_buffer: [128]u8 = undefined;
    var stdout_writer = fs.File.stdout().writer(&stdout_buffer);
    const stdout: *Io.Writer = &stdout_writer.interface;
    var it = file_json.name.map.iterator();
    while (it.next()) |entry| {
        try stdout.print("name : {s} | number : {s} | address : {s}\n", .{ entry.key_ptr.*, entry.value_ptr.number, entry.value_ptr.address });
    }
    try stdout.flush();
}

pub fn main() !void {
    var gpa: heap.DebugAllocator(.{}) = .init;
    defer _ = gpa.deinit();

    var threaded: Io.Threaded = .init(gpa.allocator());
    defer threaded.deinit();

    const argv: [][:0]u8 = process.argsAlloc(gpa.allocator()) catch |err| {
        return log.err("Fatal : {}", .{err});
    };
    defer process.argsFree(gpa.allocator(), argv);

    juicyMain(gpa.allocator(), threaded.io(), argv) catch |err| {
        return log.err("Fatal : {}", .{err});
    };

    return;
}

json example

{
  "name": {
    "Alice": {
      "number": "123456789",
      "address": "42 Rue de Zig, Paris"
    },
    "Bob": {
      "number": "+33 6 12 34 56 78",
      "address": "7 Avenue du Kernel, Lyon"
    },
    "Charlie": {
      "number": "555-0101",
      "address": "Embedded Systems Lab, Grenoble"
    }
  }
}

output :

bar ) zbr -- ./contacts.json
name : Alice | number : 123456789 | address : 42 Rue de Zig, Paris
name : Bob | number : +33 6 12 34 56 78 | address : 7 Avenue du Kernel, Lyon
name : Charlie | number : 555-0101 | address : Embedded Systems Lab, Grenoble

If you want to parse json at comptime, I don’t know if it’s possible at the moment (at least the memory can’t be reference at runtime i believe, so you can use it do to other comptime stuff, but not use that memory i think), but don’t quote me on that, if you need the file content at comptime you can use this const example = @embedFile("../contacts.json");

6

Thank you for digging it up, I am getting the “error: the following command terminated unexpectedly” again which is the same error I got when I tried to make my own allocator, however I am assuming the allocator must not be the issue this time around.

The same code ran from build.zig was succesful in providing the correct file so I am assuming my code that calls the allocator isn’t the issue either, however. Maybe it’s a limitation to comptime, I am not sure.

7

The goal is to parse the .json at comptime, I was only able to do it from build.zig and expose it as an import, but failed to so by using the comptime feautre. Thanks for the reply regardless.