Questions about reading json in zig

I’m trying to read in a json file in zig and then parse it to a struct.

The json I want to read is here, yes I have verified it is correct json.

I created these structs to hold the parsed data:

pub const Mb32Data = struct {
    name: []const u8,
    initial: Mb32Initial,
    final: Mb32Initial,
    cycles: []Mb32Cycles,
};

pub const Mb32Initial = struct {
    pc: u16,
    sp: u16,
    a: u8,
    b: u8,
    c: u8,
    d: u8,
    e: u8,
    f: u8,
    h: u8,
    l: u8,
    ime: u2,
    ei: u2,
    ram: [][]const u8,
};

pub const Mb32Cycles = struct {
    address: u16,
    data: u8,
    ignore: []u8,
};

And this is the code I’m using to parse the json:

fn open_test(path: []const u8) !void {
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    var file = try std.fs.cwd().openFile(path, .{});
    defer file.close();
    const allocator = gpa.allocator();
    const contents = try file.readToEndAlloc(allocator, std.math.maxInt(usize));
    const json = try std.json.parseFromSlice([]Mb32Data, allocator, contents, .{ .allocate = .alloc_always, .ignore_unknown_fields = true });
    const value = json.value;
    std.debug.print("name is {any}", .{value[0].name});
}

This keeps failing with very cryptic hard to understand errors like this

if (ov[1] != 0) return error.Overflow
or
return error.MissingField;

Is there an easy way to debug why the json is not parsing correctly?

edit:

The main problem was I was trying to use a named struct with fields but using anynomous structs fixed the issue. Thank you everyone for the help.

pub const Mb32Data = struct {
    name: []const u8,
    initial: Mb32Initial,
    final: Mb32Initial,
    cycles: []struct { u16, u8, []const u8 }, <---- this
};

pub const Mb32Initial = struct {
    pc: u16,
    sp: u16,
    a: u8,
    b: u8,
    c: u8,
    d: u8,
    e: u8,
    f: u8,
    h: u8,
    l: u8,
    ime: u2,
    ie: u2 = 0,
    ram: []struct { u16, u8 }, <---- and this
};

How did you verify this? Did you use something like jq to display it?

I pasted it into an online verifier like this https://jsonlint.com/

This is a repository that is used a lot for testing gameboy roms and if the json was invalid I do not think it would be recommended.

I assume this is because one of your fields is not big enough to hold a value in the json.

Ok, assuming the JSON is valid, I would look next at data types. That overflow error suggests there may be numbers in the data that are too large for their types in the struct.

I would take the JSON and extract a minimal amount of it – for the enclosing array, I would just keep one row, and try that. Once that is working, and you have convinced yourself that things are reasonably set up, try doing a binary search: keep the top part of the array in your JSON data, then the bottom, etc, until you find the data that is not correctly processed.

I figured the same and changed everything to u32 and started getting the return error.MissingField, but still it is quite hard to debug.

I also tried reducing the json to a small sample to test since the linked example is a large array but still got the missing field which was very confusing.

You can get more information about your error using std.json.Diagnostics. I have modified your function to output some more useful debugging information:

fn open_test(path: []const u8) !void {
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    var file = try std.fs.cwd().openFile(path, .{});
    defer file.close();
    const allocator = gpa.allocator();
    const contents = try file.readToEndAlloc(allocator, std.math.maxInt(usize));

    var scanner = std.json.Scanner.initCompleteInput(allocator, contents);
    defer scanner.deinit();

    var diagnostics = std.json.Diagnostics{};
    scanner.enableDiagnostics(&diagnostics);
    errdefer std.log.err("byte offset {d}\nbytes: {s}|{s}\n", .{
        diagnostics.getByteOffset(),
        contents[diagnostics.getByteOffset() -| 32..][0..32],
        contents[diagnostics.getByteOffset()..][0..32],
    });

    const json = try std.json.parseFromTokenSource([]Mb32Data, allocator, &scanner, .{ .allocate = .alloc_always, .ignore_unknown_fields = true });
    const value = json.value;
    std.debug.print("name is {any}", .{value[0].name});
}

This outputs:

error: byte offset 144
bytes: 147,"ime":1,"ie":1,"ram":[[19935|,0]]},"final":{"a":110,"b":185,"

As you can see by the location of the |, it just tried to parse a number. The error.Overflow tells you that it tried to fit a value into an integer that couldn’t hold it. In your case, this is the ram array:

    ram: [][]const u8,

19935 does not fit u8.

3 Likes

I did trying doing both of these. I first changed all the u8 to u32 and then removed all of the array values except one and removed the array from the json but got the same result error.MissingField

1 Like

For your error.MissingField, take a look at std.json.ParseOptions.

It says:

For missing fields, give the Zig struct fields default values.

Now, you have to identify which fields are missing a default value. Using the same trick with Diagnostics as before:

error: byte offset 149
bytes: ime":1,"ie":1,"ram":[[19935,0]]}|,"final":{"a":110,"b":185,"c":14

error: MissingField

We can extract the object before offset 149 using a text editor:

{
  "pc": 19935,
  "sp": 59438,
  "a": 110,
  "b": 185,
  "c": 144,
  "d": 208,
  "e": 190,
  "f": 240,
  "h": 131,
  "l": 147,
  "ime": 1,
  "ie": 1,
  "ram": [
    [
      19935,
      0
    ]
  ]
}

You misspelled the ie field as ei:

    ei: u2,

Renaming it to ie causes this error to go away, and you will have to find the next one.

1 Like

Thanks this is really nice. The std.json.Diagnostics is exactly what I was looking for. Just one quick question what do you mean by

We can extract the object before offset 149 using a text editor:

I don’t know how your text editor does it, but I can just go to column 149 and select everything backwards until I’ve arrived at the start of the object,{.