Here is my mostly complete JSON parsing utility

This allows runtime type inference of random key-value pairs, with an intuitive syntax. Given any JSON data, return the entire string or the value of any key with 2 statements:

   val = json.get("hello");
   try val.unpack();

The first statement accepts a key as query and obtains the value to a specified depth by chaining get functions as needed. The second processes and outputs the value. If the value is an array you can use the index function built into the structure:

   val = json.get("hello").get("hash").index(0);
   try val.unpack();

So from here I need to see about what I can do to optimize this. And also what I’m going to use it for. I’m leaning towards REST framework or some kind of scripting or query language. Also my test data is relatively small and only goes 2 or 3 levels deep, so I’m not sure what other limitations there are here.

This was quite a lot of work since I was also learning Zig fundamentals along the way. So I’m relieved that I was able to finally come up with a simple implementation. Also a big thank you to everybody who helped me along the way. My motivation for this was trying to implement some basic ideas of JavaScript into Zig. So the intent is to use this with some kind of web server or proxy server to process request bodies formatted as JavaScript objects. Here is the full code:

const std = @import("std");

const my_json =
   \\{
   \\   "hello": { "name":"hello", "id": "okay", "hash": [null, 2.33e+77, false, -5] },
   \\   "okay": [0, 1, "maybe", { "name":"hello", "id": "okay", "hash": [true, 2.55e+99, null, -7] }]
   \\}
;

const T = struct {
   x: ?std.json.Value,

   pub fn init(self: std.json.Value) T {
      return T {
         .x = self
      };
   }

   pub fn get(self: T, query: []const u8) T {

      if (self.x.?.object.get(query) == null) {
         std.debug.print("ERROR::{s}::", .{ "invalid" });
         return T.init(self.x.?);
      }

      return T.init(self.x.?.object.get(query).?);
   }

   pub fn unpack(self: T) !void {
      var out = std.ArrayList(u8).init(std.testing.allocator);
      defer out.deinit();

      switch (self.x.?) {
         .string => |i| {
            const P = struct { value: []const u8 };
            try std.json.stringify(P{ .value = i }, .{ }, out.writer());
         },

         .integer => |i| {
            const P = struct { value: i64 };
            try std.json.stringify(P{ .value = i }, .{ }, out.writer());
         },

         .bool => |i| {
            const P = struct { value: bool };
            try std.json.stringify(P{ .value = i }, .{ }, out.writer());
         },

         .float => |i| {
            const P = struct { value: f64 };
            try std.json.stringify(P{ .value = i }, .{ }, out.writer());
         },

         .null => {
            const P = struct { value: ?usize = null };
            try std.json.stringify(P{ }, .{ }, out.writer());
         },

         .array => |i| {
            const P = struct { value: []std.json.Value };
            try std.json.stringify(P{ .value = i.items }, .{ }, out.writer());
         },

         .object => {
            const i = self.x.?;
            const P = struct { value: ?std.json.Value };
            try std.json.stringify(P{ .value = i }, .{ }, out.writer());
         },

         else => { }
      }

      std.debug.print("{s}\n", .{ out.items });
   }

   pub fn index(self: T, i: usize) T {
      if (i > self.x.?.array.items.len) {
         std.debug.print("ERROR::{s}::", .{ "index out of bounds" });
         return T.init(self.x.?);
      }

      return T.init(self.x.?.array.items[i]);
   }
};

test {

   var val: T = undefined;

   const parsed = try std.json.parseFromSlice(std.json.Value, std.testing.allocator, my_json, .{ });
   defer parsed.deinit();

   std.debug.print("\n", .{ });

   const json = T.init(parsed.value);
   try json.unpack();

   val = json.get("hello");
   try val.unpack();

   val = json.get("okay");
   try val.unpack();

   val = json.get("okay").index(2);
   try val.unpack();

   val = json.get("okay").index(3);
   try val.unpack();

   val = json.get("hello").get("id");
   try val.unpack();

   val = json.get("hello").get("hash");
   try val.unpack();

   val = json.get("hello").get("hash").index(0);
   try val.unpack();

   val = json.get("hello").get("hash").index(1);
   try val.unpack();

   val = json.get("invalid");
   try val.unpack();

   val = json.get("hello").get("hash").index(8);
   try val.unpack();

}

Any feedback is appreciated :+1:

6 Likes

Hey! Welcome back :slight_smile:

Something you may find helpful is to capture your optionals in an if statement.

var option: ?usize = 10;

if (option) |value| {
    // do something with the value...
} else {
    // ye ol' eject button...
}

In your case… something like…

if(self.x.?.object.get(query)) |item| {
    // use item...
} else {
    // you know the drill.
}

As a matter of fact, you can do this ā€œcapture-by-ifā€ trick anytime you’re calling on of those ā€œ?ā€ operators (just capture by if instead… if you find it necessary). You may already know that, just wanted to point it out. Remember, you can capture by pointers if you’re worried about making copies of a big item.

I think you’ve got a little bug right here in this line…

i > self.x.?.array.items.len

Since we’re zero indexed, the length is always 1 past the last valid index. I think what you’re looking for is greater than or equal.

Finally, have you considered passing in your array via a parameter to your function? Personally, I’d like to have something like:

pub fn unpackInto(self: *const T, buffer: *std.ArrayList(u8)) !void {

That way, we don’t make a copy of the ā€œselfā€ argument because it may actually have a fair bit of data in it in your case. Not sure if it will, but it might ĀÆ\ _ (惄) _/ĀÆ

The more important point there is you can clear the array (which will not remove it’s capacity… so the underlying memory is saved, it just reuses it again).

So if you do something like so…

   var buffer = std.ArrayList(u8).init(std.heap.page_allocator);
   
   buffer.ensureTotalCapacity(100); // or some number you find appropriate

   val = json.get("hello").get("hash").index(1);
   try val.unpackInto(&buffer);

   // as Sze pointed out, this is more direct than resize(0)
   buffer.clearRetainingCapacity(); // (keeps the same memory allocated, just sets .len = 0)

   val = json.get("invalid");
   try val.unpackInto(&buffer);

Anyhow, definitely an improvement. Keep up the good work!

5 Likes

Very nice example! I would also move the declaration of my_json into the test, for better scoping and easier reading.

4 Likes

I think this should be:

buffer.clearRetainingCapacity();
2 Likes

Yup, this is even more direct. I’ll edit my original post to include your answer.

1 Like

Thanks everybody for the suggestions and support. Especially about the function parameters. I should be able to get this updated soon.

Hello,
j’ai rĆ©glĆ© le problĆØme comme Ƨa ???

je suis trĆØs intĆ©ressĆ© par Json, il faut que je modĆ©lise afin d’enregistrer dans un fichier a plat tel que Json , lecture, c’est fait et que je le rende dynamique(ecriture)…( model static) et ce que tu as fait, je le test .

il y a beaucoup de json dans github mais trĆØs compliquĆ©. Surtout je n’essaye pas de faire un browser de json.

pub fn index(self: T, i: usize)  T {

    switch (self.x.?) {
      .array => {
        if (i > self.x.?.array.items.len) {
          std.debug.print("ERROR::{s}::\n", .{ "index out of bounds" });
          return T.init(self.x.?);
        }
      },
      else => {
        std.debug.print("ERROR::{s}:: {s}\n", .{ "Not array", @tagName(self.x.?) });
        return T.init(self.x.?);
      }
    }
    return T.init(self.x.?.array.items[i]);
  }

1 Like

Hey JPL, I’m going to translate this to English for myself and others :slight_smile:

I think JPL here is pointing out how complex JSON can become for these browsers.

2 Likes

Hello, Just a slightly more complex piece of code with a record, a file…
I think Json is for encapsulating BOMs, for example screen definition, or file definition for SQL.
we should be able to do better, but if you do it too well, it becomes readable only by the person who wrote it. KEEPING IT SIMPLE is my motto It is better to have small modules that solve their problems than something that can do everything: think about maintenance.

Thank you all, (ps: you fell right in my research)

const std = @import("std");

const allocator = std.heap.page_allocator;






const Rvals = enum {
  testing,
  production,
};

const Rbutton = enum {
  Tkey,
  title,
};
const Rdata = enum {
  vals,
  uptime,
  hello,
  numberString,
  button
};


const DEFBUTTON = struct {
    Tkey : [] const u8,
    title: [] const u8,
  };
const DEFVALS = struct {
    testing : i32,
    production : i32 ,
  };

const Rvalue = struct {
  vals : DEFVALS,

  uptime : i32,
  hello  : [] const u8,
  numberString: [] const u8,

  button:std.ArrayList(DEFBUTTON) ,

};

const T = struct {
  x: ?std.json.Value,

  pub fn init(self: std.json.Value) T {
      return T {
        .x = self
      };
  }

  pub fn get(self: T, query: []const u8) T {
    
    if (self.x.?.object.get(query) == null) {
      std.debug.print("ERROR::{s}::", .{ "invalid" });
      return T.init(self.x.?);
    }

    return T.init(self.x.?.object.get(query).?);
  }

  pub fn unpack(self: T) !void {
    var out = std.ArrayList(u8).init(allocator);
    defer out.deinit();

    switch (self.x.?) {
      .string => |i| {
          const P = struct { value: []const u8 };
          try std.json.stringify(P{ .value = i }, .{ }, out.writer());
      },
      .integer => |i| {
          const P = struct { value: i64 };
          try std.json.stringify(P{ .value = i }, .{ }, out.writer());
      },

      .bool => |i| {
          const P = struct { value: bool };
          try std.json.stringify(P{ .value = i }, .{ }, out.writer());
      },

      .float => |i| {
          const P = struct { value: f64 };
          try std.json.stringify(P{ .value = i }, .{ }, out.writer());
      },

      .null => {
          const P = struct { value: ?usize = null };
          try std.json.stringify(P{ }, .{ }, out.writer());
      },
      .array => |i| {
          const P = struct { value: []std.json.Value };
          try std.json.stringify(P{ .value = i.items }, .{ }, out.writer());

      },

      .object => {
          const i = self.x.?;
          const P = struct { value: ?std.json.Value };
          try std.json.stringify(P{ .value = i }, .{ }, out.writer());
      },

      else => {}
    }

    std.debug.print("{s}\n", .{ out.items });
  }

  pub fn index(self: T, i: usize)  T {

    switch (self.x.?) {
      .array => {
        if (i > self.x.?.array.items.len) {
          std.debug.print("ERROR::{s}::\n", .{ "index out of bounds" });
          return T.init(self.x.?);
        }
      },
      else => {
        std.debug.print("ERROR::{s}:: {s}\n", .{ "Not array", @tagName(self.x.?) });
        return T.init(self.x.?);
      }
    }
    return T.init(self.x.?.array.items[i]);
  }

  pub fn unpackButton(self: *const T) !void {

    switch (self.x.?) {
          .array => {
              const i = self.x.?.array.items.len;
              var n : usize = 0;
              while(n < i) : (n += 1 ) {
                var val = self.index(n).get("Tkey");
                try val.unpack();
                val = self.index(n).get("title");
                try val.unpack();

              }
          },
          else => {
            std.debug.print("ERROR::{s}:: {s}\n", .{ "Not array", @tagName(self.x.?) });
          }
        }

  }
};

pub fn indexSize(self: T )  usize {

  switch (self.x.?) {
        .array => {
            return self.x.?.array.items.len;
        },
        else => {
          std.debug.print("ERROR::{s}:: {s}\n", .{ "Not array", @tagName(self.x.?) });
          return 0;
        }
      }
}




pub fn jsonRecord(my_json : []const u8) !void {

  var val: T = undefined;
  var ENRG : Rvalue =undefined ;
  ENRG.button = std.ArrayList(DEFBUTTON).init(allocator);

  const parsed = try std.json.parseFromSlice(std.json.Value, allocator, my_json, .{ });
  defer parsed.deinit();

  std.debug.print("\n", .{ });


  const json = T.init(parsed.value);

  try json.unpack();

  



  std.debug.print("----------------------------\r\n",.{});
  std.debug.print("----------------------------\r\n",.{});

  const Record = std.enums.EnumIndexer(Rdata);

  const Record_vals = std.enums.EnumIndexer(Rvals);

  const Record_button = std.enums.EnumIndexer(Rbutton);

  var n: usize = 0 ;
  var v: usize = 0 ;
  while (n < Record.count) : ( n +=1 ) {

      switch(Record.keyForIndex(n)) {
        Rdata.vals =>  { 
              v =0;
              while (v < Record_vals.count) : ( v +=1 ) {
                switch(Record_vals.keyForIndex(v)) {
                  Rvals.testing => {
                      val = json.get("vals").get(@tagName(Record_vals.keyForIndex(v)));
                      
                      //std.debug.print("{d}\r\n",.{val.x.?.integer});
                      ENRG.vals.testing = @intCast(val.x.?.integer);
                  },
                  Rvals.production => {
                      val = json.get("vals").get(@tagName(Record_vals.keyForIndex(v)));
                      
                      //std.debug.print("{d}\r\n",.{val.x.?.integer});
                      ENRG.vals.production =  @intCast(val.x.?.integer);
                  }
                }
              }
        },
        Rdata.uptime => {
          val = json.get(@tagName(Record.keyForIndex(n)));
                
          //std.debug.print("{d}\r\n",.{val.x.?.integer});
          ENRG.uptime =  @intCast(val.x.?.integer);

        },
        Rdata.hello => {
          val = json.get(@tagName(Record.keyForIndex(n)));
                
          //std.debug.print("{s}\r\n",.{val.x.?.string});
          ENRG.hello = val.x.?.string;

        },
        Rdata.numberString => {
          val = json.get(@tagName(Record.keyForIndex(n)));
                
          //std.debug.print("{s}\r\n",.{val.x.?.string});
          ENRG.numberString= val.x.?.string;

        },
        Rdata.button => {
          val = json.get(@tagName(Record.keyForIndex(n)));

          var btn : DEFBUTTON = undefined;
          var y = val.x.?.array.items.len;
          var z : usize = 0;

          //std.debug.print("{d}\r\n",.{y});
            while(z < y) : (z += 1 ) {
              //std.debug.print("{d}\r\n",.{z});
              v =0;
              while (v < Record_button.count) : ( v +=1 ) {

                switch(Record_button.keyForIndex(v)) {
                  Rbutton.Tkey => {
                      val = json.get("button").index(z).get(@tagName(Record_button.keyForIndex(v)));
                      //std.debug.print("{s}\r\n",.{val.x.?.string});

                      btn.Tkey =  val.x.?.string;
                  },
                  Rbutton.title => {
                      val = json.get("button").index(z).get(@tagName(Record_button.keyForIndex(v)));
                      //std.debug.print("{s}\r\n",.{val.x.?.string});
                      
                      btn.title = val.x.?.string;
                      ENRG.button.append(btn) catch unreachable;
                  }
                }
              }
            }
        }

      }
    }
    std.debug.print("{d}\r\n",.{ENRG.vals.testing});
    std.debug.print("{d}\r\n",.{ENRG.vals.production});
    std.debug.print("{d}\r\n",.{ENRG.uptime});
    std.debug.print("{s}\r\n",.{ENRG.hello});
    std.debug.print("{s}\r\n",.{ENRG.numberString});
    std.debug.print("{s}\r\n",.{ENRG.button.items[0].Tkey});
    std.debug.print("{s}\r\n",.{ENRG.button.items[0].title});
    std.debug.print("{s}\r\n",.{ENRG.button.items[1].Tkey});
    std.debug.print("{s}\r\n",.{ENRG.button.items[1].title});

}



pub fn main() !void {



    var out_buf: [1024]u8 = undefined;
    var slice_stream = std.io.fixedBufferStream(&out_buf);
    const out = slice_stream.writer();

    var w = std.json.writeStream(out, .{ .whitespace = .indent_2 });
    defer w.deinit();

    try w.beginObject();

      try w.objectField("vals");
      try w.beginObject();
        try w.objectField("testing");
        try w.print("  {}", .{1});
        try w.objectField("production");
        try w.print("  {}", .{42});
      try w.endObject();

      try w.objectField("uptime");
      try w.print("  {}", .{9999});
      try w.objectField("hello");
      try w.print("  \"{s}\"", .{"bonjour"});
      try w.objectField("numberString");
      try w.print("  \"{s}\"", .{"71.10"});

      try w.objectField("button");
      try w.beginArray();

        try w.beginObject();
        try w.objectField("Tkey");
        try w.print("  \"{s}\"", .{"F3"});
        try w.objectField("title");
        try w.print("  \"{s}\"", .{"F3 exit"});
        try w.endObject();

        try w.beginObject();
        try w.objectField("Tkey");
        try w.print("  \"{s}\"", .{"F9"});
        try w.objectField("title");
        try w.print("  \"{s}\"", .{"F9 enrg"});
        try w.endObject();

      try w.endArray();

    try w.endObject();

    const result = slice_stream.getWritten();

    //std.debug.print("{s}\r\n",.{result});

    var my_file = try std.fs.cwd().createFile("fileJson.txt", .{ .read = true });
    defer my_file.close();

    _ = try my_file.write(result);

      var buf : []u8= allocator.alloc(u8, result.len) catch unreachable ;

      try my_file.seekTo(0);
      _= try my_file.read(buf[0..]);
      std.debug.print("{s}\r\n",.{buf});

    jsonRecord(buf) catch unreachable;



}

It remains to use the ā€œunpackā€ module as a typing control function, I did not make the modifications.

1 Like

I would like there to be bits of code maybe not in ā€œhelpā€ that would allow us to move forward and give ideas

JPL thank you for your input. Just so you know I am specifically trying to avoid having any data structures defined in the code for now, but I think your example might be useful for dealing with increasingly complex JSON data. Ideally it will be completely random data, most likely sourced over HTTP. Also I am not quite at the point yet where I need to be writing files directly, but reading a file should be acceptable in place of reading an HTTP source. Also I got started today implementing everybody’s suggestions, so i should have a good update soon. Probably over the weekend.

1 Like

If you want, I’ll let you know about my progress, if you’re interested, because I need for my project to encapsulate data from a model in order to be able to return them ā€œWhat you see, what you getā€.
I’m very happy that you published some code, it made me want to take the plunge :wink:

1 Like

Thats fine. Im glad it was motivating for you. You are welcome to share your work :slight_smile: . I will see what I can learn from it.

depots

I drop there, you will see a Json module arrives

in preparation for :slightly_smiling_face:

1 Like