Best way of reading json from stdin

Help
,
1

My program reads a json from stdin, and outputs a json to stdout. What exactly doesn’t really matter here, but in short the input json describes a database query, and the output json describes all the data that matches that query.

This way I can do cat query.json | bin and have the result printed to stdout.
This works, but when trying to call this from Godot, I run into issues. The Godot API for running executables is a bit peculiar, and stdin and stdout are grouped together. This means that closing stdin also causes stdout to be closed, and vice versa. I’ve had to work around this by outputting to stderr instead, so I can (1) close stdin (and unfortunately stdout) and (2) read from stderr

This exposed the fact that I seem to rely on stdin being closed, before I actually start parsing the json, doing the DB query, and ouptutting.

    var buffer: [1024]u8 = undefined;
    var reader: Io.File.Reader = .init(.stdin(), io, &buffer);
    const stdin = &reader.interface;

    var json_reader: std.json.Scanner.Reader = .init(gpa, stdin);
    defer json_reader.deinit();

    const parsed = try std.json.parseFromTokenSource(Query, gpa, &json_reader, .{});
    defer parsed.deinit();

I think some programs continuously read and write (like grep) so they don’t rely on EOF before anything happens. Is this generally desired and how would I achieve something in zig?
When taking this streaming approach, it’s not clear to me how my program is supposed to exit; If you can continuously read, how to know when to stop except through EOF?

2

by having a data format where you know the end in some way, json is such a format; but the json parser seems to care about EOF, so you will have to work around that.

You could stream into an allocating writer, and inspect the data while you do to determin the end, then parse that pre-read slice. You dont need to support arbitrary json, instead just the json you actually deal with.

3

With JSON, you can get away with counting the { and } with a simple counter to know when you have received the full object. Then you would’t have to detect an stdin close.

4

No, you can’t because there’s no requirement that curlies inside strings are balanced.

5

For a robust handling, yes. In this case, it sounds like the OP controls both sides, so you can know ahead of time if that’s the case. Thanks for calling that out.

6

Indeed in this case I control both in- and output so I could get away with just counting brackets.
I’ll give that a try, thanks!

7

One nice thing about JSON is that it is extremely easy and cheap to correctly keep track of what you are reading with a simple stack, which means that you can always identify when a JSON stream of data ends… except for malformed JSON, which forces you to take additional care.