Call and spawn new process with stdin

Help
1

I want to call a process and provide stdin and get stdout how to do that?

The pipe gives and takes a file and I do not know how to read it or create one from just hard coded string, this is somewhat pseudo code, any ideas how to make it work:

const std = @import("std");

pub fn main(init: std.process.Init) !void {
	const io = init.io;
	const input = "Hello from zig";
    var child = try std.process.spawn(
        io,
        .{
            .argv = &[_][]const u8{"/bin/cat"},
            .stdin = .pipe,
            .stdout = .pipe,
        },
    );
    child.stdin = input
    var stdout_buffer: [1024]u8 = undefined;
    if (child.stdout) |stdout| {
		_ = try stdout.readPositionalAll(io, &stdout_buffer, 1024);
    }
    _ = try child.wait(io);
	stdout_buffer == input
}
2

A dangerous working snippet for illustration purposes. You still need check optional entries here.

I did need to read the standard library example to understand this. To be able to pass the stdin you need to write to child.stdin using the writer interface, and after that you need to close it to “send EOF” otherwise the cat process get stuck waiting for it.

const std = @import("std");

pub fn main(init: std.process.Init) !void {
    const io = init.io;
    const input = "Hello from zig";
    var child = try std.process.spawn(
        io,
        .{
            .argv = &[_][]const u8{"/bin/cat"},
            .stdin = .pipe,
            .stdout = .pipe,
            .stderr = .ignore,
        },
    );

    var writer = child.stdin.?.writer(io, &.{});
    try writer.interface.writeAll(input);
    child.stdin.?.close(io);
    child.stdin = null;

    var stdout_buffer: [512]u8 = undefined;
    var stdout_reader: std.Io.File.Reader = .initStreaming(child.stdout.?, io, &stdout_buffer);
    const stdout = &stdout_reader.interface;
    stdout.readSliceAll(&stdout_buffer) catch |err| switch (err) {
        error.EndOfStream => {},
        else => return err
    };

    const term = try child.wait(io);

    if (term.exited != 0) {
        std.debug.print("process failed, exit code {}", .{ term.exited });
        return;
    }

    std.debug.print("return: {s}", .{ stdout_buffer[0..stdout_reader.pos] });
}

Some of things I didn’t understand: I tried to get the stdout.bufferedLen but it’s always returning 0 (My guess is this zeroed after child wait but I need to dig more into this), so I used stdout_reader.pos instead to delimiter until where I should read the output, but I believe this is not the ideal nor safe.

3

I think that could be improved slightly by readSliceShort(), which returns how many bytes were read. In fact, this is the definition of readAll() :

pub fn readSliceAll(r: *Reader, buffer: []u8) Error!void {
    const n = try readSliceShort(r, buffer);
    if (n != buffer.len) return error.EndOfStream;
}

I feel like the usecase of All() might be something like if you’re reading in a binary format and you know the exact size of the input such that any deviation is an encoding error.

I find the “All” confusing in the function name because it’s unclear “all what?” Only by reading the implementation do I understand it means to copy into the whole buffer or fail.

In my own code, I call fill(1) and read from buffered(). Although, looking at the library code right now, maybe fillMore() is more explicit.

    while (true) {
        reader.fill(1) catch |err| switch (err) {
            error.EndOfStream => break,
            else => return err,
        };

        const buf = try allocator.dupe(u8, reader.buffered());
        // ...
        reader.tossBuffered();
    }

EDIT: Forgot to include tossBuffered()

1 Like
4

I just remembered something. The reason I chose not to use the readSlice functions is because if you look at what they’re doing,

pub fn readSliceShort(r: *Reader, buffer: []u8) ShortError!usize {
    const contents = r.buffer[r.seek..r.end];
    const copy_len = @min(buffer.len, contents.len);
    @memcpy(buffer[0..copy_len], contents[0..copy_len]);
    r.seek += copy_len;
    if (buffer.len - copy_len == 0) {
        @branchHint(.likely);
        return buffer.len;
    }
    // unhappy path: actually reading from the stream

It’s primary purpose is to copy from the reader buffer into the argument buffer. But using it like this,

It’s just copying to and from the same buffer. When I was researching the IO system, I came across one or two code snippets like this, but it was really confusing why they were initializing the reader with the same buffer they were passing to readSliceAll().

1 Like
5

Yes, to be honest I didn’t understand it very well too, and I copied exactly like it’s done in the Zig codebase (I assume the right way it’s how it’s done there), I’m still in the process of understanding how reader/writer interface work in this kind of use case

6

Thanks, I went with the following, it works and that is enough for my use case:

const std = @import("std");
const restricted = @import("restricted");

pub fn main(init: std.process.Init) !void {
    const io = init.io;
    const input = "Hello from zig";
    var child = try std.process.spawn(
        io,
        .{
            .argv = &[_][]const u8{"/bin/cat"},
            .stdin = .pipe,
            .stdout = .pipe,
            .stderr = .pipe,
        },
    );

    var writer = child.stdin.?.writer(io, &.{});
    try writer.interface.writeAll(input);
    child.stdin.?.close(io);
    child.stdin = null;

    var stdout_buffer: [1024]u8 = undefined;
    var stdout_reader: std.Io.File.Reader = .initStreaming(child.stdout.?, io, &stdout_buffer);
    const stdout = &stdout_reader.interface;
    stdout.readSliceAll(&stdout_buffer) catch |err| switch (err) {
        error.EndOfStream => {},
        else => return err,
    };
    try std.testing.expect(std.mem.eql(u8, stdout_buffer[0..stdout_reader.pos], input));

    var stderr_buffer: [1024]u8 = undefined;
    var stderr_reader: std.Io.File.Reader = .initStreaming(child.stderr.?, io, &stderr_buffer);
    const stderr = &stderr_reader.interface;
    stderr.readSliceAll(&stderr_buffer) catch |err| switch (err) {
        error.EndOfStream => {},
        else => return err,
    };
    try std.testing.expect(std.mem.eql(u8, stderr_buffer[0..stderr_reader.pos], ""));

    const term = try child.wait(io);
    if (term.exited != 0) {
        std.debug.print("process failed, exit code {}", .{term.exited});
        return;
    }
}
7

It might not be needed in your case, but in general, this can deadlock both processes, if for example, the subprocess needs to write to stderr more data than the system buffer allows and only then close stdout. You would be waiting on stdout to close, the subprocess to witing on stderr to get drained.

For a proper solution, you use use something like this:

https://ziglang.org/documentation/master/std/#std.Io.File.MultiReader

Or spawn two extra tasks for reading stdout/stderr.

Actually, I just reread the example, yes, you will deadlock it if stdin is larger than the system buffer without draining stdout in between stdin writes.

1 Like