Is this a bug with my terminal or with Zig?

Help
1

I have this weird glitch with this piece of code:

    var line = std.ArrayList(u8).init(allocator);
    defer line.deinit();
    const lw = line.writer();

    const stdin = std.io.getStdIn().reader();
    while (true) {
        std.debug.print("zjlox >> ", .{});
        if (stdin.streamUntilDelimiter(lw, '\n', null) == error.EndOfStream)
            break;
        try run(allocator, line.items);
        line.clearRetainingCapacity();
    }

I would expect it to read from StdIn until it finds a newline, which it does but only if the input has an odd numbered length. I don’t know if this is an issue with Zig or with my terminal. I am on windows which has \r\n as its newline instead of \n, but it should still work no??

1 Like
2

Strange, streamUntilDelimiter code seems pretty straighforward:

        while (true) {
            const byte: u8 = try self.readByte();
            if (byte == delimiter) return;
            try writer.writeByte(byte);
        }
3

It seems to work perfectly fine here. I suspect Windows/terminal shenanigans.

You said:

but you didn’t say what it does do instead. What behavior are you observing?

1 Like
4

I was using this pattern for dealing with windows terminal.

var bytes = std.ArrayList(u8).init(allocator);
defer bytes.deinit();

while (reader.streamUntilDelimiter(bytes.writer(), '\n', 1000) != error.EndOfStream) {
   if (bytes.items.len > 0 and bytes.items[bytes.items.len - 1] == '\r') {
      bytes.shrinkRetainingCapacity(bytes.items.len - 1);
   }

   try bytes.append('\n');
   bytes.clearRetainingCapacity();
}

This method avoids using std.trim to remove the trailing /r by using ArrayList functions exclusively. The check for /r at bytes.items[bytes.items.len - 1] will handle the windows terminal without interfering with a Linux scenario. Checking that bytes.items.len > 0 is also necessary in case of an empty line. After that I re-append the /n because I was still using it in this case. Whether using my while condition or simply using while(true) shouldn’t make a difference. However by checking for while(streamUntilDelimiter != error.EndOfStream), you can also avoid manually specifying the break. Either way I’ve found it to be reliable for Windows and Linux.

Not sure if this will help in your case, but I won’t have a chance to check out your code until later tonight, if you don’t get it solved by then.

1 Like
5

For inputs that has even-numbered length, it works as expected, it reads all the bytes until the line feed (even the carriage return).
But if the input has an odd-numbered length, it doesn’t read a line feed, so it doesn’t stop reading input. It waits until it gets an input with an even-numbered length.

If I were to print out the bytes after reading the input, it would look like this:

input >> 123                       // odd-numbered length
{ 49, 50, 51, 13 }                 // works as expected
input >> 12                        // even-numbered length
                                   // doesnt stop reading bytes
123                                // feed odd-numbered length
{ 49, 50, 13, 13, 49, 50, 51, 13 } // seems to find carriage returns with no line feeds
6

We might need to see more of your code. I adapted the sample you gave and it seems to be working fine for even or odd length inputs.

const std = @import("std");

pub fn main() !void {
   const allocator = std.heap.page_allocator;

   var line = std.ArrayList(u8).init(allocator);
   defer line.deinit();

   const lw = line.writer();

   const stdin = std.io.getStdIn().reader();

   while (true) {
      std.debug.print("zjlox >> ", .{});
      try stdin.streamUntilDelimiter(lw, '\n', null);
      //try run(allocator);
      std.debug.print("{d}\n", .{ line.items });
      line.clearRetainingCapacity();
   }
}

clearRetainingCapacity() does not free the memory, so sometimes weird things can happen if you don’t handle your ArrayList properly. Also if you don’t remove the carriage return, sometimes it will cause the characters on the next line to be overwritten. In this case you are using the decimal formatting specifier, so it shouldn’t matter.

1 Like
7

I’ll try to make a minimal reproducible example later, but I’m busy right now. But I’m pretty sure the code you made is similar to my full code but instead of page allocator, I used General Purpose Allocator.
What I find weird, is that this code doesn’t work only for Windows Terminal specifically. If I run this code in PowerShell or any other terminal, it works as expected. But I don’t think it’s a Windows Terminal bug, since if I write the same exact code in other programming languages, it works as usual.

8

Okay now I see what you mean. I was using Windows console, but when I run it under Windows Terminal, either with PowerShell or Command Prompt interface, it gives me the same problem you were talking about.

So this is most likely an issue with Windows Terminal, although it’s possible it could be resolved on the Zig end as well.

9

I’ve run into this as well. The kernel32.ReadFile call seems to stop after the \r on even length inputs (and never returns the \n unless the \n is on an odd offset, e.g. ab<enter><space><enter> will get it to read the \n but ab<enter><enter>... will never read the \n):

With some debug printing in std.os.windows.ReadFile and the following code:

    const stdin = std.io.getStdIn().reader();
    var stdout = std.io.getStdOut().writer();

    var input_buf: [512]u8 = undefined;
    while (try stdin.readUntilDelimiterOrEof(&input_buf, '\n')) |input| {
        try stdout.writeAll(input);
        try stdout.writeAll("\n");
    }

ab
amt_read: 1, byte: 97
want_read_count: 1
amt_read: 1, byte: 98
want_read_count: 1
amt_read: 1, byte: 13
want_read_count: 1

abc
amt_read: 1, byte: 97
want_read_count: 1
amt_read: 1, byte: 98
want_read_count: 1
amt_read: 1, byte: 99
want_read_count: 1
amt_read: 1, byte: 13
want_read_count: 1
amt_read: 1, byte: 10
abc

Unsure exactly where the bug is, but there’s definitely something going wrong.

Note that using a buffered stdin reader fixes it (though might still struggle on the exact boundary of the buffer, haven’t tested that):

    var buffered_stdin = std.io.bufferedReader(std.io.getStdIn().reader());
    const stdin = buffered_stdin.reader();

    // same code as above
4 Likes
10

Thanks! That’ll do for now!