Different behavior when reading from stdin on arm64 vs amd64

Help
,
1

I am taking in input from stdin and then printing that out to stdout. However, on my laptop(amd64), only the end of the bytes are outputted, while on my android phone via Termux(arm64), it echos as expected.

Edit: Im compiling with zig version 0.16.0 on both

 pub fn scanf(io: std.Io) ![]u8 {                                      
     var stdin_buffer: [1024]u8 = undefined;
     var stdin_reader = std.Io.File.stdin().reader(io, &stdin_buffer);
   
     const stdin = &stdin_reader.interface;
     //setup reader
 
     const line = try stdin.takeDelimiterExclusive('\n');
 
     return line;
 }

pub fn main(init: std.process.Init) !void {                                  
    print("My first Zig program  {s}\n", .{"YAYY!"});
    //   print("gameboard {any}\n",.{gameboard});
    const gameboard = Board.init(DBS);
    display_board(DBS, gameboard);
    const ng = Game.new();
    print("{}\n\n\n\n", .{ng});
    print("> ", .{});
                    
    var stdout_buffer: [1024]u8 = undefined;
    var stdout_writer = std.Io.File.stdout().writer(init.io, &stdout_buffer);
    const stdout = &stdout_writer.interface;
    // _ = stdout;
 
    const line = try scanf(init.io);
 
    //print("{s}\n", .{line});
    try stdout.print("{s}\n", .{line});
    try stdout.flush();
}

amd64 output

Results

My first Zig program YAYY!
0 0 0 0 0
1 0 0 0 0
2 0 0 0 0
3 0 0 0 0

0 1 2 3
.{ .players = .{ .items = { }, .capacity = 0 }, .active_resource = .Brick }

> hello world

rld

arm64 output (Expected)

Results

My first Zig program YAYY!
0 0 0 0 0
1 0 0 0 0
2 0 0 0 0
3 0 0 0 0

0 1 2 3
.{ .players = .{ .items = { }, .capacity = 0 }, .active_resource = .Brick }

> hello world
hello world

2

Hello, its because what try stdin.takeDelimiterExclusive('\n') returns is a slice pointing to the buffer of the reader.
And since your buffer is invalidated once you leave the scanf(init.io), its UB.
The stack your buffer lies in, gets overwritten with further function calls, so based on platform it will have different behaviors #UB.

You can either:

  • allocate the buffer (but you still have issues that it will error out if the line is larger than the buffer)
  • have one global stdin reader / buffer on stack, but you still have problem of the line size > buffer_size.
  • if you want really universal solution, i think the cleanest way is to use std.Io.Writer.Allocating and stream the line to it up to the delimiter (there is a function for that), then you can just refer to the written() on the alloc. writer if in scope, or convert it to ArrayList(u8) for example and work with it like that if you prefer (again there is a method for that).

Robert :blush:

2 Likes
3

Ohhhhh, I havent allocated anything to the heap! thank you for pointing that out. This is my first time working with a language with manual memory management. Im learning alot!

edit: what do you mean by UB?

2 Likes
4

For your example, you can also declare a buffer in main and pass a pointer to it as a []u8 parameter to the function scanf. The function should return a []const u8 slice, but now it would be a slice that “points” to this buffer, which has the same lifetime as main.

1 Like
5

UB: Undefined Behaviour

1 Like
7

hmmm, this might be what I do. I started allocating in the function, but then realized I would have to destroy it in function or have it left dangling otherwise(memory leak).

8

UB means Undefined Behavior, quite literally “we cant guarantee what will happen” state.

Its something non low level programmers often think its the absolute devil, they are wrong.
It can happen in a lot of languages that are not manually managed (but its harder, of course).

Basically every programming language has some things you are not supposed to do, like for example here, using stack allocated variables past the scope they are declared in.

My favourite example of UB, and why it is actually important is this:
(assume all numbers are integers, signess is not relevant for this)

const num_modified = (num * 2) / 2;

This is something that has UB, the overflow is undefined, Zig will panic with stack trace in Debug build if it happens, but it allows the compiler to transform it to the expected:

const num_modified = num;

However if you define the overflow, like this (using Zig code, but this is for example defaut behavior in Rust):

const num_modified = (num *% 2) /% 2;

This now cannot be safely transfromed to the “logical” optimalization, and will be instead optimized to something like this:

const num_modified = (num + num) >> 1;

It will smartly avoid the expensive multiplication and division, by adding the numbers together and then shifting the bits right, but it still is not the “logically” optimised operation.

Now the important WHY?

Imagine our datatype is u8, so what will happen if we multiply 128 by 2? we get 256, which cant fit to u8 and then overflows to 0. Then we divide it by 2 and get 0 again.

So if we dont define overflow, we can get more optimal code, at the cost of wrong result if it were to happen.

Hope that helps, Robert :blush:

PS: if you have any more questions, or overall need more help with anything, feel free to ask here :heart:

2 Likes
9

Or, if you dont “sub-slice” aka. the allocation is actually exactly sized as the line you are returning from the scanf(), you can just return slice to it and then have:

// stdin being a global/top level buffered Io.Reader, so you dont have isssues when calling the function multiple times over
// (the buffer would fill as much as it can, and consume more of the input than you parsed, overall you dont want to have multiple buffered readers/writers to streaming file descriptors like stdin/stdout, you never know how much of it they actually consumed, so use 1 ideally)
const line = try scanf(allocator, stdin);
defer allocator.free(line);

You can achieve that in multiple ways, one of them being just allocator.dupe() the buffer before returning and then freeing the one in the function.

If you want to return only the slice and want something better than dupe (less wastefull), i would personally (a bit more complicated), streamed the line to the allocating writer till the delimiter, transformed it to the arraylist via .toArrayList(), then called .shrinkToLen() on it, and then returned from the function result of .toOwnedSlice() from the arraylist.

If you are not keen on returning a slice, I would personally just return arraylist directly from your scanf(), yes it may be not as clean as just a slice, but if you do any more transformations on the line further, it may be a better fit. (just a reminder, you then need to defer deinit() on the returned arraylist from the scanf)

PS when i am talking about defer free/deinit, it just means the caller of the function is then responsible for cleaning up the returned resource.

Robert :blush:

10

just an fyi, that zig does not use the terminology “undefined behaviour”, instead zig uses “illegal behaviour” (IB) because

  1. it clearly communicates zig programs are not supposed to do those things
  2. it may actually be defined, but still be bad/not allowed, e.g. index out of bounds.

Zig also has the distinction of checked illegal behaviour, where zig is able to add safety checks --in safe build modes-- to catch it early and with more useful information; and unchecked illegal behaviour, where zig does not currently have safety checks, and may never for some IB.

You encountered unchecked illegal behaviour (UIB)

11

But wait! There’s more!

  • pass the buffer as a parameter, now it’s the callers’ responsibility to ensure it’s valid long enough
  • pass the reader interface as a parameter, now it works for any source of data + it’s the callers’ responsibility to ensure it is valid long enough
1 Like