Is having a centralized stdout/stderr location no longer recommended?

Hey all,

I normally centralize the stdout/stderr buffer/writer/interface in a file called utils.zig in its global scope. When the program starts it can set all of that up, and I can start using my own print API in order to print to stdout/stderr without the consumers having to worry about all of that. Since 0.16.0 introduces the Io DI/IoC strategy, the “writer()” requires an std.Io which makes sense. However this means that I would need to get the init.io from main once it starts up, and then initialize a publicly accessible variable at the top of theutils.zig (but the flow would need to change given that we need to take the address of the instance of the writer returned by the writer() function. I’m wondering if this centralization is no longer recommended or if I need to move the core stdout/stderr wiring buffers, and other stuff back to main and then try to centralize it as mentioned above?

Current Implementation in utils.zig:

https://codeberg.org/fearedbliss/Cantaloupe/src/branch/main/src/utils.zig#L29

const std = @import("std");
const builtin = @import("builtin");

const BASIC_BUFFER_SIZE = 1024;

var stdout_buffer: [BASIC_BUFFER_SIZE]u8 = undefined;
var stderr_buffer: [BASIC_BUFFER_SIZE]u8 = undefined;

var stdout_writer = std.fs.File.stdout().writer(&stdout_buffer);
var stderr_writer = std.fs.File.stderr().writer(&stderr_buffer);

const out = &stdout_writer.interface;
const err = &stderr_writer.interface;

/// Prints a message to stdout. Messages won't be outputted in tests.
pub fn print(comptime message: []const u8, args: anytype) !void {
    if (builtin.is_test) return;
    try out.print(message, args);
    try out.flush();
}

/// Prints a message to stderr. Messages won't be outputted in tests.
pub fn eprint(comptime message: []const u8, args: anytype) !void {
    if (builtin.is_test) return;
    try err.print(message, args);
    try err.flush();
}

Thank you and stay safe,

Jonathan

1 Like

The way I understand it the centralization is primarily not recommended for code that might be re-used and handed in a different Io strategy. Much like handing in an Allocator instead of relying on some global instance.

Given that, for your own code it should be fine to get init.Io from main, as long as you don’t expect utils.zig to be reused by others (or by yourself with different strategies).

My strategy in these cases is to upgrade gradually. Since print and eprint differ ony wrt out and err, you only need one

ng_print(dest: *std.Io.Writer, comptime message: []const u8, args: anytype) !void {
    if (builtin.is_test) return;
    try dest.print(message, args);
    try dest.flush();
}

And then start using it one at a time, which means adapting every function signature in the call stack to include : *Io.Writer and passing it on. Once everything is done you can remove print and eprint and rename ng_print globally. (Depending on your editor you might be better of temporarily use something like p_rint() instead of ng_print(), so searching for print will not match against the new function).

Global variables are, and were, never recommended! But for code that is not going to be reused outside the project it does not particularly matter.

@vulpesx I know global variables in general are not recommended (this isn’t really anything new in software engineering), however I do remember reading or hearing somewhere that when it comes to printing (out/err) it was better to create the instance globally and share that across the app.

@Anthon Thank you, I’ll need to research a bit more and think about how I want to proceed.

Hey @Anthon and @vulpesx, I was able to update both of my projects today from 0.15.2 to 0.16.0 using a solution that I’m mostly happy with (except for unit tests, I’m not 100% happy with the solution but it works and seems to be necessary with the current lay of the land):

src/utils.zig (https://codeberg.org/fearedbliss/Cantaloupe/src/branch/main/src/utils.zig#L36):

//! Provides centralized utility functions.
const Utils = @This();

const std = @import("std");
const builtin = @import("builtin");

// Continue to keep the stdout/stderr printing centralized, but moving the
// IO setup logic back to main since we need to get the IO implementation
// from main anyways as of 0.16.0. It also allows us to simplify the IO
// interface/writer lifetimes as well.
io: std.Io,
stdout: *std.Io.Writer,
stderr: *std.Io.Writer,

/// Creates a new Utility structure.
pub fn init(io: std.Io, stdout: *std.Io.Writer, stderr: *std.Io.Writer) Utils {
    return .{
        .io = io,
        .stdout = stdout,
        .stderr = stderr,
    };
}

/// Prints a message to stdout. Messages won't be outputted in tests.
pub fn print(self: Utils, comptime message: []const u8, args: anytype) !void {
    try self.base_print(self.stdout, message, args);
}

/// Prints a message to stderr. Messages won't be outputted in tests.
pub fn eprint(self: Utils, comptime message: []const u8, args: anytype) !void {
    try self.base_print(self.stderr, message, args);
}

/// Prints a message to the destination. Messages won't be outputted in tests.
pub fn base_print(_: Utils, dest: *std.Io.Writer, comptime message: []const u8, args: anytype) !void {
    if (builtin.is_test) return;
    try dest.print(message, args);
    try dest.flush();
}

src/main.zig (https://codeberg.org/fearedbliss/Cantaloupe/src/branch/main/src/main.zig#L42):

pub fn main(init: std.process.Init) !void {
    var aalloc = init.arena;
    const arena = aalloc.allocator();
    const io = init.io;

    var stdout_buffer: [1024]u8 = undefined;
    var stderr_buffer: [1024]u8 = undefined;
    var stdout_writer = std.Io.File.stdout().writer(io, &stdout_buffer);
    var stderr_writer = std.Io.File.stderr().writer(io, &stderr_buffer);
    const stdout = &stdout_writer.interface;
    const stderr = &stderr_writer.interface;
    const utils = Utils.init(io, stdout, stderr);
...

All references to Utils.print(..) were tweaked to use their instanced equivalents which allows the newly captured “pointer to stdout/err/in” references to be used from one centralized location. Allowing all consumers of the utils to not have to worry about printing internals.

My Honeydew app uses the same thing but it needs stdin as well for user input. The same technique worked but just need to also bring in the stdin * hooks.

...
var stdin_buffer: [1024]u8 = undefined;
...
var stdin_reader = std.Io.File.stdin().reader(io, &stdin_buffer);
...
const stdin = &stdin_reader.interface;
const utils = Utils.init(io, stdout, stderr, stdin);

One of the downsides of this is that any function that needs to print to stdout/stderr
needs to have the utility instanced passed down, but I suppose that is fine given that
regardless the “io” instance would have needed to be passed down as well. The part
that bothers me more is specifically when it comes to testing. Since a few of my functions
do print to stdout/stderr inside of their function blocks (depending if I want to alert
the user or something or if there is an error that should be reported to the console),
every single test needs to have the entire wiring defined as well. At the moment I’m
not sure if there is any way to make this shorter other than me removing the “*_buffer”
variables and having the writer() just use &.{} so that buffering is disabled,
but that only saves me two lines.

You can see that for this test, 4 of the lines is just setting up the stdout/stderr wiring so I can actually initialize the rest of the system (6 if we include stdin). This is also taking into account that I’m not including the lines for the dedicated buffer space if I were to define those as well. Ultimately, it was easier to keep all of them together (in the same test scope and in main() during real execution) since it ensures that all lifetimes and pointers are valid:

src/root.zig (https://codeberg.org/fearedbliss/Cantaloupe/src/branch/main/src/root.zig#L329):

test "get_snapshots_from_source_should_return_organized_snapshots_for_datasets_with_specific_label" {
    var aalloc = ArenaAllocator.init(std.testing.allocator);
    defer aalloc.deinit();
    const arena = aalloc.allocator();
    const io = std.testing.io;
    var stdout_writer = std.Io.File.stdout().writer(io, &.{});
    var stderr_writer = std.Io.File.stderr().writer(io, &.{});
    const stdout = &stdout_writer.interface;
    const stderr = &stderr_writer.interface;
    const utils = Utils.init(io, stdout, stderr);
    var datasets = [_][]const u8{
        "tank/leslie",
    };
    var coms = Communicator{};
    coms.get_snapshots_from_system_for_dataset_return_value = &.{
        "tank/leslie@2024-06-23-1500-00-CHECKPOINT",
        "tank/leslie@2024-07-24-1500-00-CHECKPOINT",
        "tank/leslie@2024-08-12-1200-00-LOL",
    };
    var result_buckets = StringHashMap([]Snapshot).init(arena);

    try get_snapshots_from_source(
        arena,
        io,
        &coms,
        utils,
        &datasets,
        "CHECKPOINT",
        &result_buckets,
    );

    try expect(result_buckets.contains(datasets[0]));
    const snapshots = result_buckets.get(datasets[0]).?;
    try expect(snapshots.len == 2);
    try expect(std.mem.eql(u8, snapshots[0].entry, "tank/leslie@2024-06-23-1500-00-CHECKPOINT"));
    try expect(std.mem.eql(u8, snapshots[1].entry, "tank/leslie@2024-07-24-1500-00-CHECKPOINT"));
}

@vulpesx I was also able to find one reference in the 0.15.1 release notes regarding the recommendation of making the stdout buffer global, but I may have misinterpreted in that maybe it just meant that the dedicated buffer space should be global and now the stdout() and stdout.writer(&buffer) lines as well:

I’m also wondering if making the stdout buffer global thread safe with the current writers (and readers?).

Zig newbie. I was wrestling with this as well. My instinct as always is that globals are bad and we should avoid. But passing alloc+io+env to hundreds of fns doesn’t feel great either. For 0.15.2 I too had a util helper with globals - stdxxx/buffers, an env map, etc.

As part of my 0.16 upgrade this week I switched to an App struct that’s built from juicymain (or the test equivalent). I will regret this if I decide to package my project as a lib, which is def possible. I suppose I could convert to a Contextinstead, another common pattern.

I checked out several other prominent Zig projects and I would tentatively say there is no consensus yet.

One additional consideration - it’s desirable to develop strong language conventions over time. We should document and advocate. This makes the LLMs improve rapidly, which also increases adoption of Zig. As anecdata, during my 0.16 upgrade this was the major pain point for codex - figuring out how to pass around io stuff.