An edge case for Io.Dir.realpath*() functions

Explain
1

Hi everyone,

This was the topic that finally motivated my first post here. Not to try and litigate it, but try and understand it, and perhaps ask what to do about an edge case if the long considered removal were to be accepted one day.

I’m pretty new to Zig, but feel I’ve gotten pretty far. I’ve written a couple personal projects and my own C library bindings, including fun with comptime, and have gotten very good at grepping my way through the stdlib.

It is clear the I/O module’s filesystem handling is moving in a specific direction, with common StackOverflow suggestions like getFdPath being removed in 0.16. And I can see the issues in both efficiency and compatibility with the realpath family. So as I learn Zig, I’m taking its discouragement to heart. Especially because when I’ve previously gotten frustrated something was missing, there was a better way.

(For instance, by digging into how realpath is implemented, I learned that nearly every OS in the posix module has a magic file descriptor number for the working directory. Neat!)

Zig’s general philosophy seems to suggest: if you really need to do something with paths a manner more specific than Dir offers, you should be using OS-specific APIs from the start. Dir is an interface for ease of use and broad compatibility, but it has a “narrow contract” in return.

I actually did use OS specific APIs when I had to integrate a C library into a project, which only accepts paths. Treating them as opaque tokens passed by the user works well.

But what about writing unit tests for said C library integration?

Then it becomes a lot trickier. Because all of the std.testing utility functions use the Dir API, requiring things like this:

test "open" {
    // verify that calling open() creates a new file from the user's path input
    const io = std.testing.io;
    const dir = std.testing.tmpDir(.{});
    // hack: because there is no way to get an fd path anymore, move around cwd to make everything relative
    const cwd = std.Io.Dir.cwd();
    defer std.process.setCurrentDir(io, cwd) catch {};
    try std.process.setCurrentDir(io, dir.dir);

    var handle = try henrylib.open("test"); // use a relative path to keep it in the tmpDir
    defer handle.close() catch {};

    try dir.dir.access(io, "test", .{}); // verify that the file was created
}

It works, but as someone who’s written a lot of C, Python, and Rust in the past, it feels very strange and clunky. (The latter language making Path a first-class opaque object with OS-specific behaviors.)

Is this really what the Dir API intends to encourage in users of the language?

Would it be possible to have other types of path utilities for std.testing at least?

Or is this less strange and clunky in your opinions than in mine?

1 Like
2

My personal advice would be to handle removal of realpath if it happens, and otherwise use it if you think you need it. That issue and the current doc comment IMO is more about discouraging use of realpath than saying realpath has no use cases.

(i.e. if it gets removed you could always pull its functionality into your project directly or via a library, but the added friction of that would be the point).

See the “Why Dir-based for everything?” section of this issue for some stuff to consider (note: the proposal was rejected, but that section reamains relevant with regards to the difference between path-based APIs and Dir/sub_path-based APIs).

Note that this won’t do what you expect on all systems. cwd() returns AT_FDCWD on POSIX systems, which isn’t a real file descriptor (as you mentioned), and setCurrentDir handles this specifically by making it a no-op (see https://github.com/ziglang/zig/pull/17616 for context, see https://codeberg.org/ziglang/zig/pulls/36215 and the issues it links to for more context). Also, it happening to work on Windows may be considered questionable.

To get it to do what you expect, you’d need to open a real FD to the CWD:

const orig_cwd = try std.Io.Dir.cwd().openDir(io, ".", .{});
defer orig_cwd.close();

defer std.process.setCurrentDir(io, orig_cwd) catch {};
try std.process.setCurrentDir(io, dir.dir);

Technically, that functionality still exists in File.realPath/Dir.realPath (which calls into Io.VTable.fileRealPath/Io.VTable.dirRealPath).

2 Likes
3

Note that this won’t do what you expect on all systems.

Thanks for the catch on the no-op special case. I saw special handling of it other places but didn’t spot this one.

See the “Why Dir-based for everything?” section of this issue for some stuff to consider

I think the realPath removal discussion also touched on WASI and absolute paths, which is what set my expectations. That, and other comments on that issue talked about the (relatively speaking) higher overhead that such calls required. (I also wondered about weird embedded libc while reading through it, but those would probably require many more changes anyway.)

I did not realize it was TOCTOU originally driving the design, so that was interesting context.

My personal advice would be to handle removal of realpath if it happens, and otherwise use it if you think you need it.

I did start out thinking if there was one place where realPath was most reasonable, it would be a case like this. Not only consuming a Dir based API, but also unit tests. Discovering that the tests don’t compile after a future lang version upgrade is a much lower impact than having to redesign a library binding.

I will definitely consider this a nudge of encouragement. Thanks.

1 Like