Where do I find clarity on the future (and current) relationship between fs.File and io.File? The “arrow” is fairly clear, and certain things are stated, like preferring std.Io.Dir.cwd().openFile over the older fs version, but 0.15.1 release notes, despite focusing on Io, also discusses fs.File.Reader and fs.File.Writer and how they fulfill the std.Io.Reader and .Writer interfaces. But will they eventually be completely replaced by Io.File? Somehow I’ve missed that clarity, though it’s probably in clear sight.
Ah.. maybe this is it
andy@bark ~/s/zig (std.Io-fs)> gd master --stat
CMakeLists.txt | 2 -
build.zig | 35 +-
lib/compiler/build_runner.zig | 14 +-
lib/std/Build.zig | 7 +-
lib/std/Build/Cache.zig | 83 +--
lib/std/Build/Cache/Directory.zig | 10 +-
lib/std/Build/Cache/Path.zig | 24 +-
lib/std/Build/Step.zig | 10 +-
lib/std/Build/Step/Compile.zig | 35 +-
lib/std/Build/Step/InstallDir.zig | 7 +-
lib/std/Build/Step/Options.zig | 42 +-
lib/std/Io.zig | 34 +-
lib/std/Io/Dir.zig | 1261 ++++++++++++++++++++++++++++++++++++++++--
lib/std/Io/File.zig | 628 ++++++++-------------
lib/std/Io/Threaded.zig | 2006 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
lib/std/debug.zig | 80 +--
lib/std/fs.zig | 69 +--
lib/std/fs/AtomicFile.zig | 94 ----
lib/std/fs/Dir.zig | 2066 ---------------------------------------------------------------------
lib/std/fs/File.zig | 1050 +----------------------------------
lib/std/fs/test.zig | 15 -
lib/std/os.zig | 130 -----
lib/std/posix.zig | 1184 ++-------------------------------------
lib/std/std.zig | 3 +
lib/std/zig/system.zig | 6 +-
25 files changed, 3585 insertions(+), 5310 deletions(-)
std.fs.File and std.fs.Dir will be deleted soon
14 Likes
To clarify, std.Io.File doesn’t exist in 0.15.x.
In the master branch, the fs.File → Io.File transition is a work-in-progress, and was initially introduced in
Relevant follow-up issue:
3 Likes
Ha! Now that’s horizontal-scrollbar-clear!
In this case, though there are some snippets in the release notes and elsewhere, I’d like critique on a couple of possible basic usage examples for reading a text file line-by-line assuming \n delimiters:
First, an allocator-less one that requires a big-enough buffer:
const io = std.testing.io;
var file = try std.Io.Dir.cwd().openFile(io, "test.txt", .{});
defer file.close(io);
var buf: [1024]u8 = undefined; // must be big enough for an entire line
var reader: std.Io.File.Reader = file.reader(io, &buf);
while(reader.interface.takeDelimiter('\n')) |line| {
if (line) |l| { std.debug.print("line: {s}\n", .{l}); }
else { break; }
} else |err| return err;
Second, using Writer.Allocating:
const io = std.testing.io;
var file = try std.Io.Dir.cwd().openFile(io, "test.txt", .{});
defer file.close(io);
var gpa = std.heap.DebugAllocator(.{}){};
defer _ = gpa.deinit();
const alloc = gpa.allocator();
var line = std.Io.Writer.Allocating.init(alloc);
defer line.deinit();
var buf: [1]u8 = undefined; // ridiculously small for illustration
var reader: std.Io.File.Reader = file.reader(io, &buf);
while(reader.interface.streamDelimiter(&line.writer, '\n')) |count| {
_ = count;
_ = reader.interface.toss(1);
std.debug.print("line: {s}\n", .{line.written()});
line.clearRetainingCapacity();
} else |err| switch (err) {
error.EndOfStream => {}, // while loop done
error.WriteFailed, error.ReadFailed => |e| return e,
}
if (line.written().len > 0) {
std.debug.print("tail: {s}\n", .{line.written()});
}
Please criticize without reservation. I’m not suggesting it’s a model.
Those are good approaches. And the only additional case is to handle any remaining data beyond the last delimiter. There can be some pathological files ending with non-LF characters.
See the last part in this blog.
1 Like
Ah, right, of course, the poor puppy’s tail. I’ll glue it back on….
Ok, I fixed the above.
Adding a tail processor to the Allocating variant was straightforward enough, but the stack-based variant proved more interesting. It contracted substantially by making use of takeDelimiter rather than takeDelimiterInclusive - in this case, the ‘line’ is returned even if EOF is encountered (EOF is treated like the delimiter case). This simplifies the else |err| handling, as all remaining errors are real errors, and it might be commonplace to just pass them up.
This inspired two splinters.
First: should the documentation for takeDelimiterInclusive, which says “advancing the seek position past the delimiter,” say, instead, “advancing the seek position past the delimiter, unless an error is encountered”? as the code does seem to suggest that the seek position is not advanced if a peek() fails to find the delimiter before encountering EOF. Then, another (more verbose) way of dealing with this would be to keep my takeDelimiterInclusive code, but add tail-processing code below, as done with the heap version.
Second: I consulted this documentation and this documentation on optionals, and didn’t see detail on !?T handling. Elsewhere it’s (erroneously?) suggested that the while loop breaks when an optional conditional is null, but in this interesting case with else |err| handling, I found, here, at least, that |line| simply remains an optional, so, as you see, I have to if-unwrap it to its raw string value within the while braces. This compiles, so I assume it’s right, but, again, I’m curious about the documentation.. maybe it wants to gain a short paragraph about this double-whammy scenario.
(Third?!: I suppose I feel like making one more proposal/question: why not call takeDelimiter (and variants) takeToDelimiter(), since that’s what’s actually being done. I wonder if, since this is fresh on the press, it would be a good time to do that now, early?)
For the record, since I modified the stack case substantially, above, here’s the old inferior bit:
// INFERIOR / discarded version:
while(reader.interface.takeDelimiterInclusive('\n')) |line| {
std.debug.print("line: {s}\n", .{line});
} else |err| switch (err) {
error.EndOfStream => {}, // while loop done
error.StreamTooLong, error.ReadFailed => |e| return e,
} // note: there's still a tail to process!!
I’m also interested in more followup advice on the reader-to-writer pattern, with my:
Should documentation offer some guidance on that Reader buf, in the event of direct-pathing the read-to-write stream, where there’s a heap-allocated buffer to receive data?
Thanks for sharing. I enjoyed reading the series of blog posts!!
2 Likes
Finally, I’ve collected the takeaways into an article
https://codeberg.org/jmcaine/zig-notes/src/branch/main/file-io.md
This is my first of the sort, and stems from not finding something like it. Please critique liberally (I’d be honored). I reference official doc and std doc heavily, and cite (and thank) pedropark99’s zig book, chapter 12 and williamw520’s I/O “basics” article, though both of those currently reference 0.15 rather than 0.16.
4 Likes
This is awesome. Now we will be able to route newbies (like myself) asking for a howto on reading/writing to files, to your article. Your “stream pattern” is helping me. Thanks.
I have 2 tiny observations (not mistakes), for your judgment to see if it is worth mentioning:
-
On the section “Reading a whole file”, just to mention that the result type of
std.Io.Dir.readFileis a slice (pointer) to buf and not a new allocated thing, just to reinforce zig’s policy of no hidden allocations; -
When one decides to allocate a buffer, instead of using one on the stack like you do on your examples, has to pay attention to the proper sintax:
const buf_siz = 1024 * 4;
var buf:[]u8 = try allocator.alloc(u8, buf_siz);
var reader: std.Io.File.Reader = file.reader(io, buf);
in this case, buf is passed to Reader without the & because it is already a pointer.
Also thanks to take my attention to std.mem.tokenizeSequence, I was not aware of it.
Indeed, this is the kind of thing that becomes “natural” too quickly, and is still foreign to noobs; I shall add a clarifying phrase.
Right… I’ll consider adding a little more, without detouring too much, but think at least a reference to Allocator documentation is in order. Thanks.