This is my first Zig project. zlist
It’s a simple alternative to ls. If you like this project, could you please give it a star to encourage me? Feel free to give me feedback if you encounter any problems.
Also, this project is built on Zig 0.16.
8 Likes
Very cool project, I also implemented ls once for fun 
If you do call it ls on your system though - instead of say zls - you will run into issues running scripts that rely on the POSIX specification of ls. But you may already know this and plan to make it fully POSIX compliant 
Some style suggestions:
pub fn main(init: std.process.Init) !void {
const allocator = std.heap.page_allocator;
If you’re using juicy main, why not use the arena provided for you in init.arena? page_allocator is going to allocate 4 KiB minimum per allocation.
The same with:
var threaded: std.Io.Threaded = .init_single_threaded;
const io = threaded.io();
Why not use init.io and set single_threaded in build.zig?
If you just want the args and environment, then you should probably be using std.process.Init.Minimal instead.
\\-s, --sort <u8> Sort results. Options: 0-name(asc, Default) 1-name length(asc).
I believe clap supports enums.
const ext = std.fs.path.extension(name);
if (std.mem.eql(u8, ext, ".zig")) {
return " ";
} else if (std.mem.eql(u8, ext, ".go")) {
return " ";
This might be a good use for StaticStringMap.
pub const Color = struct {
std.Io.Terminal exists, and would allow a path for non-posix operating system support in the future.
const c = @cImport({
Consider migrating to using an addTranslateC build step rather than @cImport, as there’s a pretty good chance that @cImport will be axed. In your case it might not even be necessary, as getuid, getpwuid, getgid and getgrgid are already available in the std.c namespace.
1 Like
Thank you for your suggestions. I’ll take a look at these points!
Regarding the use of page_allocator, my initial thought was to allocate more memory at once to reduce the number of subsequent memory allocations. However, it seems that this ls tool doesn’t need 4KB of memory. Perhaps another allocator could be used instead.
The initial implementation of the getIcon method used a HashMap(StaticStringMap), but I forgot why I changed it to a simple if-else statement.
As for the other points you mentioned, many of them are unfamiliar to me as a beginner in Zig. I’ve moved these questions to the issue section of the repository. Thank you so much for your suggestions.
Hello, thank you for your reply.
Yes, I previously considered naming it simply “ls” as not being ideal. This is because some other scripts might also depend on “ls,” and if the custom tool doesn’t support it, it might throw errors.
However, other naming conventions each have their drawbacks (zls would conflict with lsp, and zlist is a bit long). But now I prefer zlist.
Regarding the POSIX specification you mentioned, that’s definitely something that needs to be implemented. However, it might take some time. I will continue to iterate on this tool. Thank you so much for your reply.
1 Like
My take on projects like this is that they are great learning projects, but having a readme that sells it as the fastest, greatest, most featureful tool is asking for trouble.
The Rust community is making a bad name for itself because there’s a wave of “rewrite it in rust”. Somebody decided rewriting GNU fileutils was a good exercise. Everybody else copied and now there’s hundreds of abandoned half done versions of these tools around. I don’t want to see Zig fall in the same trap.
Making stuff public to get feedback and showcase your skills is good. Just write a readme that’s honest about why the project exists. You’re learning / experimenting. That’s ok. Everybody needs to do that.
Trust me. You don’t want people demanding support on your first project in a language.
8 Likes
You can do that like this:
const allocator = init.arena.allocator();
_ = try allocator.alloc(u8, PRE_HEAT);
arena.reset(.retain_capacity);
Where PRE_HEAT is however many bytes you expect to use.
The page_allocator allocates onto a fresh page, every time. If you ask it for 64 bytes, it allocates a page, and give you 64 bytes of it. This is a common thing to get tripped up on FWIW.
1 Like
Depending on your reasons for pre-heating, you might also want to touch every page allocated:
const allocator = init.arena.allocator();
const allocated = try allocator.alloc(u8, PRE_HEAT);
const page_size = std.heap.pageSize();
const page_count = allocated.len / page_size;
for (0..page_count) |i| {
allocated[i * page_size] = 0;
}
arena.reset(.retain_capacity);
Otherwise you also risk incurring page fault overhead on subsequent allocations, which may be comparable to the syscall overhead from just doing the allocations without a pre-heating step.
For an ls utility, I wouldn’t concern myself with this though. I’d just use the init.arena allocator directly. It’s designed to present things to the user on a terminal that is at max a few hundred times a few hundred characters across. Directory entries are small. You’ll be allocating on the order of a few hundred kilobytes. Latency for the allocations will be imperceptible to the user. Your program has already probably already reserved far more than that in virtual memory anyway, and if ls is causing your system to start swapping, it’s probably not ls’s fault.
Disclaimer: I’m also guilty of taking micro-optimizations and premature optimizations too far as well. I recently at work wrote a TCP proxy that takes advantage of FreeBSD’s ability to splice one TCP connection to the other, and handle it in entirely in kernel-space. I managed to bring it down to about 25-bytes heap allocation per connected client (I’ve thought of a way of bringing it down further than involves a technique like a XOR-linked list, but I’m not getting that passed a PR review), and still saturate the network card, for tens of thousands of simultaneous connections on some very underpowered CPUs. The userspace side only uses a single core (the kernel uses 16-threads for the copying, admittedly, and I’ve no idea how much memory it dedicates to this). I was pretty proud of getting it to the point where the entire program state fits can fit in L1d-cache for our typical use cases.
But it’s entirely unnecessary, because the smallest VM we could run it on will still have 128MiB of memory.
But it is fun.
The eternal quandary.
1 Like
Hi @WeeBull .
Thank you for your suggestion. Reflecting on my actions, I admit that I boasted about being the “fastest” in the README to gain more attention. However, it was just my first step in Zig, and it’s not as good as I originally claimed. The README document has been revised accordingly.
Another point is that I didn’t realize Zig is a relatively new language and still needs contributions from the community. My exaggerated claims about this small tool might have risked negatively impacting Zig’s image. Thank you for your suggestion.
7 Likes
Thank you so much to everyone who provided feedback!
I’ve carefully read through all the suggestions and made corrections to the main branch (tags: v0.0.2 and v0.0.3).
This is a very vibrant community. I really wanted to tag everyone who provided feedback, but I was afraid of bothering them. So, Happy New Year to everyone reading this comment!
Fun thing, the original version of the readme, i.e. the claim being very fast motivated me to write a little benchmark for the std.time.epoch functionality you use Since you need to derive a formatted date/time for every file and directory, this code runs rather often. So I thought this might be worth a look and compared it to the algorithm from C++ date / H.Hinnant (“hin”) and a more recent version by Neri & Schneider (“neri”; the “cpp” means translated from their C++ reference). Some results:
zdt/benchmark
❯ zig15 build run --release=fast
benchmark runs total time time/run (avg ± σ) (min ... max) p75 p99 p995
-----------------------------------------------------------------------------------------------------------------------------
std epochday-2-date 100000 246.556ms 2.465us ± 1.603us (2.29us ... 394.748us) 2.328us 6.554us 14.239us
[...]
hin rd-2-date 100000 65.52ms 655ns ± 131ns (636ns ... 22.488us) 654ns 662ns 690ns
ner rd-2-date (cpp) 100000 20.241ms 202ns ± 52ns (192ns ... 7.353us) 203ns 209ns 211ns
So we could roughly improve the performance of std.time.epoch by x4 (hin) or x12 (neri). Note though that I’m not sure about the contribution this has to your program’s performance overall. Might be worth a look anyways.
The code for the results above can be found over here: They speak. - you’ll need to run it with zig 0.15.2, 0.16-master does not work at the moment.
EDIT: link to the code above is wrong. It’s in the zig-0.16 branch of zdt; With knees.
1 Like
Hi @FObersteiner .
Thanks! Just seeing this now. I’ll check it out later.
If it speeds things up, I’ll make an issue for it and add the optimization.
Thank you again for your suggestion!
Have fun The reference implementation is here: eaf/algorithms/neri_schneider.hpp at main · cassioneri/eaf · GitHub - it is MIT-licensed, if you decide to use it. My Zig version is just a translation and public domain, so no need to attribute that.
2 Likes
Thanks for all the feedback! I’ve implemented most of the suggestions
Project is getting more stable now.
Would love more feedback!
Im just started working on a private project which should become kind of a ls tui. Therefore, i got a question: can you imagine making the dir/file parsing part of your code available as lib?
Of course, i could reuse your code thanks to MIT licensing, but would prefer an integrated solution for proper referencing and staying up-to-date with changes/enhancements on your code
Hi @lukeflo .
I’ve thought about it — thanks for the suggestion. Sounds like a good idea.
Yeah, I’d definitely be open to that. I’m planning to extract the dir/file parsing part (currently in src/files.zig, src/file.zig, src/git.zig, src/opt.zig) into a reusable module so it can be used as a library.
I’ll likely expose it as a Zig package so it can be added via zig fetch --save. I still need to refactor things a bit, but that’s the direction I’m heading.
It might take a few weeks since I’m a bit busy recently. I’ll open an issue to track this and close it once it’s done.
Just for fun and the purpose of learning Zig I tried to implement this algorithm for the named zlist project:
1 Like
Good idea, I’ve already replied to the corresponding PR on GitHub.
2 Likes