Hello! It’s been a while. I was shown the beauty of a source documentation generator called docco.coffee and suddenly I had to create my third go at a Zig Stdlib website generator.
Note that for rapid development (a single night), this tool was written in Ruby 
, not Zig. 
Here’s the repo:
Here’s the results:
Very interesting. Can you speak to the maintainability of this documentation?
For instance, does it pick-out doc-comments above the code and provide them or do we need to provide information for each function that we’re trying to document?
I’m curious about this because it seems like Ziglang is trying to take a hands-off approach to documentation generation. To be clear, it’s not that they don’t provide some explanation (many things are still being developed), but it seems like they ultimately want to have documentation that stays automatically synchronized with source.
In my opinion, this is the holy grail of software documentation. It’s alive; not dead letter that can become out-of-sync with the actual code. @kristoff 's work on the stdlib autodoc has been impressive and monumental given that with Zig, you can’t just parse the source to extract whtat you want, you have to go deeper into the Zig IR level because of comptime and the way types are handled. A real challenge.
It really is the goal. I’m interested in seeing how it pans out.
There’s always information that you’d like to have available about things that doesn’t seem appropriate for source code (examples of each function, related functions… etc). Those seem to be the job of an extrinsic tagging system that can keep track of relationships.
I think the standard line with Zig really should be “if you want more information, just read the code” lol. That can seem like a disappointing answer to some, but we’ve already mentioned how approachable the source actually is.
@ratfactor , really cool how it generates the sub-headings on the left to line-up with the functions in the source on the right!
Very interesting. Can you speak to the maintainability of this documentation?
Sure! It’s 100% automated. There’s nothing stopping me from putting this on a cron job to download the nightly release and re-generate these.
For instance, does it pick-out doc-comments above the code…
Yup! That’s exactly what it does! It’s intentionally dumb as a box of rocks. Here’s the part that collects doc comments: zstd-browse2/makepage.rb
I think tests can be a good way to show example usage.
An exhaustive test suite is probably too long to be included in the automatic docs in its entirety. But maybe some tests could be tagged to be included in automatic docs.
I thought about putting the names of tests on the documentation side too. It’s nice that they have names. I would style them differently.
In case you’re not familiar with them, tests that refer to a decl name are meant to be documentation:
const Foo = struct { ... };
test Foo {
// usage examples, meant to be shown in docs
}
test "foo - functionality 1" {
// normal test not meant to be shown in docs
}
Although I dunno how that should work visually with your styling.
Anyway, this a very neat way of displaying the source code, I tend to think of it as the “literate programming”-style of visualizing code.
Autodoc is going for something very different in its main interface. Maybe it would make sense to use this visualization style for the source viewer, but this is a competing idea with other things that could be done with it, so I can’t say for sure if this is the preferred approach.
I do actually want to do something vaguely along these lines for doc test bodies: since doc tests are meant to be shown as docs, I think it would be interesting to treat normal comments as doc comments in that context and provide a “jupyter notebook” style of visualization (which is vaguely similar to what you are doing with zstd-browse2).
Given this doc test:
test MyStruct {
// MyStruct needs an allocator so we must get one.
// In this test I'm using the testing allocator but you would normally
// want to use the GeneralPurposeAllocator.
const ally = std.testing.allocator;
// Create an instance of the struct:
var t = MyStruct.init(ally);
// bla bla bla
t.blabla();
}
This is how it would show:
MyStruct needs an allocator so we must get one.
In this test I’m using the testing allocator but you would normally want to use the GeneralPurposeAllocator.
const ally = std.testing.allocator;
Create an instance of the struct:
var t = MyStruct.init(ally);
bla bla bla
t.blabla();
I actually wish I had more time & brain power to dedicate to Autodoc. Unfortunately there are a lot of other things that for me have to take priority, which really makes it hard to sit down and reason lucidly about all the language stuff.
I wouldn’t mind for somebody to take on some of that :^)
(not-so-subtle wink to all the people in this thread that already have significant projects and/or experience writing educational material for Zig)
I noticed some issues on the mobile version of the std docs, which could probably be addressed wih a few css rules
indeed. lol
I would love to help in anyway I can but have no idea where to start looking. Could you point to an entry point of sorts where I can start studying the inner workings of Autodoc? I thought about this a while back but was scared off by my total lack of knowledge about Zig IR.
In case you’re not familiar with them, tests that refer to a decl name are meant to be documentation:
Thanks! I guess I’d seen that without even realizing what I was looking at. I’ve got a TODO in my notebook to add tests to the left “documentation” column. I’ll probably just display them in the order they appear in the source for this little tool.
Anyway, this a very neat way of displaying the source code
Thanks! And to be clear: this is absolutely not meant to be a competitor to real documentation like Autodoc is aiming for. This is strictly meant to be an aide to reading the source of the standard library.
I tend to think of it as the “literate programming”-style of visualizing code.
Yeah, exactly. Though I’ve learned not to call it that or Literate Programming (capital letters) pedants will yell at you on the Internet if you do. 
As it happens, I recently made a tiny literate programming experiment (also in Ruby) and it was a pretty amazing experience “in the small”:
The clickbait title is “This README is a program!” 
A standard library in that style would be incredible, but would also be a whole different kind of thing requiring tooling to even compile it!
There is something I like in Elixir: a convention for writing examples of use in the documentation of a function, which is both used in the documentation and automatically in the test suite of the library. Really great to be sure that examples actually work.
Couldn’t agree more! A single example can often be worth 1000 words of explanation.
A recent piece of documentation I saw that really drove this home is the Recurrence Rules for the ical spec. Rather than trying to explain the billion ways the rules can interact, they just have a huge list of practical examples: iCalendar.org - 3.8.5.3. Recurrence Rule
I think Zig’s test keyword is going to be huge for helping foster this kind of semi-self-documenting code.
Okay, I’ve made the aforementioned improvements.
The Decl-named tests (found some in json.zig) display like this:
(Also shows that I added formatting for the Markdown-like backtick code highlighting.)
The “string”-named tests (where the test name can be quite long) display like this:
(Also shows that I’m now including indented doc comments for struct/enum methods.)
Gosh, this is lookin’ kinda pretty.
And for you fellow terminal lovers, I’m wrapping the code chunks (right column) in <pre> tags now so it’ll display somewhat reasonably in a textmode browser like w3m (Lynx treats table cells as vertical blocks, so it’s not great with this style):
The repo and full library results (links in original post) have been updated. There are very likely some bugs in this output, but I didn’t see any. 
Oh wow I really like this, it’s nice to navigate and poke around
Maybe there is some meta page that could allow you to navigate to things outside “std” like “builtin” builtin.zig - Zig standard library
Thanks! And yes, a human-written meta page is exactly what it’s missing. I have a start on that and hope to be able to post a follow-up here in the next couple days. 
Done! Here’s my human-curated guide to the standard library. I’ve done my best to organize things by section and give brief explanations:
(Book graphic made in Inkscape.)
Yooo, this’s nice!!
Looking really good so far!
