Request for feedback on draft blog post: What Zig and TypeScript can Learn from Each Other

Last December, I did the 2017 and 2023 Advent of Code in Zig. (Many thanks to ziggit.dev for the help along the way!) I finally wrote a much-delayed blog post about the experience. Since I primarily work in TypeScript, the frame is “What can Zig and TypeScript learn from each other?”

I haven’t posted it yet because I’d love some input from the Zig side. All feedback is appreciated, but I’d particularly like to know if I’m mischaracterizing anything or making factually incorrect statements about Zig.
Here’s the draft: (draft) What TypeScript and Zig Can Learn from Each Other · GitHub

The post is a bit long, but the high level summary is:

  • Both comptime and Detectable Illegal Behavior would be really interesting in the context of TypeScript.
  • Zig could learn from the high priority that the TypeScript team places on the language service and other aspects of developer experience like error messages and documentation.
5 Likes

Hey! Nice Article. I added some feedback in the comments of the gist.

1 Like

Hi danvk, thank you for sharing your writing, I think your comments about having TS introduce runtime checks for things in debug mode is a truly good argument to make.

I’ll post some comments about things that I noticed while reading, most are nitpicks / small comments, I only have a more structured comment about the dev tooling section of the blog post.

– begin

Zig also embraces best practices that have emerged over the past few decades: immutability as a default,

Zig really only defaults to immutability when it comes to function arguments, most of the rest is mutable by default, most notably pointers: *Foo is mutable, *const Foo is immutable.
I’ll talk about why function arguments are immutable when commenting on a later part of the post.

Now the safety checks are off and the integer overflow is allowed to happen.

To be precise, integer overflow is not expected to happen. In Zig integer overflow is considered UB so, in unsafe release modes, the optimizer is allowed to leverage the assumption that it won’t happen, resulting in arbitrarily bad outcomes if that expectation is violated (although in practice it usually just means that the integer wraps around, which can also be super bad in reality, but in a somewhat less chaotic way than “fully” unpredictable UB).

To get integer wrapping (as a guaranteed, expected behavior, not an accidental one), you must use dedicated operators like +% and -% (there’s also saturating equivalents of these operators, like +|).

I will come back to this point again in a later comment.

Inserting runtime checks would allow TypeScript to flip over to an “innocent unless proven guilty” model like Zig’s, which would result in fewer false positives and make noUncheckedIndexedAccess easier to adopt.

This is so good, great point.

  1. Language service

I’m used to seeing these things being called Language Servers, are those usually called services in TS lingo?

He also mentions that he uses vim and does not use a language server, so maintaining one would be a tax on his time with no benefit.

Regardless of Andrew’s personal preferences when writing code (which I believe are actually not well represented here, more on this later), there is a plan to get good code inteligence from the Zig compiler and the main blocker is just that it’s a bit too early to start working on it yet.

Check out this blog post I’ve written recently on how to get much better diagnostics from ZLS, in there I also mention the long term roadmap plan for a complete language server solution.

My point in general is just that, while it might not look like so from the outside, there are still a ton of foundational pieces missing in the Zig compiler toolchain that must be implemented before we can get to DX tools. As an early adopter one feels some pain points and that makes one think that solving those should be prioritized, but that very rarely aligns in a meaningful way with the development roadmap.

As a small-scale example I’m working on a static site generator with its own custom templating language and I spent the last few weeks writing a HTML parser. I can totally imagine a super-early adopter getting burned by my error messages (eg @panic("TODO: explain that super must have a template attr")), but at the same time I know that makes no sense to prioritize fixing those templating-language-specific TODOs over first making sure that I can provide excellent diagnostics for basic HTML (which btw led me to publishing a HTML LSP).

Going back to Andrew’s stance on dev tooling, he actually has mentioned multiple times that he wishes for IDE-level support for stuff like “refactor into a function”, “rename symbol”, etc.
The problem is that at the moment we don’t have that yet and the Zig compiler has some pretty taxing files (eg Sema.zig, almost 40k lines of code) that are hard to handle for ZLS (or at least were at some point in time, I doubt he tried recently)

It wasn’t obvious to me why some failure modes (out of memory) are handled with explicit errors, while others (integer overflow) are handled via detectable illegal behavior.

This gets us back to the problem of “allowed to happen” vs “not expected to happen”.

You don’t get an error because the idea is that by declaring overflow as UB, the compiler can potentially elide all kinds of checks, which is mutually exclusive with the idea of handling overflow with an error. More in genral a Zig error value is for expected conditions (that are usually part of the unhappy path), while panics are for unexpected conditions, also known as programming errors. Triggering an integer overflow with a non-wrapping operator is a programming error.

If you do want to have math operations that are expected to potentially fail, std.math has all of them as functions that return errors for overflow, division by zero, etc.

Zig implicitly copies data all the time. Sometimes this can be subtle. Function parameters are passed by value.

The previous point (right before the sentences I quoted here) about Zig copying values is correct, but when it comes to function parameters things are more nuanced. When you pass an argument to a function by value, the Zig compiler is allowed to implicitly pass it as a pointer instead. Roughly speaking, small structs will be copied, big structs will be passed by reference. This is why function arguments are immutable: mutating them when copied would not be a problem, but mutating a referenced copy would be, and so you can’t modify them no matter what.

Just to reiterate for clarity: it’s an internal optimization so you treat the value as if it were a copy and, if it’s instead passed by value in the machine code, the compiler will make all the necessary adjustments to maintain correct semantics.

– end

I think you wrote a compelling article regardless of any comment I made here, but if you spend a little energy to work out the nuance around panics vs errors, etc, you will help people get a more correct mental model of how Zig works.

In case you might find them useful, here are a couple more links for you:

https://zig.news/kristoff/what-s-undefined-in-zig-9h
https://www.youtube.com/watch?v=TOIYyTacInM

11 Likes

Great post!

Some minor clarifications:

I believe what you’re looking for is either std.SinglyLinkedList (for LIFO) or std.DoublyLinkedList (which used to be named TailQueue, and before that just LinkedList). EDIT: Or std.fifo.LinearFifo for FIFO, see this comment

There’s been quite a bit of discussion around the naming/implementation of these data structures, but it’s rather spread out. Here’s a few relevant links:

I’d instead say “pointer to an array of N items.” Might not seem like much of a difference, but since arrays are value types in Zig, it makes the difference between *[N]T (where N must be comptime known) and []T with a len of N a bit more clear.

Here’s a relevant thread: Are sentinels only intended for null terminators?

@kristoff gave the better explanation here, but an additional thing that’s slightly interesting to note is that the language itself is not aware of heap allocation at all–heap allocation (and therefore out-of-memory) is fully a “userland” concept, so there’s currently no way for the compiler to insert checks for out-of-memory conditions even if it wanted to.

(this somewhat ties into something I’ve written about before)

4 Likes

Great post, thanks! There’s a bunch of stuff I’d say if I had more time, but I want to address one point:

Of course, a big difference between TypeScript and Zig is that Microsoft’s annual revenue is nearly 500,000 times greater than the Zig Foundation’s. This means that the Zig team needs to make harder choices about prioritization.

I think that gets at a smaller difference between the two situations, that of available resources. The bigger difference is the relative stage of development.

Right now, zig is unfinished and changes a lot. It still runs “gradient descent” in the search of the best systems programming language. In contrast, TypeScript is stable: it has well defined surface protected by comparability promise.

While you are in the “iteration on design” space, it might be a good idea to skip non-core features, such as docs, or perfect error messages. Besides the first order effect of opportunity cost, there’s second order effect that “feature laden” code is just harder to change, so, even acccepting someone else contribution in this are can slow you down long term.

I’ve seen this fist-hand with rustc: its error messages are great, but they do make it harder to extract re-usable libraries out of compiler, because error messages greatly increase the observable surface area that should not change during refactors.

5 Likes

Thanks for all the great feedback and links. I’ll do my best to incorporate it into the final blog post when it goes live, hopefully sometime next week. This forum has certainly been a great part of the Zig experience.

A few specific points I wanted to respond to…

WOW. I really wish I’d known about this while I was doing the Advent of Code last year. Setting this up in my repo was as easy as advertised (here’s the commit) and it does, indeed, get me type errors, compile-time overflow checks, etc. Maybe zls and vscode-zig could advertise this as part of the setup process? Just knowing this was possible would have made my Zig experience so much better.

This is a really good point. Since I only used Zig for a brief period (a month), I didn’t experience how the language changes. But things like removing async functions from the language do seem like a pretty early-stage / experimental move.

Two counterpoints I can see here are:

  1. Taken to its extreme, this would suggest that almost nothing is worth working on except core language features. There has to be some balance between flexibility in the future and developer experience at present. You need adoption and usage to see how well those language features are working out. I guess my grumpiness about DX just reflects the current balance that Zig has chosen, and that may very well be the right balance.

  2. It might be harder to bolt on a language server after the fact than to develop it alongside the compiler. There might be some features that work better in a batch, command-line compiler than in a language server. The needs of the language server might influence the design of the language. I’ve never built a language server myself, so I don’t know how likely this is to be the case.

I’m not sure how much I’ll change the three suggestions around “what Zig can learn from TypeScript” in the final post, since they do reflect my experience using Zig. But I will certainly add some caveats about why Zig may have made the tradeoffs it has, and why this might be the right choice given its current stage of development.

Thanks again for reading the post and for all the great feedback on it!

1 Like

For some context: Microsoft develops TypeScript, VSCode, and published the Language Server Protocol, which is literally defined in TypeScript. That doesn’t make it an unfair comparison, other languages (Rust) have truly excellent language servers, but TypeScript is the one language where if its language server needed or needs anything at all, that thing was or will be added to the specification, since it’s all the same group of people.

I’ve found ZLS to be pretty good at what it does, a bit better than Julia’s, a more mature and established language. There is plenty of room for improvement: a lot of what makes polishing the experience up difficult is comptime, which TypeScript doesn’t have any equivalent of (Julia’s server struggles in a similar way with macros: it’s simply a hard problem).

Which isn’t to contradict you: Language servers are good, and on an almost tautological level, any language which has one learned this from TypeScript, for the reasons alluded to above. I don’t consider the first-party vs third party distinction all that important, but reasonable people might disagree.

There is one thing I would like you to change though: you link to a year-old Reddit post about a missing feature which ZLS currently has: namely, it now offers autocomplete for imported packages which aren’t std. That would be misleading, and unfair to the maintainers of ZLS. I’m sure you can find a more current complaint about ZLS to include if you so choose.

1 Like

I have complicated thoughts here (I also know what I am talking about here, as I did language server for rust).

In general, yes, this is the failure path of pretty much every pre-2010 language: we build a batch compiler with the idea that we’ll slap as good an IDE as JetBrains on top of it later, and then we learn that that doesn’t actually work because a compiler and an ide are only superficially similar.

And Zig sort-of does follows this path. Except that the end game that Zig aims at actually goes quiet beyond just the language server, all the way up to full live code reloading. And that’s not a wishful thinking, the entire architecture is geared toward that goal.

I am still not sure whether this’ll work for things like snappy code completion. I think here’s a sizable chance that the end results would be fast enough for a live reload after you hit save, but not quite instant for seamless code-completion before you safe. But there’s at least as large a chance that this will be fast enough, and that, feature wise, compiler server for Zig will blow rust-analyzer out of the water.

And then, there’s a separate feature of just how useful can you make an IDE given Zig’s semantics, with “instantiation time generics”. I am optimistic and curious here!

5 Likes

The post is up now: A TypeScripter's Take on Zig (Advent of Code 2023)

I’ve fixed a few technical inaccuracies that were mentioned in this thread and added a “Caveats” section to “What Zig can learn from TypeScript.”

I’m really glad I asked for feedback here before publishing. The post is much better thanks to all your input. Thanks! :heart:

6 Likes

Sorry to interject but std.fifo.LinearFifo is pretty close to a built-in queue IMHO. At least that’s what I’ve been using and it seems to be working fine.

Edit: corrected “std.LinearFifo” to “std.fifo.LinearFifo”

1 Like

You’re right, that’s probably the better answer (technically it’s std.fifo.LinearFifo btw). Edited my comment to include that.

1 Like

I’m not happy with the API or name of LinearFifo. It’s definitely something that needs to be completely reworked. For example, it shouldn’t take the API as a comptime enum parameter. It should be broken into 3 different data structures instead.

4 Likes

Thank you for the article. Definitely very useful introduction to zig for newcomers such as myself.

One thing I do find confusing on the terminology is “undefined behaviour” vs “illegal behaviour”. Is it an official terminology or sort of a colloquial way of saying things in zig?

I say this because I don’t really see the terminology of “detectable illegal behavior” being used in ziglang language reference documentation, Documentation - The Zig Programming Language (ziglang.org).

Using CTRL+F, I see “undetectable illegal behavior” appearing 1 time, “illegal behavior” appearing 3 times, “undefined behavior” appearing 38 times.

Not being a language nazi, I’m just trying to be correct in understanding zig’s approach here, so I don’t say the wrong things when promoting zig to others.

The only place i see that feels more “official” is the github issue here terminology update: use the phrase “detectable illegal behavior” rather than “safety-checked undefined behavior” · Issue #2402 · ziglang/zig (github.com), but it is very far away from the language reference.

Just my 1 cent.

Note that that issue you linked is an accepted proposal, meaning changing the language reference to use the term “detectable illegal behavior” is planned.

2 Likes

The name is admittedly a little out of place but having the 3 different variations as a single API is a good demonstration of the power of comptime.

Would it be a good idea to change the API to a general purpose purpose double ended queue which can be used as both a queue and a stack at the same time?