Language Oriented Zig?

[Sze]

Why this array syntax?

I guess it’s good to focus on a minimal syntax while Zig is still heavily in development, but I really hope that at a later point there can be a little reversal in philosophy to get some balanced amount of syntax sugar into the language. Because I think the philosophy to implement things in the stdlib that should be language features is what ruined both modern C++ and Rust Both those languages also suffer from the problem that there’s no clear separation between the standard library and…

This reminds me a bit of the conversation between the Odin and Elixir creator in this video.

Basically Odin tries to build everything needed into the language with its specific syntax, instead Elixir tries to have a minimal core and use meta programming to add things (including syntax) to the language.

And Zig is somewhere in the middle between the two.
But the thing is that people disagree widely on what are missing features vs unneeded features and how/if they should be added. (Which makes sense to me because people need different things)

While I can understand wanting a declarative syntax for certain initialization patterns, it doesn’t necessarily match Zig’s imperative focus, personally my preferred approach would be if it was possible to make these things possible through meta programming, but with the current meta programming possibilities it is more clunky then having it in the language directly.

One part of me likes the idea of turning current Zig into a core/base language and future Zig into a multi-paradigm/language-oriented-programming language,
basically asking the question:

What if we could use the buildsystem to create different languages (including dialects) that can be used along-side zig?

So that @floooh could use Zig + C99 decl syntax as a language for writing c-api style code and game developers could use Zig + gamemath, while machine learning people would use Zig + machine learning math those could share most of the implementation and differ in some details.

Adding the right operators and datatypes for every use case bloats the language (and is unlikely, more likely a lot of things get rejected)

Where adding the ability to customize the language through the buildsystem (while keeping interoperability) would give a way to do anything in a well defined way, without needing to clutter the core language or be rejected because it is too niche.

For example a Zig + webdev language could add syntax for sql.

One way this could work is that those extensions basically have either their custom parser and compile to Zig or an ast, or even lower level formats (technically there could be a lot of different options here), or Zig could get syntax for embedding dsl-language snippets something like:

const people = #sql`SELECT * FROM people`;

Where the implementation for sql would be defined for the module via the build system. Zig already isn’t a (classical) strictly imperative language, it uses laziness, sometimes comptime meta programming is easier in a functional style.

Maybe it would make sense to have an option between use Zig (and live with some metaprogramming uses that are clunky) and create your own programming language from scratch.

I think Ziggy and SuperHTML are good examples that creating your own language can make a lot of sense, but even there it seems to me that there should be a mechanism to add ziggy as a file extension in the build system, which then would allow you to use ziggy files in your code base and for example import them from Zig files.

Basically I would want that the sort of integration that exists for zon, could also be added for ziggy (or other custom languages), ideally in a way that makes these languages equal to another (instead of one being seen as a builtin language and the other as second class).

Still I would find it reasonable to enforce that build.zig.zon always has to be zon. (So that bootstrapping, buildsystem, package manager and third-party tooling can remain simpler)

I think it could make a lot of sense, especially if you think about something like a game where this would allow you to for example add a scripting language (and scripts written in it) to your project which then would integrate with the normal zig code and incremental compilation / hotreload.

Once we have a compile-server serving info via a protocol (instead of LSP) it would also make sense that the editor then could show you completions for things that are declared within the scripts from zig files. So basically having imports that work across language barriers, like it would be expected for zon.

I also think if the Zig language adopted something like this, a lot of ideas could be more easily solved by finding what level of support Zig has for that idea, instead of fighting over full or no support.

Basically custom languages would become an escape hatch that allow the community to explore different alternatives to what is supported in the core language. The package manager could even get a feature that allows you to throw an error if any dependency contains anything except core (if that would help with managing complexity).

[hvbargen]

Interesting idea.
My inner programming language nerd would love this.
OTOH I’d bet that this is a bad idea in reality.
We’d end up with dozens of dialects, each of these understood only by a small fraction of the developers, probably sooner or later conflicting with each other, In particular if you want to use 32 or 3 libraries which depend on several other libraries and some of theme depend on conflicting language extensions.

But I like the idea of string literal prefixes eg for SQL. Some other languages support this.

[Sze]

@hvbargen

We’d end up with dozens of dialects, each of these understood only by a small fraction of the developers

I think that is one of the possible downsides, that probably only can be mitigated to some degree by having some kind of process of forming community consensus.

Basically the further you drift away from a core language the more difficult it gets to have something that isn’t only preferable to a single individual.

I think that there would be various degrees of general usefulness vs individual expression that can be practical, instead of being total chaos all the time.

---

However I don’t think they would conflict in language extensions, because each module would be written using what ever language it declared (or default to zig). (Similar to how racket can use modules written in different languages in the same program)

The part that is potentially messy is when somebody uses an obscure language extension that is of poor quality. But that is already a problem with using dependencies in general, that the code could be bad and you should consider whether you actually want that extra dependency, or write your own code instead.

It still would add complexity (having more than one language), but it also could reduce complexity (by using languages that better fit the actual problem).

For example sometimes it is possible to use languages that are designed to make certain errors impossible, using those can be beneficial over having to emulate such a language by somehow expressing it in a more general language.
(which adds syntactic noise)

[Southporter]

@Sze

The part that is potentially messy is when somebody uses an obscure language extension that is of poor quality. But that is already a problem with using dependencies in general, that the code could be bad and you should consider whether you actually want that extra dependency, or write your own code instead.

Isn’t this one of the problems with Haskell? They have lots of different language extensions that aren’t compatible with each other as each researcher creates what they need and drops if after the research paper. I think this would lead to a lot of incompatibilties when you get cross overs between communities (i.e. Zig + webdev trying to include Zig + machine learning for some project).
Ideally, you can keep the incompatibility down for the major flavors.

I think the main way you are going down is interesting though. If the Compiler could handle multiple custom languages and link them correctly, that could be a big win. Similar to how it now handles C, C++, and Zig code to compile together, you could have a hypothetical future where you can have a addTranslateZiggy() or b.addTranslateStep(module_to_handle_translation, options); to allow for new language features to be integrated.

[pierrelgol]

Maybe it will happen in the future. One thing that could be interesting, if if you could create builtins at comptime, as a separate build step, sort of like a pre-comptime, where you compile your own builtins, that can interract with the compilation process / parsing / codegen / linking etc in more details.

[npc1054657282]

I worry this could lead to a language fragmentation similar to Lisp.
If this were to happen, I believe the dialect concept would be best managed under the centralized control of a “second-party” organization. While maintaining the streamlined core of the first-party standard library, the second-party library would be officially guaranteed for quality and supply chain security, developed by the community, and address more specific domain needs.

[IntegratedQuantum]

Isn’t this already possible? You can just add a custom build step that preprocesses the code and e.g. parses all the custom code segments and transforms them into valid Zig code by possibly transforming all the operators to function calls.

The only problem so far is that there is no tooling support (→ ZLS, debug info)

[floooh]

Tbh I would prefer a single syntax. I don’t think what people want is entirely random, but instead a fairly fixed set of syntax features that turned out to work well in languages that are quite closely related to Zig, e.g. a compiled, statically typed language that produces high-performance code.

For instance people want to use Zig as a C replacement, or to program GPUs, or for game development, or embedded, or maybe high-performance apps living on servers - those use cases are all not that much different, e.g. I doubt that Zig will ever be a popular choice for bulding vanilla CRUD webpages. These are also all not new use cases but have been explored since decades and the experiments what turned out to be useful and what not had already been made.

I don’t find async/await all that useful for my use cases for instance, since when you have a recurring ‘game tick’ anyway, completion callbacks work just fine with much less machinery and for regular commandline tools, blocking IO also works fine, but I don’t mind that Zig will get async/await since it will be useful for some types of event-driven applications - I realize though that most of the new async-await mechanism isn’t about syntax - but think about it more like “what’s the current fancy feature the Zig team focuses on”

Some sort of ‘langauge extension mechanism’ with build-time ‘syntax plugins’ could work, and might be useful for people to explore ideas - but this sounds too much like a ‘language construction toolkit’ to me - e.g. in the end not much different than operator overloading in C++, which turned out a bad idea except for one very specific usecase: vector math libraries.

[hvbargen]

I consider Zig (once it’s stable) a possibly good choice for logic inside databases.
Here, performance matters and correctness matters.
A language feature which I would like to see added is distinct numeric types.
Distinct types are easy with Zig, but distinct numeric types are pretty useless without operator overloading.

[glfmn]

@Sze

Zig could get syntax for embedding dsl-language snippets something like:

const people = #sql`SELECT * FROM people`;

Would it be possible to implement SQL in comptime?

const query: SqlQuery =
    sql.compile("SELECT * FROM people"); // arg is comptime

You could also get access to the data structure that represents the query itself somehow and do some cool stuff.

[KilianHanich]

What you describe here, reminds me of Greenspun’s tenth rule:

Any sufficiently complicated C or Fortran program contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.