Thanks for the warm welcome.
Well I’m glad to hear it doesn’t take comptime as that would have opened a can of worms for very limited gains.
While your suggestion of searching for @import makes sense it does require me to check out the entire merge/pull request to run proper tools on it. In the era of gitlab/GitHub Open Source that branch is rarely available in the same repo.
I fail to see a good rationale for allowing this in the middle of the file.
The rationale is that Zig allows declaration anywhere and restricting that would require some soecial casing in the compiler I think, so it’s more natural to just let it be and trust programmers to structure their code in an understandable way.
Split up std so that it doesn’t contain a ton of functionality that can be used for harmful stuff. (DynLib, elf, fs, io, process, and maybe others )
If you want to do this, it is already possible using const namespace = @import("std").namespace to only include the parts of the std lib you actually use. Semantically I don’t think it makes a difference to the compiler, since Zig is evaluated lazily.
“no naked imports” should be a pretty simple thing to enforce/add to contribution guidelines for an open source project.
But I see that more as a failure of open source tools providing a good seamless workflow, I think the fix is to make it easy to run the branch with the proper tools, instead of relying on in-browser workflows too much.
Maybe custom static analyzers that lock down certain files and modules so that they can just use specific imports, could help reduce the amount of code that needs to be checked manually. And those could be a tool you can run both manually and as part of CI.
Of course responsibility is with GitHub and gitlab. The problem is that they are preoccupied with trying to compete and make $$$. Companies tends to know exactly who they’re employing so I’m not super worried about hostile code at work.
With open source I am worried about a hundred people sharing pseudonyms and then agreeing that it takes three of them to run code on my system.
I also worry that writing a solid safety scanner would take significant effort that few want to fund.
I understand that this goes against some of the philosophy, however I will patiently repeat my sentiments in the hope that given time others will recognize that we either need a solid safety checker or make a few low impact changes to help open source maintainers protect against hostile intents.
I consider the latter much more realistic but would love to be proven wrong.
Do you mean exploits that are hidden behind several harmless looking commits and different pseudonyms, that only after a long time eventually result in something that combines into a chain of attacks?
I think for that you could do things like specify which files can be read by which processes/build-steps within a sandboxed build-environment and by strictly specifying what is allowed to access what, you at least could significantly reduce the attack surface.
I think Zig plans to eventually have a sandboxed build mode for repeatable builds and I think that could then maybe get added features to harden it and have some sort of capability based security.
But so far I haven’t worked on something, where I need to review vastly more code than I write, so I don’t really have practical experience with what helps when you need to review loads of code.
I tend to walk a middle ground. @import expression is, above all, a dependency declaration and I would like to know my dependencies upfront, hence @import are in the beginning of the file. Pub aliases and public declarations are also in the beginning because they are public interfaces. Non-pub declarations are towards the end together with non-pub aliases. Tests are right after relevant declarations, for they serve as usage documentation.
Modern IDEs to a great extend alleviate the pain of hunting for definitions. In VScode with zig extension hovering over a function reveals its signature and the doc-comment. If it is not enough, pressing F12 jumps to its definition. Pressing ctrl- jumps back. These little tricks relieves one from the pressure to perfectly organize the files structure.
It was a character customization system for the unity game engine. Lot’s of serialization of mesh parts, textures, customization values, colors, equipment, etc.
So we had a ripe mix of binary data, funky reflection code, editor code, runtime code, all in all a perfect place to hide an attack.
One pull request in particular was hundreds of files, not particularly well written, but the functionality (a container for one or more exact locked pieces, complete with rendering icons for the pieces) was in high demand and I was swamped with kids in real life. So I ended up safety checking it and allowing it as an option, mostly because I knew full well I wouldn’t have time to make something I considered a good solution for more than a year. Eventually I came to the conclusion that the demands of my family meant stepping down and handing over the reigns.
Disagree. Imports are variables that can have any name, and you can reference an import within another file because of this. For example:
const blah = @import("std");
const blahblah = blah.io;
Which means that not all imports, or more correctly, uses of code within different modules/files, are using @import(). In addition, this allows for quite a bit of obfuscation.
Anyways, I’m sure you can create some parser tool that can find this stuff much easier than trying to do it by eye.
It doesn’t make a lot of sense to disagree with something by talking about another, unrelated thing.
All imports into a program use @import, and a string which tells you precisely what was imported. So you can find all of them with a grep, and go from there.
Sure, attempts to obfuscate from there can be made, by using various misleading names for things. But, so what? Security begins at the edges, and if you see a line like:
const innocuous = @import("std").fs;
When reviewing a patch, what else do you need to know? Shenanigans are going on, demand an explanation or just refuse the patch.
This leaves me curious what you think a language which doesn’t have this ‘problem’ would even look like.
We should follow “top imports style”.
I would even suggest formater (or zig build)
search all @imports a put them sorted by name, remove unused as GoImports do
And Java( or Java IDE) as I remember from those Java days.
Tests must be at the bottom.
i really like both the “files as structs” pattern and the “imports at the bottom” pattern. it’s not unusual for me to open a Zig file on the web, where the ability for my editor to hide import statements is moot.
i don’t like the suggestion to disallow @import statements outside of top-level scope. i’m sympathetic to the problems made possible by allowing @import anywhere a declaration can be made, but it’s nice to enforce, for instance, that only testing a file relies on std by only importing std from within test blocks.
in general, i think it’s useful to remember that mechanical style rules cannot be a total substitute for an interpersonal agreement to prioritize similar goals with regard to readability.
Personally I think status quo is great and we don’t need any additional rules on where the imports are being made, the flexibility is useful and the rest can be done via people agreeing on a per project basis or tools that people can add to their projects/build steps, or they can create linters if they want to.
I like to organize my declarations so that things that build on another are grouped closely together and form a chain that can be read easily, but with the linear nature of files that has sometimes limits, still it is better than having to do a nearly random walk between a lot of functions.
I don’t think we should enforce things like alphabetical order, if you want alphabetical order than customize your editor to display it like that or format it like that.
I don’t think there is any inherent value in using an alphabetical order, I often find it much more useful to have imports right above the 2 functions that use it, that way you quickly can break down what makes use of what. It depends a bit whether it is just some example project with multiple sections in a single file, or some more complicated project that separates into more files.
And please lets not take organization advice from Java, I am enjoying Zig, can’t say the same about Java.
The module system already tells you which modules can import what, based on which module imports you have attached to a module.
For a situation where you don’t want to allow access to all of "std", you can create your own modules that re-export some of what is available in "std" and then force people to only use your modules.
My 2 cents are, lets not force annoying rules on everybody, just because someone wants their favorite three rules to be king. If there are any more rules added, these should have clear benefits, instead of just being someones personal preference.
This is a strong worded statement that flies against Zig practice as demonstrated by the std library. Zig tests labeled with the declaration are doc-tests and become part of documentation together with doc comment. Keeping doc-comments, declaration and doc-test physically together is the current practice that makes lot of sense. Other tests can go to the end if the file or to a separate file (my personal choice).
Creators of this language have a lot of weird bikeshedding tendencies.
If its supposed to be a C successor, then it should follow at least some of the things C does, imo.
You can’t use a variable before declaring it, so why should you be able to use a import before importing it. Inconsistencies are not good, you should be able to logically deduce the behaviour of the language for ease of use reasons.
Though in the end, it doesn’t really matter, just move them to the top if it bothers you.
I am explicitly not weighing on the wider topic, but I want to explain just this one thing.
The reason is cyclic dependencies: you do often have to write where foo depends on bar which depends on baz. The simplest form this comes up with is in mutually recursive functions, but you can also have things like recursive data types.
So any language needs some way of using stuff before it is defined. In C, the solution are forward declarations. In ML – let rec. In Python, the semantics that module is evaluated from top to bottom, and that functions are lookup dynamically in module’s dict when the function is invoked, not statically when the calling function is parsed.
But by far the most simple and natural solution is to say that top-level declarations are order-independent.
Zig has variables, and declarations. They do use the same syntax, but they aren’t the same thing.
Declarations can be used before they’re declared, but variables can’t be.
An import can be either to a declaration, or to a variable. Imports as declarations follow the rules for declarations, imports as variables follow the rules for variables.
So there’s no inconsistency, because imports aren’t special. All declarations and variables work the same way.
@matklad gave a good explanation of why this is useful, but:
It’s not just top-level declarations, it’s any declaration at any scope.
Unrelated? I thought we were talking about imports?
The fact is NOT all explicit imports use @import which is what you falsely stated. That’s not true. My use of the term disagree was putting it kindly, because it’s flat out objectively false.
And just to be clear, for the purposes of this particular conversation, import here means using a module/package/file/whatever-zig-wants-to-call-it in a file, because the usage is what’s most relevant here, not the fact that std imoprts crypto, and so anything that imports std also “imports” crypto. Yes, this might be different from the compiler. Humans aren’t compilers and linguistic concepts transcend the compiler.
In the example you give, you import both std and fs. This means both that you import std and, implicitly, everything that std imports, which is not quite what every language does, and you can explicitly “import” (with quotes) anything std imports into your own file.
Regardless of this import vs. use nonsense, the whole point of the conversation is about obfuscation - getting away with using insecure or ill-intentioned things in a patch without anyone seeing it. And it’s very easy to obfuscate specific parts of std.
It’s weird to not distinguish between importing std, which is perfectly fine, from explicitly importing everything else that is underneath std, like sockets, networking, crypto, tls, http, etc.
It is not sensible to restrict all of std for the sake of protecting against a subset of the standard library.
As for other languages, some languages, when they import a module or package, allow setting that import as public, or exported. The equivalent of Zig’s system in this type of language would be making every import publicly exported by default. There are clearly downsides to this approach, just like there are always tradeoffs for most everything. And you and many others might be fine with the downsides, but pretending like there’s no downsides does nobody any favors.
I do not believe this approach will ever work, and here’s why: a std library sub-module can suddenly include a new import, and it becomes automatically accessible to every single file that imports that module.
So, even if I created this new module that re-exports specific things, a new “sub-module” might be added to any module I re-export, or a new import in those modules might be added, and then you’ve instantly broken this faux security mechanism.
Because now these modules that I’ve re-exported are automagically exporting more things then I intended. And this is why some languages don’t auto-export every single freaking import! 
Not even Golang does this. I don’t recall Odin doing this either, but I might be misremembering.
To add to the larger conversation, I don’t see how placing imports at the bottom makes any sense. They aren’t exactly indexes. They are used within the file, and so should be organized as such, especially if you are importing things outside of the std library. Placing them at the top has been done for a long time likely because it’s natural.
I find it more natural to read dependencies from top to bottom. Things that are depended on by a lot of other things go at the top. Things that are less depended on go towards the bottom. Imports are some of the most depended on things in code. They should be at the top.
Also, I find almost everyone I’ve talked to in university about endnotes vs. footnotes prefers footnotes over endnotes as long as the footnotes are not too extensive. The reason being that it’s easier to flip to the back of a chapter than it is to flip to the back of the book. One might say declarations should be closer to where they are used.
I know the why’s, I’m just saying its inconsistent logically, sorry for not explaining properly, my fault.