Not familiar with him/his work, but I will say that I’m sure there are tons of great engineers working at Microsoft. The problem seems to be that the system there doesn’t incentive or enforce quality.
Worth a person to recognize: Dave Cutler - Wikipedia
2 Likes
As a starter, I’d say just avoid posting links to AI generated stuff. Go ahead, use it for fining documents, blogs, source code etc. But turn that into processed material by a human. What I mean by that is, understand it, reference it the way you understood it. Not just plain AI output.
I am using AI myself, but I’m always processing what it’s giving me, finding different solutions with out it, etc.
It’s a tool. Use it as a tool, but at the end give your own human touch to it 
7 Likes
Hi, this has probably been explained in some form or other, but just for clarification:
Does this mean that if I use the zig stl (from 0.16 onwards I guess), and release a game with it, then Microsoft changes something in their implementation or API in this “core” dll, my game will break for every owner that gets this Windows update? (provided I hit these APIs of course) And that this is intentional on part of MS - i.e. these APIs are documented as unstable in this sense? And that Zig’s reason for going directly for these APIs is performance?
As far as I understand, I think it’s a combination of performance and consistency. The higher-level APIs change more often, hide things, and don’t match the documentation. So while the risk you describe is real, so too is the risk they avoid by taking this path.
I’m betting they’re right but it’s worth a thought for “what if they’re wrong?” If they’re wrong, I’d hope we could expect that either the core team or a third party team would then release yet another implementation of Io that you could plug in. And other than selecting an Io implementation (and any Io interface changes for as long as it keeps evolving - I expect it to stabilize but not yet), I’d expect that change to work without even modifying your application code.
2 Likes
That would be a cold day in hell when kernel32.dll, user32.dll etc… suddenly break compatibility 
(at most they add more functions, like CreateWindow => CreateWindowEx, but I’m not aware of changing an existing function interface all the way back to the 90s) => that’s basically the key to Window’s legendary backward compatibility.
I would also be surprised if ntdll breaks compatibility, but AFAIK Microsoft explicitly doesn’t guarantee backward compatibility for ntdll the way they do for the Win32 DLLs (which is the ‘official’ Windows system interface, like it or not).
E.g. the situation is reversed from Linux: Linux guarantees to never break the syscall layer, but the higher level system DLLs don’t have that strict guarantee, while Windows guarantees backward compatibility in the higher level Win32 API, but not for the lower level syscall layer.
Whether ntdll ever breaks is hard to figure out from googling. Personally I would feel better if Zig would offer two Windows ‘subsystem targets’ for it’s stdlib platform shims, one that goes through Win32 and is the default (even if that’s slower or generally sucks more), and one optional that goes through ntdll but is ‘at your own risk’. In any case I would prefer if ntdll usage is reduced to a minimum, to those function that really show an obvious advantage versus their Win32 wrappers.
PS: Windows might not even be important enough in a couple of years for such decisions to be relevant though, e.g. for the last bastion where Windows still matters (PC gaming) I would be more concerned about my Windows code working well on Linux with WINE+Proton instead of native Windows E.g. if WINE/Proton implements a fully compatible and high performance ntdll, then all is good, if not: problem (e.g. I wonder if WINE or Proton do ‘shortcuts’ and implement some things entirely in the Win32 layer, which would mean that WINE’s ntdll might not be as robust / well tested or performant as the Windows ntdll).
1 Like
Would it be possible to gather many different versions of ntdll over the years and compare their ABI to get objective information about how often it broke compatibility since its creation ? It wouldn’t cover the case where a function’s behavior changes while its signature doesn’t, but it would be a start …
1 Like
By higher-level APIs changing more often, I was referencing this from the Devlog
Unfortunately, starting with Windows 8, the first time you call this function, it dynamically loads
bcryptprimitives.dlland calls ProcessPrng. If loading the DLL fails (for example due to an overloaded system, which we have observed on Zig CI several times), it returns error 38 (from a function that hasvoidreturn type and is documented to never fail).
As for whether ntdll ever breaks, I’ll accept that the Devlog didn’t prove it doesn’t. I’m thinking Microsoft uses their internal APIs often enough that they would also avoid changing those, but I agree it isn’t proven.
As for whether we’d benefit from a second version of IO that avoided ntdll, I’m betting from this conversation that if the core team doesn’t then somebody else will, and I’m betting that if they’re proven wrong here the core team will also change their minds when it happens.
I also agree that we might want a different one for WINE even if their arguments are right for Windows.
The idea that all of this should be supported without application changes, incidentally, makes me a big believer in the Io interface as a whole.
1 Like
Mainly the motivation is to avoid bugs. Collecting entropy should not crash the application, but using the “stable” APIs leads to that situation. This isn’t a hypothetical, it happened multiple times in reality. From that perspective, which APIs are more “stable”? If we had already migrated to ntdll for entropy, it would not have happened.
You either gamble on the higher level APIs or gamble on the lower level APIs. Worst case scenario, you have to rebuild your application in the future and ship a fix for a newer Windows version. But I’m confident in where I’ve placed my bet.
This isn’t something that Zig has started doing recently, the std lib has been depending on ntdll since 2018 (8 years ago!). Zig has the ability to do file system operations based on open directory handles. That capability is offered by NtCreateFile but not by any kernel32 functions. This helps applications avoid file system TOCTOU bugs.
Let’s just say, underneath the layers of crufty API, there is a much smaller and cleaner operating system struggling to get out.
18 Likes
One other famous example is Rust’s mio crate (used in Tokio for example) and NodeJS using \Device\Afd to implement polling on windows.
2 Likes
It’ll never happen, but it would fantastic for OS competition if Microsoft were to just go all in on NT interfaces; and rather than encourage users to move to Linux-y interfaces like WSL, makes NTDLL.DLL the definitive interface to the kernel.
It won’t happen. So we’re probably stuck with POSIX forever.
At least it’s free software. I’ll take what I can get.
I like that reference to C++ ahah
1 Like