Zig Devlog ⚡ Bypassing Kernel32.dll for Fun and Nonprofit

Zig Devlog :high_voltage: Bypassing Kernel32.dll for Fun and Nonprofit

25 Likes

I wonder what the Zig maintainers would do when a few years down the line things just break because Microsoft changed these ioctl codes.

I mean, the codes used here are (afaik) private and undocumented API, so I wouldn’t be surprised (or mad at them). And at least Apple does this stuff in their private APIs regularly with the purpose to stop people from using them.

It wouldn’t even be surprising to me if somebody working on Windows sees this blog post and does such a change on purpose because they don’t like that people do this.

2 Likes

Nah, Microsoft never do this. They’re very committed to backward compatibility. This old post by Raymond Chen provides some good examples.

4 Likes

Note: Post edited

Rust uses NtReadFile and NtWriteFile to do Windows file operations in its std lib. Rust had to drop down to use ntdll 5 years ago to fix (File implementation on Windows has unsound methods · Issue #81357 · rust-lang/rust · GitHub) after a Microsoft engineer (jstarks) noted a soundness bug in the higher level windows API call, and subsequently “blessed” the use of ntdll file functions directly in (Fix unsound `File` methods by ChrisDenton · Pull Request #95469 · rust-lang/rust · GitHub) by saying:

”NtReadFile and NtWriteFile are publicly documented and so are fine from the Windows team’s perspective.” (Fix unsound `File` methods by ChrisDenton · Pull Request #95469 · rust-lang/rust · GitHub).

Golang also links against ntdll directly, with the following functions in its standard libary in use: NtCreateFile, NtOpenFile, NtSetInformationFile, NtQueryInformationFile, RtlNtStatusToDosErrorNoTeb, RtlGetVersion, RtlIsDosDeviceName_U .

please do not use ai in ziggit, you are way smarter and you will learn more if you do your own research.

7 Likes

15 Likes

Reading from the blog, while it seems like they wouldn’t change the API, they could still end up changing that Zig doesn’t get what they wanted from the switch of making the ioctl call to the \Device\CNG driver directly.

(For the record, that’s the thing I am scared about here. The NtRead-/-WriteFile functions are fine since Microsoft at least documents them, are aware that they are used and likely does promise stability.)

Also, this actually makes sense if you know Microsoft (and Windows’) history:

What we’ve learned empirically is that the ntdll APIs are generally well-engineered, reasonable, and powerful, but the kernel32 wrappers introduce unnecessary heap allocations, additional failure modes, unintentional CPU usage, and bloat. Using ntdll functions feels like using software made by senior engineers, while using kernel32 functions feels like using software made by Microsoft employees.

The WIN32 API was created during the times of MS-DOS, when Microsoft did in fact mostly have Junior Developers.

The NT API on the other hand was created during the development of Windows NT, quite a few decades later.

1 Like

This is how I research.

1 Like

The more we avoid such tools together, we encourage and grow each other’s skills. In a world where everybody’s brains are atrophying from LLM overuse, we can stand out by keeping our minds sharp.

You don’t have to agree; you’re the boss of yourself. But here’s some gentle, social pressure to challenge yourself :slightly_smiling_face:

37 Likes

You are giving them too much credit they can’t even ship a windows update without breaking half the computers this days. I doubt they have the skills required to be mean (on purpose)

1 Like

I totally agree that it’s important to keep our minds sharp. Thank you for the gentle social pressure.
I am curious what should the proper method be for answering a question like:

“How do other languages like go or Rust do file operations on Windows?”

An approach without AI could be to- download the language sources, trace the call graph for the file operations manually and determine what the Windows calls are. Is it ok to ask AI do that part, and point me to the code to look at for comparison zig’s approach? What would you consider to be a healthier method to maintain my sharpness? I am currently using AI to do what I consider “grunt work” like this because its way easier. Thanks in advance.
:thinking:

1 Like

Learning to read foreign codebases is a great skill. And there’s often lots of goodies you can accidentally notice along the way of achieving your original objective.

Plus if you have questions you can ask Go or Rust community members; a tiny act of bridging the gap between otherwise isolated groups of people. You might even make a friend!

14 Likes

The takeaway is that the Windows developers do crazy stuff. Why shouldn’t they? If their software breaks after a Windows upgrade, people will blame Microsoft anyway.

1 Like

It doesn’t have to be so black and white, there’s also a middle ground, where you can do a little internet searching (or even AI prompting) to find good secondary sources. These could be official docs, blog posts, forum threads, videos, etc.

1 Like

Using ntdll functions feels like using software made by senior engineers, while using kernel32 functions feels like using software made by Microsoft employees.

I love this.

5 Likes

It may sound counter-intuitive but Dave Cutler actually is a Microsoft employee.

1 Like

It’s a vice, I know, but us millennials have been shamelessly poking fun at M$ Winblows since we were teenagers and it brings us joy and you can’t take it away from us! :face_with_tongue:

19 Likes

In my personal opinion, you don’t answer the question in Ziggit if you don’t know the answer. Someone who actually are experts in Go or Rust may show up and hand type the answer.

Months ago, someone here asked “how aircraft computers or any life critical computers are programmed and ensured they are running extremely safe and stable…” something like that… If you do AI research, AI will tell you a lot of different ways… I bumped into the post, and told them that actually aircraft computers are not that safe and stable, and their programs are nothing much different than what everyone else writing… We have FMGC freezing every single day… and manufactures release OEB every now and then… I answered that post and corrected most people’s believes in terms of aircraft computers or something just because I know that for sure because I work in this industry.

What I want to say is that if people know that field, they should be able to answer questions without relying too much on AI.

Also, just my two cents here: Zig community doesn’t like AI generally anyways…

8 Likes

Luckily the code for go and rust is open so no experts are required. My question was about the “approved by the zig community” way of doing research to avoid dismissal. In this case, how other more established, stable languages treat their Windows dependencies happens to be pertinent to the discussion.

2 Likes