I’d like to open a discussion on how to treat file descriptors. In my runtime, I can’t do non-blocking file read/write on any system outside of Linux (io_uring) or Windows (iocp). These need to go through a thread pool, there is no way around it. However, for pipes/devices, those can be open in non-blocking mode and the thread pool can be avoided by poll + read/write. Standard library currently doesn’t distinguish between these two cases.
I’m thinking about adding Pipe struct to zio that will have the non-blocking semantics, while File is expected to be physical disk-based file.
Do you think it makes sense to add this distinction to std? Or what’s the plan for dealing with these cases on the kqueue event loop implementation?
While it is an important distinction, I think you are overlooking that most operating systems treat them all as files, a file in the context of an OS is quite abstract.
There is a distinction, but it tends to only go as far as types of files, or files opened/created with certain flags.
Grouping them all as just a file does simplify both the OS API and the std lib, for the common case at least.
This abstraction does not prevent you from doing non-blocking, some files just won’t support it.
that is mostly the current reality, it is certainly oversimplified and lacks nuance.
As for my opinion, I’m on the fence on what I think OS’s and std should do.
I think it’s best to follow what current OS’s generally do.
The thing is, OSes that actually need this have O_NONBLOCK and the access pattern for blocking and non-blocking is different. It doesn’t matter on Linux and Windows, but this mostly impacts BSDs that don’t have async file I/O.
I’m thinking about abusing fileReadStreaming for this. It would essentially become poll+read on BSDs, but at the cost that it’s going to block for real files. I guess with enough documentation, that’s reasonable.