State of networking in `std.Io`

Brainstorming
, ,
1

I’ve been writing about this in the past, but after the 0.16 release, things already got worse. The interface is missing several key features, that people are hacking around, tying to a particular std.Io implementation, so even if the library accepts io parameter, it doesn’t mean it will work with any std.Io implementation.

  1. Ability to configure socket options like keepalive, nodelay, etc.

  2. Ability to have read/write timeouts

  3. Ability to run poll as an operation

I have several open PRs for improving networking in std.Io, plus others waiting to get the initial ones merged, since they are stacked, but it seems that networking is not a priority. I realize that reviewing is hard and there are limited resources, I’m not trying to push, I’m just trying to understand the goal/priorities.

I guess I’m just trying to decide whether to invest further into networking libraries based on std.Io, or I give up and use my own networking APIs for now.

23 Likes
3

Let’s not do this in this thread, please. I like std.Io, it solves a real problem, it’s the first language outside of Go that can have networking libraries working on any backend. Let’s not make this into a wider topic. I just want to see if fixing networking in std.Io is a priority or not. And I mean just the interface layer, I personally don’t really care about any of the implementations in stdlib.

11 Likes
4

I have been working on a PR for a type safe setsockopt wrapper that (IMO) is a bit more zig like. Can you share links to your networking PRs? I would like to look them over. It would help focus the discussion too.

Speaking generally, I think the state of PRs is a problem, and Andrew has acknowledged this in the past. I don’t necessarily think your experience is Io/networking related, and might just be PR related.

I can’t personally be mad because reviewing PRs is a bit of a chore and always less fun than implementing something yourself. It can be demotivating for PR authors though.

3 Likes
5

Not just mine, but here is a list of important PRs:

Plus, there is a buch of open PRs for unix sockets / UDP, which I don’t care that much about, so I’ll not look for them.

4 Likes
6

I faced this very same thing building an epoll based TCP server. To set NONBLOCK on the socket I had to resort to direct std.os.linux.accept4, bypassing Io entirely.
The code ends up being a mix of Io and direct syscall using .socket.handle, which doesn’t particularly bother me, but perhaps defeats the purpose of Io, at least partially.

Truth be told, I don’t think I’m running into murky waters here as long as everything is single-thread, but as soon as I cross that line the mixed use of Io and lower level primitives might bite me back. Another aspect of this is that I’m intentionally not writing portable code, but instead linux targeted specifically, which maybe also defeats the purpose of or need for Io in this context (single threaded, single target :person_shrugging:).

But I guess the bottom line is: not having everything in the Io interface hasn’t stopped me from achieving what I wanted, just made a few select parts of the code non-portable and potentially non thread safe, which are compromises I can live with for now.

2 Likes
7

That’s the anti-pattern, which is acceptable in an application, but not in libraries. Currently, most libraries assume std.Io.Threaded, so they call syscalls like this, even blocking ones, like poll. Which in turns means I can’t use them with my own implementation of std.Io, at which point I just want to give up and use zio APIs exclusively, essentially building my own stack fully, ignore std.Io. But I feel that very little is missing to actually avoid these hacks and have cross-backend libraries, it just needs some dedication to bring featues into std.Io.

7 Likes
8

imo, don’t give up :slight_smile: i think it’s gonna just be a good bit slower than we want for now.

3 Likes
9

A couple of issues which I’ve run into recently while working on zigmirror:

  • Lack of ability to specify timeouts in a lot of places, which you’ve already mentioned. Working around this with std.Io.Select and a separate watchdog task works, but in a lot of cases it’s inefficient and tends to make the code rather messy
  • Inability to open a dgram unix socket to implement sd_notify without linking libsystemd
  • Lack of sendfile support for std.Io.net.Stream.Writer (though it looks like there’s been some progress on this recently thanks to you)

But overall I’ve enjoyed the experience of building a server on top of std.Io. Being able to write everything once and have it just work reasonably efficiently, regardless of what OS or event loop you run want to run it on, is super nice.

2 Likes
10

Good reminder, now that it’s merged, I could implement the blocking sendfile for std.Io.Threaded as I am curious how it compares to my zio user-mode implementation, which is currently much faster than the fallback used in std.Io.Writer.

3 Likes
11

I appreciate you bringing this up. I don’t personally have time to give these issues the amount of attention to detail that I think they deserve for a couple more weeks. Other team members may have time, it’s up to them. But at that point let’s strategize about releases. Maybe we tag 0.17.0 without solving these problems, but we try to do another “short” cycle (at least by intention) that aims only to address std.Io networking before the next tag.

30 Likes
Ancillary data w/ Io
12

I came across a similar problem and talking to a couple of guys I think the real problem is more organisatorial than technical.

I love your approach of making Zig uncompromisingly what you want it to be. So what I’m saying is not “hurry up” or “it’s good enough” or “grow the team”.

What I mean to say is that the project needs to scale without any of the above, because the current state is structurally problematic for the ecosystem. I’m using Zig a lot lately and I’m in love with it. But all of my problems take the same shape: I vendor almost all of my dependencies and add hacks and workarounds for issues that are often rooted in unfinished or problematic abstractions in std.*. The problem with “abstractions” is that you can work around bug or implementation details easily, but if the abstraction is flawed, this is much harder. A workaround then leaks through all libraries and creates a maintenance debt everywhere. The more such debts accumulate, the more Zig as a whole gets a bad reputation. The damage grows at every level.

A concrete example for this: I need to create an outgoing connection bound to a specific IP address. This is not directly supported in std.Io. It’s easy to create a socket manually, but std.Io combines create + connect so you cannot use this socket. You can create and connect the socket outside of std.Io and then use it, but then you need to integrate the connect itself in std.Io (you don’t want to block a thread for several seconds waiting for a connect to complete). I can do that in my own code, but when I use an external library, they too do not have abstractions for this, e.g. httpx. They don’t want to add hacks for features that are obscure to most and important to some users. And if they adopt a workaround, they have to remove it again next zig release.

The solution could be to raise the prio for std, to grow the team or to just be patient.

None of that really works for reasons.

But what about adopting the Linux theme of module-maintainers? The scope of Zig is huge and eventually, it will need to scale or it will remain niche. I personally would like Zig to become more popular, it’s absolutely lovable and popularity is what drives availability of libraries, something Zig still lacks. Being a maintainer is certainly not as much fun as hacking, but it allows to scale. I don’t know if this is an option, but it’s the only way I see to accelerate things without pressuring the creative and innovative branch into delivering this or that tomorrow because it’s needed now.

In the area of std.Io, I believe the most important issue is the interface itself, much more than any concrete implementation. If the abstraction is correct and complete enough, anybody can provide an implementation. Once this is there, the community can evolve without blocking the core team.

Just my 2 cents…

8 Likes
13

I understand the criticism, at the same time, it’s hard to innovate if you don’t ship something, that people can use, and criticize, I don’t think anybody, can masterfully think of all use cases, all the nuances, in which someone, or something may or may not need to use what you do.

Therefore, It’s probably easier, to let your user test it for you, and give you informed feedback, which you can then use to iterate on the design and eventually settle on something that pleases the most people, there’s always going to be tradeoff, but I think that’s part of the process right ?

But I do agree that the scope is huge, and I think splitting more of the compiler work, into maybe a tier system, for example compiler is only for tier 1 contributors, std design for tier 2, std bug for tier 3, zig pieces that are shipped but not part of the compiler, as tier 4 and so on. Or what I really like is to have an std like it is, and a contrib friendly std, where everyone, can add the code.

2 Likes
14

I did not mean any of what I said as critique. My point is that I believe some parts of zig need acceleration, especially std.Io and I don’t want that to mean worse quality. As it stands today, the quality is cool, but the pace is lacking, meant as an observation.

The proposal to look at maintainers managing contributions and ensuring they meet the quality standards is the only way out of the dilemma I can see. It comes at a price, somebody had to be willing and able to take the role. And that’s a big ask. I just don’t see another solution, other than the wait or go away choice. Neither one is attractive for users.

“nobody can masterfully” - Sure. But if somebody focuses on the completeness and correctness aspects of the interface as a dedicated goal instead of distributing efforts on the right abstraction and meaningful implementations at the same time and is willing and able to scale up by reviewer community contributions, things accelerate.

If the answer to “Can I do networking with std.Io” is: yes if using an implementation supporting these features, then things are good. If the answer is “Yes, if std.Io interface were different, you could, but as it is you can’t.” then things are bad. In the first case, you can roll your own std.Io implementation. In the second, you can’t. You had to fork std.Io and compete with it, which is really bad.

5 Likes
15

I think the really critical part is where std abstracts over system/os interfaces. Incorrect or incomplete abstracts have really crippling effects there.

Most other parts of std are not all that critical. Data structures can be replaced by alternatives easily. But things like allocators and Io are critical, Io especially because it wraps async/concurrent and that has perversive effects

16

I see, more clearly, and I think you are right, having said that, without users you don’t get information and clear signal about whether the abstraction works or not, this is a tough nut to crack, and I think async/await is the 4 or 5th iteration, and I understand how they do it.

The way I code is similar, I consider the problem, implement the solution, realize that my mental model of the problem wasn’t right, re-implement it again further, realize, that it’s still missing pieces, redo it again, etc etc.

I think this is how you design great stuff, by iterating, using it, seeing it’s defect, and design with those new information in mind. And as you said it yourself, Zig doesn’t compromise, and so it’s expected, to take more time, to be incomplete, and sometimes broken, or crippling because it didn’t consider something, at the same time, if it wasn’t for you or others idk if Andrew and others would have noticed, so in a way you noticing is also part of helping zig go faster

17

I work the same way, unless I work on things like protocols or generally things that have a well defined surface.

For std.Io that basically means the syscall/sysapi surface of posix/linux/mac/*bsd/win. This is ridiculously huge. But it’s also the kind of task where you can let AIs do some research and find problems you would overlook until reaching the 5th iteration of whatever you do.

I think my split between creative/innovative and mechanical/scale out is just that. All the layers that require adaptation to exising things (http protocol, system API, libc, …) are the kind of things that need to be correct and complete and as soon as possible.

I don’t enjoy the part of my work that is interfacing or wrapping alien code much, but in most of my work, it takes most of the time. I still enjoy it, it’s just not quite as cool as working on new stuff. Somebody has to do it to get it done though.

1 Like
18

I wonder if a version of Bevy’s “open code review” process could help with the PR backlog? I don’t have firsthand experience trying to work on Zig, so perhaps this wouldn’t address the specific issues here. But it’s an interesting model that could potentially complement the “module maintainers” approach. And the decentralized spirit of open code review seems like a good cultural fit for Zig! :face_with_tongue:

2 Likes
19

From the std.Io perspective, the PR backlog is a secondary issue. The primary issue, as far as I see it, is that the interface is still lacking in many areas, and in order to improve it, it needs a coherent design. You can’t do that by individual PRs from outside people.

One design area is timeout handling at multiple levels. Both making it easy to have per-operation timeouts, especially insider reader/writer, as that’s how most people interact with network and files, but also on task level. The later could be done by concurrentTimeout, or awaitTimeout, both work, it’s a matter of deciding where to put it. The per-operation timeouts mean adding some timeout control to reader/writer (I have an exploratory PR open for that), but also finishing the work of moving accept and connect to std.Io.Operation, and finishing the design of std.Io.Batch.

And related to the std.Io.Batch is an implementation detail in std.Io.Threaded. Timeouts via operateTimeout in std.Io.Threaded currently work just by luck. It still uses blocking sockets. It polls with the timeout, but then it does the blocking syscall and if the readiness was drained somewhere else, it will block outside of the timeout. In order to fix it, sockets in std.Io.Threaded would need to become non-blocking.

Another thing are socket options. For any production code, it’s essential to have control over the nagle’s algorithm, keep-alive, in some cases kernel buffer sizes. It’s not clear how it would fit the interface. Maybe it could be combined with the issue mentioned by @mutex, there is a real need to sometimes connect or accept a socket that is already bound to a specific address. Maybe if the vtable is split into POSIX-like operations netCreate/netBind/netListen/netConnect/netAccept, it would feel less strange to have netSetOption and netGetOption.

For integration with existing C libraries, it would be extremely useful to have a poll mechanism in the interface. Then you could integrate e.g. libpq, since it needs to do it’s own reads/writes, it just needs to be able to block on poll externally.

Additionally, the current split between netRead/netReceive is kind strange. It would make more sense if there was just one socket receive operation, with optional address. If the address is empty, it’s a stream read, otherwise it’s a datagram receive.

13 Likes
20

To illustrate the blast radius of this particular issue:

I have a use case, where I need to make http requests from a virtual IP to a device.

std.Io does not support that, consequently httpx does not expose this either. I can create the socket myself and set it up to connect nonblocking and then just inject the result as if it was created through std.Io. Non-blocking because I’m not using the threaded std.Io (zio).

On first write to that socket, the write op then waits until the connection is completed and the std.Io just works. However, now the write operation might fail with ConnectFailed, which is not an error of write but one of connect.

The std.Io implementation I’m using classifies this error as UnexpectedError. This classification is correct in regards to the std.Io contract (where the socket is already connected by construction) and incorrect in regards to Linux, where a write might fail with ECONNREFUSED if nonblocking.

I need to know if the device I’m contacting is reachable and distinguish this from a failed write, both errors I encounter frequently. In the first case I need to alert, in the second just retry.

To work around the issue in std.Io, zio would have to incorrectly pass through a ConnectFailed in write (and other) operations, httpx would have to support a hack most users don’t need, and I have to try to convince both of them to do that, knowing that it’s a dirty hack.

And then three people have to remember that there is this hack and remove it once the original problem is fixed. The first to remove it breaks the two using it. And meanwhile, regular users of all three libraries adapt to write ops raising ConnectFailed and have to be fixed if it no longer does so.

The alternative for me is to vendor both libraries, which means I have to constantly merge upstream changes, verify that my hack doesn’t break other things and if my stuff is a library, others wouldn’t want to use it because of obscure hackery on my part.

7 Likes
21

this sounds like a PR to std?

1 Like