I will be building http proxy and I don’t have much experience with multithreading, so I want to keep it as easy as possible but I also need it performant. Since each request should be separate from others I want to avoid sharing memory between threads, maybe except some metrics counters. I also want to use event loop (zio) within the threads so one thread can serve many requests simultaneously. So I am asking more experienced devs here. 1. Does it make sense having event loop instance per thread? And 2. does it make sense trying to keep handling of one request within 1 thread or is that just unnecessary complication?
How fast do you want it to be?
If you are going for maximum performance, you need to work with the event loop directly. That means callbacks and completions, rather than threads and fibers. See the architecture of nginx or envoy. It’s not a very nice way of writing code, but it’s fast.
Oh the other hand, if you use fibers/threads, you are gaining some developer experience, things are easier to reason about, but you are leaving performance on the table. It’s a trade-off.
2 Likes
I need it to be fast but time to delivery of the finished program is also important. I believe green threads should be good enough so it is a trade off I am willing to take. But I will be doing experiments before committing. I want to focus on bigger picture stuff that won’t make the development actively harder for me. I just don’t want to do too many of those experiments so I am asking here 
Honestly, the answers really depend on the rest of the architecture. For example, if you want to use zio as the event loop, then you are tied to the event loop per thread setup, as you can’t use one event loop across multiple threads. I don’t see why would you need to even think about multithreading in a HTTP proxy, as it’s all I/O bound, and there is pretty much no cross-thread communication. The one thing that could be shared is a pool of upstream connections, and maybe some configuration. I’d just use std.Io for this and use a backend that runs in green threads.
1 Like