From the summary, it sounds like async is always as good or better than true threads. So when should I use a thread instead of async?
I think this is a limited view. What I’m seeing here is that threads, an arguably simpler concurrency primitive, can scale quite competitively on modern operating systems.
Therefore, while you should continue to use async/epoll etc. when there is a true need (e.g. building web facing software like haproxy, nginx, envoy, caddy), many use cases will run competitively using a simpler blocking threads model and it might be worth evaluating whether you’d prefer working in that paradigm.
Async requires separating I/O-bound tasks from CPU-bound ones (you need to specifically spawn a task on a threadpool if it’s going to hog the CPU). If you can’t reliably isolate the two workloads, you’re going to have I/O latency problems. In that case it’s better to sacrifice threads for mixed workload.
Similarly if you’re dealing with FFI or non-async libraries, you have no guarantee what they’ll do, so it’s best to use a real thread just in case.
async/await in Rust creates a single state machine for every possible await point. If your code is complex with lots of on-stack state intermixed with awaits, then this state machine can get large. Recursion requires heap allocation (because it ceases to have compile-time-fixed call graph depth, so the state becomes unbound). If your code is complex, with lots of state and recursion, a real thread with a real stack may be better.
If I understand rust’s async correctly or rather that it follows other languages’ async/concurrency patterns, you use async to improve single thread/core performance, it allows you to continue if the thread hits something like an io wait or similar non-cpu task. Multi-threading can do this but is better when processing for durations, like a browser.
cli tools that are intended to do one thing and exit quickly like ls for example may degrade performance by adding threads, but may improve performance by adding async. Now if as mentioned you have a browser and you have already maxed out what a single thread can do using async it’s time to split using threads to allow you to use more of the available cpu cores to do its job.
golang explanation of concurrency vs parallelism: https://blog.golang.org/waza-talk
Performance is only one metric. It is indeed true, to the first approximation, that async is not worse than threads in terms of performance. The most convincing argument for the opposite is that async might have worse latency, as it’s easier to starve tasks, but I haven’t seen this conclusively demonstrated for Rust.
Another important metric is ergonomics or “is programming model sane?”. In this regard, async, at least in Rust, is not a sane programming model. Stack traces do not make sense, you get extra “failure modes” (any await might not return (when the corresponding task is dropped)), and there are expressively restrictions (recursion, dynamic dispatch, and abstraction are doable, but awkward).
To play the devil’s advocate, sync, blocking model is also far from perfect. It doesn’t handle cancellation or selection between several sources of events. That is, if you do a blocking read call, there isn’t a nice way to cancel it.
With async, because you need to redo the entirety of IO anyway, you can actually go and add proper support for cancellation throughout.
When a syscall/FFI blocks, or when you might have to crunch a lot of numbers are good cases.