1. 17
  1. 6

    Project Loom is getting closer! A quick bit of background for those who don’t follow developments in the Java/JVM world:

    Virtual threads are an attempt to bridge the gap between the simple flow of control of traditional thread-based concurrency and the high scalability of asynchronous programming. They offer an execution environment that looks to the application code like a thread-based model with synchronous blocking operations. Under the hood, though, the virtual machine implements those threads using user-space continuations that run on a pool of underlying OS threads. Blocking I/O operations become asynchronous internally, but present synchronous behavior to application code. Creating a virtual thread is an extremely cheap operation and even modest hardware can easily support millions of active virtual threads. Converting an existing application from traditional threads with blocking synchronous I/O to virtual threads with async I/O can be as simple as a one-line code change in some cases.

    1. 1

      So would we expect a performance boost in case of traditional frameworks, servers and libraries, just because (virtual) threads will be less expensive to use ?

      1. 10

        With the caveat that I’m just an enthusiastic outsider, not someone actually working on it: You should see a performance improvement in a lot of common cases, but it’ll depend on the workload and on what aspects of performance you care about.

        The ideal scenario where virtual threads will clearly be a big win for very little effort is if the workload is mostly I/O-bound, most I/O is done synchronously, concurrency is thread-based, and the number of concurrent operations is artificially capped because not enough threads can be spawned (e.g., because their stacks would eat too much memory). In that kind of situation, switching to virtual threads should cause an immediate capacity increase.

        If the workload is primarily CPU-bound, virtual threads won’t buy you much. They might even cause undesirable side effects like starvation if you have more active threads than CPU cores since (to my knowledge) virtual threads don’t get switched preemptively. A given OS thread will keep executing its currently-active virtual thread until the virtual thread hits a suspend point (I/O operation, etc.) You might save a modest amount of memory but that’d be about it.

        But assuming you’re not CPU-bound, aside from increased capacity, you might also see some performance differences thanks to the fact that virtual threads don’t require much memory. I’d expect to see total memory footprint go down in a lot of cases, and as a result you might see improved performance from better cache locality, though that’d be hugely dependent on what the code is doing.

        In a nontrivial application, the performance impact is going to get complicated, I think. For example, being able to spawn millions of virtual threads with low overhead won’t cause you to have millions of available database connections in your connection pool. On the other hand, I think any application that’s using traditional threads on big metal probably already hits most of those kinds of resource contention bottlenecks. Virtual threads won’t make things any worse; they just won’t be a panacea.

        The reason I’m quite excited about virtual threads is parochial: I find it annoying to work with code that solves the scalability problem by making everything asynchronous. It’s especially awful in Java and only slightly better on other JVM languages (heaven help you if you have to read a stack trace from RxJava code) but even when there’s first-class language and runtime support for async execution, it’s added cognitive overhead and a thing you can get subtly wrong without noticing.

        Ideally, I don’t think application code should have to explicitly deal with the mechanics of concurrency unless it happens to be code that is specifically solving concurrency-related problems, but right now evidence of the concurrency model is strewn like trash all over async-style application code. I once spent a couple days tracking down a highly intermittent bug in TypeScript code that turned out to be due to me omitting the await keyword on one line; by leaving that out, I’d created a race condition without knowing it. Virtual threads will mean I can write that code in synchronous style but still get the scaling benefits of the async style.