This is a wonderfully clear explanation of why futures are such a useful tool for the creation of concurrent applications. In fact, futures are the most FP-like way of encapsulating concurrency that I’ve seen (although I haven’t played around with much FP concurrency, so if I’m entirely off-base please tell me).
Well they’re usually unbounded which isn’t great (although you can pool them) and the “more FP” way of doing it is concurrent pipes/channels. You see this with concurrent FRP, Haskell’s pipes-concurrency, concurrent-machines, etc etc.
Futures are in Haskell basically less flexible/useful MVars - they don’t go that far, but they’re fine to use when appropriate. Pretty limited.
Twitter Futures don’t work that way. They aren’t backed by a threadpool by default, and they don’t represent asynchronous computation, they represent asynchronous values. As the paper states, they aren’t ersatz threads, so it’s not meaningful to stay that they’re unbounded.
Twitter Futures are more like dataflow variables, which it turns out is a pretty nice abstraction for doing asynchronous programming.
Don’t Block (unless you do it the right way)
Finagle automatically juggles threads to keep your service running smoothly. However, if your service blocks, it can block all Finagle threads.
If your code calls a blocking operation (apply or get), use a Future Pool to wrap the blocking code. This runs the blocking operation in its own thread pool, giving you a Future for the completion (or failure) of that operation which you can compose with other Futures.
Most Finagle/Futures code I’ve seen has involved used unbounded thread pool for the ExecutorService. Even with the unboundedness that aside, Futures are not the most structured way to handle concurrency and aren’t as powerful or efficient as you’d need in many cases.
Stuff like Haskell’s MVar and STM are more powerful and general. We also have green threads roughly as cheap as Erlang’s but with the option of forking a “real” thread with or without CPU pinning if we want it.
An incomplete list of some of the tools Haskell has for attacking concurrency problems:
If most of the work you’re doing is blocking, then probably Futures are the wrong abstraction for you to use, I agree. However, most uses of Futures in finagle are done when receiving requests or making requests, both of which are asynchronous. If you have to do a blocking computation and mix it with an asynchronous one, then it’s probably simpler for you to back it by a thread pool–but it’s trivial to do it in a bounded way, either with an AsynchronousQueue paired with an unbounded FuturePool, or a bounded FuturePool, and it’s necessarily intentional. It’s quite easy to avoid using an unbounded pool–if you want to, you have to use FuturePool.unboundedPool, which seems pretty explicit.
Nobody is claiming that Futures are the only useful way to do any kind of concurrent programming. The claim is simply that Futures are a powerful tool for concurrency, much like the way that Threads are.
MVar is clearly very powerful–my impression is that it’s like a composable version of a java SynchronousQueue? I agree this is very powerful, but it’s unclear to me why it’s inherently more structured or efficient than a Future-based implementation.
STM is a joy to use–if I could, I would definitely prefer to use STM to using locks, which can be very thorny to think about. However, as far as I can tell, nobody has gotten the optimistic locking in STM to perform all that well under significant load, especially when you’re composing many chunks of STM’d code. Fundamentally, it seems like there are some problems which are simpler with pessimistic locking. Clearly some progress has been made since “STM: Why is it only a research toy” but I don’t think it would be the first tool I would reach for when building a concurrent system until more work has been done.
I’m sure that haskell has many useful abstractions for doing concurrent programming–I have heard good things about green threads, pipes, and machines. Futures are essentially backed by IVars–this used to be explicit in the code but was effectively inlined in our Promise implementation.
, nobody has gotten the optimistic locking in STM to perform all that well under significant load,
Nobody has gotten Ruby to perform all that well under significant load either, but people still use it a lot. Similarly, when you need easy-to-reason about concurrency and your performance/fairness expectations aren’t strong, STM can still perform very well.
In particular, a lot of the problems with STM center on contention with a single STM container. That’s the pathological case and not really how you want to use STM. Similarly, you wouldn’t have a single global lock like MongoDB in a database you designed right?
STM’s fine for most concurrency problems, but the default thing resorted to is MVar. Which is efficient, simple and provably fair. Non-Haskell users don’t understand that Haskellers use STM when there’s something complicated or they need to eliminate the possibility of deadlocks. Most other language communities don’t have this option at all and no amount of dissembling about the “problems” with STM is going to change the fact that you aren’t even given the choice.
Do the simple thing that works correctly first, then make it more complicated it has to be. That’s the design ethic that introduces STM as a nice concurrency abstraction.
Similarly, when I wrote Blacktip, I used a single MVar wrapped two pieces of data. I could’ve made it more granular and put the timestamp in an MVar and the sequence into an atomic counter for performance, but it didn’t really matter.
A take and a put from an MVar takes 14 nanoseconds on most machines, for a sense of scale. Generating a single unique id in Blacktip takes ~1-1.2 microseconds, I could make that 2-3x faster. The MVar would still be a minuscule part of the time to return a unique id.
The point is that in Haskell, there is no one concurrency fixit.
We have almost everything. You can go really high level with things like Par and STM, or really low-level with locks and semaphores and FFI.
The point is that we have more choices that are fast, nice to use, and can be reasoned about.
This is a wonderfully clear explanation of why futures are such a useful tool for the creation of concurrent applications. In fact, futures are the most FP-like way of encapsulating concurrency that I’ve seen (although I haven’t played around with much FP concurrency, so if I’m entirely off-base please tell me).
Well they’re usually unbounded which isn’t great (although you can pool them) and the “more FP” way of doing it is concurrent pipes/channels. You see this with concurrent FRP, Haskell’s pipes-concurrency, concurrent-machines, etc etc.
Futures are in Haskell basically less flexible/useful MVars - they don’t go that far, but they’re fine to use when appropriate. Pretty limited.
Twitter Futures don’t work that way. They aren’t backed by a threadpool by default, and they don’t represent asynchronous computation, they represent asynchronous values. As the paper states, they aren’t ersatz threads, so it’s not meaningful to stay that they’re unbounded.
Twitter Futures are more like dataflow variables, which it turns out is a pretty nice abstraction for doing asynchronous programming.
Come on.
https://github.com/twitter/util/blob/master/util-core/src/main/scala/com/twitter/util/FuturePool.scala
https://twitter.github.io/scala_school/finagle.html specifically:
Most Finagle/Futures code I’ve seen has involved used unbounded thread pool for the ExecutorService. Even with the unboundedness that aside, Futures are not the most structured way to handle concurrency and aren’t as powerful or efficient as you’d need in many cases.
Stuff like Haskell’s MVar and STM are more powerful and general. We also have green threads roughly as cheap as Erlang’s but with the option of forking a “real” thread with or without CPU pinning if we want it.
An incomplete list of some of the tools Haskell has for attacking concurrency problems:
One of the best books on parallelism and concurrency I’ve ever read is all free and all Haskell: http://chimera.labs.oreilly.com/books/1230000000929
MVar (powerful hybrid of a promise and queue)
scalaz-streams was based on Kmett’s Haskell library machines - http://hackage.haskell.org/package/machines
http://hackage.haskell.org/package/stm-conduit
If most of the work you’re doing is blocking, then probably Futures are the wrong abstraction for you to use, I agree. However, most uses of Futures in finagle are done when receiving requests or making requests, both of which are asynchronous. If you have to do a blocking computation and mix it with an asynchronous one, then it’s probably simpler for you to back it by a thread pool–but it’s trivial to do it in a bounded way, either with an AsynchronousQueue paired with an unbounded FuturePool, or a bounded FuturePool, and it’s necessarily intentional. It’s quite easy to avoid using an unbounded pool–if you want to, you have to use FuturePool.unboundedPool, which seems pretty explicit.
Nobody is claiming that Futures are the only useful way to do any kind of concurrent programming. The claim is simply that Futures are a powerful tool for concurrency, much like the way that Threads are.
MVar is clearly very powerful–my impression is that it’s like a composable version of a java SynchronousQueue? I agree this is very powerful, but it’s unclear to me why it’s inherently more structured or efficient than a Future-based implementation.
STM is a joy to use–if I could, I would definitely prefer to use STM to using locks, which can be very thorny to think about. However, as far as I can tell, nobody has gotten the optimistic locking in STM to perform all that well under significant load, especially when you’re composing many chunks of STM’d code. Fundamentally, it seems like there are some problems which are simpler with pessimistic locking. Clearly some progress has been made since “STM: Why is it only a research toy” but I don’t think it would be the first tool I would reach for when building a concurrent system until more work has been done.
I’m sure that haskell has many useful abstractions for doing concurrent programming–I have heard good things about green threads, pipes, and machines. Futures are essentially backed by IVars–this used to be explicit in the code but was effectively inlined in our Promise implementation.
Nobody has gotten Ruby to perform all that well under significant load either, but people still use it a lot. Similarly, when you need easy-to-reason about concurrency and your performance/fairness expectations aren’t strong, STM can still perform very well.
In particular, a lot of the problems with STM center on contention with a single STM container. That’s the pathological case and not really how you want to use STM. Similarly, you wouldn’t have a single global lock like MongoDB in a database you designed right?
STM’s fine for most concurrency problems, but the default thing resorted to is MVar. Which is efficient, simple and provably fair. Non-Haskell users don’t understand that Haskellers use STM when there’s something complicated or they need to eliminate the possibility of deadlocks. Most other language communities don’t have this option at all and no amount of dissembling about the “problems” with STM is going to change the fact that you aren’t even given the choice.
Do the simple thing that works correctly first, then make it more complicated it has to be. That’s the design ethic that introduces STM as a nice concurrency abstraction.
Similarly, when I wrote Blacktip, I used a single MVar wrapped two pieces of data. I could’ve made it more granular and put the timestamp in an MVar and the sequence into an atomic counter for performance, but it didn’t really matter.
A take and a put from an MVar takes 14 nanoseconds on most machines, for a sense of scale. Generating a single unique id in Blacktip takes ~1-1.2 microseconds, I could make that 2-3x faster. The MVar would still be a minuscule part of the time to return a unique id.
The point is that in Haskell, there is no one concurrency fixit.
We have almost everything. You can go really high level with things like Par and STM, or really low-level with locks and semaphores and FFI.
The point is that we have more choices that are fast, nice to use, and can be reasoned about.