in javascript, if you have an async function, you cannot call it from a sync-function and get it’s return-value in a sync way.
in rust this is not true, you can achieve this.
therefore, rust does not have the coloring-problem.
this might be true, but the more fundamental problem is still there: you should not mix&match sync and async functions too much, otherwise bad things can happen (even the article mentions this). so i think the coloring problem is still there.
this is not something any language (that has both sync and async functions) can escape from i think. for example, async-functions assume they never block (for a long time). so every time you call a sync-function from an async-function, you have to know that it will return shortly.
This argument doesn’t even hold up against JavaScript when using Node.js. Although it is a hack and not at all recommended for production use, on Node.js you can use something like deasync to block on promises and other things.
It’s nice to bring some nuance to the discussion: some languages and ecosystems have it worse than others.
To add some more nuance, here’s a tradeoff about the “throw it in an executor” solution that I rarely see discussed. How many threads do you create?
Well, first, you can either have it be bounded or unbounded. Unbounded seems obviously problematic because the whole point of async code is to avoid the heaviness one thread per task, and you may end up hitting that worst case.
But bounded has a less obvious issue in that it effectively becomes a semaphore. Imaging having two sync tasks, A and B where the result of B ends up unblocking A (think mutex) and a thread pool of size 1. If you attempt to throw both on a thread pool, and A ends up scheduled and B doesn’t, you get a deadlock.
You don’t even need dependencies between tasks, either. If you have an async task that dispatches a sync task that dispatches an async task that dispatches a sync task, and your threadpool doesn’t have enough room, you can hit it again. Switching between the worlds still comes with edge cases.
This may seem rare and it probably is, especially for threadpools of any appreciable size, but I’ve hit it in production before (on Twisted Python). It was a relief when I stopped having to think about these issues entirely.
Imaging having two sync tasks, A and B where the result of B ends up unblocking A (think mutex)
Isn’t this an antipattern for async in general? Typically you’d either a) make sure to release the mutex before yielding, or b) change he interaction to “B notifies A”, right?
Changing the interaction to “B notifies A” doesn’t fix anything because presumably A waits until it is notified, taking up a threadpool slot, making it so that B can never notify A. Additionally, it’s not always obvious when one sync task depends on another, especially if you allow your sync tasks to block on the result of an async task. In my experience, that sort of thing happens when you have to bolt the two worlds together.
But bounded has a less obvious issue in that it effectively becomes a semaphore. Imaging having two sync tasks, A and B where the result of B ends up unblocking A (think mutex) and a thread pool of size 1. If you attempt to throw both on a thread pool, and A ends up scheduled and B doesn’t, you get a deadlock.
I’ve never designed a system like this or worked on a system designed like this before. I’ve never had one task depend on the value of another task while both tasks were scheduled simultaneously. As long as your tasks spawn dependent tasks and transitively one of those eventually dependent tasks does not have to wait on another task, we can ensure that the entire chain of tasks will finish 1. That said, running out of threads in a thread pool is a real problem that plagues lots of thread-based applications. There are multiple strategies here. Sometimes we try to acquire a thread from the pool with a deadline and we retry a few times to grab a thread from the pool, eventually failing the computation if we just cannot grab a thread from the pool. Other times we just spawn a new thread, but this can lead to scheduler thrashing if we end up spawning too many threads. Another common solution is to create multiple thread pools and allocate different pools to different workloads, so that you can make large thread pools for long running threads and make smaller thread pools of short running tasks.
Thread-based work scheduling can, imo, be just as complicated as async scheduling. The biggest difference is that async scheduling makes you pay the cost in code complexity (through function coloring, concurrency runtimes, etc) while thread-based scheduling makes you pay the cost in operational and architectural complexity (by deciding how many thread pools to have, which tasks should run on which pools, how large each pool should be, how long we should wait to retry to grab a thread from the pool, etc, etc). While shifting the complexity to operational and architectural complexity might seem to shift the work up to operators or some dedicate operationalizing phase, in practice the context lost by lifting decisions up to this level can make tradeoffs for pools and tasks non-obvious, making it harder to make good decisions. Also, as workloads change over time, new thread pools may need to be created and these new pools necessitate rebalancing of other pools, which requires a lot of churn. Async has none of these drawbacks (though to be clear, it has its own unique drawbacks.)
I’ve never designed a system like this or worked on a system designed like this before. I’ve never had one task depend on the value of another task while both tasks were scheduled simultaneously.
Here’s perhaps a not-unreasonable scenario: imagine a cache with an API to retrieve some value for a key if it exists and otherwise compute, store, and return it. The cache exports an async API and the callback it runs to compute the value ends up dispatching a sync task to a threadpool (maybe it’s a database query using a sync library). We want the cache to be able to be accessed from multiple threads, so it is wrapped in a sync mutex.
Now imagine that an async task tries to use the cache that is backed by a threadpool of size 1. The task disaptches a thread which acquires the sync mutex, calls to get some value (waiting however on the returned future), and assuming it doesn’t exist, the cache blocks forever because it cannot dispatch the task to produce the value. The size of 1 isn’t special: this can happen with any bounded size thread pool under enough concurrent load.
One may object to the sync mutex, but you can have the same issue if the cache is recursive in the sense that producing a value may depend on the cache populating other values. I don’t think that’s very far fetched either. Alternatively, the cache may be a library used as a component of a sync object that is expected to be used concurrently and that is the part that contains the mutex.
In my experience, the problem is surprisingly easy to accidentally introduce when you have a code base that frequently mixes async and sync code dispatching to each other. Once I started really looking for it, I found many places where it could have happened in the (admittedly very wacky) code base.
Fair enough that is a situation that can arise. Those situations I would probably reach for either adding an expiry to my threaded tasks or separating thread pools for DB or cache threads from general application threads. (Perhaps an R/W Lock would help over a regular mutex, but I realize that’s orthogonal to the problem at hand here and probably a pedagogical simplification.) The reality is that mixing sync and async code can be pretty fraught if you’re not careful.
I have seen similar scenarios without a user visible mutex: you get deadlocks if a thread on a bounded thread pool waits for another task scheduled on the same thread pool
Of course, there are remedies, e.g. never schedule subtasks on the same thread pool. Timeouts help but still lead to abysmall behavior under load because your threads just idle around until the timeout triggers.
Isn’t that equivalent to either a threadpool of size 1 or going back to epoll style event loops? If it’s the former, you haven’t gained anything, and if it’s the latter, you’ve thrown out the benefits of the async keyword.
Async has always been a syntax sugar for epoll-style event loops. Number of threads has nothing to do with it, e.g. tokio can switch between single and multi-threaded execution, but so can nginx.
Async gives you higher-level composability of futures, and the ease of writing imperative-like code to build state machines.
Want to call an async function blockingly from a sync function? -> Throw it on an executor!
This is colored, no? If it wasn’t you would just call the async function without needing an executor. Cannot you do something like this in every language that has futures-based async?
You can fire-and-forget async functions in any language with “colors”, but you can’t block synchronously until they finish.
In JS there’s no way to get a result of a Promise without unwinding your stack first (microtasks are processed only after the entire call stack unwinds, so no return, no results).
In Rust you can block a thread waiting for anything async that you want. That’s not even a feature of async executors, but merely the fact that you’re in full control OS threads, so you can always build on top of that.
No, it’s typed. Rust, like C# and others, avoid the color mess by just unifying it with the type system. It has special syntax but under the hood it’s all the same. It’s simpler (no separate color checker needed) and more powerful (eg sync manipulation of async functions).
// "async" color
async function a() {
return "hello";
}
// not "async" color, but calls an async-color function
function b() {
a().then(greeting => console.log(`${greeting} world`));
}
You can also easily return values calculated in sync functions that depend on data from async functions, because an async function is just syntax sugar for returning a Promise, and so your non-async function can just return a Promise as well:
// "async" color
async function a() {
return 1;
}
// not "async" color
function b() {
return a().then(x => x + 1);
}
// not "async" color
function c() {
b().then(val => console.log(val));
}
c();
// prints 2
Or, since await is just syntax sugar for waiting for a Promise, you could even mix and match, e.g. c could be rewritten as:
async function c() {
const val = await b();
console.log(val);
}
Even though b was never explicitly tagged with the async color marker.
Well, it really depends what you mean by “async”, and I think precision matters, as this topic is trickier in JS than some give it credit for. This might be pedantic on my part, but just to be super clear here, a function “is async” in JS when it uses the “async” keyword – that’s it. The only thing that the async keyword does is:
It makes the function return a promise
It enables the use of await and try/catch/finally as syntactic sugar for .then/.catch/.finally.
As a result, when an async function is desugared, it decomposes into an outer function which chains together a number of callbacks. This is what makes the async function feel async; any one statement within the async function body could belong to a number of different callbacks after desugaring.
If you’re using promises (via then, etc) the function is async, not sync, no matter which syntax you’re using…
I don’t think “the function” is precise enough to be meaningful because it could mean two things:
this function composes and returns a promise chain, and the process by which that chain runs to completion is asynchronous with respect to other work happening in the event loop
this function constructs a value, calls a method on it, and then returns that value; the value may be a promise but that’s neither here nor there when it comes to each statement in the outer function body getting executed synchronously
To get a Promise in JS, not returning it and neither handling any rejection of it is dangerous, as that will swallow any exceptions thrown within that Promise chain.
It also won’t execute the content of the Promise chain synchronously, so any code depending on that execution won’t be able to wait for it to be done.
This is a real issue in JS that’s been discussed in the context of eg CLI tools, where a sync execution can be preferable from a simplicity view point.
I’m curious how many languages have “colored” functions in this blog post’s sense of you can’t feasibly call one from the other. For example, the post brings up JavaScript, but JS is pretty similar in terms of the functions not actually being “colored” in a strict sense; the async keyword is just syntax sugar for wrapping the returned object in a Promise, and await is similarly syntax sugar for calling .then on a promise. A non-async function can call an async function just fine:
async function a() {
return 1;
}
function b() {
a().then(x => console.log(x));
}
Are there languages where you really can’t call an async function from a non-async function at all? Even Python has a Rust-like awkward solution of throwing things on an event loop (personally, I find the JS/TypeScript approach of everything-is-a-Promise to be more elegant, though).
I guess my take is a bit ironic given that the original “What color is your function” blog post was focused on JavaScript — albeit before it had shipped async/await, although the original post also says async/await is bad, so /shrug. But in terms of contrasting the original blog post and this Rust-focused one… I mean, Rust’s functions do seem colored in the async/await sense? It is more annoying to call an async function from a non-async function, considerably more so than even JS. I guess neither me nor the author is convinced that’s a huge downside — it’s more verbose, but the payoff of async code is nice in that you can avoid threads while still getting concurrency.
The fact that bridging the two worlds requires a bit of syntax is a deliberate design in Rust.
.await points have important semantic meaning, because the caller driving the async code can abort it at any .await point. Just like with ? for handling errors, Rust wants to make it locally explicit where functions can diverge.
The compiler won’t automagically spawn async fn on an executor, because unlike Node/Zig/Golang, Rust doesn’t have any global built-in executor.
Rust by design avoids “magic” runtime behaviors. It doesn’t even do heap allocations or refcounting automatically. Implicitly running event loops is very far from the “portable assembler” mindset.
Rust supports all kinds of platforms, from embedded to WASM, so one standard built-in executor wouldn’t work well for all of them.
In network applications that need low latency you also wouldn’t want random parts of code and 3rd party libraries implicitly throw extra work onto your precious executor’s queue.
A standard API for abstracting executors for 3rd party libraries is an open design problem in Rust. It’s not a limitation of the language. It’s just people haven’t agreed yet how it should look like. So far it’s been agreed that global mutable state is bad, so the API needs to be more sophisticated than hooking up something to a global spawn.
[[tangential]] what’s with all the swearing nowadays? Has it become impossible to drive a technical point without swearing? I’m maybe becoming one hell of a grizzli bear but I find that to be a major turn-off.
There’s no swearing here by typical standards. There is a Bowdlerized expletive in the title, but that word has explicitly been neutered. The author committed a crime against words in order to avoid swearing in front of you.
It’s a shame that the static site generator Gatsby needs a plug-in to make RSS feeds. Byproduct of the shitty JavaScript age we live in these days, sniff. /s
the basic argument of the article is this:
this might be true, but the more fundamental problem is still there: you should not mix&match sync and async functions too much, otherwise bad things can happen (even the article mentions this). so i think the coloring problem is still there.
this is not something any language (that has both sync and async functions) can escape from i think. for example, async-functions assume they never block (for a long time). so every time you call a sync-function from an async-function, you have to know that it will return shortly.
This argument doesn’t even hold up against JavaScript when using Node.js. Although it is a hack and not at all recommended for production use, on Node.js you can use something like deasync to block on promises and other things.
It’s nice to bring some nuance to the discussion: some languages and ecosystems have it worse than others.
To add some more nuance, here’s a tradeoff about the “throw it in an executor” solution that I rarely see discussed. How many threads do you create?
Well, first, you can either have it be bounded or unbounded. Unbounded seems obviously problematic because the whole point of async code is to avoid the heaviness one thread per task, and you may end up hitting that worst case.
But bounded has a less obvious issue in that it effectively becomes a semaphore. Imaging having two sync tasks, A and B where the result of B ends up unblocking A (think mutex) and a thread pool of size 1. If you attempt to throw both on a thread pool, and A ends up scheduled and B doesn’t, you get a deadlock.
You don’t even need dependencies between tasks, either. If you have an async task that dispatches a sync task that dispatches an async task that dispatches a sync task, and your threadpool doesn’t have enough room, you can hit it again. Switching between the worlds still comes with edge cases.
This may seem rare and it probably is, especially for threadpools of any appreciable size, but I’ve hit it in production before (on Twisted Python). It was a relief when I stopped having to think about these issues entirely.
Isn’t this an antipattern for async in general? Typically you’d either a) make sure to release the mutex before yielding, or b) change he interaction to “B notifies A”, right?
Changing the interaction to “B notifies A” doesn’t fix anything because presumably A waits until it is notified, taking up a threadpool slot, making it so that B can never notify A. Additionally, it’s not always obvious when one sync task depends on another, especially if you allow your sync tasks to block on the result of an async task. In my experience, that sort of thing happens when you have to bolt the two worlds together.
It’s a general problem. It can happen whenever you have a threadpool, no matter whether it’s sync or async.
I’ve never designed a system like this or worked on a system designed like this before. I’ve never had one task depend on the value of another task while both tasks were scheduled simultaneously. As long as your tasks spawn dependent tasks and transitively one of those eventually dependent tasks does not have to wait on another task, we can ensure that the entire chain of tasks will finish 1. That said, running out of threads in a thread pool is a real problem that plagues lots of thread-based applications. There are multiple strategies here. Sometimes we try to acquire a thread from the pool with a deadline and we retry a few times to grab a thread from the pool, eventually failing the computation if we just cannot grab a thread from the pool. Other times we just spawn a new thread, but this can lead to scheduler thrashing if we end up spawning too many threads. Another common solution is to create multiple thread pools and allocate different pools to different workloads, so that you can make large thread pools for long running threads and make smaller thread pools of short running tasks.
Thread-based work scheduling can, imo, be just as complicated as async scheduling. The biggest difference is that async scheduling makes you pay the cost in code complexity (through function coloring, concurrency runtimes, etc) while thread-based scheduling makes you pay the cost in operational and architectural complexity (by deciding how many thread pools to have, which tasks should run on which pools, how large each pool should be, how long we should wait to retry to grab a thread from the pool, etc, etc). While shifting the complexity to operational and architectural complexity might seem to shift the work up to operators or some dedicate operationalizing phase, in practice the context lost by lifting decisions up to this level can make tradeoffs for pools and tasks non-obvious, making it harder to make good decisions. Also, as workloads change over time, new thread pools may need to be created and these new pools necessitate rebalancing of other pools, which requires a lot of churn. Async has none of these drawbacks (though to be clear, it has its own unique drawbacks.)
Here’s perhaps a not-unreasonable scenario: imagine a cache with an API to retrieve some value for a key if it exists and otherwise compute, store, and return it. The cache exports an async API and the callback it runs to compute the value ends up dispatching a sync task to a threadpool (maybe it’s a database query using a sync library). We want the cache to be able to be accessed from multiple threads, so it is wrapped in a sync mutex.
Now imagine that an async task tries to use the cache that is backed by a threadpool of size 1. The task disaptches a thread which acquires the sync mutex, calls to get some value (waiting however on the returned future), and assuming it doesn’t exist, the cache blocks forever because it cannot dispatch the task to produce the value. The size of 1 isn’t special: this can happen with any bounded size thread pool under enough concurrent load.
One may object to the sync mutex, but you can have the same issue if the cache is recursive in the sense that producing a value may depend on the cache populating other values. I don’t think that’s very far fetched either. Alternatively, the cache may be a library used as a component of a sync object that is expected to be used concurrently and that is the part that contains the mutex.
In my experience, the problem is surprisingly easy to accidentally introduce when you have a code base that frequently mixes async and sync code dispatching to each other. Once I started really looking for it, I found many places where it could have happened in the (admittedly very wacky) code base.
Fair enough that is a situation that can arise. Those situations I would probably reach for either adding an expiry to my threaded tasks or separating thread pools for DB or cache threads from general application threads. (Perhaps an R/W Lock would help over a regular mutex, but I realize that’s orthogonal to the problem at hand here and probably a pedagogical simplification.) The reality is that mixing sync and async code can be pretty fraught if you’re not careful.
I have seen similar scenarios without a user visible mutex: you get deadlocks if a thread on a bounded thread pool waits for another task scheduled on the same thread pool
Of course, there are remedies, e.g. never schedule subtasks on the same thread pool. Timeouts help but still lead to abysmall behavior under load because your threads just idle around until the timeout triggers.
Note that you can also run async Rust functions with zero (extra) threads, by polling it on your current thread. A threadpool is not a requirement.
Isn’t that equivalent to either a threadpool of size 1 or going back to epoll style event loops? If it’s the former, you haven’t gained anything, and if it’s the latter, you’ve thrown out the benefits of the async keyword.
Async has always been a syntax sugar for epoll-style event loops. Number of threads has nothing to do with it, e.g. tokio can switch between single and multi-threaded execution, but so can nginx.
Async gives you higher-level composability of futures, and the ease of writing imperative-like code to build state machines.
This is colored, no? If it wasn’t you would just call the async function without needing an executor. Cannot you do something like this in every language that has futures-based async?
The key part is blockingly.
You can fire-and-forget async functions in any language with “colors”, but you can’t block synchronously until they finish.
In JS there’s no way to get a result of a
Promisewithout unwinding your stack first (microtasks are processed only after the entire call stack unwinds, so noreturn, no results).In Rust you can block a thread waiting for anything async that you want. That’s not even a feature of async executors, but merely the fact that you’re in full control OS threads, so you can always build on top of that.
No, it’s typed. Rust, like C# and others, avoid the color mess by just unifying it with the type system. It has special syntax but under the hood it’s all the same. It’s simpler (no separate color checker needed) and more powerful (eg sync manipulation of async functions).
Just because Rust and C# hardcoded
async/awaitinto the language doesn’t mean their functions aren’t colored anymore.Calling async functions from sync code in javascript is no easy feat
Wait, what? It’s much easier than Rust:
You can also easily return values calculated in sync functions that depend on data from async functions, because an async function is just syntax sugar for returning a Promise, and so your non-async function can just return a Promise as well:
Or, since
awaitis just syntax sugar for waiting for a Promise, you could even mix and match, e.g.ccould be rewritten as:Even though
bwas never explicitly tagged with theasynccolor marker.If you’re using promises (via then, etc) the function is async, not sync, no matter which syntax you’re using…
Well, it really depends what you mean by “async”, and I think precision matters, as this topic is trickier in JS than some give it credit for. This might be pedantic on my part, but just to be super clear here, a function “is async” in JS when it uses the “async” keyword – that’s it. The only thing that the
asynckeyword does is:awaitandtry/catch/finallyas syntactic sugar for.then/.catch/.finally.As a result, when an
asyncfunction is desugared, it decomposes into an outer function which chains together a number of callbacks. This is what makes the async function feel async; any one statement within the async function body could belong to a number of different callbacks after desugaring.I don’t think “the function” is precise enough to be meaningful because it could mean two things:
Pedantry noted. I should have said if your function returns a promise it is async.
To get a Promise in JS, not returning it and neither handling any rejection of it is dangerous, as that will swallow any exceptions thrown within that Promise chain.
It also won’t execute the content of the Promise chain synchronously, so any code depending on that execution won’t be able to wait for it to be done.
This is a real issue in JS that’s been discussed in the context of eg CLI tools, where a sync execution can be preferable from a simplicity view point.
I’m curious how many languages have “colored” functions in this blog post’s sense of you can’t feasibly call one from the other. For example, the post brings up JavaScript, but JS is pretty similar in terms of the functions not actually being “colored” in a strict sense; the
asynckeyword is just syntax sugar for wrapping the returned object in a Promise, andawaitis similarly syntax sugar for calling.thenon a promise. A non-asyncfunction can call anasyncfunction just fine:Are there languages where you really can’t call an
asyncfunction from a non-asyncfunction at all? Even Python has a Rust-like awkward solution of throwing things on an event loop (personally, I find the JS/TypeScript approach of everything-is-a-Promise to be more elegant, though).I guess my take is a bit ironic given that the original “What color is your function” blog post was focused on JavaScript — albeit before it had shipped
async/await, although the original post also saysasync/awaitis bad, so /shrug. But in terms of contrasting the original blog post and this Rust-focused one… I mean, Rust’s functions do seem colored in theasync/awaitsense? It is more annoying to call an async function from a non-async function, considerably more so than even JS. I guess neither me nor the author is convinced that’s a huge downside — it’s more verbose, but the payoff of async code is nice in that you can avoid threads while still getting concurrency.The fact that bridging the two worlds requires a bit of syntax is a deliberate design in Rust.
.awaitpoints have important semantic meaning, because the caller driving the async code can abort it at any.awaitpoint. Just like with?for handling errors, Rust wants to make it locally explicit where functions can diverge.The compiler won’t automagically spawn
async fnon an executor, because unlike Node/Zig/Golang, Rust doesn’t have any global built-in executor.A standard API for abstracting executors for 3rd party libraries is an open design problem in Rust. It’s not a limitation of the language. It’s just people haven’t agreed yet how it should look like. So far it’s been agreed that global mutable state is bad, so the API needs to be more sophisticated than hooking up something to a global
spawn.[[tangential]] what’s with all the swearing nowadays? Has it become impossible to drive a technical point without swearing? I’m maybe becoming one hell of a grizzli bear but I find that to be a major turn-off.
There’s no swearing here by typical standards. There is a Bowdlerized expletive in the title, but that word has explicitly been neutered. The author committed a crime against words in order to avoid swearing in front of you.
If you declare yourself a fan of hobos, swearing is de rigueur.
It’s a shame that the static site generator Gatsby needs a plug-in to make RSS feeds. Byproduct of the shitty JavaScript age we live in these days, sniff. /s