I don’t see what the difference is between that and async/await, I rewrote your code and there don’t seem to be any major changes other than run() being implicit. People used to use generators like you’re doing with libraries like co before async/await was widely available but now everyone seems to have moved on because async/await does basically the same thing without any dependencies.
async function main() {
/*
* Spawn some pre-existing balls.
*/
ball(100, 200);
ball(200, 300);
ball(300, 400);
/*
* Main loop.
*
* Block on click events, spawn a new ball on each click.
*/
while (true) {
const event = await new Promise((resolve) =>
document.addEventListener("click", resolve, { once: true }),
);
ball(event.pageX, event.pageY);
}
}
window.onload = () => main();
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
async function ball(x, y) {
const ball = document.createElement("div");
ball.classList.add("ball");
ball.style.left = x + "px";
ball.style.top = y + "px";
document.body.appendChild(ball);
const xdelta = -5 + rand(0, 10);
const ydelta = -5 + rand(0, 10);
while (true) {
x += xdelta;
y += ydelta;
ball.style.left = x + "px";
ball.style.top = y + "px";
await sleep(50);
}
}
function rand(min, max) {
return Math.random() * (max - min) + min;
}
I could be smarter with the events too, it was just faster to put in a oneliner like that :P
The only differences I can think of is Promises (and by extension, await) always call their callbacks as a microtask and not synchronously, the browser handles the scheduling, and also apparently there’s a .return() method on the result of a generator function, which I did not know about until today. Otherwise generators and async functions are almost equivalent, you can implement each in terms of the other
Exactly. With delimited continuations, if you pass a synchronous function then the continuation resolves synchronously without getting put on the event loop.
If you use async/await you don’t have a choice: it’s going on the event loop.
This might not matter for 90% use cases but when it does matter it is critically important.
Out of curiosity, how would you detect multiple functions trying to read an event with promises, and how would you detect there not being any function waiting for an event when one happens? I can’t really wrap my head around it. Is it doable? Or is that a semantic difference between generators and async/await?
One pleasant difference is that when you write a function as a coroutine the sync and the async variant and be the same function. To be sync, just evaluate using a coroutine that blows up if it is yielded a promise.
In iter-tools I don’t use that trick, and the cost I pay is that I have to write every single function twice (I wrote a macro system to do it).
I’m currently writing a parser, and having a macro copy all the parser code into sync and async variants would be absurd, but generator functions let me do it: define behavior once but in a what that propagates promises if they occur
I don’t see what the difference is between that and async/await
Coming back to this question after some learning, the differences are:
With promises, there is always an implicit asynchrony. The handlers for promise fulfillment and rejection always run via the microtask queue. With generators, this is not the case. This would be observable in the get_next_event() function of my demo, which doesn’t use any promises (unlike the sleep() function which does, and is therefore of course async).
I was just talking about this the other day in the BABLR discord! I think the author made exactly one mistake: All the places they’ve written yield*, yield would actually be more correct.
To understand why, look at how you would use this pattern to code a recursive algorithm in a way that you don’t need to worry about stack overflows. If you write yield* you would still get stack overflows, but with yield everything “just works” (and potentially quite a lot more performantly than yield*)!
It’s an inversion of control pattern really. yield* says, “I’m doing the calling here on my call stack,” but to make a call with yield you’re giving up control to the VM, saying, “You keep a call stack and do the bookkeeping for me.”
Most importantly this leaves you free to have a distinction between describing work that you want to be done and actually doing that work, which is where the real power of the pattern lies.
In my mind this one example illuminates the reasoning behind virtually every choice made in the generator spec, including one that drives me a bit crazy when debugging: the body of the generator function doesn’t begin executing until the first call to next(). That is the source of laziness that makes it all work. The example also explains the presence and mechanics of both generator.return() and generator.throw(), and the mechanics for passing values back through those methods from the VM to the yielder.
All the places they’ve written yield*, yield would actually be more correct.
I don’t think so. I use yield* because I need to delegate to another generator, and that generator may itself yield multiple times - I need to pass all those yields to the function’s original caller.
const arrayLast = arr => arr[arr.length - 1];
const { toString } = Object.prototype;
const isGenerator = val => toString.call(val) === '[object Generator]';
function *trampolineGenerators(rootFn) {
const stack = [rootFn];
let fn = rootFn, step = fn.next();
while (true) {
while (step.done) {
// a function's control flow terminated
stack.pop();
fn = arrayLast(stack);
if (!fn) {
// the call stack is exhausted, we are done
return step.value;
}
// a value is returned from a call
step = fn.next(step.value);
}
if (isGenerator(step.value)) {
// function call: generator lazily invoked another generator
stack.push(step.value);
fn = arrayLast(stack);
} else {
// generator yields a value to the caller of trampolineGenerators
// (this is the part you were wondering about)
yield step.value;
}
step = fn.next();
}
}
Nothing about using yield instead of yield* actually prevents the generator you invoke yielding more than once, in fact with yield you have the exact same set of mechanics as normal function calling!
Think about it: If you call a function (the normal way), even though you can only get one return value back, the function is allowed to call as many functions as it wants in order to compute that return value, and it doesn’t need permission – each function has a direct relationship with the VM so that its parent won’t even know what it is doing.
yield* is the weird pattern where theoretically your parent reads all your mail before it gets delivered to the VM
So the point is yes, it’s perfectly possible that a function yields to the VM once and then the VM then yields many values to its caller, some of which were perhaps produced by a nested (yielded-to) function call.
CPython’s core gave up on it because of the ergonomic issues (initial .send(), handling exceptions, etc.) and the lack of readability. We also know that Twisted’s community implemented all three, including coroutines on Twisted, and the Twisted community stuck with callbacks because they are fastest, followed by @inlineCallbacks, and finally coroutines are slowest. The difference isn’t that large, but it was enough to cause folks to stop trying coroutines.
And yes, Twisted has an entire family of event-loop implementations precisely because Python’s VM doesn’t include a scheduler. Somebody’s gotta write it at some level of the stack.
Using iterator / generator machinery for async under the covers is fine, but only providing this interface quickly grows unwieldy and confusing for users, especially when you start layering sync and async e.g. asynchronous iterators were absolutely heinous and very error prone as syntactic support is limited and the desugared operations are quite tricky,
async for foo in bar:
pass
becomes something like
bar_iter = yield from bar.iter()
for foo_future in bar_iter:
try:
foo = yield from foo_future
except StopAsyncIteration:
break
pass
I don’t see what the difference is between that and async/await, I rewrote your code and there don’t seem to be any major changes other than
run()being implicit. People used to use generators like you’re doing with libraries like co before async/await was widely available but now everyone seems to have moved on because async/await does basically the same thing without any dependencies.Very interesting response, thanks!
I wonder what the exact differences are, semantically.
For example, in my version I can detect when two generators try to read an event concurrently, and throw an error if this happens. https://github.com/manuel/delimgen/blob/5005c4be6e458226829aed43b841bafc9ebb6c66/event.mjs#L39
I can also detect when an event happens, and no-one is waiting for one. https://github.com/manuel/delimgen/blob/5005c4be6e458226829aed43b841bafc9ebb6c66/event.mjs#L59
The Redux Saga project is also using generators, and they have repeatedly stated that async/await wouldn’t work for them, they need generators. https://github.com/redux-saga/redux-saga/issues/987 https://github.com/redux-saga/redux-saga/issues/1373
I could be smarter with the events too, it was just faster to put in a oneliner like that :P
The only differences I can think of is Promises (and by extension, await) always call their callbacks as a microtask and not synchronously, the browser handles the scheduling, and also apparently there’s a
.return()method on the result of a generator function, which I did not know about until today. Otherwise generators and async functions are almost equivalent, you can implement each in terms of the otherExactly. With delimited continuations, if you pass a synchronous function then the continuation resolves synchronously without getting put on the event loop.
If you use async/await you don’t have a choice: it’s going on the event loop.
This might not matter for 90% use cases but when it does matter it is critically important.
Out of curiosity, how would you detect multiple functions trying to read an event with promises, and how would you detect there not being any function waiting for an event when one happens? I can’t really wrap my head around it. Is it doable? Or is that a semantic difference between generators and async/await?
Almost the same way you did it with generators!
here’s the full program using these (paste so i don’t make the thread too long :P )
async generators might be a good fit here though! https://paste.owo.codes/21gov45.ts (typescript this time)
Thanks a lot, this really helps me understand the two approaches!
It’s all about control. When you use async await, every yield point must be a promise. If you use generators you can yield whatever you want.
I gave a talk at MichiganTS about delimited continuations if you are interested: https://youtu.be/uRbqLGj_6mI?si=Ajff3t0Mip8e_2CC
One pleasant difference is that when you write a function as a coroutine the sync and the async variant and be the same function. To be sync, just evaluate using a coroutine that blows up if it is yielded a promise.
In iter-tools I don’t use that trick, and the cost I pay is that I have to write every single function twice (I wrote a macro system to do it).
I’m currently writing a parser, and having a macro copy all the parser code into sync and async variants would be absurd, but generator functions let me do it: define behavior once but in a what that propagates promises if they occur
Coming back to this question after some learning, the differences are:
With promises, there is always an implicit asynchrony. The handlers for promise fulfillment and rejection always run via the microtask queue. With generators, this is not the case. This would be observable in the
get_next_event()function of my demo, which doesn’t use any promises (unlike thesleep()function which does, and is therefore of course async).Unlike promises, suspended generators are cancellable via their
return()orthrow()methods. See https://lobste.rs/s/fedhvm/await_event_horizon_javascript for more.(lol, just re-read your comment
easrngand saw that you have already pointed out exactly those two point ;-))I was just talking about this the other day in the BABLR discord! I think the author made exactly one mistake: All the places they’ve written
yield*,yieldwould actually be more correct.To understand why, look at how you would use this pattern to code a recursive algorithm in a way that you don’t need to worry about stack overflows. If you write
yield*you would still get stack overflows, but withyieldeverything “just works” (and potentially quite a lot more performantly thanyield*)!It’s an inversion of control pattern really.
yield*says, “I’m doing the calling here on my call stack,” but to make a call withyieldyou’re giving up control to the VM, saying, “You keep a call stack and do the bookkeeping for me.”Most importantly this leaves you free to have a distinction between describing work that you want to be done and actually doing that work, which is where the real power of the pattern lies.
In my mind this one example illuminates the reasoning behind virtually every choice made in the generator spec, including one that drives me a bit crazy when debugging: the body of the generator function doesn’t begin executing until the first call to
next(). That is the source of laziness that makes it all work. The example also explains the presence and mechanics of bothgenerator.return()andgenerator.throw(), and the mechanics for passing values back through those methods from the VM to the yielder.I don’t think so. I use
yield*because I need to delegate to another generator, and that generator may itself yield multiple times - I need to pass all those yields to the function’s original caller.Here’s the code that does it:
Here is a rather contrived but more complete example usage: https://gist.github.com/conartist6/263606de6d1af1a12f83e4ff582cd96e
The real usage I care about is streaming parsing, which I have been able to implement on top of this pattern
Nothing about using
yieldinstead ofyield*actually prevents the generator you invoke yielding more than once, in fact withyieldyou have the exact same set of mechanics as normal function calling!Think about it: If you call a function (the normal way), even though you can only get one return value back, the function is allowed to call as many functions as it wants in order to compute that return value, and it doesn’t need permission – each function has a direct relationship with the VM so that its parent won’t even know what it is doing.
yield*is the weird pattern where theoretically your parent reads all your mail before it gets delivered to the VMSo the point is yes, it’s perfectly possible that a function yields to the VM once and then the VM then yields many values to its caller, some of which were perhaps produced by a nested (yielded-to) function call.
Python tried this approach before it added async/await syntax. I forget the reason they gave up on it. Probably just scheduling problems.
CPython’s core gave up on it because of the ergonomic issues (initial
.send(), handling exceptions, etc.) and the lack of readability. We also know that Twisted’s community implemented all three, including coroutines on Twisted, and the Twisted community stuck with callbacks because they are fastest, followed by @inlineCallbacks, and finally coroutines are slowest. The difference isn’t that large, but it was enough to cause folks to stop trying coroutines.And yes, Twisted has an entire family of event-loop implementations precisely because Python’s VM doesn’t include a scheduler. Somebody’s gotta write it at some level of the stack.
TIL Stackless Python is still if not exactly alive certainly not exactly dead either!
Using iterator / generator machinery for async under the covers is fine, but only providing this interface quickly grows unwieldy and confusing for users, especially when you start layering sync and async e.g. asynchronous iterators were absolutely heinous and very error prone as syntactic support is limited and the desugared operations are quite tricky,
becomes something like
but I’m not sure I’ve covered all the edge cases.