I’m just a web nerd, so the most exciting thing I saw there was a benchmark getting 2 million requests/s from a HTTP server. I’m super tempted by rust but to be totally honest I find it really difficult to write. Also zero cost abstractions are awesome, but I’m willing to pay for my coroutine/blocking semantics abstraction. After a year writing twisted I get flashbacks every time I see .and_then().
But if select has 2 futures in it, and one of them is blocked on a syscall or other long-running process, what happens? Is the other call interrupted? How is it freed if it’s holding a lock or something like that?
so if you have a future representing something other thread that’s blocked in a syscall then there’s of course no way to cancel that. the idea is that futures representing pending computation can be canceled via drop, and then anything which represents parllel computation can communicate that. but you’d need a protocol for transferring the drop signal to the production side. If you’re doing something that’s going to block, it should go into a thread pool and no, there’s no way to cancel computations that are actively running on a thread pool. But that’s not how any I/O happens by default, for example. e.g. tcp reads/writes are canceled by just dropping the associated future
To be clear: if the other Rust future in the select isn’t done, that means the whole rust frame will block when it goes out of scope, correct? I’m worried that you’re going to get these deadlock-looking situations that that are impossible to debug or defend against. The suggestion of select() with timeout futures only works if nothing will ever block. And it’s not just syscalls, you could be hosed by a mutex that’s never released, right?
Why can’t drop be used as the cancelation signal? Or maybe add an explicit cancel to the Future trait (which can be used with or in addition to drop)? It seems like good hygiene is to make sure that everyone who’s implementing a future has ensured that they are immediately cancelable in all cases and the caller will never have their stack blocked by Future clean-up.
Drop is the cancellation signal for the future, but if the call is blocking, you may not actually have a way to cancel it. That’s why you put it in the pool, which your future would then be checking in on in a non-blocking manner. You could then cancel that future, which would tell the pool “clean up that worker” and return, not blocking the rest of your stuff.
What I interpret @steveklabnik as saying is, you can do the cancellation if you have a thread pool; but not if you don’t; and if you do you end up moving these calls into the pool.
This is really nice - I’m looking forward to the maturation of Tokio (https://github.com/tokio-rs/tokio), a Finagle-like network programming library built on top of this future abstraction.
How is the specification of behavior related to the implementation?
in Rust, different language features have very different performance profiles, and that also comes with restrictions on what you can do. So this library, for example, was designed to be as low-overhead as possible, and therefore, has certain behaviors that allow it to achieve that basically-zero performance.
In a language that’s not as performance-sensitive, you do a lot of things that can add overhead. So for example, imagine that the specification of the behavior here required runtime reflection. Doing that is extremely difficult in Rust due to performance reasons.
Can you pin-point a rule which would require runtime reflection to implement?
I haven’t read this particular spec yet, I was spec-ulating (near-pun intended) based on my previous experience: no GC means that my APIs end up very differently in Rust than they do, say, Ruby.
Because behavior affects how APIs are developed. For example, a spec that says “this can return any value as long as it implements a foo() method” is simple in dynamically typed languages, but more complicated in statically typed ones. And especially if the specification isn’t performance-oriented, it might require behavior that’s significantly less performant than other behavior.
I’m just a web nerd, so the most exciting thing I saw there was a benchmark getting 2 million requests/s from a HTTP server. I’m super tempted by rust but to be totally honest I find it really difficult to write. Also zero cost abstractions are awesome, but I’m willing to pay for my coroutine/blocking semantics abstraction. After a year writing twisted I get flashbacks every time I see .and_then().
How is memory freed in the select() case where one Future returns and the other is left hanging around?
In Rust, memory is always freed once its owner is dropped. So select returns the other future, and whenever you let it go out of scope, it’s free’d.
But if select has 2 futures in it, and one of them is blocked on a syscall or other long-running process, what happens? Is the other call interrupted? How is it freed if it’s holding a lock or something like that?
So. I asked the authors about it:
Does that make sense? (I think it does to me…)
Thanks for looking into it.
To be clear: if the other Rust future in the select isn’t done, that means the whole rust frame will block when it goes out of scope, correct? I’m worried that you’re going to get these deadlock-looking situations that that are impossible to debug or defend against. The suggestion of select() with timeout futures only works if nothing will ever block. And it’s not just syscalls, you could be hosed by a mutex that’s never released, right?
Why can’t drop be used as the cancelation signal? Or maybe add an explicit cancel to the Future trait (which can be used with or in addition to drop)? It seems like good hygiene is to make sure that everyone who’s implementing a future has ensured that they are immediately cancelable in all cases and the caller will never have their stack blocked by Future clean-up.
Drop is the cancellation signal for the future, but if the call is blocking, you may not actually have a way to cancel it. That’s why you put it in the pool, which your future would then be checking in on in a non-blocking manner. You could then cancel that future, which would tell the pool “clean up that worker” and return, not blocking the rest of your stuff.
In my understanding.
If the call is really blocking – like a syscall – how can you cancel it?
You can send a signal to the process blocked on the call and the call will end with EINTR.
Well, to be clear: I assumed one constraint was, all code is running in the same process.
See pthread_kill
What I interpret @steveklabnik as saying is, you can do the cancellation if you have a thread pool; but not if you don’t; and if you do you end up moving these calls into the pool.
This is really nice - I’m looking forward to the maturation of Tokio (https://github.com/tokio-rs/tokio), a Finagle-like network programming library built on top of this future abstraction.
If you’re interested in learning what programming using this kind of abstraction is like, Marius Eriksen(author of Finagle)’s writings about it are really interesting - check out RPC Redux, Your Server as a Function, and Systems Programming at Twitter.
Hmm. I must admit to a small stomach ache at the thought of a lambda with side effect being executed at some unknown future time.
Do the streams implement the reactive streams standard?
Is there a standard? I’m only aware of the collection of libraries that fall under that banner, not a spec.
It’s a standard in the same way JSON API is a standard ;)
http://www.reactive-streams.org
Interesting! I’ll have to dig into that page. Thanks!
(Oh, and I guess this is implied, but I’m almost certain that the answer is no. This makes really heavy use of rust-specific features.)
How is the specification of behavior related to the implementation?
in Rust, different language features have very different performance profiles, and that also comes with restrictions on what you can do. So this library, for example, was designed to be as low-overhead as possible, and therefore, has certain behaviors that allow it to achieve that basically-zero performance.
In a language that’s not as performance-sensitive, you do a lot of things that can add overhead. So for example, imagine that the specification of the behavior here required runtime reflection. Doing that is extremely difficult in Rust due to performance reasons.
That different language features might have different performance is true, but it’s completely unrelated to the spec.
There are 43 rules. Can you pin-point a rule which would require runtime reflection to implement?
I haven’t read this particular spec yet, I was spec-ulating (near-pun intended) based on my previous experience: no GC means that my APIs end up very differently in Rust than they do, say, Ruby.
Sure, but the API can look anyway it wants. Why would the spec care how the API looks like? Provide the most idiomatic, rusty API you can think of.
Because behavior affects how APIs are developed. For example, a spec that says “this can return any value as long as it implements a foo() method” is simple in dynamically typed languages, but more complicated in statically typed ones. And especially if the specification isn’t performance-oriented, it might require behavior that’s significantly less performant than other behavior.
Can you be specific which of the rules would cause issues?
Then why keep speculating?