I quite like Perl 6. This video on Perl 6 concurrency gives a good overview of the concurrency features it offers. The futures/promises has a very Alice ML or Mozart/Oz dataflow feel.
Don’t think I’d include Rust in that. M:N threading was a feature of the language early-on, but it was removed because it complicated the runtime. What you link is a third-party library that tries to provide a library implementation of green threads, but 1) it’s not part of the standard language, 2) it’s alpha-quality code with no official release, and 3) according to its own continuous-integration tests, it doesn’t even compile currently.
Rust has semantic support for those patterns, though, through Send and Sync.
The M:N-threading runtime was not removed because it complicated the runtime, but because of the decision to fully remove any active runtime. (There’s a remaining minimal runtime that does stack unwinding and such)
It’s certainly lower level, but Rust knows what that pattern is and assists you there. I’m not sure if I want to recommend it for learning, but those systems can and have been written well in Rust. It’s certainly no “here’s a library, just use it” state of things, I would guess that’s still a year or so out.
I was not specific enough so you are technically correct. What I meant to ask is if rust has a way to preempt any of the running threads - something like go. Afaics futures-cpupool is cooperative multithreading, not preemptive.
Yes, that is true that Go scheduler is not completely preemptive (although I don’t think it’s useful to call it cooperative - I don’t recall seeing any code which had to manually mark preemption points).
If you want example of completely preemptive scheduler in M:N system the best example I know of is Erlang. You can read about it here. Some additional languages with this property are also mentioned there.
huh! I stand corrected (about the content of the article), I missed that paragraph. My apologies.
That said, preemptive multitasking involves preemption; that doesn’t happen in Erlang. Schedulers let processes run, possibly infinitely, until they (a) have used up a reduction metric and (b) encounter a scheduling decision point (e.g. an erlang function call). There is no way to guarantee preemption or get particularly near hard-real-time scheduling. So I would say jlouis is wrong in that particular assertion although the rest of the article is correct.
The problem here is that Rust is operating at a level lower: as Rust has no runtime, it also has no concept of “reductions”, as Erlang has. Erlang does not manually mark these preemption points, it just defines a language primitive as such (function calls). Rust being a low-level language, we just can’t do that. Suspension itself at that level is a tricky problem.
So, without further work, no, Rust cannot have these semantics in-language. You could build them in Rust, though. Futures are kind of a way towards that: small, composable parts with natural points where preemption can happen.
I am currently not aware of a system that builds such a large system to a final state. For preemption, we currently rely on the system scheduler. So, you could use futures-cpupool to put futures on it that emulate the Erlang semantics. (the last part probably being a big piece of work)
To circle back to my initial point: just as in many aspects, Rust has semantics support for many of these things, without forcing a solution.
I never doubted that rust can be used to build things you mentioned. I was just surprised by @superpat suggestion that rust can provide preemptive M:N threading via crates. As I suspected and you have confirmed this can’t be done without runtime.
Go measure the cost of real threads on a modern linux kernel, compare to Golang, rethink some of the beliefs you have been parroting from Golang dogma. It’s an antipattern to spin up 10k goroutines anyway.
Erlang has a much better situation, but still, once you’re getting into the realm of high-throughput you shouldn’t be erlanging.
Real threads are cheap enough for 99.9% of workloads, and incur a lot less CPU overhead for steady state execution under high concurrency. Don’t drink the kool-aid.
this is something i see people often ranting about in absolute contexts which drives me crazy, although i have observed problems related to high frequency thread creation/churn can be an issue for truly cpu bound workloads. i would say most people’s workloads are largely idle or IO bound, having thousands of largely unused threads is nbd.
i wish i had a nickel for every compute instance w/ a daily max cpu usage of less than 60%.
This may seem like a trivial gripe, but it’s been a real issue for me when writing Perl 6: I can’t find an editor whose syntax highlighting can keep up with Perl6. I’m a Vim guy, and Vim’s Perl6 syntax highlighting is s l o w, causing the editor to lag, making using the language a chore in that editor.
I’ve tried:
VScode in Vim mode, but it doesn’t (at least, when I used it) have support for Perl6 highlighting and Perl5 highlighting was just SLIGHTLY off.
Spacemacs; I wasn’t able to find a Perl6 layer for it.
So um, any suggestions? Perl6 is crazy cool, but I just can’t find a means to write it without getting frustrated at laggy/incorrect syntax highlighting.
Howdy, also a vim user. I moved to neovim a while back and it allows aync syntax highlighting through plugins like chromatica. Since leveraging this along with ale things have become significantly snappier, especially on larger files.
Concurrency is hard and if you want M:N thread multiplexing (i.e. WEB SCALE CODE, where application threads aren’t pinned to pthreads) your options today are precisely Erlang, Go, .NET, and Perl 6.
Funny, I’ve been doing it in Scala for years (and know people who’ve been doing it in TCL for decades). More to the point, the article seems to claim there’s something bad/difficult about doing this in Python/Ruby, but Python has any number of decent ways of doing it.
Huh? CPython and MRI Ruby are utterly useless at this. Sure you can spawn pthreads and multiplex coroutines onto them but the only thing you can do with multiple pthreads is wait on I/O — if you want to perform work on them, say hi to your friend the GIL, who says nope :D
I quite like Perl 6. This video on Perl 6 concurrency gives a good overview of the concurrency features it offers. The futures/promises has a very Alice ML or Mozart/Oz dataflow feel.
The video is helpful. Never looked seriously into Perl, but this topic motivates me to study the language. Thanks!
And JVM, GHC Haskell, Rust, Pony…
Don’t think I’d include Rust in that. M:N threading was a feature of the language early-on, but it was removed because it complicated the runtime. What you link is a third-party library that tries to provide a library implementation of green threads, but 1) it’s not part of the standard language, 2) it’s alpha-quality code with no official release, and 3) according to its own continuous-integration tests, it doesn’t even compile currently.
Rust has semantic support for those patterns, though, through Send and Sync.
The M:N-threading runtime was not removed because it complicated the runtime, but because of the decision to fully remove any active runtime. (There’s a remaining minimal runtime that does stack unwinding and such)
It’s certainly lower level, but Rust knows what that pattern is and assists you there. I’m not sure if I want to recommend it for learning, but those systems can and have been written well in Rust. It’s certainly no “here’s a library, just use it” state of things, I would guess that’s still a year or so out.
In rusts defense, the language tends to support external crates for most extended features.
Do you mean that there is a crate that provides M:N threading?
futures-cpupool and others, yes.
I was not specific enough so you are technically correct. What I meant to ask is if rust has a way to preempt any of the running threads - something like go. Afaics futures-cpupool is cooperative multithreading, not preemptive.
Can you give me an example? AFAIK, the Go scheduler is only lightly preemptive.
(The reference I just dug out, I’m not really into go in detail)
https://github.com/golang/go/issues/11462
Yes, that is true that Go scheduler is not completely preemptive (although I don’t think it’s useful to call it cooperative - I don’t recall seeing any code which had to manually mark preemption points).
If you want example of completely preemptive scheduler in M:N system the best example I know of is Erlang. You can read about it here. Some additional languages with this property are also mentioned there.
Erlang schedulers are not preemptive, as the linked article explains.
Does it really say that? To quote the article:
To me it sounds like the article says that Erlang is one of the few runtimes with preemptive scheduling.
huh! I stand corrected (about the content of the article), I missed that paragraph. My apologies.
That said, preemptive multitasking involves preemption; that doesn’t happen in Erlang. Schedulers let processes run, possibly infinitely, until they (a) have used up a reduction metric and (b) encounter a scheduling decision point (e.g. an erlang function call). There is no way to guarantee preemption or get particularly near hard-real-time scheduling. So I would say jlouis is wrong in that particular assertion although the rest of the article is correct.
The problem here is that Rust is operating at a level lower: as Rust has no runtime, it also has no concept of “reductions”, as Erlang has. Erlang does not manually mark these preemption points, it just defines a language primitive as such (function calls). Rust being a low-level language, we just can’t do that. Suspension itself at that level is a tricky problem.
So, without further work, no, Rust cannot have these semantics in-language. You could build them in Rust, though. Futures are kind of a way towards that: small, composable parts with natural points where preemption can happen.
I am currently not aware of a system that builds such a large system to a final state. For preemption, we currently rely on the system scheduler. So, you could use futures-cpupool to put futures on it that emulate the Erlang semantics. (the last part probably being a big piece of work)
To circle back to my initial point: just as in many aspects, Rust has semantics support for many of these things, without forcing a solution.
I never doubted that rust can be used to build things you mentioned. I was just surprised by @superpat suggestion that rust can provide preemptive M:N threading via crates. As I suspected and you have confirmed this can’t be done without runtime.
Well, the crate would provide the runtime :).
[Comment removed by author]
Sure, but low-level languages are built to support multiple eco-systems on top of them.
Go measure the cost of real threads on a modern linux kernel, compare to Golang, rethink some of the beliefs you have been parroting from Golang dogma. It’s an antipattern to spin up 10k goroutines anyway.
Erlang has a much better situation, but still, once you’re getting into the realm of high-throughput you shouldn’t be erlanging.
Real threads are cheap enough for 99.9% of workloads, and incur a lot less CPU overhead for steady state execution under high concurrency. Don’t drink the kool-aid.
this is something i see people often ranting about in absolute contexts which drives me crazy, although i have observed problems related to high frequency thread creation/churn can be an issue for truly cpu bound workloads. i would say most people’s workloads are largely idle or IO bound, having thousands of largely unused threads is nbd.
i wish i had a nickel for every compute instance w/ a daily max cpu usage of less than 60%.
This may seem like a trivial gripe, but it’s been a real issue for me when writing Perl 6: I can’t find an editor whose syntax highlighting can keep up with Perl6. I’m a Vim guy, and Vim’s Perl6 syntax highlighting is s l o w, causing the editor to lag, making using the language a chore in that editor.
I’ve tried:
So um, any suggestions? Perl6 is crazy cool, but I just can’t find a means to write it without getting frustrated at laggy/incorrect syntax highlighting.
Howdy, also a vim user. I moved to neovim a while back and it allows aync syntax highlighting through plugins like chromatica. Since leveraging this along with ale things have become significantly snappier, especially on larger files.
Currently Atom seems to have the best Perl 6 plugins.
Try Elixir! It’s all the fun of Erlang but in a much better package.
The author has definitely tried it. First link in the “If you liked this you might also enjoy” section after the text.
Funny, I’ve been doing it in Scala for years (and know people who’ve been doing it in TCL for decades). More to the point, the article seems to claim there’s something bad/difficult about doing this in Python/Ruby, but Python has any number of decent ways of doing it.
Huh? CPython and MRI Ruby are utterly useless at this. Sure you can spawn pthreads and multiplex coroutines onto them but the only thing you can do with multiple pthreads is wait on I/O — if you want to perform work on them, say hi to your friend the GIL, who says nope :D
Sure, but WEB SCALE is all about waiting on I/O. For performing work 1:1 threads are fine, if anything they tend to perform better than M:N.
I got real curious about this sentence so I looked up the C function that does case-changing in MoarVM. Can’t quite figure out where the cleverness is yet, but if you do drop me a comment reply.