Deadlocks are easy compared to data races. You just connect a debugger, and you see exactly what is deadlocked, and which stack traces lead to this situation. That is a walk in the park compared to some bytes somewhere sometimes briefly having wrong value, which might not even be reproducible with a debugger attached.
I’ve had my share of deadlocks in Rust, and I’ve been able to solve them all. OTOH in my last C project I’ve had a data race, and the best I could do was to give up and remove all non-trivial OpenMP uses. Was it my bug? Was it compiler bug? I couldn’t know. I couldn’t even reproduce the data race myself.
Three years ago I fought with a deadlock in a distributed system. Never learned of a good way to debug this. My problem magically disappeared due to some seemingly unrelated change.
Deadlocks are easy compared to data races. You just connect a debugger, and you see exactly what is deadlocked, and which stack traces lead to this situation. That is a walk in the park compared to some bytes somewhere sometimes briefly having wrong value, which might not even be reproducible with a debugger attached.
I’ve had my share of deadlocks in Rust, and I’ve been able to solve them all. OTOH in my last C project I’ve had a data race, and the best I could do was to give up and remove all non-trivial OpenMP uses. Was it my bug? Was it compiler bug? I couldn’t know. I couldn’t even reproduce the data race myself.
distributed system? Everything is more complicated with micro-services and other distributed systems. Rust can’t save you there.
While I’d agree with your overall point that data races are more insidious and typically harder to debug, deadlocks can certainly be nondeterministic and arbitrarily difficult to reproduce with a debugger attached too.
The nice thing about deadlocks is that they’re eminently fixable but still sufficiently annoying that after the third time you start asking “why am I getting deadlocks all the time?” and rekajigger your design so that it can’t happen any more. In my experience it’s another case of confused ownership, just expressed differently.
Did you try clang thread sanitizer? Last time I used it was over 5 years ago and it very effectively told me exactly which threads and exactly what structures / bytes had a race.
I use address sanitizer every time I code C and C++ now and it has changed how I code… I would call it “fearless memory management”
That’s the experience you get in Rust, but at compile time. The important improvement is that you don’t need to make the race (or use-after-free) happen for it to be caught. This helps ensure that also rare situations you haven’t thought about testing are correct too.
I wouldn’t. I’ve used valgrind since the early 2000s and asan since I knew it existed (probably around 2010/2011 but I’m not sure). I’ve lost count of how many memory management bugs went undetected until a segfault led me to investigate.
asan is huge for C and C++. But it’s only as good as your test suite and its coverage, and even then…
Compile-time verification that the whole program (and dependencies!) are free from data races is without a doubt a major improvement over C, C++ and Go. It is a way stronger guarantee than most other languages offer with “just be careful” and runtime sanitizers (and Rust also supports thread sanitizer). On top of that Rust has a rich ecosystem of libraries with robust concurrency primitives, so you can make programs multi-threaded without even touching any lock yourself.
But your entire argument hangs only on a literal interpretation of an advertising slogan. Of course a slogan doesn’t contain every possible caveat. And arguing about it is arguing about who fears what, which is unconstructive (there’s always someone afraid of everything, but this doesn’t change what Rust does).
The reasonable interpretation is that we all know data races are a major source of hard-to-fix bugs that are commonly feared, and Rust improves in this area so much that addition of concurrency to programs is no longer a major risk (but Rust does not guarantee bug-free programs, and never claimed that).
I don’t think Go is even close to being in the same category as C or C++ in this regard.
rich ecosystem of libraries
I don’t think any existed at the time I wrote the deadlock (I looked). I ended up using mio directly. Things certainly seem to have change for the better since then.
we all know data races are a major source of hard-to-fix bugs that are commonly feared,
In certain languages, sure. I’ve written more concurrency bugs than I can count in C++, and I’m glad that if I write Rust those wouldn’t be possible. But there are other languages which make such bugs rare or non-existent. Rust is literally the only language I’ve every written a deadlock in and I did it on my first day.
Again, you’re hung up on that one deadlock, as if it invalidated everything else. “Fearless” doesn’t mean “can be totally careless” or “guaranteed bug-free”.
I don’t think you will find any language that can use low-level mio/libuv that can do better here, so let’s not pit real shipping Rust against an imaginary ideal.
And you don’t have to use mio. You can use Rust with data-bound parallelism where you can’t make that bug. You can use Rust with an actor framework. Probably you could even hook it up to the Pony runtime.
BTW, Pony is deadlock-free on a technicality. It doesn’t solve the problem, but merely reshapes it to “two actors will send a message only after receiving a message from each other”.
I always assume that Rust’s “fearless” is more about “your thread will not accidentally use data that it shared with someone else and it was freed in the meantime” than preventing you from writing deadlocks – just like its regular safety features will prevent you from breaking the memory-related stuff, not from actual logic bugs.
I don’t think that’s a fair comparison. I can trust Rust to not let me violate memory safety. I can’t trust it to prevent me from writing deadlocks. In one case, it’s fearless memory management, in the other it’s fearless concurrency except for deadlocks.
Plenty of languages make it easy to not accidentally used data that is shared with someone else.
Really? How many languages provide shared-memory concurrency with the promise that you won’t have data races? I can’t think of any other than rust. Erlang is another case but it doesn’t really share memory.
I think you partially answered your own question. From my personal experience, also Haskell and D. From reading the tutorial, Pony. The fact that Erlang doesn’t share memory is a good thing.
Does D protect you from data races if you use mutex?
Does pony have locks? It seems to me it’s only about channels and sharing immutable references (which you can also do in rust, no need for locks if you share & Foo, just have to ensure it lives long enough)
Erlang doesn’t share memory at all, which is good, except when you try to write something very efficient I guess, like a shared cache. Besides you can get deadlocks with actors too, if two actors are waiting for messages from one another.
Haskell provides shared memory abstractions, but apart from the STM I don’t think it does anything special for the deadlock issue.
In this list, I still don’t see anything that offers what rust does (race-free shared memory concurrency, statically guaranteed, where multiple threads can still modify the shared structures concurrently). Channels don’t protect from deadlocks at a higher level, and even then, rust has them too if they fit your needs.
Does D protect you from data races if you use mutex?
That’s not the usual way of writing concurrent code, which is to send messages between threads. You can only send immutable or shared data. The former guarantees a lack of data races, the latter can’t really be used without locks. That’s an oversimplified explanation though.
Does pony have locks?
No.
but apart from the STM
I think STM is their secret weapon when it comes to concurrency.
That’s not the usual way of writing concurrent code, which is to send messages between threads.
But in rust you can do that too, and the move semantics even allows you to send mutable things since you give up their ownership. And you can still get deadlocks. I fail to see what D does that rust doesn’t.
Detecting deadlocks statically is equivalent to solving the halting (heh) problem, so it’s never gonna be 100%.
That being said, it is possible to detect deadlocks once they’ve occurred or are about to…how does Rust handle a deadlock situation? Does it do runtime deadlock detection and panic, or just lock up?
(Forgive me if I sound pedantic. Not my goal. Low on coffee and it’s a pain to buy more right now, what with the apocalypse and all…)
Rust as a language doesn’t even know that locks exist. They’re purely a library feature. Data-race safety of the locked data is expressed via Send/Sync traits, but there isn’t more to it. The stdlib just wraps pthread mutexes, but you’re free to use whatever implementation you want. parking_lot from WebKit is a popular alternative.
You can constraint the programmer such that deadlocks are impossible by construction. X10 started like this. Later they introduced old-fashioned locks via library so the guarantee can be broken (like unsafe in Rust).
X10 has async { } to spawn of an activity (think thread). There is “finish { }” which waits for all (recursively) spawned activities inside the block at the end of the block. You also get some barrier synchronization with a smart API design. If that is all you use, then you cannot have deadlocks.
The Pony programming language guarantees absence of deadlocks.
Right, I was referring to classic free-for-all threads-with-locks models; there are ways to prevent deadlocks by construction or restriction.
Not deterministically.
I’m rapidly approaching the edge of my knowledge, so absolutely correct me if I’m wrong, but at least for traditional threads-with-locks, you can generate resource-acquisition graphs to determine if a deadlock has occurred, right?
You can’t determine a priori if a deadlock will occur by static analysis of an arbitrary threads-and-locks, at least not in the general case, though, you’re right; that’s what I was alluding to above.
Most schemes to tame concurrency still seem to be able to deadlock and livelock. My favorite, though, is still SCOOP. Heck, I”ll throw in a submission on it since SCOOP appears portable across languages.
Seemed like a great language for maintainable, business applications. Took a while for me to understand the non-technical reasons languages like it don’t get much adoption.
Bertrand Meyer did a great job alienating the entire OOP community. Like to the point of even refusing to use common, widespread terminology any of his papers.
I know it was still cutting edge 25 years ago, but has it kept that up since? I remember reviewing it and being pretty disappointed. Even its contract system doesn’t feel that great now that we have Racket and Dafny.
You know me, I’m a technical Luddite. Anything newer than 25 years is inexcusable newfangled decadence.
But, to answer your question, no, I don’t think it’s kept up. Last time I looked at the “official” Eiffel implementation, they were mainly focusing on interop and compiler speedups, though that was several years ago.
Until you write a deadlock, which I managed to do within 30min of writing Rust code.
Deadlocks are easy compared to data races. You just connect a debugger, and you see exactly what is deadlocked, and which stack traces lead to this situation. That is a walk in the park compared to some bytes somewhere sometimes briefly having wrong value, which might not even be reproducible with a debugger attached.
I’ve had my share of deadlocks in Rust, and I’ve been able to solve them all. OTOH in my last C project I’ve had a data race, and the best I could do was to give up and remove all non-trivial OpenMP uses. Was it my bug? Was it compiler bug? I couldn’t know. I couldn’t even reproduce the data race myself.
Three years ago I fought with a deadlock in a distributed system. Never learned of a good way to debug this. My problem magically disappeared due to some seemingly unrelated change.
distributed system? Everything is more complicated with micro-services and other distributed systems. Rust can’t save you there.
While I’d agree with your overall point that data races are more insidious and typically harder to debug, deadlocks can certainly be nondeterministic and arbitrarily difficult to reproduce with a debugger attached too.
The nice thing about deadlocks is that they’re eminently fixable but still sufficiently annoying that after the third time you start asking “why am I getting deadlocks all the time?” and rekajigger your design so that it can’t happen any more. In my experience it’s another case of confused ownership, just expressed differently.
Did you try clang thread sanitizer? Last time I used it was over 5 years ago and it very effectively told me exactly which threads and exactly what structures / bytes had a race.
I use address sanitizer every time I code C and C++ now and it has changed how I code… I would call it “fearless memory management”
That’s the experience you get in Rust, but at compile time. The important improvement is that you don’t need to make the race (or use-after-free) happen for it to be caught. This helps ensure that also rare situations you haven’t thought about testing are correct too.
I wouldn’t. I’ve used valgrind since the early 2000s and asan since I knew it existed (probably around 2010/2011 but I’m not sure). I’ve lost count of how many memory management bugs went undetected until a segfault led me to investigate.
asan is huge for C and C++. But it’s only as good as your test suite and its coverage, and even then…
Even assuming I agree, I don’t see how that changes my point that if I can write deadlocks it’s not fearless.
That’s not been my experience. And that’s if the bug is easily reproducible, which tends to not be the case with concurrency.
Why compare to C? C is terrible at writing concurrent programs. If the bar for Rust is “better than C”…
Compile-time verification that the whole program (and dependencies!) are free from data races is without a doubt a major improvement over C, C++ and Go. It is a way stronger guarantee than most other languages offer with “just be careful” and runtime sanitizers (and Rust also supports thread sanitizer). On top of that Rust has a rich ecosystem of libraries with robust concurrency primitives, so you can make programs multi-threaded without even touching any lock yourself.
But your entire argument hangs only on a literal interpretation of an advertising slogan. Of course a slogan doesn’t contain every possible caveat. And arguing about it is arguing about who fears what, which is unconstructive (there’s always someone afraid of everything, but this doesn’t change what Rust does).
The reasonable interpretation is that we all know data races are a major source of hard-to-fix bugs that are commonly feared, and Rust improves in this area so much that addition of concurrency to programs is no longer a major risk (but Rust does not guarantee bug-free programs, and never claimed that).
I don’t think Go is even close to being in the same category as C or C++ in this regard.
I don’t think any existed at the time I wrote the deadlock (I looked). I ended up using mio directly. Things certainly seem to have change for the better since then.
In certain languages, sure. I’ve written more concurrency bugs than I can count in C++, and I’m glad that if I write Rust those wouldn’t be possible. But there are other languages which make such bugs rare or non-existent. Rust is literally the only language I’ve every written a deadlock in and I did it on my first day.
Again, you’re hung up on that one deadlock, as if it invalidated everything else. “Fearless” doesn’t mean “can be totally careless” or “guaranteed bug-free”.
I don’t think you will find any language that can use low-level
mio/libuvthat can do better here, so let’s not pit real shipping Rust against an imaginary ideal.And you don’t have to use
mio. You can use Rust with data-bound parallelism where you can’t make that bug. You can use Rust with an actor framework. Probably you could even hook it up to the Pony runtime.BTW, Pony is deadlock-free on a technicality. It doesn’t solve the problem, but merely reshapes it to “two actors will send a message only after receiving a message from each other”.
I always assume that Rust’s “fearless” is more about “your thread will not accidentally use data that it shared with someone else and it was freed in the meantime” than preventing you from writing deadlocks – just like its regular safety features will prevent you from breaking the memory-related stuff, not from actual logic bugs.
I don’t think that’s a fair comparison. I can trust Rust to not let me violate memory safety. I can’t trust it to prevent me from writing deadlocks. In one case, it’s fearless memory management, in the other it’s fearless concurrency except for deadlocks.
Plenty of languages make it easy to not accidentally used data that is shared with someone else.
Really? How many languages provide shared-memory concurrency with the promise that you won’t have data races? I can’t think of any other than rust. Erlang is another case but it doesn’t really share memory.
I think you partially answered your own question. From my personal experience, also Haskell and D. From reading the tutorial, Pony. The fact that Erlang doesn’t share memory is a good thing.
& Foo, just have to ensure it lives long enough)In this list, I still don’t see anything that offers what rust does (race-free shared memory concurrency, statically guaranteed, where multiple threads can still modify the shared structures concurrently). Channels don’t protect from deadlocks at a higher level, and even then, rust has them too if they fit your needs.
That’s not the usual way of writing concurrent code, which is to send messages between threads. You can only send immutable or shared data. The former guarantees a lack of data races, the latter can’t really be used without locks. That’s an oversimplified explanation though.
No.
I think STM is their secret weapon when it comes to concurrency.
But in rust you can do that too, and the move semantics even allows you to send mutable things since you give up their ownership. And you can still get deadlocks. I fail to see what D does that rust doesn’t.
Detecting deadlocks statically is equivalent to solving the halting (heh) problem, so it’s never gonna be 100%.
That being said, it is possible to detect deadlocks once they’ve occurred or are about to…how does Rust handle a deadlock situation? Does it do runtime deadlock detection and panic, or just lock up?
(Forgive me if I sound pedantic. Not my goal. Low on coffee and it’s a pain to buy more right now, what with the apocalypse and all…)
Rust as a language doesn’t even know that locks exist. They’re purely a library feature. Data-race safety of the locked data is expressed via
Send/Synctraits, but there isn’t more to it. The stdlib just wraps pthread mutexes, but you’re free to use whatever implementation you want.parking_lotfrom WebKit is a popular alternative.You can constraint the programmer such that deadlocks are impossible by construction. X10 started like this. Later they introduced old-fashioned locks via library so the guarantee can be broken (like unsafe in Rust).
X10 has
async { }to spawn of an activity (think thread). There is “finish { }” which waits for all (recursively) spawned activities inside the block at the end of the block. You also get some barrier synchronization with a smart API design. If that is all you use, then you cannot have deadlocks.The Pony programming language guarantees absence of deadlocks.
Not deterministically.
Right, I was referring to classic free-for-all threads-with-locks models; there are ways to prevent deadlocks by construction or restriction.
I’m rapidly approaching the edge of my knowledge, so absolutely correct me if I’m wrong, but at least for traditional threads-with-locks, you can generate resource-acquisition graphs to determine if a deadlock has occurred, right?
You can’t determine a priori if a deadlock will occur by static analysis of an arbitrary threads-and-locks, at least not in the general case, though, you’re right; that’s what I was alluding to above.
Most schemes to tame concurrency still seem to be able to deadlock and livelock. My favorite, though, is still SCOOP. Heck, I”ll throw in a submission on it since SCOOP appears portable across languages.
Eiffel is the best of all the OOP languages. It’s got so much going for it, it’s sad that it isn’t more popular.
Seemed like a great language for maintainable, business applications. Took a while for me to understand the non-technical reasons languages like it don’t get much adoption.
For Eiffel the two big ones were
Carl Sassenrath, is that you?!?
stares wistfully at REBOL
I tried to learn it, because I’m a huge fan of contracts, only to find out you couldn’t get it up and running without purchase, so I binned the plan.
There were a few open source compilers over the years but, as far as I know, none are still under much active development.
D has contracts.
I know it was still cutting edge 25 years ago, but has it kept that up since? I remember reviewing it and being pretty disappointed. Even its contract system doesn’t feel that great now that we have Racket and Dafny.
You know me, I’m a technical Luddite. Anything newer than 25 years is inexcusable newfangled decadence.
But, to answer your question, no, I don’t think it’s kept up. Last time I looked at the “official” Eiffel implementation, they were mainly focusing on interop and compiler speedups, though that was several years ago.
Pony doesn’t allow deadlocks to happen.
Well, it uses the actor model instead of multithreading with shared state that I recall. Great that it stops deadlocks, though.