It’s a new language, you have to go slow. I don’t get why people think they should automatically be productive in a new language. Yes, it sucks to be slapped upside the head by the ownership model. But if you want the safety advantage (which, presumably, you do, given that you’re using Rust), that means a compiler that you have to please and think like in order to make it to executable code. If you don’t, use C.
I guess I find it immensely confusing that people clamor for tools to save them from themselves and then summarily reject them when they aren’t lenient enough. It’s clear that, collectively, we believe less in good practices than good tools. After all, what is React but a way to enable lots of [junior] coders to work on parts of a page without stepping on each other?
I don’t get why people think they should automatically be productive in a new language.
I agree. I am by no means a Rust expert, but I have done some small projects in Rust. The ownership problem in the last example seems trivial to solve. Either you do an early return if you have cached the entry, something like:
if let Some(cached) = self.cache.get(host) {
return Ok(cached);
}
// Now the immutable borrow is gone.
Or you use the entry function of HashMap. If you find that the entry is occupied, you use get on OccupiedEntry. Otherwise, you have your mutable handle via VacantEntry and you can use it to insert the results into the cache.
I understand the author’s frustration. I have also been through the two week ‘how do I please the borrow checker’-hell. But once you understand the rules and the usual ways to address ownership issues, it’s relatively smooth sailing. The guarantees that the Rust ownership provides (like no simultaneous mutable/immutable borrows) allow you to do really cool things, like the peek_mut method of BinaryHeap:
Basically it restores the heap property through the ‘Drop’ (destructor) trait. It is safe to do this, because it’s a mutable borrow, blocking immutable borrows and thus inconsistent views of the data when the top of the heap is changed and the sift-down operation has not been applied yet. The other day, I used the same trick in some code where I have a sparse vector data structure, which is a wrapper around BTreeMap that automatically removes vector components when their values are changed to 0.
I get your point, but you are blatantly misquoting the author here. They spent multiple evenings learning rust for their true toy, not an hour. The hour was spent getting 30 lines of code for a different toy example into something that looked like reasonable rust, which didn’t then compile.
Please don’t quote people out of context, especially when it is so easy to catch.
I’m thinking about Haskell and wondering why this is, because the difference is pretty stark. I didn’t expect to ever actually learn Haskell, so when it finally seemed to be happening after months of messing with it, I was pretty overjoyed. Nobody had told me it would be easy; in fact, I had assumed it would be really difficult and I might not be able to get there. I’m not paying close enough attention, are they marketing Rust as something you can learn pretty easily? That’s not the sense one gets from reading blogs.
A lot of people move between Java and JavaScript or C#, and they’re just not prepared for something totally different. This guy is pretty competent though, I don’t think that’s what’s happening here.
Maybe systems people are just really impatient for progress?
Ultimately, all this negative press will work to its benefit. Programmers like to be elitist about knowing hard things (like Haskell, in years past) so if Rust develops a reputation of impenetrability, that just means in two or three years there will be a lot of young Rust programmers.
Ultimately, all this negative press will work to its benefit.
Unfortunately, this was exactly the sort of negative press (right down to an ESR hatchet job quoted endlessly) that killed Ada in industry.
For whatever reason “I spent a whole hour writing code that in the end didn’t compile” is considered a damning indictment of a language, whereas “I spent a whole hour writing code that in the end compiled with many subtle correctness issues unaddressed that later bit me in the ass” is considered “wow so productive!”.
whereas “I spent a whole hour writing code that in the end compiled with many subtle correctness issues unaddressed that later bit me in the ass” is considered “wow so productive!”.
Incidentally, the C++ code that the OP wrote actually contains undefined behavior for violating the strict aliasing rule. (There’s a cast and dereference from char* to trie_header_info*, where trie_header_info has a stricter alignment than char.)
Actually, the Rust code from the OP (linked in comments) contains the same error, but it is annotated with unsafe. :-)
ESR definitely ran into the issue described by @brinker above, but apart from that I thought he was reasonably fair about what issues he had. I did not read as particularly spiteful to me, just that he found Go more productive and Rust frustrating and immature. That makes sense; Rust is younger and less mature and Go prioritizes productivity above many other things.
I’m not sure what “killed” Ada in industry but lots of good languages failed during that era for reasons that had nothing to do with technical superiority. Personally I find Ada kind of dull to read and the situation with GNATS has always confused me. I thought Eiffel looked more like “Ada done right” when I was in college; now I think Ada probably had a better emphasis on types than Eiffel did that probably lent itself to reliability more directly than Eiffel’s design-by-contract stuff. But this is way out on the fringe of anything I know about really; I would greatly enjoy hearing more about Ada now.
ESR’s post generated substantial frustration in the Rust community because a number of the factual claims or statements he makes about Rust are wrong. Members of the Rust community offered corrections for these claims, but ESR reiterated them without edit in his follow up post. Reasonable criticism based on preference or disagreement with design decisions is one thing, claiming untrue things about a language as part of an explanation of why not to use it is another thing entirely.
Well, if it makes you feel better, the big thing I got from it was “NTPsec relies on a certain system call that isn’t a core part of Rust” which isn’t going to weigh on my mind particularly hard. I agree with you that criticism should be factual.
ESR definitely ran into the issue described by @brinker above, but apart from that I thought he was reasonably fair about what issues he had.
That wasn’t really my impression: he was factually incorrect about a number of issues in both his initial rant and his follow up (string manipulation in particular), and he mostly seemed to be criticizing Rust for not being what he wanted rather than on its own terms (ok, Rust doesn’t have a first-class syntax for select/kqueue/epoll/IO completion ports – but that’s obviously by design, because Rust is intended to be a runtimeless C replacement, not something with a heavy-weight runtime that papers over the large semantic differences between those calls. If you went into Rust just wanting Go, then just use Go).
I’m not sure what “killed” Ada in industry but lots of good languages failed during that era for reasons that had nothing to do with technical superiority.
If I had a nickel for every time someone quoted ESR’s “hacker dictionary” entry on Ada being lol so huge and designed by comittee, I’d have…well, a couple of dollars, anyways. You’ll still hear them today, and Ada is still on the small side of languages these days, and still isn’t designed by committee, while a lot of popular languages are.
It had some attention on it in the late ‘90s, but every time a discussion got going around it, you’d hear the same things: people parroting ESR with no direct experience of their own, people (generally students, which is understandable, but also working professionals who should have known better) complaining that the compiler rejected their code (and that’s obviously bad because a permissive compiler is more important than catching bugs), etc.
Just a general negative tone that kept people from trying it out, which kept candidates for jobs at a minimum, which kept employers from ever giving it serious thought.
I encountered Ada in the early 90s (in college). At that time, Ada was considered a large, bondage and discipline language where one fought the compiler all the way (hmm … much like Rust today). At the time, C has just been standardized and C++ was still years away from its first real standard so Ada felt large and cumbersome. At the time, I liked the idea of Ada, just not the implementation (I found it a bit verbose for my liking). ESR was writing about Ada around this time, and he was was parroting the zeitgeist of the time.
Yeah, it’s funny the way initial perceptions sink something long after they cease to be true.
I was having these conversations in 1998-1999, after the Ada ‘95 and C++ '98 standardizations, which were really easy to point at and say that “no, actually, Ada’s a lot smaller than C++ at the moment”, but it didn’t really matter, because an off-the-cuff riff on Ada as it had stood in maybe the late '80s was the dominant impression of the language, and no amount of facts or pointing out that paying a bit of cost up front in satisfying the compiler was easily better than paying 10x that cost in chasing bugs and CVEs were capable of changing that.
This is more or less what I’m concerned is the unclimbable hill Rust now faces.
I have one Ada anecdote. When I was an undergrad, one of the CS contests in my state had a rule that you had to use C, but you could use Ada instead if you had written a compiler for it and you used that compiler. Apparently students from the military institute would occasionally show up to the contest with their own Ada compiler, and they were allowed to use it.
C and C++ programmers, hearing “systems language,” expect Rust to be a lot easier for them than it is. Yes, Rust is a systems language that offers the same degree of performance that they do, but it is in many respects wildly different. This difference between expectation and reality leads to a lot of confusion and frustration.
On the other hand, people coming from languages like Python and Ruby expect Rust to be hard, and often find that it is easier than they anticipated. Not easy, but easier.
I might amend that to hearing “systems language that will make you unbelievably productive”. The common thread to many rust complaints I’ve seen is that “after considerable investment studying the borrow checker” is relegated to a tiny footnote.
There’s quite a gap between “correct” and “provably correct” code. It’s easy to write the former, but convincing the compiler of the latter is difficult. It doesn’t really feel like progress.
To your first point, I do think that Rust could be more up front about the complexity. For example, I think that there could be more done to encourage Rust programmers to read The Rust Programming Language book before attempting a new project. There are a number of common stumbling blocks addressed in the book, and things like changes to Rust’s website could encourage more people to read it.
On the second point, I disagree. While there are C and C++ programmers who can write safe code without the ceremony of Rust to back them up, I don’t think this is true for most programmers. That is, while the gap between “correct” and “provably correct” may be large, there is also a large gap between “looks correct but isn’t” and “actually correct” code, and Rust helps the layperson write “actually correct” code with confidence.
This book could also be a source of the mismatch between users that didn’t have much trouble getting over the borrow checker (e.g. me) and those that did.
When I was learning rust, I had multiple tabs open with the book all the time and it helped tremendously. However, I’m also familiar with a lot of FP languages (not necessarily fluent; e.g. Haskell, Scala, Clojure, OCaml) so the type system wasn’t an additional point of confusion like it may be for those who’ve spent the majority of their time in C/C++/Java.
By “correct” and “provably correct”, you mean “provably correct by hand” and “mechanically provably correct”, respectively, right? I have no idea how you could write code you know is correct without at least an informal proof sketch in your head that it’s indeed correct. (Or are you saying it’s easy to write correct programs by accident? I’m pretty sure that’s not true.) An informal proof might not satisfy a mechanical proof checker, but it’s a proof nevertheless.
Maybe systems people are just really impatient for progress?
That statement sounds so weird after a life-time of C/C++ being virtually the only game in town. I know about Ada and the like, but for non-specialized systems programming, yeesh.
the hacker news comments made a really good point - while on the whole rust may not be more complex than say c++ or haskell, it makes you pay most of that complexity up front as the price of admission, whereas in most languages you can start off by using a small subset of the features relatively easily and have working code while you ramp yourself up to the full language.
So true. Going from Java -> Ruby took me months to get something near idiomatic code. Ruby -> Golang was much faster (about a month) but still took time. Like you, I’ve realized it takes weeks or months for these ideas to percolate through your brain and become second nature.
I think it will make these comparisons more honest if we show what the borrow checker actually replaces: formal verification of correct, heap usage via tools like separation logic. Here’s Wikipedia page of that:
Now, compare using that with C language (i.e. Microsoft’s VCC) to whatever heuristics people are given to get programs like OP’s through the borrow checker. Im betting the latter is going to be a dramatic improvement in usability over the learning curve of formal proof, separation logic, and barely-documented tools like VCC.
Now, that said, it will still be harder than writing unsafe or GC’d code. That’s not the goal, though. Goal is safe, memory management with no GC or other overhead. Apples to apples is Rust vs C + separation logic or other formal method. Or vs C with static analysis that’s sound & structure limitations they impose on C programs if such tools exist by now. Rust comes out as “Way better with room for improvement.”
This is a good point and you don’t hear this perspective enough. After decades of research we haven’t been able to do much better than substructural type systems (i.e., linear & affine types) for automated verification of memory safety in GC-less languages. Rust’s complexity is essential. Which is why I think if our industry is to get serious about security, our collective response to “Rust is too hard” must either be: 1) use a GCed language, or 2) too bad. If you can’t grok linear types you probably aren’t writing safe C.
Rust’s complexity is only essential if you take the “zero-cost abstractions” notion at face value and reject GC outright. I think that this is a mistake. Even with the borrow checker, as soon as you introduce unbounded data structures, you introduce dynamism. A growable vector is still going to need to perform dynamic memory management with malloc and free. There are runtime costs to these and Rust’s design starts with the (IMO, false) assumption that the runtime costs of malloc and free are necessarily less than the run-time costs of automatic memory management.
The entire design space of heap models between C-style and Lisp/SmallTalk/Java-style is practically unexplored. What limited exploration has occurred here, mostly by the “managed OS” academics, has been extremely promising despite generally starting with the complex foundation of a C#-style object system (ie. dynamic objects plus unboxed value types).
Rust’s complexity is only essential if you take the “zero-cost abstractions” notion at face value and reject GC outright.
The borrow checker – linear typing – is there to demonstrate that variables are accessed “safely”, which is to say: only once in a mutable way. What does that have to do with GC? It’s true that Rust also checks whether or not you’re trying to use a stack allocated variable in an unsafe way, like returning a reference to it. It could promote it to the heap but that kind of magic would likely be unergonomic behavior for systems programmers.
1) My comment was focused on GC because the comment I replied to suggested “use a GCed language” and because the lack of GC is one of Rust’s most touted features. I believe my argument about zero-cost (read: static) abstractions actually having some dynamic cost applies equally well elsewhere.
2) The borrow checker is about ownership. The two main elements of ownership are lifetime and, yes, mutation. The former is obviously about memory management. The latter is, frankly, about mutation’s nature to behave like concurrency. Entire categories of mutation/concurrency problems go away if you are able to make cheap, immutable snapshots, but such a mechanism necessarily requires dynamic allocation…
3) It is only necessary to perform heap promotion if you’re only willing to accept static escape analysis. If you’re willing to entertain dynamic mechanisms, you have other options. For example, you could perform something akin to a compacting garbage collection on the call frame at return time. This isn’t very much different than performing a normal struct-returning copy, except you follow pointers within the stack and move those too.
4) I’ll concede that what I’m proposing may not reasonably fit in traditional language designs. That’s an argument for more research and experimentation, not doubling down on static-only.
The kind of mutability you’re describing would seem to be resolvable to SSA, given that it’s based on “…cheap, immutable snapshots…”. If the mutation is really a copy, it would seem to give you a style of programming that’s really close to Haskell most of the time – the only observable from a function is its return value.
How do you think shared data structures should be handled?
In a paradigm like you’re describing, I imagine the compiler would insert code to generate the snapshot and hook writes and turn them into copies. Is the overhead of this comparable to something like ARC? I ask because my understanding is that Swift benefits a lot from skipping that overhead when it’s able to perform stack promotion.
I mostly agree. However, many of us made correct or secure code at times without knowing linear types, etc. Do it with careful modeling, contracts at interfaces, FSM’s, etc. Need to go full formal for best results but keeping stuff simple gets 90% there.
Take me as an example where looking earlier at some linear types PDF’s had a pile of mathematical gibberish I don’t understand. I dont even know what a type is in terms of implementation or higher meaning. Some tell me it’s just a number. Some offer huge tomes that will teach me if I read everything in them. Nothing like my experience with static types in imperative languages, DbC, Ada’s protections, a PlusCal demo, etc. Those I understood a lot as material could be absorbed by beginners with clear connection between abstract and concrete stuff.
So, we may not be stuck with accepting Rust to get to the goal. Maybe we just need better material on helping programmatic, non-mathematical folks (like me) understand useful stuff like linear types or stronger specs for feeding imperative code into solvers SPARK-style. I think, like SPARK or Dafny, the language might improve to point where little is required of operator. Maybe a bunch of pre-certified patterns for hard things borrow checker cant handle that can be discharged through a prover. User just specifies the pattern with it and code turned into VC’s.
So, both tooling and education might improve the usability situation for stuff like borrow checker. Meanwhile, they gotta tough it up as you said using whatever tools achieve the safety/security goal they want to achieve. Right now, that’s about the only game in town if one wants temporal safety without GC and with good ecosystem.
However, many of us made correct or secure code at times without knowing linear types, etc. […] Take me as an example where looking earlier at some linear types PDF’s had a pile of mathematical gibberish I don’t understand.
At some intuitive level you have to be familiar with linear reasoning or your code would be littered with use-after-free bugs. I don’t know what PDFs you were looking at but there is a good probability the intended audience is people already familiar with type theory. I hope there are more appropriate resources out there for people wanting to understand Rust’s type system or we have a long way to go in the education department.
I think, like SPARK or Dafny, the language might improve to point where little is required of operator.
Right, manual/computer-aided proof is always an option–I didn’t mention this because your first post did. I’m not super familiar with this area but I’ve been meaning to read Mostly-Automated Verification of Low-Level Programs in Computational Separation Logic. I’m pessimistic about this approach being easier for average programmers–it may accept more programs than linear typing would, but I’m worried that it won’t fail gracefully. How is a programmer incapable of linear reasoning going to discharge proof obligations that are too hard for the prover?
How is a programmer incapable of linear reasoning going to discharge proof obligations that are too hard for the prover?
They won’t. There will always be gaps. I’m saying there could be some extra patterns and analyses covered past the borrow checker that only require basic specs or automated tooling. The improvements the SPARK team did over time increasing automation in the paper below are main inspiration for this comment. There were also results elsewhere like doing some HOL proofs using SAT (or SMT?) solvers after an automated conversion to what they can take.
When I see these “I tried Rust for a week and I was frustrated” articles I try to imagine how frustrated I’d be after a week of trying to write equally safe code in C++. That’s with >10 years of C++ experience, but not much since 2000 or so—I would be trying to sort out the “modern” safe way to use C++ (there is one by now, right?) from the other 1000 ways to use it.
I found Rust frustrating too, but I totally felt it would be worth it to get those guarantees with such a comparatively simple language.
I experienced something similar to this in Rust maybe a year ago. I was trying to write a baby LISP interpreter as a way to get more comfortable in the language. It felt like I was fighting the compiler the whole way through, but it was because I was so used to programming in a C++ way. The thing that really killed the project for me was something so painfully simple that it had me scratching my head as to how this passed under peoples' noses: getting a substring from some base string.
There is no .substr() method, or any equivalent that’s built in.
You can’t take the slice version of the string, allocate a new slice, and copy index-to-index; slices are indexed by byte, and not by character. Furthermore, string sizes are done by character, and not by byte.
So the only solution was to write my own substring method (accounting for Unicode/multibyte character weirdness described above), or bring in a new dependency to the project. It felt ridiculous that I wasn’t able to do this basic string manipulation that’s a pretty common operation in most languages. My solution was equally ridiculous; chop off N front characters, reverse the string, chop off M back characters, and reverse again.
Maybe there’s a better way to do this, but it just felt like an extreme solution to something that the standard library should be able to solve. The first hit from googling “rust substring” gives this link detailing a few possible solutions, but nothing conclusive. It was a frustrating experience and it really turned me off to the language for a little while.
If you can, please see my response to @azrazalea. Rust strings are UTF-8 encoded, which means we give up fast & easy substring indexing, but the rust devs have decided that the gains are worth the losses.
Taking a substring in Rust by byte ranges is simple. If s is a String, then &s[i..j] is a &str substring of s from bytes i to j. If either i or j aren’t on a valid UTF-8 encoded codepoint boundary, then the expression causes a panic.
Doing this by Unicode codepoint index is generally considered an anti-pattern and something you should try to avoid. Both Rust and Go leave this type of slicing/indexing out of their standard library, so Rust is not alone in this. But if you wanted to do it, the way I’d suggest would be to get the byte offset of the corresponding codepoint, and then slice it that way. You can get byte offsets by using the s.char_indices() iterator. For example, s.char_indices().nth(5).unwrap().0 will give you the byte offset of the beginning of the 5th codepoint.
The reason why this type operation is not something you should generally do is because “Unicode codepoint” isn’t necessarily “one character” from the perspective of humans. A better approximation of “character” is a grapheme cluster, as defined by the Unicode standard.
As I see it, there are two good reasons to make this operation very explicit:
To give you appropriate pause that maybe it’s not what you want to do.
To make the performance costs obvious. Slicing by bytes is O(1), but slicing by codepoint is O(n) given how Rust represents strings internally.
it just felt like an extreme solution to something that the standard library should be able to solve
I’m a beginner/hobbyist in rust, but from what I understand their philosophy is to have very very little in the standard library and for you to pull in a dependency for many things that other languages have in their standard library.
In addition, strings are indeed just bytes in rust but from what I understand they went that way to avoid setting a “blessed” encoding for the language. Since they are just bytes, libraries can implement UTF-8, UTF-16, whatever support.
As far as unicode support in standard library, the situation isn’t that great with programming languages in general. I don’t see rust as much of an outlier.
There should be a real book for learning Rust for systems programmers. And there should be a real book for learning Rust for people new to systems programming.
I’m not sure why the heap api is still unstable. Maybe they’re waiting for the place API to stabilize. Also, I imagine for the poster’s use-case, one could have easily used a boxed slice. But it’s still interesting that an easy way to allocate n-bytes is not available in stable rust.
As far as I can tell, the OP’s use case is perfectly satisfied by a normal Vec<u8>. I have no idea why they reached for RawVec. I asked in the comments section so that we can fix whatever led them to that data type. RawVec is super duper specialized and definitely not recommended for generic “bundle of bytes” usage like this. That’s why it’s not stable.
From my experience, creating recursive data structures is quite difficult and requires a good handle of the borrow checker. I had a similar experience with early success and sudden halt when I reached this point. Its also difficult to know after you have something working if it would be considered idiomatic. I don’t see this as a fault of the Rust language or community.
It’s a new language, you have to go slow. I don’t get why people think they should automatically be productive in a new language. Yes, it sucks to be slapped upside the head by the ownership model. But if you want the safety advantage (which, presumably, you do, given that you’re using Rust), that means a compiler that you have to please and think like in order to make it to executable code. If you don’t, use C.
I guess I find it immensely confusing that people clamor for tools to save them from themselves and then summarily reject them when they aren’t lenient enough. It’s clear that, collectively, we believe less in good practices than good tools. After all, what is React but a way to enable lots of [junior] coders to work on parts of a page without stepping on each other?
Edit: misremembered quote, removed that, thanks angersock
I agree. I am by no means a Rust expert, but I have done some small projects in Rust. The ownership problem in the last example seems trivial to solve. Either you do an early return if you have cached the entry, something like:
Or you use the
entryfunction ofHashMap. If you find that the entry is occupied, you usegetonOccupiedEntry. Otherwise, you have your mutable handle viaVacantEntryand you can use it to insert the results into the cache.I understand the author’s frustration. I have also been through the two week ‘how do I please the borrow checker’-hell. But once you understand the rules and the usual ways to address ownership issues, it’s relatively smooth sailing. The guarantees that the Rust ownership provides (like no simultaneous mutable/immutable borrows) allow you to do really cool things, like the
peek_mutmethod ofBinaryHeap:https://doc.rust-lang.org/std/collections/struct.BinaryHeap.html#method.peek_mut
Basically it restores the heap property through the ‘Drop’ (destructor) trait. It is safe to do this, because it’s a mutable borrow, blocking immutable borrows and thus inconsistent views of the data when the top of the heap is changed and the sift-down operation has not been applied yet. The other day, I used the same trick in some code where I have a sparse vector data structure, which is a wrapper around BTreeMap that automatically removes vector components when their values are changed to 0.
I get your point, but you are blatantly misquoting the author here. They spent multiple evenings learning rust for their true toy, not an hour. The hour was spent getting 30 lines of code for a different toy example into something that looked like reasonable rust, which didn’t then compile.
Please don’t quote people out of context, especially when it is so easy to catch.
You are right, I double-checked the context and fixed my mistake. Point still stands, however.
I’m thinking about Haskell and wondering why this is, because the difference is pretty stark. I didn’t expect to ever actually learn Haskell, so when it finally seemed to be happening after months of messing with it, I was pretty overjoyed. Nobody had told me it would be easy; in fact, I had assumed it would be really difficult and I might not be able to get there. I’m not paying close enough attention, are they marketing Rust as something you can learn pretty easily? That’s not the sense one gets from reading blogs.
A lot of people move between Java and JavaScript or C#, and they’re just not prepared for something totally different. This guy is pretty competent though, I don’t think that’s what’s happening here.
Maybe systems people are just really impatient for progress?
Ultimately, all this negative press will work to its benefit. Programmers like to be elitist about knowing hard things (like Haskell, in years past) so if Rust develops a reputation of impenetrability, that just means in two or three years there will be a lot of young Rust programmers.
Unfortunately, this was exactly the sort of negative press (right down to an ESR hatchet job quoted endlessly) that killed Ada in industry.
For whatever reason “I spent a whole hour writing code that in the end didn’t compile” is considered a damning indictment of a language, whereas “I spent a whole hour writing code that in the end compiled with many subtle correctness issues unaddressed that later bit me in the ass” is considered “wow so productive!”.
Our industry is still very immature.
Incidentally, the C++ code that the OP wrote actually contains undefined behavior for violating the strict aliasing rule. (There’s a cast and dereference from
char*totrie_header_info*, wheretrie_header_infohas a stricter alignment thanchar.)Actually, the Rust code from the OP (linked in comments) contains the same error, but it is annotated with
unsafe. :-)ESR definitely ran into the issue described by @brinker above, but apart from that I thought he was reasonably fair about what issues he had. I did not read as particularly spiteful to me, just that he found Go more productive and Rust frustrating and immature. That makes sense; Rust is younger and less mature and Go prioritizes productivity above many other things.
I’m not sure what “killed” Ada in industry but lots of good languages failed during that era for reasons that had nothing to do with technical superiority. Personally I find Ada kind of dull to read and the situation with GNATS has always confused me. I thought Eiffel looked more like “Ada done right” when I was in college; now I think Ada probably had a better emphasis on types than Eiffel did that probably lent itself to reliability more directly than Eiffel’s design-by-contract stuff. But this is way out on the fringe of anything I know about really; I would greatly enjoy hearing more about Ada now.
ESR’s post generated substantial frustration in the Rust community because a number of the factual claims or statements he makes about Rust are wrong. Members of the Rust community offered corrections for these claims, but ESR reiterated them without edit in his follow up post. Reasonable criticism based on preference or disagreement with design decisions is one thing, claiming untrue things about a language as part of an explanation of why not to use it is another thing entirely.
Well, if it makes you feel better, the big thing I got from it was “NTPsec relies on a certain system call that isn’t a core part of Rust” which isn’t going to weigh on my mind particularly hard. I agree with you that criticism should be factual.
That wasn’t really my impression: he was factually incorrect about a number of issues in both his initial rant and his follow up (string manipulation in particular), and he mostly seemed to be criticizing Rust for not being what he wanted rather than on its own terms (ok, Rust doesn’t have a first-class syntax for select/kqueue/epoll/IO completion ports – but that’s obviously by design, because Rust is intended to be a runtimeless C replacement, not something with a heavy-weight runtime that papers over the large semantic differences between those calls. If you went into Rust just wanting Go, then just use Go).
If I had a nickel for every time someone quoted ESR’s “hacker dictionary” entry on Ada being lol so huge and designed by comittee, I’d have…well, a couple of dollars, anyways. You’ll still hear them today, and Ada is still on the small side of languages these days, and still isn’t designed by committee, while a lot of popular languages are.
It had some attention on it in the late ‘90s, but every time a discussion got going around it, you’d hear the same things: people parroting ESR with no direct experience of their own, people (generally students, which is understandable, but also working professionals who should have known better) complaining that the compiler rejected their code (and that’s obviously bad because a permissive compiler is more important than catching bugs), etc.
Just a general negative tone that kept people from trying it out, which kept candidates for jobs at a minimum, which kept employers from ever giving it serious thought.
I encountered Ada in the early 90s (in college). At that time, Ada was considered a large, bondage and discipline language where one fought the compiler all the way (hmm … much like Rust today). At the time, C has just been standardized and C++ was still years away from its first real standard so Ada felt large and cumbersome. At the time, I liked the idea of Ada, just not the implementation (I found it a bit verbose for my liking). ESR was writing about Ada around this time, and he was was parroting the zeitgeist of the time.
Compared to C++ today? It’s lightweight.
Yeah, it’s funny the way initial perceptions sink something long after they cease to be true.
I was having these conversations in 1998-1999, after the Ada ‘95 and C++ '98 standardizations, which were really easy to point at and say that “no, actually, Ada’s a lot smaller than C++ at the moment”, but it didn’t really matter, because an off-the-cuff riff on Ada as it had stood in maybe the late '80s was the dominant impression of the language, and no amount of facts or pointing out that paying a bit of cost up front in satisfying the compiler was easily better than paying 10x that cost in chasing bugs and CVEs were capable of changing that.
This is more or less what I’m concerned is the unclimbable hill Rust now faces.
I have one Ada anecdote. When I was an undergrad, one of the CS contests in my state had a rule that you had to use C, but you could use Ada instead if you had written a compiler for it and you used that compiler. Apparently students from the military institute would occasionally show up to the contest with their own Ada compiler, and they were allowed to use it.
In my experience, the problem is thus:
C and C++ programmers, hearing “systems language,” expect Rust to be a lot easier for them than it is. Yes, Rust is a systems language that offers the same degree of performance that they do, but it is in many respects wildly different. This difference between expectation and reality leads to a lot of confusion and frustration.
On the other hand, people coming from languages like Python and Ruby expect Rust to be hard, and often find that it is easier than they anticipated. Not easy, but easier.
I might amend that to hearing “systems language that will make you unbelievably productive”. The common thread to many rust complaints I’ve seen is that “after considerable investment studying the borrow checker” is relegated to a tiny footnote.
There’s quite a gap between “correct” and “provably correct” code. It’s easy to write the former, but convincing the compiler of the latter is difficult. It doesn’t really feel like progress.
To your first point, I do think that Rust could be more up front about the complexity. For example, I think that there could be more done to encourage Rust programmers to read The Rust Programming Language book before attempting a new project. There are a number of common stumbling blocks addressed in the book, and things like changes to Rust’s website could encourage more people to read it.
On the second point, I disagree. While there are C and C++ programmers who can write safe code without the ceremony of Rust to back them up, I don’t think this is true for most programmers. That is, while the gap between “correct” and “provably correct” may be large, there is also a large gap between “looks correct but isn’t” and “actually correct” code, and Rust helps the layperson write “actually correct” code with confidence.
This book could also be a source of the mismatch between users that didn’t have much trouble getting over the borrow checker (e.g. me) and those that did.
When I was learning rust, I had multiple tabs open with the book all the time and it helped tremendously. However, I’m also familiar with a lot of FP languages (not necessarily fluent; e.g. Haskell, Scala, Clojure, OCaml) so the type system wasn’t an additional point of confusion like it may be for those who’ve spent the majority of their time in C/C++/Java.
By “correct” and “provably correct”, you mean “provably correct by hand” and “mechanically provably correct”, respectively, right? I have no idea how you could write code you know is correct without at least an informal proof sketch in your head that it’s indeed correct. (Or are you saying it’s easy to write correct programs by accident? I’m pretty sure that’s not true.) An informal proof might not satisfy a mechanical proof checker, but it’s a proof nevertheless.
That statement sounds so weird after a life-time of C/C++ being virtually the only game in town. I know about Ada and the like, but for non-specialized systems programming, yeesh.
I mean, impatient to make progress on their problem, not impatient for new languages and paradigms.
Ah, that makes more sense.
I dunno, webbers I run into seem to have stronger time preference than systems people.
Systems is harder initially I think.
the hacker news comments made a really good point - while on the whole rust may not be more complex than say c++ or haskell, it makes you pay most of that complexity up front as the price of admission, whereas in most languages you can start off by using a small subset of the features relatively easily and have working code while you ramp yourself up to the full language.
So true. Going from Java -> Ruby took me months to get something near idiomatic code. Ruby -> Golang was much faster (about a month) but still took time. Like you, I’ve realized it takes weeks or months for these ideas to percolate through your brain and become second nature.
I think it will make these comparisons more honest if we show what the borrow checker actually replaces: formal verification of correct, heap usage via tools like separation logic. Here’s Wikipedia page of that:
https://en.m.wikipedia.org/wiki/Separation_logic
Now, compare using that with C language (i.e. Microsoft’s VCC) to whatever heuristics people are given to get programs like OP’s through the borrow checker. Im betting the latter is going to be a dramatic improvement in usability over the learning curve of formal proof, separation logic, and barely-documented tools like VCC.
Now, that said, it will still be harder than writing unsafe or GC’d code. That’s not the goal, though. Goal is safe, memory management with no GC or other overhead. Apples to apples is Rust vs C + separation logic or other formal method. Or vs C with static analysis that’s sound & structure limitations they impose on C programs if such tools exist by now. Rust comes out as “Way better with room for improvement.”
This is a good point and you don’t hear this perspective enough. After decades of research we haven’t been able to do much better than substructural type systems (i.e., linear & affine types) for automated verification of memory safety in GC-less languages. Rust’s complexity is essential. Which is why I think if our industry is to get serious about security, our collective response to “Rust is too hard” must either be: 1) use a GCed language, or 2) too bad. If you can’t grok linear types you probably aren’t writing safe C.
Rust’s complexity is only essential if you take the “zero-cost abstractions” notion at face value and reject GC outright. I think that this is a mistake. Even with the borrow checker, as soon as you introduce unbounded data structures, you introduce dynamism. A growable vector is still going to need to perform dynamic memory management with malloc and free. There are runtime costs to these and Rust’s design starts with the (IMO, false) assumption that the runtime costs of malloc and free are necessarily less than the run-time costs of automatic memory management.
The entire design space of heap models between C-style and Lisp/SmallTalk/Java-style is practically unexplored. What limited exploration has occurred here, mostly by the “managed OS” academics, has been extremely promising despite generally starting with the complex foundation of a C#-style object system (ie. dynamic objects plus unboxed value types).
The borrow checker – linear typing – is there to demonstrate that variables are accessed “safely”, which is to say: only once in a mutable way. What does that have to do with GC? It’s true that Rust also checks whether or not you’re trying to use a stack allocated variable in an unsafe way, like returning a reference to it. It could promote it to the heap but that kind of magic would likely be unergonomic behavior for systems programmers.
Several things to unpack here:
1) My comment was focused on GC because the comment I replied to suggested “use a GCed language” and because the lack of GC is one of Rust’s most touted features. I believe my argument about zero-cost (read: static) abstractions actually having some dynamic cost applies equally well elsewhere.
2) The borrow checker is about ownership. The two main elements of ownership are lifetime and, yes, mutation. The former is obviously about memory management. The latter is, frankly, about mutation’s nature to behave like concurrency. Entire categories of mutation/concurrency problems go away if you are able to make cheap, immutable snapshots, but such a mechanism necessarily requires dynamic allocation…
3) It is only necessary to perform heap promotion if you’re only willing to accept static escape analysis. If you’re willing to entertain dynamic mechanisms, you have other options. For example, you could perform something akin to a compacting garbage collection on the call frame at return time. This isn’t very much different than performing a normal struct-returning copy, except you follow pointers within the stack and move those too.
4) I’ll concede that what I’m proposing may not reasonably fit in traditional language designs. That’s an argument for more research and experimentation, not doubling down on static-only.
If mutation is always “handled” with cheap copies doesn’t that describe Haskell?
No. I said cheap snapshots, not copies. Implies you have something mutable and then make it copy-on-write in O(1) time.
The kind of mutability you’re describing would seem to be resolvable to SSA, given that it’s based on “…cheap, immutable snapshots…”. If the mutation is really a copy, it would seem to give you a style of programming that’s really close to Haskell most of the time – the only observable from a function is its return value.
How do you think shared data structures should be handled?
In a paradigm like you’re describing, I imagine the compiler would insert code to generate the snapshot and hook writes and turn them into copies. Is the overhead of this comparable to something like ARC? I ask because my understanding is that Swift benefits a lot from skipping that overhead when it’s able to perform stack promotion.
I mostly agree. However, many of us made correct or secure code at times without knowing linear types, etc. Do it with careful modeling, contracts at interfaces, FSM’s, etc. Need to go full formal for best results but keeping stuff simple gets 90% there.
Take me as an example where looking earlier at some linear types PDF’s had a pile of mathematical gibberish I don’t understand. I dont even know what a type is in terms of implementation or higher meaning. Some tell me it’s just a number. Some offer huge tomes that will teach me if I read everything in them. Nothing like my experience with static types in imperative languages, DbC, Ada’s protections, a PlusCal demo, etc. Those I understood a lot as material could be absorbed by beginners with clear connection between abstract and concrete stuff.
So, we may not be stuck with accepting Rust to get to the goal. Maybe we just need better material on helping programmatic, non-mathematical folks (like me) understand useful stuff like linear types or stronger specs for feeding imperative code into solvers SPARK-style. I think, like SPARK or Dafny, the language might improve to point where little is required of operator. Maybe a bunch of pre-certified patterns for hard things borrow checker cant handle that can be discharged through a prover. User just specifies the pattern with it and code turned into VC’s.
So, both tooling and education might improve the usability situation for stuff like borrow checker. Meanwhile, they gotta tough it up as you said using whatever tools achieve the safety/security goal they want to achieve. Right now, that’s about the only game in town if one wants temporal safety without GC and with good ecosystem.
At some intuitive level you have to be familiar with linear reasoning or your code would be littered with use-after-free bugs. I don’t know what PDFs you were looking at but there is a good probability the intended audience is people already familiar with type theory. I hope there are more appropriate resources out there for people wanting to understand Rust’s type system or we have a long way to go in the education department.
Right, manual/computer-aided proof is always an option–I didn’t mention this because your first post did. I’m not super familiar with this area but I’ve been meaning to read Mostly-Automated Verification of Low-Level Programs in Computational Separation Logic. I’m pessimistic about this approach being easier for average programmers–it may accept more programs than linear typing would, but I’m worried that it won’t fail gracefully. How is a programmer incapable of linear reasoning going to discharge proof obligations that are too hard for the prover?
They won’t. There will always be gaps. I’m saying there could be some extra patterns and analyses covered past the borrow checker that only require basic specs or automated tooling. The improvements the SPARK team did over time increasing automation in the paper below are main inspiration for this comment. There were also results elsewhere like doing some HOL proofs using SAT (or SMT?) solvers after an automated conversion to what they can take.
http://www.spark-2014.org/uploads/itp_2014_r610.pdf
When I see these “I tried Rust for a week and I was frustrated” articles I try to imagine how frustrated I’d be after a week of trying to write equally safe code in C++. That’s with >10 years of C++ experience, but not much since 2000 or so—I would be trying to sort out the “modern” safe way to use C++ (there is one by now, right?) from the other 1000 ways to use it.
I found Rust frustrating too, but I totally felt it would be worth it to get those guarantees with such a comparatively simple language.
I experienced something similar to this in Rust maybe a year ago. I was trying to write a baby LISP interpreter as a way to get more comfortable in the language. It felt like I was fighting the compiler the whole way through, but it was because I was so used to programming in a C++ way. The thing that really killed the project for me was something so painfully simple that it had me scratching my head as to how this passed under peoples' noses: getting a substring from some base string.
So the only solution was to write my own substring method (accounting for Unicode/multibyte character weirdness described above), or bring in a new dependency to the project. It felt ridiculous that I wasn’t able to do this basic string manipulation that’s a pretty common operation in most languages. My solution was equally ridiculous; chop off N front characters, reverse the string, chop off M back characters, and reverse again.
Maybe there’s a better way to do this, but it just felt like an extreme solution to something that the standard library should be able to solve. The first hit from googling “rust substring” gives this link detailing a few possible solutions, but nothing conclusive. It was a frustrating experience and it really turned me off to the language for a little while.
If you can, please see my response to @azrazalea. Rust strings are UTF-8 encoded, which means we give up fast & easy substring indexing, but the rust devs have decided that the gains are worth the losses.
Taking a substring in Rust by byte ranges is simple. If
sis aString, then&s[i..j]is a&strsubstring ofsfrom bytesitoj. If eitheriorjaren’t on a valid UTF-8 encoded codepoint boundary, then the expression causes a panic.Doing this by Unicode codepoint index is generally considered an anti-pattern and something you should try to avoid. Both Rust and Go leave this type of slicing/indexing out of their standard library, so Rust is not alone in this. But if you wanted to do it, the way I’d suggest would be to get the byte offset of the corresponding codepoint, and then slice it that way. You can get byte offsets by using the
s.char_indices()iterator. For example,s.char_indices().nth(5).unwrap().0will give you the byte offset of the beginning of the 5th codepoint.The reason why this type operation is not something you should generally do is because “Unicode codepoint” isn’t necessarily “one character” from the perspective of humans. A better approximation of “character” is a grapheme cluster, as defined by the Unicode standard.
As I see it, there are two good reasons to make this operation very explicit:
I’m a beginner/hobbyist in rust, but from what I understand their philosophy is to have very very little in the standard library and for you to pull in a dependency for many things that other languages have in their standard library.
In addition, strings are indeed just bytes in rust but from what I understand they went that way to avoid setting a “blessed” encoding for the language. Since they are just bytes, libraries can implement UTF-8, UTF-16, whatever support.
As far as unicode support in standard library, the situation isn’t that great with programming languages in general. I don’t see rust as much of an outlier.
This is incorrect. Strings in rust (meaning the String and &str types) are guaranteed to be UTF-8. In the stdlib, you can get the individual characters with
.chars()and get their indices with.char_indices(). If you want to deal with graphemes, you need to useunicode-segmentation: https://unicode-rs.github.io/unicode-segmentation/unicode_segmentation/index.html . Manishearth had a great post on dealing with unicode in strings recently: http://manishearth.github.io/blog/2017/01/14/stop-ascribing-meaning-to-unicode-code-points/edit: wrong manishearth page! it is now corrected
(also cc @intercal for all of the above)
Thanks for the correction! The need of a library for graphemes is what I was thinking of.
There should be a real book for learning Rust for systems programmers. And there should be a real book for learning Rust for people new to systems programming.
I’m not sure why the heap api is still unstable. Maybe they’re waiting for the place API to stabilize. Also, I imagine for the poster’s use-case, one could have easily used a boxed slice. But it’s still interesting that an easy way to allocate n-bytes is not available in stable rust.
As far as I can tell, the OP’s use case is perfectly satisfied by a normal
Vec<u8>. I have no idea why they reached forRawVec. I asked in the comments section so that we can fix whatever led them to that data type.RawVecis super duper specialized and definitely not recommended for generic “bundle of bytes” usage like this. That’s why it’s not stable.What is a good proper use of
RawVec?When you want to build your own
Vec-like generic data structure. If you just want a buffer of bytes thenVec<u8>itself is perfectly suitable.From my experience, creating recursive data structures is quite difficult and requires a good handle of the borrow checker. I had a similar experience with early success and sudden halt when I reached this point. Its also difficult to know after you have something working if it would be considered idiomatic. I don’t see this as a fault of the Rust language or community.