Maybe I’m just not over the hump yet, but my experience with Rust hasn’t been super happy. Last month I built both a Rust and a Nim binding to a C API [that I wrote and own.] I’m a newbie a both languages. The Nim binding was a joy to write, simple and clean. The Rust one involved tons of fighting with the borrow checker, and a couple of areas I just had to leave as ugly hacks because Doing it right created bizarre errors I couldn’t understand — something about interactions between lifetimes and generics.
I don’t think I’m having the typical newbie problems understanding lifetimes. I’ve been using C since 1981 and C++ since 1990, and I and understand stacks and move semantics and where data lives. I appreciate the work Rust has done to make delicate lifetime dependencies checkable by the compiler! But it seems to have made the language extremely complex. I found a blog post about “What newbies get wrong about understanding lifetimes” last week, and it made me basically give up — trying to understand the subtle distinctions between what I thought was going on and what is actually going on just made my brain hurt, in the same way it hurts when I try to understand template metaprogramming or hyperbolic geometry.
Honestly I’m not sure the complexity of the lifetime analysis in Rust is worth it, compared to the minor performance overhead of using some ref-counted objects (as in Nim, or my current C++ code.) But I’m willing to be convinced otherwise.
Your post surprised me. I’ve been using C++ for more than a decade and the transition to Rust was largely painless. After reading half the Rust Book and 2 weeks of experimenting, I felt fairly comfortable with the language. Because I was already using C++11 features like unique_ptr and move semantics, the transition to Rust didn’t require me to change anything at all about my programming style¹: The lifetime management in C++ is largely the same, just without strict compiler checks. Because of that, I assumed that coming from modern C++ is a great shortcut to picking up Rust. I did have some trouble understanding the lifetime syntax like 'a and so on, but the language has some excellent documentation that helped me understand it and the Rust channel on Discord is full of super nice and helpful people.
At this point, I’m already about 3-5x more productive with Rust than I am with C++², even though I started looking into Rust less than a year ago. I can churn out software of a size and level that was impossible to churn out before. That is incredibly empowering. Rust made me feel like when I was a kid learning my first programming language. I think it’s a real, life-changing innovation. I haven’t felt this excited about programming since I was a kid.
¹ This programming style is such that your C++ code does not contain a single new, delete, malloc or free. Instead, you use make_unique and make_shared and clean up by letting the unique_ptr and shared_ptr go out of scope. It is pure RAII, exception-safe and prevents leaks and double deletes by default.
² For various reasons:
The standardized build system with cargo makes cross-platform development and library usage so much simpler.
In particular, cross-platform support is basically free: I developed on Windows for months and cargo run worked on Linux first try without a single change. This is unheard of in the C++ world for any nontrivial program.
The Rust standard library is much better than the C/C++ standard library.
Rust has a proper, UTF-8 encoded string type.
The error messages that are emitted by rustc are almost always very detailed and accurate. When I was a beginner, I would literally just copy-paste what the compiler proposed, and it would make my code compile and work properly.
Error handling is simple thanks to the powerful Result enum (sum type). Additionally, everyone uses Result, so there is a standardized way to return errors across the entire ecosystem, something that C++ lacks.
The compiler prevented countless errors (and thus debugging) on my part.
Yup, everything you’re saying is what I want to get out of Rust — the goal is to port some complex, finicky, concurrent networking code that uses this API I’m wrapping, because it’s too complicated, messy and hard to maintain in C++. And I’ve really enjoyed parts of Rust like Cargo and the wonderfully informative compiler error messages, which feel like a TA helping teach me. (Until the point where I get the same error over and over no matter how I tweak the code…)
I think I need to find a forum where I can post some snippets that don’t work and ask for help. I’m not really sure where to go, though, as Rust is so big there isn’t a single obvious place (like the Nim forum, for Nim.) Do I just go directly to SO?
(And yeah, my C++ style leans heavily on unique_pointer, moves, and on homemade RefCounted and Ref<> classes. I’ve been kind of writing Rust-like code for years, especially after I read about Rust in 2014(?) and began envying it.)
Me, I once fell into an even more frustrating case, of a “bistable error message/hint”: the compiler showed me an error message with a helpful suggestion how to change my code. I changed it as suggested, just to be greeted with a new error message with a helpful suggestion to change it… back to the exactly same shape of code I had first :(
Another thing that made me put it down for the time being, until I regenerate enough psychical stamina to try and attack the hump once again, was a feeling that Rust requires me to sacrifice the ideal of being able to write simple APIs (i.e. hiding complexity from their users/callers). In other words, I found there seems sometimes to be just one way I can structure an API to make it safe and correct, and it makes my API super complex, and requires the caller to know some nontrivial Rust incantations to actually be able to use it.
Really sorry I can’t provide you with a concrete example of a code in question now. I have it buried somewhere in the inaccessible past comments on lobste.rs :( I discussed them with some Rust user here on lobste.rs and they seemed to agree my cases were some rough spots and they didn’t have an idea how to help me resolve them. Although I do also sometimes think, that maybe if I learn and internalize enough Rust some day, I migh find a way to structure my API somehow completely differently to render the whole issue void?
I hear you! I hate to give the unsatisfying answer, but your description of your experience so far sounds an awful lot like mine before I got “over the hump”.
I would posit also that your extensive experience with C and C++ may even be hindering you, in one particular way, and that is this:
I’ve been using C since 1981 and C++ since 1990, and I and understand stacks and move semantics and where data lives.
trying to understand the subtle distinctions between what I thought was going on and what is actually going on just made my brain hurt
I suspect the process you’re going through is one of relinquishing control over your understanding of where data lives, its owners, and its lifetimes, and giving it to the compiler. As someone who has been writing C and C++ for a long time, and has therefore been forced to reason about these things entirely in your own head to the point where it’s second nature now, I can imagine the process of unlearning that is uncomfortable and painful.
I wasn’t an experienced user of C or C++ when I came to Rust, though, so I’m interested to hear what you think about my perspective on this. Am I totally off base here?
relinquishing control over your understanding of where data lives, its owners, and its lifetimes
Probably part of the problem is I’m wrapping a library that already owns and manages data, with some tricky lifetimes (interior pointers and such.) Part of the appeal of Rust is to make this API safer — in C++ you have to keep track of some of the dependencies by hand and be aware that releasing X will invalidate Y. I know I can teach the borrow checker about this, it’s what it was made to do, but it’s a more difficult task than what most people probably start out with in Rust.
In Rust, more so than other languages, I strongly think it’s best to learn the language the standard way, by reading the official Rust book or one of the major alternative introductory books, prior to doing any sort of major project with it. In my experience, people coming from C or C++ struggle the most with Rust because they expect it to be similar, and expect their knowledge to transfer. Rust is more different than they expect, and they get frustrated.
In your case, you chose a project which particularly hits against one of the harder challenges in Rust when you haven’t learned the language well enough yet: writing safe FFI bindings for another library. Doing so in an idiomatic way in Rust requires you to have a sense of what ownership structures are easy to represent in Rust, and which ones are hard, and having that architectural sense is difficult when you’re new to the language.
I’m not blaming you, by the way, this approach to learning a new language is a common one. In my opinion, it works poorly for learning Rust. Rust has a learning curve, and there aren’t shortcuts over it. Pick an easier problem to learn the language better first, or go through the book (without working on your current project) and return when you’ve finished it. That’s my two cents.
Yeah, it definitely sounds like you’ve jumped in at the deep end, in the place where Rust’s unique features benefit you least :)
I suppose you’re fighting not only your own brain attempting to override the suggestions the compiler is making, but also the mismatch between Rust and C++’s understanding of that same ownership information.
You can use ref-counted objects in Rust too, and in fact I recommend people having trouble with borrow checker to use it. Reference counting is well supported in Rust, although currently it looks ugly. I believe making reference counting looks more beautiful and natural is the important next step for Rust, since it is often the right choice.
Usually, at some point it made click and I had a deeper understanding of what I did before and I made it work. But not always. Sometimes, I really hated the way out. So I can relate to the frustration.
What newbies get wrong about understanding lifetimes” was a great resource to me but I found it confusingly written at times. I used it more as an invitation for what to think about. Then figured things out on my own. So don’t think that you are the only one.
Using Arc, Rc, clone is totally fine in most cases. Don’t try too hard to avoid them.
I don’t like how complicated rust is, but it really does deliver things other languages don’t:
Memory safety without a GC.
Data race safety.
Deterministic resource freeing.
Top tier performance.
Rust has a near monopoly on this combination so can afford to fail a bit in other areas. Not even meaning to say rust is bad at anything, but that it could probably be worse and still offer something other tools cannot.
I often hear people say things like “Rust is complicated”. Could you elaborate on what you’re referring to when you say that?
I tend to find Rust is less complicated than a language like C or a language like Python, because in both of those (admittedly very different) languages, I feel there are a lot of things under the surface that I need to hold in my head in order to write and reason about code effectively. Maybe I’ve just accepted that any language is going to come with a certain level of complexity, but personally prefer it to be clearly visible up front, rather than hidden within the design of the language in a way that I’ll only discover after I’ve used it for a while.
Often when reading about core issues, like the Pin api and different discussions about soundness of the compiler, I find it difficult to follow, and I feel like only a small number of people really understand what is going on and are qualified to contribute to important decisions.
I feel that tools like cargo actually make some things more difficult, especially for people trying to write package managers.
I personally have written a crappy, but self hosting C compiler - https://github.com/andrewchambers/c . I don’t feel like writing a rust compiler is in my capability at all. This puts it in another tier of complexity in my mind.
That being said I don’t think rust is that hard to use as an end user.
I think I understand what you’re saying better now, thanks - you’re referring more to complexity of implementation rather than complexity of the end product? I agree with that to some degree; I think some of that complexity is inherent to the task of offloading work from the programmer’s brain into the compiler.
I feel that tools like cargo actually make some a lot of things more difficult, especially for people trying to write package managers.
I’m not sure this is a perspective I’ve heard before! What sorts of things does it make more difficult?
It expects network access at build time unless you make special provisions.
It throws away many conventions used by long standing tools such as .a files, .so files and pkg-config.
It does not make it easy to share libraries between different programs that are packaged by a system package manager.
Rust:
Has a culture of bypassing system package managers via rust up, making deployments using alternative package managers harder.
The rust compiler itself takes a huge amount of ram, cpu and disk space time to build from source, making it cumbersome to deal with for build from source package managers.
Had incorrect links on the rust website for building from source, this wasn’t fixed for something like a year or more despite a ticket being open, making it even harder to build from source.
Cannot be built unless you already have rust, introducing a circular dependency.
Lots of little things and considerations add up to form the nebulous ‘complicated’ label.
When I hear this, I always imagine a person who writes code, but never has to maintain, test or debug it. :-)
For me, debugging is one of the hardest things I have to do – using a language that minimizes the times I have to debug is a net win for me.
“I could write this in X, but not in Rust” makes me look at all the stuff that other languages silently accept and turn into UB – all of which is likely to be rejected in Rust at compile time.
When I hear this, I always imagine a person who writes code, but never has to maintain, test or debug it. :-)
Just perhaps you might stop and consider that people can disagree with your views for reasons other than being stupid.
Personally I don’t find Rust that hard to write the first time. But changing it is a nightmare because it forces you to rewrite huge swathes of code because you’ve slightly changed the way you store some data. Yes perhaps in theory there’s some P: AsRef<T: Blah Blah Blah> crap you can do to abstract it and not have to change it later but that just reinforces the point: it’s a complicated language where doing things right the first time is hard and takes a lot of experience. It’s not as simple as just having a parameter that is type T and passing any T to it like it is in 99% of other programming languages.
It forces you to rewrite huge swathes of code because you’ve slightly changed the way you store some data
Huh. How does Rust compare to C here? In Rust you can at least insert indirection easily because there is no difference between . and -> as in C. Unifying . and -> was done specifically so that changing the way you store data does not cause huge rewrite.
Replacing . with -> is completely trivial compared to rewriting the type signatures of half your programme and then realising you have to actually make even more invasive changes than that, possibly rewriting the whole thing.
The design must be simple, both in implementation and interface. It is more important for the implementation to be simple than the interface. Simplicity is the most important consideration in a design.
While Unix and C follows this principle, Rust does not. In that sense, Rust is complicated.
No Rust is complicated in every sense of the word complicated. Rust code is complicated, Rust’s implementation is complicated, Rust’s syntax is complicated, Rust’s semantics are complicated (and undocumented). Rust is just a complicated beast of complication.
It’s not a complicated implementation of a simple interface. It’s an implementation of a very complicated interface.
I have high hope for Borrowing Safe Pointers from Rust in SPARK to make it non-monopoly, but besides technical achievements, SPARK probably won’t ever be as popular as Rust.
I don’t believe that rust solves the right problems in the right ways. This is specifically with respect to the single-owner raii/lifetime system; the rest of the language is imo pretty nice (aside from the error messages, which are an implementation problem).
For starters, ATS and f* both provide much stronger safety guarantees, so if you want the strongest possible guarantees that your low-level code is correct, you can’t stop at rust.
Beyond that, it’s helpful to look at the bigger picture of what characteristics a program needs to have, and what characteristics a language can have to help facilitate that. I propose that there are broadly three program characteristics that are affected by a language’s ownership/lifetime system: throughput, resource use, and ease of use/correctness. That is: how long does the code take to run, how much memory does it use, and how likely is it to do the right thing / how much work does it take to massage your code to be accepted by the compiler. This last is admittedly rather nebulous. It depends quite a lot on an individual’s experience with a given language, as well as overall experience and attention to detail. Even leaving aside specific language experience, different individuals may rank different languages differently, simply due to different approaches and thinking styles. So I hope you will forgive my speaking a little bit generally and loosely about the topic of ease-of-use/correctness.
The primary resource that programs need to manage is memory[1]. We have several strategies for managing memory:
(Note: implicit/explicit below refers to whether something something is an explicit part of the type system, not an explicit part of user code.)
implicitly managed global heap, as with malloc/free in c
implicit stack-based raii with automatically freed memory, as in c++, or c with alloca (note: though this is not usually a general-purpose solution, it can be. But more interestingly, it can be composed with other strategies.)
explicitly managed single-owner abstraction over the global heap and possible the stack, as in rust
explicit automatic reference counting as an abstraction over the global heap and possibly the stack, as in swift
implicit memory pools/regions
explicit automatic tracing garbage collector as an abstraction over the global heap, possibly the stack, possibly memory regions (as in a nursery gc), possible a compactor (as in a compacting gc). (Java)
custom allocators, which may have arbitrarily complicated designs, be arbitrarily composed, arbitrarily explicit, etc. Not possible to enumerate them all here.
I mentioned before there are three attributes relevant to a memory management scheme. But there is a separate axis along which we have to consider each one: worst case vs average case. A tracing GC will usually have higher throughput than an automatic reference counter, but the automatic reference counter will usually have very consistent performance. On the other hand, an automatic reference counter is usually implemented on top of something like malloc. Garbage collectors generally need a bigger heap than malloc, but malloc has a pathological fragmentation problem which a compacting garbage collector is able to avoid.
This comment is getting very long already, and comparing all of the above systems would be out of scope. But I’ll make a few specific observations and field further arguments as they come:
Because of the fragmentation problem mentioned above, memory pools and special-purpose allocators will always outperform a malloc-based system both in resource usage and throughput (memory management is constant-time + better cache coherency)
Additionally, implicitly managed memory pools are usually easier to use than an implicitly managed global heap, because you don’t have to think about the lifetime of each individual object.
Implicit malloc/free in c should generally perform similarly to an explicit single-owner system like rust’s, because most of the allocation time is spent in malloc, and they have little (or no) runtime performance hit on top of that. The implicit system may have a slight edge because it has more flexible data structures; then again, the explicit single-owner system may have a slight edge because it has more opportunity to allocate locally defined objects directly on the stack if their ownership is not given away. But these are marginal gains either way.
Naïve reference counting will involve a significant performance hit compared to any of the above systems. However, there is a heavy caveat. Consider what happens if you take your single-owner verified code, remove all the lifetime annotations, and give it to a reference-counting compiler. Assuming it has access to all your source code (which is a reasonable assumption; the single-owner compiler has that), then if it performs even basic optimizations—this isn’t a sufficiently smart compiler-type case—it will elide all the reference counting overhead. Granted, most reference-counted code isn’t written like this, but it means that reference counting isn’t a performance dead end, and it’s not difficult to squeeze your rc code to remove some of the rc overhead if you have to.
It’s possible to have shared mutable references, but forbid sharing them across threads.
The flexibility gains from having shared mutable references are not trivial, and can significantly improve ease of use.
Correctness improvements from strictly defined lifetimes are a myth. Lifetimes aren’t an inherent part of any algorithm, they’re an artifact of the fact that computers have limited memory and need to reuse it.
To summarize:
When maximum performance is needed, pools or special-purpose allocators will always beat single-owner systems.
For all other cases, the performance cap on reference counting is identical with single-owner systems, while the flexibility cap is much higher.
File handles and mutex locks also come up, but those require different strategies. Happy to talk about those too, but tl;dr file handles should be avoided where possible and refcounted where not; mutexes should also be avoided where possible, and be scoped where not.
The problem is this: people are writing code in a language generally already know how to program and already know the language. (And if you don’t know how to program, you should likely start with something like python or lisp.) So really, an error message shouldn’t give you an essay about why your code is wrong. You knowwhy your code is wrong. What you need is to know what about your code is wrong. So the most useful error message draws your eye directly to the site of the error and gives you only minimal information beyond that, because what’s interesting to look at is your own code, not the compiler’s error message.
I’m reminded of this comment about gcc 10’s static analyzer. There was a demonstration of some setjmp code with convoluted control flow that had a memory leak, and the analyzer had a two-page-long output with ascii-art and tracebacks, showing how it proved there was a memory leak. Then a commenter compared that output with the output from valgrind; the latter was a simple 4-line traceback, and it was easier to see the problem with the code from that.
I guess I probably think they are too verbose too, for me personally. But this is spoken by someone who has used the language daily, continuously, for about six years now. I can’t really remember the last time I really had to scrutinize a rustc error carefully. I can just quickly pattern match and know the error immediately.
These days though, my editor (vim) just tells me the compiler errors via rust-analyzer, every time I save the file. It’s nearly instant. I often don’t even need to read the status bar with the first line of the error message. I’ll just know the problem by virtue of where it exists in the code and fix it. This means I rarely even look at errors (or warnings) coming from rustc itself in a terminal. If I were, I probably would have written a quick wrapper script or something that condenses the error messages by now.
I’ll happily abide the verbose error messages, because I do really think it helps folks who are in the process of learning the language and coming face-to-face with anything like the borrow checker for the first time.
Strongly disagree. As a veteran programmer who’s new to Rust, I found the descriptive messages very useful, because they clarify what’s happening at a higher level: not just what’s wrong, but how it went wrong across multiple lines of code. This is a pretty savvy feature since it helps people get over the learning curve.
I’m sure it’d be nice to compress these for experts. Maybe they can add a compiler flag to do that.
So really, an error message shouldn’t give you an essay about why your code is wrong. You knowwhy your code is wrong. What you need is to know what about your code is wrong.
I think the point of an error message generally is that you don’t, in fact, know why your code is wrong, or you likely wouldn’t have written the code that produced the error in the first place. Particularly in the case of Rust, there are whole classes of errors (read: lifetime/ownership errors) that even people who, as you say “already know how to program” may not ever have encountered before. Even someone like me, who has been using Rust for around 4 years now, still encounters lifetime errors that I absolutely need the verbose explanation in order to understand.
That’s not to mention that sometimes (again, read: lifetime/ownership errors), the question of “what about your code is wrong” is inextricable from “why your code is wrong”.
People are writing code in a language generally already know how to program and already know the language. (And if you don’t know how to program, you should likely start with something like python or lisp.)
This is a bold assumption to make. Should error messages all be tailored to people who already know how to program? What about the people who are using the language as their first (or one of their first) language? (I don’t necessarily agree that everyone should start with Python or Lisp; I think, for some beginners, Rust is a great choice. A lot of people would have a lot fewer issues understanding ownership if they didn’t need to first unlearn the model that other languages teach.)
I’ve spent a lot of time in the past helping classmates and mentees of mine learn languages like C, and even some of the most basic error messages are so concise (perhaps “terse” would be a better word) that they’re utterly baffling to someone who’s new to programming. The problem I see here is that for a beginner, that situation is a complete roadblock, whereas for someone who is already experienced at programming, error messages that are too verbose are an annoyance at worst.
I don’t see a problem necessarily with the idea of adding a “concise” mode to the Rust compiler so it outputs more straight-to-the-point error messages. But I think the way they are right now is a sensible default, and I know a lot of people who work tirelessly to make sure the error messages are as explanatory and high-quality as possible, including soliciting feedback from the community. I don’t think I’ve ever before heard someone complain that they’re too explanatory.
I’m reminded of this comment about gcc 10’s static analyzer.
I agree with you about this error message. I don’t know if I agree that the problem is that it’s too verbose, though; I think it’s just a bad error message.
Maybe I’m just not over the hump yet, but my experience with Rust hasn’t been super happy. Last month I built both a Rust and a Nim binding to a C API [that I wrote and own.] I’m a newbie a both languages. The Nim binding was a joy to write, simple and clean. The Rust one involved tons of fighting with the borrow checker, and a couple of areas I just had to leave as ugly hacks because Doing it right created bizarre errors I couldn’t understand — something about interactions between lifetimes and generics.
I don’t think I’m having the typical newbie problems understanding lifetimes. I’ve been using C since 1981 and C++ since 1990, and I and understand stacks and move semantics and where data lives. I appreciate the work Rust has done to make delicate lifetime dependencies checkable by the compiler! But it seems to have made the language extremely complex. I found a blog post about “What newbies get wrong about understanding lifetimes” last week, and it made me basically give up — trying to understand the subtle distinctions between what I thought was going on and what is actually going on just made my brain hurt, in the same way it hurts when I try to understand template metaprogramming or hyperbolic geometry.
Honestly I’m not sure the complexity of the lifetime analysis in Rust is worth it, compared to the minor performance overhead of using some ref-counted objects (as in Nim, or my current C++ code.) But I’m willing to be convinced otherwise.
Your post surprised me. I’ve been using C++ for more than a decade and the transition to Rust was largely painless. After reading half the Rust Book and 2 weeks of experimenting, I felt fairly comfortable with the language. Because I was already using C++11 features like
unique_ptrand move semantics, the transition to Rust didn’t require me to change anything at all about my programming style¹: The lifetime management in C++ is largely the same, just without strict compiler checks. Because of that, I assumed that coming from modern C++ is a great shortcut to picking up Rust. I did have some trouble understanding the lifetime syntax like'aand so on, but the language has some excellent documentation that helped me understand it and the Rust channel on Discord is full of super nice and helpful people.At this point, I’m already about 3-5x more productive with Rust than I am with C++², even though I started looking into Rust less than a year ago. I can churn out software of a size and level that was impossible to churn out before. That is incredibly empowering. Rust made me feel like when I was a kid learning my first programming language. I think it’s a real, life-changing innovation. I haven’t felt this excited about programming since I was a kid.
¹ This programming style is such that your C++ code does not contain a single
new,delete,mallocorfree. Instead, you usemake_uniqueandmake_sharedand clean up by letting theunique_ptrandshared_ptrgo out of scope. It is pure RAII, exception-safe and prevents leaks and double deletes by default.² For various reasons:
cargo runworked on Linux first try without a single change. This is unheard of in the C++ world for any nontrivial program.rustcare almost always very detailed and accurate. When I was a beginner, I would literally just copy-paste what the compiler proposed, and it would make my code compile and work properly.Resultenum (sum type). Additionally, everyone usesResult, so there is a standardized way to return errors across the entire ecosystem, something that C++ lacks.Yup, everything you’re saying is what I want to get out of Rust — the goal is to port some complex, finicky, concurrent networking code that uses this API I’m wrapping, because it’s too complicated, messy and hard to maintain in C++. And I’ve really enjoyed parts of Rust like Cargo and the wonderfully informative compiler error messages, which feel like a TA helping teach me. (Until the point where I get the same error over and over no matter how I tweak the code…)
I think I need to find a forum where I can post some snippets that don’t work and ask for help. I’m not really sure where to go, though, as Rust is so big there isn’t a single obvious place (like the Nim forum, for Nim.) Do I just go directly to SO?
(And yeah, my C++ style leans heavily on unique_pointer, moves, and on homemade RefCounted and Ref<> classes. I’ve been kind of writing Rust-like code for years, especially after I read about Rust in 2014(?) and began envying it.)
Two good and official palce to ask for help are the users forum and the the community Discord.
Me, I once fell into an even more frustrating case, of a “bistable error message/hint”: the compiler showed me an error message with a helpful suggestion how to change my code. I changed it as suggested, just to be greeted with a new error message with a helpful suggestion to change it… back to the exactly same shape of code I had first :(
Another thing that made me put it down for the time being, until I regenerate enough psychical stamina to try and attack the hump once again, was a feeling that Rust requires me to sacrifice the ideal of being able to write simple APIs (i.e. hiding complexity from their users/callers). In other words, I found there seems sometimes to be just one way I can structure an API to make it safe and correct, and it makes my API super complex, and requires the caller to know some nontrivial Rust incantations to actually be able to use it.
Really sorry I can’t provide you with a concrete example of a code in question now. I have it buried somewhere in the inaccessible past comments on lobste.rs :( I discussed them with some Rust user here on lobste.rs and they seemed to agree my cases were some rough spots and they didn’t have an idea how to help me resolve them. Although I do also sometimes think, that maybe if I learn and internalize enough Rust some day, I migh find a way to structure my API somehow completely differently to render the whole issue void?
I hear you! I hate to give the unsatisfying answer, but your description of your experience so far sounds an awful lot like mine before I got “over the hump”.
I would posit also that your extensive experience with C and C++ may even be hindering you, in one particular way, and that is this:
I suspect the process you’re going through is one of relinquishing control over your understanding of where data lives, its owners, and its lifetimes, and giving it to the compiler. As someone who has been writing C and C++ for a long time, and has therefore been forced to reason about these things entirely in your own head to the point where it’s second nature now, I can imagine the process of unlearning that is uncomfortable and painful.
I wasn’t an experienced user of C or C++ when I came to Rust, though, so I’m interested to hear what you think about my perspective on this. Am I totally off base here?
Probably part of the problem is I’m wrapping a library that already owns and manages data, with some tricky lifetimes (interior pointers and such.) Part of the appeal of Rust is to make this API safer — in C++ you have to keep track of some of the dependencies by hand and be aware that releasing X will invalidate Y. I know I can teach the borrow checker about this, it’s what it was made to do, but it’s a more difficult task than what most people probably start out with in Rust.
In Rust, more so than other languages, I strongly think it’s best to learn the language the standard way, by reading the official Rust book or one of the major alternative introductory books, prior to doing any sort of major project with it. In my experience, people coming from C or C++ struggle the most with Rust because they expect it to be similar, and expect their knowledge to transfer. Rust is more different than they expect, and they get frustrated.
In your case, you chose a project which particularly hits against one of the harder challenges in Rust when you haven’t learned the language well enough yet: writing safe FFI bindings for another library. Doing so in an idiomatic way in Rust requires you to have a sense of what ownership structures are easy to represent in Rust, and which ones are hard, and having that architectural sense is difficult when you’re new to the language.
I’m not blaming you, by the way, this approach to learning a new language is a common one. In my opinion, it works poorly for learning Rust. Rust has a learning curve, and there aren’t shortcuts over it. Pick an easier problem to learn the language better first, or go through the book (without working on your current project) and return when you’ve finished it. That’s my two cents.
Yeah, it definitely sounds like you’ve jumped in at the deep end, in the place where Rust’s unique features benefit you least :)
I suppose you’re fighting not only your own brain attempting to override the suggestions the compiler is making, but also the mismatch between Rust and C++’s understanding of that same ownership information.
You can use ref-counted objects in Rust too, and in fact I recommend people having trouble with borrow checker to use it. Reference counting is well supported in Rust, although currently it looks ugly. I believe making reference counting looks more beautiful and natural is the important next step for Rust, since it is often the right choice.
More ergonomic RC for Rust sounds interesting. Is there any work being done towards that goal?
I love Rust but I had my struggles with it.
Usually, at some point it made click and I had a deeper understanding of what I did before and I made it work. But not always. Sometimes, I really hated the way out. So I can relate to the frustration.
What newbies get wrong about understanding lifetimes” was a great resource to me but I found it confusingly written at times. I used it more as an invitation for what to think about. Then figured things out on my own. So don’t think that you are the only one.
Using Arc, Rc, clone is totally fine in most cases. Don’t try too hard to avoid them.
I don’t like how complicated rust is, but it really does deliver things other languages don’t:
Rust has a near monopoly on this combination so can afford to fail a bit in other areas. Not even meaning to say rust is bad at anything, but that it could probably be worse and still offer something other tools cannot.
I often hear people say things like “Rust is complicated”. Could you elaborate on what you’re referring to when you say that?
I tend to find Rust is less complicated than a language like C or a language like Python, because in both of those (admittedly very different) languages, I feel there are a lot of things under the surface that I need to hold in my head in order to write and reason about code effectively. Maybe I’ve just accepted that any language is going to come with a certain level of complexity, but personally prefer it to be clearly visible up front, rather than hidden within the design of the language in a way that I’ll only discover after I’ve used it for a while.
Often when reading about core issues, like the Pin api and different discussions about soundness of the compiler, I find it difficult to follow, and I feel like only a small number of people really understand what is going on and are qualified to contribute to important decisions.
I feel that tools like cargo actually make some things more difficult, especially for people trying to write package managers.
I personally have written a crappy, but self hosting C compiler - https://github.com/andrewchambers/c . I don’t feel like writing a rust compiler is in my capability at all. This puts it in another tier of complexity in my mind.
That being said I don’t think rust is that hard to use as an end user.
I think I understand what you’re saying better now, thanks - you’re referring more to complexity of implementation rather than complexity of the end product? I agree with that to some degree; I think some of that complexity is inherent to the task of offloading work from the programmer’s brain into the compiler.
I’m not sure this is a perspective I’ve heard before! What sorts of things does it make more difficult?
Cargo:
Rust:
Lots of little things and considerations add up to form the nebulous ‘complicated’ label.
When I hear this, I always imagine a person who writes code, but never has to maintain, test or debug it. :-)
For me, debugging is one of the hardest things I have to do – using a language that minimizes the times I have to debug is a net win for me.
“I could write this in X, but not in Rust” makes me look at all the stuff that other languages silently accept and turn into UB – all of which is likely to be rejected in Rust at compile time.
Just perhaps you might stop and consider that people can disagree with your views for reasons other than being stupid.
Personally I don’t find Rust that hard to write the first time. But changing it is a nightmare because it forces you to rewrite huge swathes of code because you’ve slightly changed the way you store some data. Yes perhaps in theory there’s some
P: AsRef<T: Blah Blah Blah>crap you can do to abstract it and not have to change it later but that just reinforces the point: it’s a complicated language where doing things right the first time is hard and takes a lot of experience. It’s not as simple as just having a parameter that is typeTand passing anyTto it like it is in 99% of other programming languages.Huh. How does Rust compare to C here? In Rust you can at least insert indirection easily because there is no difference between
.and->as in C. Unifying.and->was done specifically so that changing the way you store data does not cause huge rewrite.Replacing
.with->is completely trivial compared to rewriting the type signatures of half your programme and then realising you have to actually make even more invasive changes than that, possibly rewriting the whole thing.“Rust is complicated” usually means the implementation is complicated (see ac’s comments here for examples), and it is.
In the immortal words of Worse is Better:
While Unix and C follows this principle, Rust does not. In that sense, Rust is complicated.
No Rust is complicated in every sense of the word complicated. Rust code is complicated, Rust’s implementation is complicated, Rust’s syntax is complicated, Rust’s semantics are complicated (and undocumented). Rust is just a complicated beast of complication.
It’s not a complicated implementation of a simple interface. It’s an implementation of a very complicated interface.
@sanxiyn, you should wear your hat in this thread ;)
It’s great that you have such a vivid imagination.
I have high hope for Borrowing Safe Pointers from Rust in SPARK to make it non-monopoly, but besides technical achievements, SPARK probably won’t ever be as popular as Rust.
I don’t believe that rust solves the right problems in the right ways. This is specifically with respect to the single-owner raii/lifetime system; the rest of the language is imo pretty nice (aside from the error messages, which are an implementation problem).
For starters, ATS and f* both provide much stronger safety guarantees, so if you want the strongest possible guarantees that your low-level code is correct, you can’t stop at rust.
Beyond that, it’s helpful to look at the bigger picture of what characteristics a program needs to have, and what characteristics a language can have to help facilitate that. I propose that there are broadly three program characteristics that are affected by a language’s ownership/lifetime system: throughput, resource use, and ease of use/correctness. That is: how long does the code take to run, how much memory does it use, and how likely is it to do the right thing / how much work does it take to massage your code to be accepted by the compiler. This last is admittedly rather nebulous. It depends quite a lot on an individual’s experience with a given language, as well as overall experience and attention to detail. Even leaving aside specific language experience, different individuals may rank different languages differently, simply due to different approaches and thinking styles. So I hope you will forgive my speaking a little bit generally and loosely about the topic of ease-of-use/correctness.
The primary resource that programs need to manage is memory[1]. We have several strategies for managing memory:
(Note: implicit/explicit below refers to whether something something is an explicit part of the type system, not an explicit part of user code.)
implicitly managed global heap, as with malloc/free in c
implicit stack-based raii with automatically freed memory, as in c++, or c with alloca (note: though this is not usually a general-purpose solution, it can be. But more interestingly, it can be composed with other strategies.)
explicitly managed single-owner abstraction over the global heap and possible the stack, as in rust
explicit automatic reference counting as an abstraction over the global heap and possibly the stack, as in swift
implicit memory pools/regions
explicit automatic tracing garbage collector as an abstraction over the global heap, possibly the stack, possibly memory regions (as in a nursery gc), possible a compactor (as in a compacting gc). (Java)
custom allocators, which may have arbitrarily complicated designs, be arbitrarily composed, arbitrarily explicit, etc. Not possible to enumerate them all here.
I mentioned before there are three attributes relevant to a memory management scheme. But there is a separate axis along which we have to consider each one: worst case vs average case. A tracing GC will usually have higher throughput than an automatic reference counter, but the automatic reference counter will usually have very consistent performance. On the other hand, an automatic reference counter is usually implemented on top of something like malloc. Garbage collectors generally need a bigger heap than malloc, but malloc has a pathological fragmentation problem which a compacting garbage collector is able to avoid.
This comment is getting very long already, and comparing all of the above systems would be out of scope. But I’ll make a few specific observations and field further arguments as they come:
Because of the fragmentation problem mentioned above, memory pools and special-purpose allocators will always outperform a malloc-based system both in resource usage and throughput (memory management is constant-time + better cache coherency)
Additionally, implicitly managed memory pools are usually easier to use than an implicitly managed global heap, because you don’t have to think about the lifetime of each individual object.
Implicit malloc/free in c should generally perform similarly to an explicit single-owner system like rust’s, because most of the allocation time is spent in malloc, and they have little (or no) runtime performance hit on top of that. The implicit system may have a slight edge because it has more flexible data structures; then again, the explicit single-owner system may have a slight edge because it has more opportunity to allocate locally defined objects directly on the stack if their ownership is not given away. But these are marginal gains either way.
Naïve reference counting will involve a significant performance hit compared to any of the above systems. However, there is a heavy caveat. Consider what happens if you take your single-owner verified code, remove all the lifetime annotations, and give it to a reference-counting compiler. Assuming it has access to all your source code (which is a reasonable assumption; the single-owner compiler has that), then if it performs even basic optimizations—this isn’t a sufficiently smart compiler-type case—it will elide all the reference counting overhead. Granted, most reference-counted code isn’t written like this, but it means that reference counting isn’t a performance dead end, and it’s not difficult to squeeze your rc code to remove some of the rc overhead if you have to.
It’s possible to have shared mutable references, but forbid sharing them across threads.
The flexibility gains from having shared mutable references are not trivial, and can significantly improve ease of use.
Correctness improvements from strictly defined lifetimes are a myth. Lifetimes aren’t an inherent part of any algorithm, they’re an artifact of the fact that computers have limited memory and need to reuse it.
To summarize:
I’d just like to ask you about one specific thing you said:
What complaints do you have about the error messages? I’ve only heard good things about Rust’s error messages, and haven’t had any bad experiences.
They’re too verbose.
The problem is this: people are writing code in a language generally already know how to program and already know the language. (And if you don’t know how to program, you should likely start with something like python or lisp.) So really, an error message shouldn’t give you an essay about why your code is wrong. You know why your code is wrong. What you need is to know what about your code is wrong. So the most useful error message draws your eye directly to the site of the error and gives you only minimal information beyond that, because what’s interesting to look at is your own code, not the compiler’s error message.
I’m reminded of this comment about gcc 10’s static analyzer. There was a demonstration of some
setjmpcode with convoluted control flow that had a memory leak, and the analyzer had a two-page-long output with ascii-art and tracebacks, showing how it proved there was a memory leak. Then a commenter compared that output with the output from valgrind; the latter was a simple 4-line traceback, and it was easier to see the problem with the code from that.I guess I probably think they are too verbose too, for me personally. But this is spoken by someone who has used the language daily, continuously, for about six years now. I can’t really remember the last time I really had to scrutinize a rustc error carefully. I can just quickly pattern match and know the error immediately.
These days though, my editor (vim) just tells me the compiler errors via rust-analyzer, every time I save the file. It’s nearly instant. I often don’t even need to read the status bar with the first line of the error message. I’ll just know the problem by virtue of where it exists in the code and fix it. This means I rarely even look at errors (or warnings) coming from rustc itself in a terminal. If I were, I probably would have written a quick wrapper script or something that condenses the error messages by now.
I’ll happily abide the verbose error messages, because I do really think it helps folks who are in the process of learning the language and coming face-to-face with anything like the borrow checker for the first time.
Strongly disagree. As a veteran programmer who’s new to Rust, I found the descriptive messages very useful, because they clarify what’s happening at a higher level: not just what’s wrong, but how it went wrong across multiple lines of code. This is a pretty savvy feature since it helps people get over the learning curve.
I’m sure it’d be nice to compress these for experts. Maybe they can add a compiler flag to do that.
I think the point of an error message generally is that you don’t, in fact, know why your code is wrong, or you likely wouldn’t have written the code that produced the error in the first place. Particularly in the case of Rust, there are whole classes of errors (read: lifetime/ownership errors) that even people who, as you say “already know how to program” may not ever have encountered before. Even someone like me, who has been using Rust for around 4 years now, still encounters lifetime errors that I absolutely need the verbose explanation in order to understand.
That’s not to mention that sometimes (again, read: lifetime/ownership errors), the question of “what about your code is wrong” is inextricable from “why your code is wrong”.
This is a bold assumption to make. Should error messages all be tailored to people who already know how to program? What about the people who are using the language as their first (or one of their first) language? (I don’t necessarily agree that everyone should start with Python or Lisp; I think, for some beginners, Rust is a great choice. A lot of people would have a lot fewer issues understanding ownership if they didn’t need to first unlearn the model that other languages teach.)
I’ve spent a lot of time in the past helping classmates and mentees of mine learn languages like C, and even some of the most basic error messages are so concise (perhaps “terse” would be a better word) that they’re utterly baffling to someone who’s new to programming. The problem I see here is that for a beginner, that situation is a complete roadblock, whereas for someone who is already experienced at programming, error messages that are too verbose are an annoyance at worst.
I don’t see a problem necessarily with the idea of adding a “concise” mode to the Rust compiler so it outputs more straight-to-the-point error messages. But I think the way they are right now is a sensible default, and I know a lot of people who work tirelessly to make sure the error messages are as explanatory and high-quality as possible, including soliciting feedback from the community. I don’t think I’ve ever before heard someone complain that they’re too explanatory.
I agree with you about this error message. I don’t know if I agree that the problem is that it’s too verbose, though; I think it’s just a bad error message.