1. 40
  1.  

  2. 34

    Its OK to write unsafe Rust code. Honest! Compile time safety is the main innovation of Rust and its killer feature, but its ok for parts of an API to be unsafe. Rust enforces safety by guarenteeing a certain pattern of memory use, and if that is a bad fit for your problem then thats just life. The halting problem guarentees that no one-size-fits-all solution will ever exist.

    I’m currently writing unsafe Rust for graphics library code. I’m reasonably sure it can’t corrupt memory because I’ve structured it so it can’t. Rust could enforce some of the invariants I’ve imposed at compile time, but it would make the entire process viciously painful.

    Even unsafe rust has real enums, strong types, macros, pattern matching, a module system, and all the other things C (and C++) doesn’t. Using C instead is like finding out you new dirt bike gets sand in your face and going back to riding offroad on a donkey instead of wearing a helmet.

    1. 9

      The effort to write unsafe rust seems to me far more than just writing C. So if you don’t get the safety benefit, it is just a pain.

      1. 7

        But the point is that only the unsafe parts will be unsafe. Besides that, writing unsafe Rust is subjectively more fun than writing C.

        1. 3

          I think it It depends on what you are interfacing with. Interfacing with a C framework can be very hard because there will be so many assumptions and things to work around and the unsafety will be pervasive, for small C libraries it is not bad. I’ve tried both some of both types of rust wrapping, and that is just my feeling so far. .

          1. 2

            Although I have no experience in Rust, but I can still acknowledge this: I have interfaced with suite some C code from C#, and if that C code was not designed in a mindset that it will be wrapped it was really painful. Now I can imagine that strict ownership of memory references can only add a lot of more hardship and complexity.

          2. 2

            The takeaway of the pre-poop your pants bug was that unsound, unsafe code does create bugs in safe code.

            1. 2

              Yeah, this section of the rustonomicon https://doc.rust-lang.org/nomicon/working-with-unsafe.html talks about exactly that too.

          3. 3

            There are a lot more safety benefits to rust then preventing pointer dereferencing (which is primarily what unsafe allows). Too many to list from a smartphone. And Rust provides far more than just safety.

            1. 1

              I think that the difficulty lies in wrapping it with safe code which is true to all safety guarantees.

              Or do you genuinely mean writing unsafe rust all the way is more difficult than writing C? I’d love to hear why.

              1. 3

                genuinely mean writing unsafe rust all the way is more difficult than writing C?

                I found very low level rust code dealing with C types has excessive casting + far more verbose code. Interfacing with C is not idiomatic rust, needing to use types like CString and CStr and doing lots conversions, it is just more annoying for some things.

                1. 3

                  I think it is in part a fault of C library, not Rust. Null-terminated C string is not null safe and in general a bad interface to provide for libraries.

                  1. 1

                    I didn’t say it was rust’s fault, just that it can be lots of extra work to do interop.

          4. 10

            I can always just use unsafe bindings to wlroots, just like I would with any other language. However, the entire point of Rust is that it’s safe. Doing that is not an option because at that point you lose the entire benefit of the language.

            The author has much more experience with Rust than I do, but I’m still skeptical of this claim. Only the API boundary would be unsafe, not any of the other logic, right?

            But more importantly, does Rust really provide no other benefits as a language? If C++ were to suddenly achieve the same safety benefits as Rust, would C++ be the better language? I doubt it, even though I prefer C++.

            Again I don’t know much Rust, so if anyone has some insight I’d love to hear it. Do unsafe interfaces “infect” calling code in the same way callbacks / async functions do?

            1. 12

              There are plenty of other benefits, like a strong, ergonomic type system, an actual package manager, etc. See the comment from @icefox.

              1. 6

                Again I don’t know much Rust, so if anyone has some insight I’d love to hear it. Do unsafe interfaces “infect” calling code in the same way callbacks / async functions do?

                Nope. unsafe means “the onus is on the programmer to make this safe”. The standard library is mostly unsafe code; part of the decision making for what to include in std is “does this need unsafe code to implement well?”

              2. 8

                #[dehandle] looks suspiciously like a stackless coroutine, which is Coming Soon (tm) to Rust.

                Not saying that you should wait on it. There’s intentionally no schedule or deadline for unstable Rust features, and there will always be something Coming Soon (tm) that would make everything better, so if you waited, you’d be waiting forever. Just figured it would be interesting to anyone who’s watching the Rust project from the sidelines.

                1. 2

                  Cool, thanks for the link!

                  1. 1

                    Oh nice, didn’t knew there’s such things as the unstable book or rather: That it contains this nightly stuff and gives you an idea of how it works.

                  2. 10

                    I have had the view that safety is typically not the most important goal of a project (user satisfaction is). In my experience, safety can get in the way of writing software that is useful (the real world is really messy). It’s great to see a real-life case with Rust from someone who obviously tried very hard but ultimately had safety get in the way of writing the software they wanted. Hopefully, Rust will evolve to interface with the “unsafe” world in a more ergonomic way.

                    1. 9

                      I think many people obsess over Rust’s idea of safety and lose track of the point of safety. The point of safety is to create software that does not leak (performance or memory) and does not crash.

                      It is entirely possible to write software that does not leak or crash, in C. It requires good practice and good tooling to enforce that good practice, as does programming in literally any other programming language (You can write spaghetti code in any language, after all).

                      1. 6

                        That’s absolutely true, but how much faith do you have that Way Cooler will be one of the C projects that uses good practice and good tooling to minimize the amount of memory-safety bugs?

                        1. 1

                          If it’s written by somebody that is very familiar with Rust (and clearly it will be) then actually I have a lot of confidence in Rust. Writing a lot of Rust is like writing a lot of any language: it infects your mind and way of doing things, and I suspect it will have infected the author’s mind with thinking about memory management constantly, which will hopefully mean they think about it constantly during their C programming too.

                          1. 1

                            This has actually happened to me, the more I write Rust the more I can predict what the compiler will complain about. And I think about that a lot when writing C too. It’s very… interesting? when I’m writing C and I think about ownership.

                        2. 4

                          I actually think a more encompassing tool that also deals with safety, but not just safety, all other forms of bugs is Design by Contract. I’m excited to see DbC will be part of the C++ language standard for 2020. Rust had a discussion about DbC as part of the language but many feel this library fills most of the needs. While a library approach is better than nothing, having DbC as part of the language (after all, types ARE a contract) makes more sense to me. Hopefully, the Rust folks keep the discussion going and make DbC part of the language. Maybe more discussion happened but I haven’t seen it.

                          1. 3

                            Out of curiosity, how is Design by Contract different/better than just sprinkling assertions at the start/end of your functions? Most of the examples I’ve seen for it are like those in the readme of the library you linked, which are fairly trivial. How do you express more complicated contracts like, say, “The sublist returned by this list search routine is a subset of the list passed in and shares the same memory”? Do you have suggestions for more in depth things I can read?

                            1. 3

                              Most languages don’t have first-class support for Design by Contract so most implementations are glorified assertions. To express more complicated contracts you would use higher level functions in your assertion. For example, you can define a “is_subset” function and use that in your assertions.

                              I do many C++ projects and use DbC for all of them, and before C++2020 I used a small library I wrote Which I copy and past in all projects. When compilers will support native contracts in C++, I’ll use those instead of the library.

                              1. 2

                                My Dafny is a little rusty, but assuming you mean modifying the list to get the sublist, it would be something like

                                modifies array
                                ensures exists lo, hi :: out == old(array[lo..hi])
                                
                            2. 3

                              How about memory vulnerabilities leading to RCEs? Do you suggest apps with those are safe? Why “has leaks” is unsafe, but “has remote code execution vuln” is safe? I totally don’t get it. Can you explain, other than just claiming “because I say so”?

                              1. 1

                                Can you explain your position better?

                                Are you saying that GCC is guaranteed to generate code that has remote code execution vulnerabilities? And Rust doesn’t have this?

                                It’s my understanding that memory leaks are what cause RCEs (At least, the main cause of them). It is also my understanding that there is nothing inherent about C’s memory model that causes presumably safe (As tested by tooling and humans) code to magically appear with RCEs.

                                1. 4

                                  RCEs are not caused by memory leaks, but most commonly by accidental occurences of:

                                  • use after free (this is something different than memory leaks — more like opposite of memory leaks: mem leak is when you forget to free(), use after free is when you do free() and yet dereference the pointer afterwards)
                                  • out-of-bounds array reads/writes (most notoriously as a result of off-by-one errors, but not only)
                                  • Undefined Behavior (good intro: [1] [2] [3])

                                  Especially the much-too-many, casually unknown UB “fine print” cases are “something inherent about C that causes presumably safe (as tested by tooling and humans) code to magically appear with RCEs”. The most infamous example that basically proves that human/tooling testing is not enough, is the recent RCE in SQLite, a project that is often perceived as the most covered by tests & tooling & “human review” open-source C codebase in the world, with any contenders being far behind. Other than that, the PVS-Studio’s website is a great reference showcasing the proliferation of (often RCE-grade) errors in widely used open-source C codebases. (The common understanding being, that closed-source codebases are usually even worse.) Their blog is also worth a read.

                                  Rust is specifically designed with an aim towards completely eliminating whole classes of such errors. Not all errors, mind you. But especially the RCE-grade ones resulting from the reasons I listed above. IOW, yes, I’m saying that GCC/Clang is guaranteed* to generate code that has remote code execution vulnerabilities, and Rust doesn’t have this**.

                                  * — Unless: (a) maybe if the codebase were written by 1 person, who is flawless; but there is no such person; (b) also, there’s some chance military-grade hardened C code might have no RCEs (this requires super expensive and tedious dev process); (c) code written in Frama-C or similar “verified C” dialects, potentially.
                                  ** — You can still write RCEs in Rust, but it’s levels of magnitude harder to do accidentally, and close to impossible if you’re not using unsafe blocks (barring bugs in the compiler/standard library).

                                  Edit: Also, please note that I’m fully aware Rust is super annoying to work with. I myself tried, and gave up for now. I’m not claiming it’s a panaceum for any and all problems. But C, and esp. C++, those are paths I followed with devotion, deep enough that I touched the lurking madness, and came back shaken and changed forever.

                                  1. 2

                                    The supposed RCE in sqllite is dubious. Sqlite is a database that can accept executable code as an input. If your application allows unrestricted access to that database and someone then provides an input to the DB with executable code in it and the DB executes it, then the error is in the application, not the DB.

                                    1. 3

                                      If you let a user write queries, it’s expected behavior that they can modify the database.

                                      It’s not expected behavior that they can read/write unrelated files or perform network activity.

                                      1. 2

                                        that is exactly the intended functionality of the sqlite full text search

                                        1. 1

                                          Did you somehow reply to the wrong comment?

                                          How is full text search intended to let me overwrite /etc/passwd?

                                          The text search index is stored in the same database file as everything else. I expect users with access to use it to read/write that file.

                                      2. 2

                                        Sorry, but it feels like “blame the victim” mentality to me. Did SQLite at least come with warnings of “do not let users write SQL queries, as we don’t audit such scenarios for security”?

                                        1. 2

                                          It’s not at all “blame the victim”. It is: “understand what the tool does”. SQLite doesn’t even have user accounts. It’s an SQL engine and SQL is a powerful language.

                                      3. 0

                                        use after free (this is something different than memory leaks — more like opposite of memory leaks: mem leak is when you forget to free(), use after free is when you do free() and yet dereference the pointer afterwards)

                                        This is a simple error to avoid, though. Alter free() to check for NULL and don’t free() if the input is NULL (Something that is nonsensical and a check that should be done anyway), then after every single free, set the pointer to NULL. It becomes very, very difficult to use-after-free using this method. In the same way, set file descriptors to negative values before assignment and check for negative values before close(), etc.

                                        out-of-bounds array reads/writes (most notoriously as a result of off-by-one errors, but not only)

                                        Every single array should store and check against the maximum number of elements, and in some cases the current number of elements in the array.

                                        Undefined Behavior (good intro: [1] [2] [3])

                                        This is probably the only case that causes errors. However, there are tools to check for UB (One of them, PVS-Studio, you linked to. Another is available here: http://css.csail.mit.edu/stack/). It is possible to remove the UB from C, but there is a lot of resistance to it for the reason that it is useful. I personally don’t agree entirely with them, and I do not see the need to rehash those arguments here.

                                        Formal verification is something I find deeply interesting, but at some point you need to balance ‘shipping’ against ‘total safety’. I also don’t see distrusting {GCC / Clang}’s code generation as a ‘useful’ stance, given that most ‘safe’ languages eventually, somewhere down the chain, rely on the code generation from {GCC / Clang}. A case in point, Rust’s first version was written in OCaml, which descends from OCaml Light, which was written in C.

                                        If you truly believe that “GCC/Clang is guaranteed to generate code that has remote code execution vulnerabilities” (Emphasis mine), then you must extend that distrust to the OCaml Light code generator, which was generated by {Clang/GCC}, which means you have to distrust OCaml (A language cannot be safe without safe code generation), and therefore you can hardly trust Rust to be safe!

                                        1. 2

                                          Ok, from what you wrote here, I am surprised to see that we seem to be kinda coming to an agreement. Meaning, theoretically, I certainly agree, that it’s possible to have safe code emitted from GCC, iff the input C source code is perfectly flawless (as I explained in the footnote). I stand by my claim however, that in real life, i.e. practically, this cannot really be achieved (with the exceptions I stated previously). Moreover, I suppose even PVS-Studio is not perfect; I don’t know it very well, I admit, so maybe I’m in error, but I’d be really (and positively!) surprised if they claimed to eliminate all UB-based errors.

                                          Interestingly, as to generated C code (i.e. output of compilers such as OCaml Light, Nim, etc.), I believe it can be actually much easier to keep safe! The trick here is that the generated C code will probably be a relatively small/finite subset of predesigned C patterns/snippets. The author of the compiler should thus be able to enforce using only safe C constructs; interactions between the limited number of patterns can be scrutinized much better; and finally, extra protections such as bounds checks can be added fully automatically to every snippet that needs them, leaving no space for an occasional human error in perusing them. (Including the common hubris such as not setting a freed pointer to NULL “because obviously it won’t be used anywhere further, so why waste the coveted CPU cycles”.) OTOH, if a compiler author does introduce some UB, I imagine there would be a higher chance that it will get repeated automatically by the compiler, thus hopefully making it “louder”/more frequent and therefore easier to notice and fix.

                                          Edit: Also, please note, that with all the checks you suggest (in free, in arrays), you’re actually already not talking about basic C, but some special dialect of C-with-protections! Notably, some of them (e.g. bounds checks) are often laughed away by C programmers, as “costly, reducing effectiveness/speed”. And with this notion of adding reasonable protections, you can slowly get to trying to unknowingly reimplement Rust! Or Ada, more probably. Which is indeed seen as much sager language than C. (Though also more annoying, I believe even than Rust.)

                                          1. 1

                                            you’re actually already not talking about basic C, but some special dialect of C-with-protections!

                                            Wow, I didn’t realise that coding properly was called coding in a different language! Thanks! I didn’t realise that ‘Python with style-guides and tooling’ was a different language to Python! That’s amazing! /s

                                            The trick here is that the generated C code will probably be a relatively small/finite subset of predesigned C patterns/snippets. The author of the compiler should thus be able to enforce using only safe C constructs; interactions between the limited number of patterns can be scrutinized much better; and finally, extra protections such as bounds checks can be added fully automatically to every snippet that needs them, leaving no space for an occasional human error in perusing them.

                                            So what you’re essentially saying is that, you can code safely in C, albeit extremely carefully. Yes, we do seem to be agreeing.

                                          2. 1

                                            This is a simple error to avoid, though. Alter free() to check for NULL and don’t free() if the input is NULL

                                            That only works if they’re not only freeing the same spot in memory, but also freeing it through the same pointer. That’s not where most of the use-after-frees come from. The use-after-frees all come from aliased pointers, like in this code.

                                            void consume_object(object *ptr) {
                                                do_something_with(ptr);
                                                free(ptr);
                                                ptr = NULL;
                                            }
                                            void main_whatever() {
                                                object *ptr = malloc(sizeof(object));
                                                init_object(ptr);
                                                consume_object(ptr);
                                                /* ptr is not NULL, in spite of what `consume_pointer` did, because reasons */
                                                free(ptr);
                                                ptr = NULL;
                                            }
                                            
                                            1. 1

                                              because reasons

                                              But if you’re writing code like that, you don’t understand pointers or argument passing. If I write code in Haskell and expect it to be strict-evaluation, then there will be huge problems with it. That’s not a fault of the language. That’s a fault of me.

                                              Besides, as I have previously said, these problems are caught with appropriate tooling. Both cppcheck and scan-build point out this error.

                                            2. 1

                                              Alter free() to check for NULL and don’t free() if the input is NULL (Something that is nonsensical and a check that should be done anyway),

                                              Two problems:

                                              1. Developers regularly don’t do it enough to catch everything.

                                              2. There might be a performance penalty.

                                              Rust’s method increases odds they’ll have to deal with it while eliminating the need for runtime checks. If runtime checks are fine, there’s way to do those, too, while getting other benefits for memory safety.

                                      4. 2

                                        “It is entirely possible to write software that does not leak or crash, in C.”

                                        It will be a lot harder in most cases. Especially given memory safety applies on all inputs. Getting that in C usually means hand-inserting checks everywhere and/or running it through a sound static analyzer that might cost five digits.

                                        “ It requires good practice and good tooling to enforce that good practice, as does programming in literally any other programming language (You can write spaghetti code in any language, after all).”

                                        This is a false equivalence. If the code is safe Rust, your mistakes will not usually lead to code injection. Programmers will make mistakes, esp if overly casual or hurried. Field evidence indicates most will do this regularly with careful experts doing it some of the time. In C, those mistakes will increase number of successful hacks or leaks.

                                        So, you could say using something that converts most hacks into panics is using “good tooling” to get good results. Aside from borrow checker, you get most of that safety without even trying. That boosts productivity on that kind of code. If borrow checker is too much, one can downgrade to reference counting or unsafe with the other risks still mitigated automatically.

                                        1. 1

                                          Getting that in C usually means hand-inserting checks everywhere and/or running it through a sound static analyzer that might cost five digits.

                                          From personal experience, a lot of the tooling that helps you avoid these mistakes is free. But, don’t let that stop your fearmongering, please.

                                          If the code is safe Rust, your mistakes will not usually lead to code injection.

                                          [citation needed].

                                          No, really. As far as I know, there are no long-term studies on Rust projects. I don’t think you can argue with ‘field evidence’ until you can show that ‘field evidence’ proves that Rust improves safety. After all, as you say, programmers are lazy. For all you know, they could just be doing the equivalent of dumping unsafe { ... } everywhere the borrow checker complains and calling it a day.

                                          1. 1

                                            a lot of the tooling that helps you avoid these mistakes is free.

                                            Are the tools for total memory safety in C as easy as a Rust compile offering same guarantees? Especially no temporal errors or races on all inputs?

                                            It’s a trick question: I’ve submitted more tooling like that here than anybody. I read their performance and effectiveness evaluation sections. It usually takes more CPU/RAM with less certain benefits than just using Rust. Then, we have the constant vulnerabilities supporting my position. Amateurs are doing better in Rust so far since it’s just immune to a lot of those problems. The compiler yells at them and/or apps just panic in bad situation.

                                            “I don’t think you can argue with ‘field evidence’ until you can show that ‘field evidence’ proves that Rust improves safety. “

                                            The design of the language makes it immune to many classes of errors. Outside a compiler error, about any code in safe Rust will inherit those properties. If you’re talking compiler errors, that would be weird since C compilers, esp optimizing, have been buggy as hell without C programmers showing up dismissing potential benefits of C until their compilers are proven correct. Virtually no use of CompCert in GPL’d software either. Be a double standard there.

                                            “For all you know, they could just be doing the equivalent of dumping unsafe { … } everywhere the borrow checker complains and calling it a day.”

                                            Btw, the borrow checker is just one of many safety mechanisms in Rust. It covers temporal errors that show up even in OpenBSD. I mean, if they can’t avoid them…

                                            What you say might be true, is worth looking for in field data, and might give a different picture of average app in safety. That said, even if they were doing that, the code would still be safer than C given it has no protections against those same risks vs Rust apps likely combining pre-existing, borrow-checked libraries with new code, some of which is unsafe. Attack surface gets lowered just because the lazy path in Rust is pre-existing, safe libraries. The harder path, creating borrow-checked code, is strongly encouraged by its ecosystem to get code into stdlib’s, etc. Better default than C again.

                                    2. 3

                                      This is a reasonable move. It’s long known that wayland is built very much with only C clients in mind, which makes FFI to it hard. I’m not sure if I agree with the described ownership problem (Ownership of “weak” references with a forced check is a standard procedure in Rust, see for example rc::Weak), but there’s definitely diminishing returns. When checking the FFI bindings for soundness becomes a lions share of your work, you might well cut your losses and move to C. The described API surface is just staggering.

                                      Note the last paragraph: this only applies to the compositor, not the window manager itself. I’m also not sure why way-cooler is writing its own compositor instead of using what’s out there, but for that, I don’t know enough.

                                      Especially for mixed code bases, these kinds of engineering experiences are important, as there is a tradeoff happening and people need to be informed of it to make good judgement.

                                      1. 0

                                        It’s long known that wayland is built very much with only C clients in mind, which makes FFI to it hard

                                        This is false and borderline FUD. The Wayland protocol is very simple and implementations in many programming languages exist - including pure Rust. libwayland, the C implementation, is harder to write bindings for in Rust, but not so in many other languages. wlroots, because its design is based on libwayland’s design, faces similar challenges for Rust bindings, but also has many other successful language bindings. The flaw is with Rust, not with Wayland.

                                        1. 1

                                          I mean, I assume you can write bindings to Rust in the same way as other languages, I assume this naive approach:

                                          This naive style implementation could work, however it would cause a lot of branching code. Each call to wlroots will require a check to see if the handle has been dropped, even though it almost certainly has not been dropped (it can only be dropped between event callbacks, wlroots/Wayland is callback based). This means there will be unnecessary paths (that are hopefully marked cold) that either panic or return an error.

                                          Would work just fine and must be how other languages are bound to (or maybe they’re unsound?)

                                          I guess the core complaint is that there is not an easy way to provide a lifetime guarantee that can be reasoned about in a type system that currently exists, so your choices are to figure it out yourself (risk memory unsafety) or be slightly inefficient. I would question how much inefficiency there really is, but I also don’t think that the “callback hell” in timidger’s post is all that bad (the magic macros are worse imo).

                                          This all doesn’t sound like a Rust flaw, I don’t know if it’s a flaw in anything, I guess I don’t super get what all the drama is about.