1. 5

    that reminds me of the banned.h from MS during the SDLC buzz times, the old links are not working anymore, but here is copy: https://github.com/x509cert/banned/blob/master/banned.h

    1.  

      I’ve never seen preprocessor directives tabbed like that before, is that standard?

      1.  

        While most code is not like this, it’s not uncommon.

        1.  

          Yes. The standard requires that the first character of the line be a # to be recognised as a preprocessor directive (so it’s trivial for the preprocessor to differentiate between directives and source code) and then follows the normal C tokenisation rules, where # is a token, include is a token, pragma is a token and so on. C allows an arbitrary amount of whitespace between tokens (e.g. ++ i and ++i are equivalent).

          This is the only way of indenting preprocessor directives that is actually allowed by the standard. clang-format will happily do this for you.

      1.  

        I am quite interested on how eOS breaks FOSS licenses by paywalling the access to their builds which also raised the controversy in the past IIRC.

        1. 9

          How is that breaking FOSS licenses?

          1. 22

            It doesn’t. GPL doesn’t forbid re-selling software, as long as the source code is available to those users.

            1. 9

              It is baffling how many people do not understand this. People also do not understand that the GPL does not demand that you offer the software to everyone for download. It only demands that that you distribute the code to the parties receiving the software. That means I can create GPL software for a customer and as long as I given them a tarball of the source, all is fine.

              1.  

                I can download elementary OS for free. Do I get the source code?

                1. 6

                  You should be offered the code, it’s not required that the code be part of the download. Read the GPL. It’s not that hard to decipher.

        1. 1

          Here’s why Curry-Howard is important to programmers even with concrete types. When we express the properties we want at the type level, we have a mathematical guarantee that things can’t go wrong. That’s the whole point for us–that things can’t go wrong. For example, consider this code (OCaml syntax):

          type person
          type name
          
          val name : string -> name option
          val person : id:int -> name -> person
          

          This code guarantees that a person value can be constructed with any int value as an ID, but only with a valid name value, which can be constructed only by following the rules encoded in the name function. The types enforce that these logical rules must be followed. If you have a name value, you have a proof that it was constructed correctly. And if you have a person value, you also have a proof that it was constructed correctly.

          Simple, everyday code that gives you mathematical proofs that it will work correctly. That’s why C-H is important.

          1. 5

            Why do you need C-H for this? You can constrain input to sets just fine and make assertions over the domain and range of pure functions well enough without invoking C-H. TLA+ after all models algorithms using set theoretic constructions. So what is C-H giving you here?

            1. 1

              It’s giving me a simple model for thinking about my code, using a system (static types) that is mainstream and widely available. If I set up my types correctly, then the programs I construct using them can be provably correct. This is cool because this tool is accessible to working developers today.

            2. 1

              Sure, and I use a ton of these in OCaml! But this is clearly on the “program” side of things. These are not proofs of anything a mathematician would call a theorem. They’re just statements about the program, which helps understanding it and allow the compiler to remove a ton of runtime checks; but nothing more. Trying to coerce OCaml into proving non trivial stuff with its types gives you michelson whose design is heavily informed by what you can and can’t do with GADTs. If you want better proofs for OCaml, use Why3 or the upcoming gospel from the same team.

              1. 3

                The fact that you can have a type system at all is a consequence of C-H. Of course that standard programming languages’ type systems make lousy theorem provers, so what? Similarly, the theorems that arise from “regular” programs are not mathematically interesting, but again, so what? How does this make C-H a scam?

                In the end C-H is more important to mathematicians than programmers, as it allows for things like Coq to exist. I find the idea of C-H not being important very bizarre. Before C-H, the idea that mathematics could be formalized at all was a hypothesis. C-H shows not only that it is true, but also shows you how to do it. It’s one of the most profound mathematical discoveries of the 20th century.

            1. 8

              I think there are valid arguments on both sides here, but this post doesn’t seem to be grounded in experience.

              Practically speaking, users of weird architectures do contribute patches back. Those people eventually become the maintainers. When those people go away, the project drops support for certain architectures. That happened with CPython, e.g. it doesn’t support Mac OS 9 anymore as far as I remember.

              It’s sort of a self-fulfilling prophesy – if the code is in C, you will get people who try to compile it for unusual platforms. If it’s in Rust, they won’t be able to try.

              I’m not saying which one is better, just that this post misses the point. If you want to use Rust and close off certain options, that’s fine. Those options might not be important to the project. Someone else can start a different project with the goal of portability to more architectures.

              Changing languages in the middle of the project is a slightly different case. But that’s why the right to fork exists.

              1. 25

                Author here: this post is grounded in a couple of years of experience as a packager, and a couple more years doing compiler engineering (mostly C and C++).

                Practically speaking, users of weird architectures do contribute patches back. Those people eventually become the maintainers. When those people go away, the project drops support for certain architectures. That happened with CPython, e.g. it doesn’t support Mac OS 9 anymore as far as I remember.

                This is the “hobbyist” group mentioned in the post. They do a fantastic job getting complex projects working for their purposes, and their work is critically undervalued. But the assumptions that stem from that work are also dangerous and unfounded: that C has any sort of “compiled is correct” contract, and that you can move larger, critical work to novel architectures just by patching bugs as they pop up.

                1. 6

                  OK I think I see your point now. TBH the post was a little hard to read.

                  Yes, the people contributing back patches often have a “it works on my machine” attitude. And if it starts “working for others”, the expectation of support can arise.

                  And those low quality patches could have security problems and tarnish the reputation of the project.

                  So I would say that there are some projects where having the “weird architectures” off to the side is a good thing, and some where it could be a bad thing. That is valid but I didn’t really get it from the post.


                  I also take issue with the “no such thing as cross platform C”. I would say it’s very hard to write cross platform C, but it definitely exists. sqlite and Lua are pretty good examples from what I can see.

                  After hacking on CPython, I was surprised at how much it diverged from that. There are a lot of #ifdefs in CPython making it largely unportable C.

                  In the ideal world you would have portable C in most files and unportable C in other files. Patches for random architectures should be limited to the latter.

                  In other words, separate computation from I/O. The computation is very portable; I/O tends to be very unportable. Again, sqlite and Lua are good examples – they are parameterized by I/O (and even memory allocators). They don’t hard-code dependencies, so they’re more portable. They use dependency inversion.

                  1. 10

                    TBH the post was a little hard to read.

                    That’s very fair; I’m not particularly happy with how it came out :-)

                    I also take issue with the “no such thing as cross platform C”. I would say it’s very hard to write cross platform C, but it definitely exists. sqlite and Lua are pretty good examples from what I can see.

                    I’ve heard this argument before, and I think it’s true in one important sense: C has a whole bunch of mechanisms for making it easy to get your code compiling on different platforms. OTOH, to your observation about computation being generally portable: I think this is less true than C programmers generally take for granted. A lot of C is implicitly dependent on memory models that happen to be shared by the overwhelming majority of today’s commercial CPUs; a lot of primitive operations in C are under-specified in the interest of embedded domains.

                    Maybe it’s possible to truly cross-platform C, but it’s my current suspicion that there’s no to verify that for any given program (even shining examples of portability like sqlite). But I admit that that’s moving the goalposts a bit :-)

                    1. 11

                      Maybe it’s possible to truly cross-platform C, but it’s my current suspicion that there’s no to verify that for any given program (even shining examples of portability like sqlite).

                      I think the argument holds up just fine despite the existence of counterexamples like Sqlite and Lua; basically it means that every attempt to write portable and safe code in C can be interpreted as an assertion that the author (and every future contributor) is as capable and experienced as Dr. Hipp!

                      1. 6

                        A lot of C is implicitly dependent on memory models that happen to be shared by the overwhelming majority of today’s commercial CPUs

                        That’s largely a result of the CPU vendors optimising for C, due to its popularity. Which leads to its popularity. Which…

                        1. 2

                          A lot of C is implicitly dependent on memory models that happen to be shared by the overwhelming majority of today’s commercial CPUs; a lot of primitive operations in C are under-specified in the interest of embedded domains.

                          As the author of a C library, I can confirm that fully portable C is possible (I target the intersection of C99 and C++). It wasn’t always easy, but I managed to root out all undefined and unspecified behaviour. All that is left is one instance of implementation defined behaviour: right shift of negative integers. Which I have decided is not a problem, because I don’t know a single platform in current use that doesn’t propagate the sign bit in this case.

                          The flip side is that I don’t do any I/O, which prevents me from directly accessing the system’s RNG.

                          Incidentally, I’m a firm believer in the separation of computation and I/O. In practice, I/O makes a relatively small portion of programs. Clearly separating it from the rest turns the majority of the program into “pure computation”, which (i) can be portable, and (ii) is much easier to test than I/O.

                        2. 5

                          I also take issue with the “no such thing as cross platform C”. I would say it’s very hard to write cross platform C, but it definitely exists. sqlite and Lua are pretty good examples from what I can see.

                          I see this as parallel to “no such thing as memory-safe C”. Sure, cross-platform C exists in theory, but it’s vanishingly rare in practice, and I’d wager even the examples you cite are likely to have niche platform incompatibilities that haven’t been discovered yet.

                          1. 1

                            I’d wager even the examples you cite are likely to have niche platform incompatibilities that haven’t been discovered yet.

                            Portability in C is hard, but it is simple: no undefined behaviour, no unspecified behaviour, no implementation defined behaviour. If you do that, and there are still are platform incompatibilities, then the platform’s compiler is at fault: it has a bug, fails to implement part of the standard, or simply conforms to the wrong standard (say, C89 where the code was C99).

                            If we’re confident a given project is free of undefined, unspecified, and implementation defined behaviour, then we can be confident we’ll never discover further niche platform incompatibilities. (Still, achieving such confidence is much harder than it has any right to be.)

                            1. 3

                              Portability in C is hard, but it is simple: no undefined behaviour, no unspecified behaviour, no implementation defined behaviour.

                              That is a very tall order, though. Probably impossibly tall for many (most?) people. I asked how to do this and the answers I would say were mixed at best. Simple isn’t good enough if it’s so hard nobody can actually do it.

                      2. 3

                        If it’s in Rust, they won’t be able to try.

                        I think this is the most trenchant point here. If someone wants to maintain a project for their own “weird” architecture, then they need to maintain the toolchain and the project. I’ve been in that position and it sucks. In fact, it’s worse, because they need to maintain the toolchain before they even get to the project.

                        I’m particularly sensitive to this because I’m typing this on ppc64le. We’re lucky that IBM did a lot of the work for us, but corporate interests shift. There’s no Rust compiler for half the systems in this room.

                        1. 2

                          I’m not familiar with these systems. What are they used for? What kind of devices use them? What industries/sectors/etc.?

                          1. 3

                            Ppc is very common in aerospace and automotive industries. Of course there are also power servers running Linux and Aix, but those are comparatively a niche compared to the embedded market.

                            1. 6

                              Got it. Sounds like definitely something that would not be hobbyists working on side projects using mass-market hardware. I think the article was referring to this–these corporate users should be willing to pay up to get their platforms supported.

                              1. 3

                                So does that mean we should only care about architectures that have corporate backing? Like I say, this isn’t a situation where it’s only a project port that needs maintainers. The OP puts it well that without a toolchain, they can’t even start on it. If Rust is going to replace C, then it should fill the same niche, not the same niche for systems “we like.”

                                For the record, my projects are all officially side projects; my day job has nothing officially to do with computing.

                                1. 8

                                  So does that mean we should only care about architectures that have corporate backing?

                                  Yes, it does. Money talks. Open source is not sustainable without money. I can work on a pet project on the side on evenings and weekends only for a relatively short period of time. After that it’s going to go unmaintained until the next person comes along to pick it up. This is going to happen until someone gets a day job working on the project.

                                  If Rust is going to replace C, then it should fill the same niche, not the same niche for systems “we like.”

                                  C has a four-decade head start on Rust, if no one is allowed to use Rust until it’s caught up to those four decades of porting and standardization effort–for the sake of people’s side projects–then that argument is a non-starter.

                                  1. 3

                                    Yes, it does. Money talks.

                                    In such a world there would be no room for hobbyists, unless they work with what other people are using. Breakage of their interests would be meaningless and unimportant. That’s a non-starter too.

                                    But, as you say, you’re unfamiliar with these systems, so as far as you’re concerned they shouldn’t matter, right?

                                    1. 9

                                      In that (this) world, there is room for hobbyists only insofar as they support their own hobbies and don’t demand open source maintainers to keep providing free support for them.

                                2. 2

                                  OpenWrt runs on TP-Link TL-WDR4900 WiFi Router. This is a PowerPC system. OpenWrt is nearly a definition of hobbyists working on side projects using mass-market hardware.

                                  1. 2

                                    It says on that page that this device was discontinued in 2015. Incidentally, same year Rust reached 1.0.

                                    1. 2

                                      I am not sure what you are trying to argue. The same page shows it to be in OpenWrt 19.07, which is the very latest release of OpenWrt.

                        1. 4

                          Impressive. Now I want to see Lua as an x86 boot loader. :-)

                          1. 2

                            FWIW, the FreeBSD/illumos boot loader uses Lua.

                            1. 12

                              I know, I worked on it :-)

                          1. 73

                            First, their argument for Rust (and against C) because of memory safety implies that they have not done due diligence in finding and fixing such bugs. […] And with my bc, I did my due diligence with memory safety. I fuzzed my bc and eliminated all of the bugs.

                            This seems like such a short-sighted, limited view. Software is not bug-free. And ruling out a class of bugs by choice of technology as a measure of improving overall robustness won’t fix everything, but at the very least it’s a trade-off that deserves more thorough analysis than this empty dismissal.

                            1. 19

                              I think he is probably right (after all, he wrote it) when he says rewriting his bc in Rust would make it more buggy. I disagree this is an empty dismissal, since it is backed by his personal experience.

                              For the same reason, I think cryptography developers are probably right (after all, they wrote it) when they say rewriting their software in Rust would make it less buggy. So the author is wrong about this. His argument is not convincing why he knows better than developers.

                              1. 15

                                I think there’s a big difference between programs and libraries with stable requirements and those that evolve here. The bc utility is basically doing the same thing that it did 20 years ago. It has a spec defined by POSIX and a few extensions. There is little need to modify it other than to fix bugs. It occasionally gets new features, but they’re small incremental changes.

                                Any decision to rewrite a project is a trade off between the benefits from fixing the accumulated technical debt and the cost of doing and validating the rewrite. For something stable with little need of future changes, that trade is easy to see: the cost of the rewrite is high, the benefit is low. In terms of rewriting in a memory-safe language, there’s an additional trade between the cost of a memory safety vulnerability and the cost of the rewrite. The cost of Heartbleed in OpenSSL was phenomenal, significantly higher than the cost of rewriting the crypto library. In the cast of bc, the cost of a memory safety bug is pretty negligible.

                                Data from Microsoft’s Security Response Center and Google’s Project Zero agree that around 70-75% of vulnerabilities are caused by memory safety bugs. Choosing a language that avoids those by construction means that you can focus your attention on the remaining 25-30% of security-related bugs. The author talks about fuzzing, address sanitiser, and so on. These are great tools. They’re also completely unnecessary in a memory-safe language because they try to find classes of bugs that you cannot introduce in the first place in a memory-safe language (and they do so probabilistically, never guaranteeing that they’ve found them all).

                                If you’re starting a new project, then you need a really good reason to start it in C and pay the cost of all of that fuzzing.

                                1. 17

                                  Data from Microsoft’s Security Response Center and Google’s Project Zero agree that around 70-75% of vulnerabilities are caused by memory safety bugs. Choosing a language that avoids those by construction means that you can focus your attention on the remaining 25-30% of security-related bugs.

                                  There’s an implied assumption here that if a language is memory safe, those memory safe bugs will simply go away. In my experience, that is not quite true. Sometimes those memory safety bugs will turn into logic bugs.

                                  Not to pick on Rust here, but in Rust it is very common to put values into an array and use array indices instead of pointers when you have some kind of self-referential data structure that’s impossible to express otherwise using rust’s move semantics. If you simply do such a naive transformation of your C algorithm, your code will be memory safe, but all your bugs, use after free, etc, will still be there. You just lifted them to logical bugs.

                                  Rust has no good abstractions to deal with this problem, there are some attempts but they all have various practical problems.

                                  Other languages like ATS and F* have abstractions to help with this problem directly, as well as other problems of logical soundness.

                                  1. 13

                                    Right - but in lifting these from memory bugs to logic bugs, you get a runtime panic/abort instead of a jump to a (likely-attacker-controllable) address. That’s a very different kind of impact!

                                    1. 9

                                      You don’t get a panic if you access the “wrong” array index. The index is still a valid index for the array. Its meaning (allocated slot, free slot, etc), is lost to the type system, though in a more advanced language it need not be. This later leads to data corruption, etc, just like in C.

                                      1. 7

                                        It leads to a much safer variant of data corruption though. Instead of corrupting arbitrary memory in c or c++ (like a function pointer, vtable, or return address), you are only corrupting a single variable’s value in allocated and valid and aligned memory (like a single int).

                                        You would get a panic in rust for every memory corruption bug that could cause arbitrary code execution, which is what matters.

                                        1. 1

                                          This later leads to data corruption, etc, just like in C.

                                          Can you expand on this? I had expected the behaviour in Rust to be significantly safer than C here. In C, the data corruption caused by use-after free often allows an attacker to execute arbitrary code.

                                          I totally see your point about logical corruption (including things like exposing critical secrets), but I don’t follow that all the way to “just like C”. How would an array index error be exploited in Rust to execute arbitrary code?

                                          1. 11

                                            I have once written a bytecode interpreter in C++, for a garbage collected scripting language. I implemented my own two-space garbage collector. For performance reasons, I didn’t use malloc() directly, but instead allocated a big enough byte array to host all my things. If I overflew that array, Valgrind could see it. But if I messed up it’s internal structure, no dice. That heap of mine was full of indices and sizes, and I made many mistakes that caused them to be corrupted, or somehow not quite right. And I had no way to tell.

                                            I solved this by writing my own custom heap analyser, that examined the byte array and tell me what’s in there. If I see all my “allocated” objects in order, all was well. Often, I would see something was amiss, and I could go and fix the bug. Had I written it in Rust instead, I would have had to write the exact same custom heap analyser. Because Rust wouldn’t have prevented me from putting the wrong values inside my array. It’s perfectly “safe” after all, to write gibberish in that array as long as I don’t overflow it.

                                            Now could this particular bug lead to arbitrary code execution? Well, not quite. It would generate wrong results, but it would only execute what my C++/Rust program would normally execute. In this case however, I was implementing a freaking scripting language. The code an attacker could execute wasn’t quite arbitrary, but it came pretty damn close.

                                            1. 6

                                              The effects of data corruption depend on what the code does with the data. This often means arbitrary code execution, but not always. It’s not a property of C, it’s a property of the code. This doesn’t change when you change the implementation language.

                                              Fundamentally there is no semantic difference between a pointer in a C heap and an array index into a Rust array. In fact some sophisticated blog authors that explain this array technique often point out they compile to the exact same assembly code. It’s what the code does with the data that leads to exploitation (or not).

                                              Of course Rust has many additional safety advantages compared to C, buffer overflows don’t smash the stack, etc, and using references in Rust if you can is safe. And when using references, there’s a great deal of correlation between Rust’s notion of memory safety and true logic safety. This is good! But many people don’t realise that this safety is predicated on the lack of aliasing. The borrow checker is only a mechanism to enforce this invariant, it’s not an operative abstraction. It’s the lack of aliasing that gets you the safety, not the borrow checker itself. When you give up aliasing, you lose a lot of what Rust can do for you. Virtually everybody understands that if you introduce unsafe pointers, they give up safety, but less people seem to understand that introducing aliasing via otherwise safe mechanism has the same effect. Of course, the program continues to be memory safe in Rust terms, but you lose the strong correlation between memory safety and logic safety that you used to have.

                                              Not that there’s anything wrong with this, mind you, it’s just something people need to be aware of, just as they are already aware of the tradeoffs that they make when using unsafe. It does make a projection for the number of bugs that Rust can prevent in practice more difficult, though.

                                              1. 6

                                                I think this is incorrect. Arbitrary code execution does not mean “can execute an arbitrary part of my program due to a logic bug”, it means “can execute arbitrary code on the host, beyond the code in my program”. Even a rust alias logic bug dies not open up this kind of arbitrary code execution exposure because you can’t alias an int with a function pointer or a vtable or a return address on the stack, like you can in c or c++. You can only alias an int with an int in safe rust, which is an order of magnitude safer and really does eliminate an entire class of vulnerabilities.

                                                1. 6

                                                  I think this is incorrect. Arbitrary code execution does not mean “can execute an arbitrary part of my program due to a logic bug”, it means “can execute arbitrary code on the host, beyond the code in my program”.

                                                  In the security research world, we usually treat control of the program counter (the aptly named rip on x86-64) as “arbitrary code execution.” You can do a surprising of programming using code that’s already in a process without sending any byte code of your own with return-oriented programming.

                                                  1. 3

                                                    But does Rust let you do that here? What does a snippet of Rust code look like that allows attacker-controlled indexing into an array escalate to controlling the program counter?

                                                    1. 2

                                                      Surely you agree that “variables changing underfoot” implies “programs flow becomes different from what I expect”. That’s why we use variables, to hold the Turing machine state which influences the next state. A logical use after free means “variables changing underfoot”. You don’t expect a free array slot’s value (perhaps now reallocated) to change based on some remote code, but it does.

                                                      1. 3

                                                        Right, but “program flow becomes different from what I expect, but it still must flow only to instruction sequences that the original program encoded” is much much safer than “program flow can be pointed at arbitrary memory, which might not even contain instructions, or might contain user-supplied data”.

                                                        1. 2

                                                          With ROP, the program flow only goes through “instruction sequences that the original program encoded”, and yet ROP is pretty much fatal.

                                                          1. 7

                                                            ROP is not possible when you index an array wrong in rust, what is your point?

                                                            1. 6

                                                              And you can’t do rop in safe rust.

                                                              1. 2

                                                                Maybe not directly within the native code of the program itself, but I think (at least part of) 4ad’s point is that that’s not the only level of abstraction that matters (the memory bug vs. logic bug distinction).

                                                                As an example, consider a CPU emulator written entirely in safe Rust that indexes into a u8 array to perform its emulated memory accesses. If you compile an unsafe program to whatever ISA you’re emulating and execute it on your emulator, a bad input could still lead to arbitrary code execution – it’s at the next semantic level up and not at the level of your program itself, but how much does that ultimately matter? (It’s not really terribly different than ROP – attacker-controlled inputs determining what parts of your program get executed.)

                                                                That’s admittedly a somewhat “extreme” case, but I don’t think the distinction between programs that do fall into that category and those that don’t is terribly clear. Nearly any program can, if you squint a bit, be viewed essentially as a specialized interpreter for the language of its config file (or command-line flags or whatever else).

                                                                1. 2

                                                                  There’s no distinction here. If your program implements a cpu emulator then your program can execute with no arbitrary code execution at all and still emulate arbitrary code execution on the virtual cpu. If you want the virtual program executing to not possibly execute arbitrary virtual instructions, you need to generate the virtual program’s instructions using a safe language too.

                                                                  In most cases, though, arbitrary virtual code execution is less dangerous than arbitrary native code execution, though that’s beside the point.

                                                                  1. 2

                                                                    So…we agree? My point was basically that attacker-controlled arbitrary code execution can happen at multiple semantic levels – in the emulator or in the emulated program (in my example), and writing the emulator in a safe language only protects against the former, while the latter can really be just as bad.

                                                                    Though I realize now my example was poorly chosen, so a hopefully better one: even if both the emulator and the emulated program are written in memory-safe languages, if the emulator has a bug due to an array-index use-after-free that causes it to misbehave and incorrectly change the value of some byte of emulated memory, that destroys the safety guarantees of the emulated program and we’re back in arbitrary-badness-land.

                                                                    1. 1

                                                                      Sure but this is just as meaningful as talking about a cpu hardware bug that might cause a native safe program to run amok. Technically true but not very useful when evaluating the safe programming language

                                                      2. 3

                                                        Right, I agree, and safe rust aliasing that the GP described is not possible to control the program counter arbitrarily.

                                                      3. 4

                                                        Yeah exactly, this is the part I thought @4ad was arguing was possible. Eg. in C, use-after-free often allows me to make the program start interpreting attacker-provided data as machine code. I thought this is what 4ad was saying was also possible in Rust, but I don’t think that’s what they are claiming now.

                                                        To me, that’s a big difference. Restricting the possible actions of a program to only those APIs and activities the original code includes, vs C where any machine code can be injected in this same scenario, is a major reduction in attack surface, to me.

                                                        1. 4

                                                          One thing to note is that code is data and data is code, in a true, hard-mathematical sense.

                                                          The set of

                                                          the possible actions of a program to only those APIs and activities the original code includes,

                                                          and

                                                          C where any machine code can be injected in this same scenario

                                                          is exactly the same (unbounded!). Of course it is much easier in practice to effect desired behavior when you can inject shell code into programs, but that’s hardly required. You don’t need to inject code with ROP either (of course ROP itself is not possible in Rust because of other mitigations, this is just an example).

                                                          Please note that in no way I am suggesting that Rust is doing anything bad here. Rust is raising the bar, which is great. I want the bar raised even higher, and we know for a fact that this is possible today both in theory and practice. Until we raise the bar, I want people to understand why we need to raise the bar.

                                                          At the end of a day you either are type safe or you aren’t. Of course the specifics of what happens when you aren’t type safe depend on the language!

                                                          PS: arrays can contain other things than integers, e.g. they can contain function pointers. Of course you can’t confuse an int with a function pointer, but using the wrong function pointer is pretty catastrophic.

                                                          1. 3

                                                            is exactly the same (unbounded!).

                                                            I guess this is what I don’t understand, sorry for being dense. Can you show a concrete code example?

                                                            In my mind I see a program like this:

                                                            
                                                            enum Action {
                                                                GENERATE_USER_WEEKLY_REPORT,
                                                                GENERATE_USER_DAILY_REPORT,
                                                                LAUNCH_NUCLEAR_MISSILES
                                                            }
                                                            
                                                            impl Action {
                                                              pub fn run(&self) {
                                                                ...
                                                              }
                                                            }
                                                            
                                                            // Remember to remove the nuclear missile action before calling!
                                                            fn exploitable( my_actions:  &Vec<Box<Action>>, user_controlled: usize ) {
                                                              my_actions[user_controlled].run();
                                                            }
                                                            
                                                            

                                                            In my mind, there are two differences between this code in Rust and similar code in C:

                                                            1. This only allows the user to launch nuclear missiles; it does not allow them to, say, write to the harddrive or make network calls (unless one of the actions contained code that did that ofc); in C, I’d likely be able to make something like this call any system function I wanted to, whether machine code to do that was present in the original binary or not.

                                                            2. In Rust, this doesn’t allow arbitrary control flow, I can’t make this jump to any function in the binary, I can only trick it into running the wrong Action; in C, I can call run on any arbitrary object anywhere in the heap.

                                                            ie. in C, this would let me execute anything in the binary, while in Rust it still has to abide by the control flow of the original program, I thought was the case, anyway.

                                                            I think you’re saying this is wrong, can you explain how/why and maybe show a code example if you can spare the time?

                                                            1. 4

                                                              This is correct and 4ad is mistaken. I’m not sure why 4ad believes the two are equivalent; they aren’t.

                                                            2. 3

                                                              “is exactly the same”

                                                              It simply isn’t, and I’m not sure why you think it is.

                                                        2. 1

                                                          In fact some sophisticated blog authors that explain this array technique often point out they compile to the exact same assembly code.

                                                          Do you have any links on this that you recommend?

                                                2. 2

                                                  Good analysis. You didn’t use the words, but this is a great description of the distinction between stocks and flows: https://en.wikipedia.org/wiki/Stock_and_flow. I wish more people talking about software paid attention to it.

                                                  1. 2

                                                    Author here.

                                                    I would also argue that crypto should not change often, like bc. You might add ciphers, or deprecate old ones, but once a cipher is written and tested, there should be very little need for it to change. In my opinion.

                                                  2. 8

                                                    For the same reason, I think cryptography developers are probably right (after all, they wrote it) when they say rewriting their software in Rust would make it less buggy.

                                                    Have they actually rewrote anything? Or have they instead selected a different crypto library they trust better than the previous one? On the one hand, Rust has no advantage over C in this particular context. On the other hand, they may have other reasons to trust the Rust library better than the C one. Maybe it’s better tested, or more widely used, or audited by more reputable companies.

                                                    If I take your word for it however, I have to disagree. Rewriting a cryptographic library in Rust is more likely to introduce new bugs, than it is to fix bugs that haven’t already been found and fixed in the C code. I do think however that the risk is slim, if they take care to also port the entire test suite as well.

                                                    1. 7

                                                      In the Cryptography case isn’t the Rust addition some ASN.1 parsing code? This is cryptography adjacent but very much not the kind of different that your point about cryptography code makes. Parsing code unless it is very trivial and maybe not even then tends to be some of the more dangerous code you can write. In this particular case Rust is likely a large improvement in both ergonomics for the parsing as well as safety.

                                                      1. 1

                                                        You’ve got a point. I can weaken it somewhat, but not entirely eliminate it.

                                                        I don’t consider ASN.1 “modern”. It’s over complicated for no good reason. Certificates can be much, much simpler than that: at each level, you have a public key, ID & expiration date, a certificate of the CA, and a signature from the CA. Just put them all in binary blobs, and the only thing left to parse are the ID & expiration date, which can be left to the application. And if the ID is an URL, and the expiration date is an 64-bit int representing seconds from epoch, there won’t be much parsing to do… Simply put, parsing certificate can be “very trivial”.

                                                        Another angle is that if you need ASN.1 certificates, then you are almost certainly using TLS, so you’re probably in a context where you can afford the reduced portability of a safer language. Do use the safer language in this case.

                                                        Yet another angle is that in practice, we can separate the parsing code from the rest of the cryptographic library. In my opinion, parsing of certificate formats do not belong to a low-level cryptographic library. In general, I believe the whole thing should be organised in tiers:

                                                        • At the lowest level, you have the implementation of the cryptographic primitives.
                                                        • Just above that, you have constructions: authenticated encryption, authenticated key exchange, PAKE…
                                                        • Higher up still, you have file format, network packet formats, and certificates. They can (and should) still be trivial enough that even C can be trusted with them. They can still be implemented with zero dependencies, so C’s portability can still be a win. Though at that level, you probably have an idea of the target platforms, making portability less of a problem.
                                                        • Higher up still is interfacing with the actual system: getting random numbers, talking to the file system, actually sending & receiving network packets… At that level, you definitely know which set of platforms you are targetting, and memory management & concurrency start becoming real issues. At that point you should seriously consider switching to a non-C, safer language.
                                                        • At the highest level (the application), you should have switched away from C in almost all cases.
                                                    2. 2

                                                      For the same reason, I think cryptography developers are probably right (after all, they wrote it) when they say rewriting their software in Rust would make it less buggy. So the author is wrong about this. His argument is not convincing why he knows better than developers.

                                                      This is a fair point. When it comes down to it, whether I am right or wrong about it will only be seen in the consequences of the decision that they made.

                                                    3. 14

                                                      Here’s the more thorough analysis you’re asking for: this is cryptographic code we’re talking about. Many assumptions that would be reasonable for application code simply does not apply here:

                                                      • Cryptographic code is pathologically straight-line, with very few branches.
                                                      • Cryptographic code has pathologically simple allocation patterns. It often avoids heap allocation altogether.
                                                      • Cryptographic code is pathogenically easy to test, because it is generally constant time: we can test all code paths by covering all possible input & output lengths. If it passes the sanitizers & valgrind under those conditions, it is almost certainly correct (with very few exceptions).

                                                      I wrote a crypto library, and the worst bug it ever had wasn’t caused by C, but by a logic error that would have happened even in Haskell. What little undefined behaviour it did have didn’t have any visible effect on the generated code.

                                                      Assuming you have a proper test suite (that tests all input & output lengths), and run that test suite with sanitisers & Valgrind, the kind of bug Rust fixes won’t occur in your cryptographic C code to begin with. There is therefore no practical advantage, in this particular case to using Rust over C. Especially when the target language is Python: you have to write bindings anyway, so you can’t really take advantage of Rust’s better APIs.

                                                      1. 2

                                                        These bugs still occur in critical software frequently. It is more difficult and time consuming to do all of the things you proposed than it is to use a safer language (in my opinion), and the safer language guarantees more than your suggestions would. And there’s also no risk of someone forgetting to run those things.

                                                        1. 6

                                                          These bugs still occur in critical software frequently.

                                                          Yes they do. I was specifically talking about one particular kind of critical software: cryptographic code. It’s a very narrow niche.

                                                          It is more difficult and time consuming to do all of the things you proposed than it is to use a safer language (in my opinion)

                                                          In my 4 years of first hand experience writing cryptographic code, it’s really not. Rust needs the same test suite as C does, and turning on the sanitizers (or Valgrind) on this test suite is a command line away. The real advantage of Rust lies in its safer API (where you can give bounded buffers instead of raw pointers). Also, the rest of the application will almost certainly be much safer if it’s written in Rust instead of C.

                                                          And there’s also no risk of someone forgetting to run those things.

                                                          Someone who might forget those things has no business writing cryptographic code at all yet, be it in C or in Rust. (Note: when I started out, I had no business writing cryptographic code either. It took over 6 months of people findings bugs and me learning to write a better test suite before I could reasonably say my code was “production worthy”.)

                                                          1. 6

                                                            Rusts advantage goes much further than at the api boundary, but again the discussion should be around how to get safer languages more widely used (ergonomics, platform support) and not around “super careful programmers who have perfect test suites and flawless build pipelines don’t need safer languages”. To me it is like saying “super careful contractors with perfect tools don’t need safety gear”, except if you make a mistake in crypto code, you hurt more than just yourself. Why leave that up to human fallability?

                                                            1. 4

                                                              Rusts advantage goes much further than at the api boundary

                                                              Yes it does. In almost all domains. I’m talking about modern cryptographic code.

                                                              again the discussion should be around how to get safer languages more widely used (ergonomics, platform support)

                                                              Write a spec. A formal one if possible. Then implement that spec for more platforms. Convincing projects to Rewrite It In Rust may work as a way to coerce people into supporting more platforms, but it also antagonises users who just get non-working software; such a strategy may not be optimal.

                                                              not around “super careful programmers who have perfect test suites and flawless build pipelines don’t need safer languages”.

                                                              You’re not hearing me. I’m not talking in general, I’m talking about the specific case of cryptographic code (I know, I’m repeating myself.)

                                                              • In this specific case, the amount of care required to write correct C code is the same as the amount of care required to write Rust code.
                                                              • In this specific case, Rust is not safer.
                                                              • In this specific case, you need that perfect test suite. In either language.
                                                              • In this specific case, you can write that perfect test suite. In either language.

                                                              except if you make a mistake in crypto code, you hurt more than just yourself. Why leave that up to human fallability?

                                                              I really don’t. I root out potential mistakes by expanding my test suite as soon as I learn about a new class of bugs. And as it happens, I am painfully aware of the mistakes I made. One of them was even a critical vulnerability. And you know what? Rust wouldn’t have saved me.

                                                              Here are the bugs that Rust would have prevented:

                                                              • An integer overflow that makes elliptic curves unusable on 16-bit platforms. Inconvenient, but (i) it’s not a vulnerability, and (ii) Monocypher’s elliptic curve code is poorly suited to 16-bit platforms (where I recommend C25519 instead).
                                                              • An instance of undefined behaviour the sanitizers didn’t catch, that generated correct code on the compilers I could test. (Note that TweetNaCl itself also have a couple instances of undefined behaviour, which to my knowledge never caused anyone any problem so far. Undefined behaviour is unclean, but it’s not always a death sentence.)
                                                              • A failure to compile code that relied on conditional compilation. I expect Rust has better ways than #ifdef, though I don’t actually know.

                                                              Here are the bugs that Rust would not have prevented:

                                                              • Failure to wipe internal buffers (a “best effort” attempt to erase secrets from the computer’s RAM).
                                                              • A critical vulnerability where fake signatures are accepted as if they were genuine.

                                                              Lesson learned: in this specific case, Rust would have prevented the unimportant bugs, and would have let the important ones slip through the cracks.

                                                              1. 8

                                                                I’m talking about modern cryptographic code.

                                                                In this discussion, I think it is important to remind that cryptography developers are explicitly and intentionally not writing modern cryptographic code. One thing they want to use Rust on is ASN.1 parsing. Modern cryptographic practice is that you shouldn’t use ASN.1 and they are right. Implementing ASN.1 in Rust is also right.

                                                                1. 4

                                                                  I’m talking about modern cryptographic code.

                                                                  So am I.

                                                                  In this specific case, the amount of care required to write correct C code is the same as the amount of care required to write Rust code.

                                                                  I disagree.

                                                                  In this specific case, Rust is not safer.

                                                                  I disagree here too.

                                                                  In this specific case, you need that perfect test suite. In either language.

                                                                  I partially agree. There is no such thing as a perfect test suite. A good crypto implementation should have a comprehensive test suite, of course, no matter the language. But that still isn’t as good as preventing these classes of bugs at compile time.

                                                                  Rust wouldn’t have saved me.

                                                                  Not really the point. Regardless of how lucky or skilled you are that there are no known critical vulnerabilities in these categories in your code, that disregards both unknown vulnerabilities in your code, and vulnerabilities in other people’s code as well. A safe language catches all three and scales; your method catches only one and doesn’t scale.

                                                                  1. 1

                                                                    Note that I did go the extra mile and went a bit further than Valgrind & the sanitisers. I also happen to run Monocypher’s test suite under the TIS interpreter, and more recently TIS-CI (from TrustInSoft). Those things guarantee that they’ll catch any and all undefined behaviour, and they found a couple bugs the sanitisers didn’t.

                                                                    that disregards both unknown vulnerabilities in your code

                                                                    After that level of testing and a successful third party audit, I am confident there are none left.

                                                                    and vulnerabilities in other people’s code as well

                                                                    There is no such code. I have zero dependencies. Not even the standard library. The only thing I have to fear now is a compiler bug.

                                                                    your method catches only one and doesn’t scale.

                                                                    I went out of my way not to scale. Yet another peculiarity of modern cryptographic code, is that I don’t have to scale.

                                                                    1. 1

                                                                      There is no such code.

                                                                      Sure there is. Other people write cryptographic code too. Unless you are here just arguing against safe languages for only this single project? Because it seemed like a broader statement originally.

                                                                      I went out of my way not to scale.

                                                                      I mean scale as in other developers also writing cryptographic software, not scale as in your software scaling up.

                                                                      1. 1

                                                                        Sure there is. Other people write cryptographic code too. Unless you are here just arguing against safe languages for only this single project

                                                                        I was talking about Monocypher specifically. Other projects do have dependencies, and any project that would use Monocypher almost certainly has dependencies, starting with system calls.

                                                                        I mean scale as in other developers also writing cryptographic software, not scale as in your software scaling up.

                                                                        Fair enough. I was thinking from the project’s point of view: a given project only need one crypto library. A greenfield project can ditch backward compatibility and use a modern crypto library, which can be very small (or formally verified).

                                                                        Yes, other people write cryptographic code. I myself added my own to this ever growing pile because I was unsatisfied with what we had (not even Libsodium was enough for me: too big, not easy to deploy). And the number of bugs in Monocypher + Libsodium is certainly higher than the number of bugs in Libsodium alone. No doubt about that.

                                                                        Another reason why crypto libraries written in unsafe languages don’t scale, is the reputation game: it doesn’t matter how rigorously tested or verified my library is, if you don’t know it. And know it you cannot, unless you’re more knowledgeable than I am, and bother to audit my work yourself, which is prohibitively expensive. So in practice, you have to fall back to reputation and external signs: what other people say, the state of documentation, the security track record, issues from the bug tracker…

                                                        2. 6

                                                          This made me twitch!

                                                          Why make a choice which prevents an entire class of bugs when you could simply put in extra time and effort to make sure you catch and fix them all?

                                                          Why lock your doors when you can simply stand guard in front of them all night with a baseball bat?

                                                          While personally would back the cryptography devs’ decision here, I think there is a legitimate discussion to be had around whether breaking compatibility for some long-standing users is the right thing to do. This post isn’t contributing well to that discussion.

                                                        1. 16

                                                          I take a very firm and mathematically oriented stance on the matter of abstraction.

                                                          Mathematicians understand this very well, but programmers do not understand the difference between an abstraction and a wrapper and a generalization.

                                                          A set is a very weak mathematical object by itself, the power of set theory comes from the set axioms. It’s the axioms where the abstraction lies, and not in the sets.

                                                          A programmer might think that ℤ is an abstraction over ..., -1, 0, 1, ..., but the true abstraction is the fact that the ring (ℤ, +, *) exists. It’s the abstract notion of a ring where the power of abstraction lies.

                                                          Abstraction is all about semantics, and the semantics are abstract.

                                                          You can always wrap something in something else: S = {(x, 1) | x ∈ ℝ}.

                                                          You can always generalize anything: S = ℚ⁺ ⨯ ℤ⁻.

                                                          These, an abstraction do not make. If you can’t define the abstraction without invoking the elements you are abstracting over, it’s not an abstraction.

                                                          The idea of an abstraction leak is a misnomer. Abstraction can’t leak by definition, there’s nothing to leak. What’s not in the abstraction is not in the abstraction. What we call abstraction leaks happen because of insufficient or wrong abstractions. Abstraction leaks is something that happens because of the lack of abstraction. It’s not a problem with the abstraction.

                                                          You can have “abstraction leaks” in mathematics too, but the interpretation is completely different. A programmer might think that Riemann integrals are leaky because they can’t integrate every function. A mathematician knows that Riemann integrals simply don’t have enough abstractive power, and will choose other kinds of abstractions to do his or her integrals like Lebesgue integrals.

                                                          Of course the utility of calculus comes from the linear structures it generates, not from the act of calculating integrals. THAT is where the abstraction trully lies.

                                                          Operating systems and programming languages are in the business of providing abstractions. Libraries in most languages can’t create new abstractions.

                                                          Features are not abstractions. There is a trend in modern operating systems to provide features without providing abstraction. For example, eBPF exists outside the process abstraction. The set of processes plus eBPF do not have any kind of operative abstraction, they are simply a generalization.

                                                          Why do I care about all of this? Because abstraction are powerful, and because generalization or wrapping without abstraction is taking power away.

                                                          1. 11

                                                            It’s easy to ignore or dismiss lots of things that Dijkstra said (and sometimes we should), but his quote seems appropriate for agreeing with @4ad.

                                                            The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise

                                                            1. 2

                                                              To take your example and run with it, the fact that the ring of integers exists is visible from multiple perspectives in programming. Each of these perspectives corresponds to a way of maintaining and operating code. Note that, in all cases, we are trying to instantiate the abstraction of the ring onto an existing abstraction of integers, and so we are really highlighting the interface between abstractions rather than the abstractions themselves.

                                                              • We can use property tests to spot-check elements of the abstraction. I did this in Monte.
                                                              • We can use type-driven proofs to define the data of the abstraction alongside a proven type signature. This is done in Agda.
                                                              • We can use syntactic definitions to assert the existence of the abstraction as a logical entailment. This is done in Metamath.
                                                            1. 3

                                                              When CDE was removed from Solaris Express (it was never in OpenSolaris), I was very pissed, but I ran the old CDE binaries on OpenSolaris for a good amount of time.

                                                              XFCE started as a CDE clone, but very few people remember that today: https://www.youtube.com/watch?v=zr2loh51-Eg&t=730s.

                                                              1. 23

                                                                I don’t know why so many people use spaces instead of tabs for indentation. Always use tabs for indentation and if you prefer 2 spaces per tab set it as such in your editor. There’s no place anywhere where a tab is “defined” as an 8-space indent. Please learn to set your software up correctly and stop forcing your choice on the rest of the world.

                                                                As a side-note, use spaces for alignment (e.g. multiline function arguments). In the ideal case, your code-formatting then shouldn’t depend on the number of spaces per tab.

                                                                1. 13

                                                                  When people use tabs instead of spaces, and I edit their code, then what typically happens is that their code looks wrong: columns don’t line up. I have to change my tab size to match theirs before I can read the code. This happens for several reasons. Sometimes tabs and spaces are used inconsistently for indentation. Or sometimes tabs are used for internal indentation (eg, inside of //… comments). Or sometimes indentation is used to line up code in a case like this:

                                                                  int myfunction(float arg1,
                                                                                 float arg2)
                                                                  

                                                                  (Where the word ‘float’ lines up in lines 1 and 2.)

                                                                  With discipline and rigid adherence to a coding style that avoids these problems, then tabs can work. In my experience, other developers typically don’t have the discipline to write code that looks right when the tab size changes.

                                                                  Always use spaces for indentation. Then you won’t inflict this problem on other people reading your code.

                                                                  1. 20

                                                                    On the other hand, you’ll “inflict” your indentation size preferences on people if you use spaces. This is why no one can agree on this: they both have upsides and downsides, and it’s a matter of which downsides are “less bad”.

                                                                    The general solution is to use tabs for indentation and spaces for alignment, as FRIGN mentioned, which gives the benefits of both and (almost) always works well regardless of tab size. Tools like gofmt handle this automatically. There’s also “elastic tabstop”.

                                                                    1. 9

                                                                      In a given codebase, there’s plenty of stylistic choices which are taken and cannot be visually changed with a client-side switch (see for example the number of clang-format settings). Maybe it’s easier to accept that indentation is also one such stylistic choice, there’s probably too much thought wasted on this matter.

                                                                      1. 1

                                                                        Maybe; I don’t really have very strong opinions on it as such – there are more important things in programming or life to worry about – but at the same time, even though I programmed Ruby full-time for 2 years, I never really got used to the Ruby convention of 2-space indentation. It’s just too narrow for me.

                                                                    2. 15

                                                                      Always use spaces for indentation. Then you won’t inflict this problem on other people reading your code.

                                                                      You should rather use spaces for alignment. Indentation is used to semantically separate portions of code. Alignment is used to make the code more readable. They serve two different purposes, and can totally coexist withing the same code:

                                                                      /*
                                                                       * Tabs emphasized as ">·······" for clarity
                                                                       */
                                                                      int myfunction(float arg1,
                                                                                     float arg2)
                                                                      {
                                                                      >·······if (arg1 != arg2)
                                                                      >·······>·······return 0;
                                                                      
                                                                      >·······fprintf(stderr, "arg1 = %f, arg2 = %f\n",
                                                                      >·······        arg1,
                                                                      >·······        arg2);     // note the use of space AFTER the tabulation
                                                                      
                                                                      >·······return 1;
                                                                      }
                                                                      
                                                                      1. 1

                                                                        Exactly! If someone feels the need to align some code, this must be done in spaces, not tabs.

                                                                        (or use elastic tabs, if it is supported by given project and tools)

                                                                      2. 7

                                                                        The solution is to use tabs for indentation but spaces for alignment. So for the second line of your code example, tabs would be used up to the indentation level of int, and spaces would be used from that point on to align float.

                                                                        1. 3

                                                                          You’re conflating indent and alignment. In your example, both lines have the same indent and so should start with the same number of tabs. The first line has no alignment spacing, 15 alignment spaces.

                                                                          The latest versions of clang-format support this and so you can use tabs for indentation spaces for alignment and get the best of both worlds.

                                                                        2. 4

                                                                          A big reason for me for spaces is that things like diff and blame will prefix the lines. With spaces for indentation, all lines simply get shifted over by a constant amount. With tabs, the prefixes will often “eat into” a certain amount of the first level of indentation. You can end up with the first level of indentation being shorter than the rest.

                                                                          (Not to mention that tab-width may vary between different tools outside of just the editor or IDE and it can be a pain to try to get them all to be consistent. The terminal I use, for example, aligns tabs to multiples of eight spaces and has no configuration to change this.)

                                                                          1. 4

                                                                            I’m sorry – but tabs are in fact defined as 8 spaces, for all practical purposes. Look at your code with cat. Or more, or less, for example.

                                                                            1. 5

                                                                              I don’t know why

                                                                              Assuming you’re saying this earnestly and legitimately do not understand, then I can share why I don’t generally use tabs:

                                                                              • One exception: if tabs and indentation are managed by an auto-formatting tool, then I generally don’t have a problem with them.
                                                                              • In projects without an auto-formatter that used tabs, they seem to always wind up being a mess. By “mess” I mean that if your tab width isn’t set to the one used by the author, then the indentation looks totally wrong. Either that, or they didn’t mix tabs and spaces correctly.
                                                                              • Code isn’t exclusively viewed in your editor. Sometimes it’s viewed on GitHub where changing the tab settings is not possible (or, is at least not straight forward).
                                                                              • Tabs means each person has their own tab width. And that means each person experiences a fixed column limit in a different way. e.g., If I use two-space tabs, then I’m going to be able to fit a bit more code in 80 columns than my co-worker who is using eight-space tabs. And now that means my co-worker is going to see long lines.

                                                                              Overall, tabs just end up being a mess in my experience. If you’ve found a way to work with them in a consistent way among your collaborators, then all the power to ya, but I haven’t figured that out. So that’s why.

                                                                              1. 3

                                                                                As a side-note, use spaces for alignment (e.g. multiline function arguments).

                                                                                I avoid doing alignment like this because it has to be done the right way lest it look right to me and wrong to you. Instead, parameters in a multiline call just get extra indentation. If you put the first argument on a new line too and maybe treat the parens like braces, this can be aligned, readable, and work for any indentation method. Maybe that’s the kind of thing you meant by:

                                                                                In the ideal case, your code-formatting then shouldn’t depend on the number of spaces per tab.

                                                                                The twist: My projects, though they would be amenable to tabs, use spaces and always have. I imagine it’s just the environment I landed in when I started my career. I don’t really think about it. But that’s the benefit, right? You don’t have to think about the pitfalls of mixing two kinds of space character when there is only one kind of space character.

                                                                                1. 1

                                                                                  This is the default style in Rustfmt (indent_style=“Block”), and it’s also common in Python.

                                                                                  I think this is the more sensible indent style, objectively speaking.

                                                                                  Every day, I wish clang-format had an option for it (oh, how I hate clang-format).

                                                                                2. 2

                                                                                  I always strive for code consistency, even when I dislike like the particular style, but using spaces for indentation is something I can’t ever get into, and will refuse to work on. Fortunately, tabs are standard for the languages I use, but sometimes I veer of into another language which uses space, and it’s painful…

                                                                                  As a side-note, use spaces for alignment (e.g. multiline function arguments). In the ideal case, your code-formatting then shouldn’t depend on the number of spaces per tab.

                                                                                  If you want to do that, you can do that, and ideed that’s what gofmt does for struct field alignment, however note that code will be misaligned if you use a proportional font. Yes, we do exist. No, I will not use a monospace font.

                                                                                  Speaking of line length, I try to keep line length reasonable, and a quick query shows that 99.3% of my lines are under 80 columns, however, If I need a longer line, I will write a longer line! Artificially breaking the odd long line is a mistake. Not only that artificial line breaks are harder to read, but they also break grep and diff.

                                                                                  1. 1

                                                                                    There’s no place anywhere where a tab is “defined” as an 8-space indent.

                                                                                    Very little software even defaults to 8 these days. Just terminals

                                                                                    1. 1

                                                                                      Two words for you: Python. And, um, well, maybe it’s just the one word.

                                                                                      Python (and any language where indentation carries syntactic meaning) has a more nuanced definition of tabs and spaces out of necessity.

                                                                                      See Python’s lexical spec:

                                                                                      Tabs are replaced (from left to right) by one to eight spaces such that the total number of characters up to and including the replacement is a multiple of eight (this is intended to be the same rule as used by Unix).

                                                                                      So, in Python’s case, tabs are defined as an 8-space indent, sort of. That is, Python has “tabstops” every 8 characters, and a tab character will take you to the next tabstop. And apparently that’s based on a Unix standard, but I haven’t bothered to research and verify that.

                                                                                      What’s the big deal, then? People should still be able to just use only tabs, right? Well, then you run into the “8 spaces” thing, and your code no longer fits in your window. Not to mention that the Python style standard says “4 spaces” for indentation, which can’t be done with tabs alone.

                                                                                      But everyone should configure their editor, right? Well, that works fine if you’re always in complete control of your programming environment and you only have to set it up once. But if you’re on contract and you need to use client hardware, or if you need to edit a file on a server, or if you need to ever pair program at all, then using only-tabs won’t save you from the very-hard-to-read 8 spaces problem.

                                                                                      Python’s not the only language where indentation matters. Elm, CoffeeScript, haskell and elixir all have syntax-important indentation. Also Makefiles, which you may have to deal with regardless of your implementation language. They don’t all have the same complicated ruleset that Python does, but unless you’re looking up the lexical definitions, you don’t know.

                                                                                      Which means people rely on the provided configurations for editing languages like that, because those configuration authors have probably checked and made sure things work. Which leaves you with whatever spacing that author used. Unless you want to learn elisp or vimscript or whatever other configuration language your editor has.

                                                                                      Basically, your approach would be great if we were inventing programming from scratch. But we’re not, and the last time we did we made whitespace relevant because of punchcards, so we have to live in a world where your approach only works for a subset of situations and a subset of programmers.

                                                                                    1. 26

                                                                                      Avoiding culture war issues. I found any engagement to be highly stressful and rarely constructive

                                                                                      1. 4

                                                                                        I avoid it as much as I can, I never willingly participate in these issues, but sometimes I find myself explicitly targeted by combatants in this war.

                                                                                      1. 5

                                                                                        Turning off notifications on my phone.

                                                                                        Deleting all group chats app from my phone.

                                                                                        Giving up on breakfast, which by itself caused me to go from 90kg to 60kg.

                                                                                        1. 1

                                                                                          Turning off notifications on my phone.

                                                                                          I’ve started doing this since Monday this week. I was already aware of my bad smartphone usage habits and decided to take some measures against it after reading this article in which a mother shared how her daughter turned into a smartphone junkie. The easiest way to achieve this on Android is to permanently activate do not disturb mode. This also forces you to do good old phone calls instead of writing text messages and asynchronously waiting for a reply. I really forgot how easy it is to plan things with a quick phone call.

                                                                                          Deleting all group chats app from my phone.

                                                                                          I just turned off notifications, but this is the next step to go.

                                                                                        1. 3

                                                                                          Expectation: a pure text-based chat system, from a more enlightened age

                                                                                          Reality: trolls spamming channels with huge ascii-art dildos and/or swastikas, and ddos

                                                                                          1. 36

                                                                                            Reality: trolls spamming channels with huge ascii-art dildos and/or swastikas, and ddos

                                                                                            Not in my reality.

                                                                                            1. 9

                                                                                              I’m also surprised to hear that. Unless you explicitly look for troll channels, my experience has either been quiet (but quick to answer) or constantly active, and on topic.

                                                                                            2. 17

                                                                                              Never saw anything like that on freenode. Mind me asking - what channels do you visit?

                                                                                              1. 11

                                                                                                I can’t say I’ve seen the things that the grandparent comment mentioned, but they definitely wouldn’t be on Freenode. If you limit yourself to Freenode, IRC is a very safe and well-moderated experience, especially on some exemplary channels like the Haskell one.

                                                                                                I have accidentally wandered into uncomfortable conversations and much worse things on some of the other popular IRC networks, of which quite a few still exist: https://netsplit.de/networks/top100.php

                                                                                                The same thing is true of sketchy Discord servers as well; it’s not like IRC is unique in this regard.

                                                                                                1. 3

                                                                                                  A year or two back, Supernets was spamming hard on IRC networks. I forgot if Freenode was affected, but I know a lot of the smaller networks I was on were.

                                                                                                  1. 2

                                                                                                    Not OP, but I spend my time on IRCnet and EFnet since my IRC use is just to stay in touch with friends. Anyway, last year I was DDoS’d pretty hard because someone wanted my nick on EFnet.

                                                                                                    1. 1

                                                                                                      Sometimes I miss #C++ on EFnet, not enough to go back on EFnet, but I do miss it – a lot of wonderful people were there in the late 90s. Freenode feels a lot more sane in terms of management and tools for the system operators. Cloaks and nickname registration go a long way.

                                                                                                    2. 2

                                                                                                      I’m in, like, 15 networks, and never saw anything like that either.

                                                                                                  1. 58

                                                                                                    my partner submitted a patch to OpenBSD a few weeks ago, and he had to set up an entirely new mail client which didn’t mangle his email message to HTML-ise or do other things to it, so he could even make that one patch. That’s a barrier to entry that’s pretty high for somebody who may want to be a first-time contributor.

                                                                                                    I think it was actually Gmail that was a barrier. And he also couldn’t do it from Apple Mail. It is just that the modern mail client has intentionally moved towards HTML

                                                                                                    I am flabbergasted someone is able to send a patch to OpenBSD but is not able to set the web interface of GMail to sent plain-text emails. Or install Thunderbird, which might have been the solution, in that particular case.

                                                                                                    I also never used git send-email, but I don’t think it is the barrier to become a kernel maintainer.

                                                                                                    Actually, that might work as an efficient sieve to select the ones who want to be active contributors from those who are just superficial or ephemeral contributors.

                                                                                                    In my opinion, although Github supposedly decreases the barrier of first-time contribution, it increases the load on the maintainers, which has to interact with a lot of low-quality implementation, reckless hacks and a torrent of issues which might not even be. That is my experience in the repositories I visit or contributed.

                                                                                                    1. 41

                                                                                                      Patching some component or other of OpenBSD is not a skill that transfers well to making some random application do what you want.

                                                                                                      1. 40

                                                                                                        I agree with you.

                                                                                                        On the other hand, if someone knows how to install OpenBSD, use cvs (or git), program in C and navigate the kernel’s souce-code, I supposed they are capable of going to any search engine and find the answer on how to send plain-text email via the GMail interface.

                                                                                                        1. 16

                                                                                                          GMail mangles your “plain-text” messages with hard-wraps and by playing with your indentation. You’d have to install and configure thunderbird or something.

                                                                                                          1. 3

                                                                                                            One of the many reasons why I use Mutt.

                                                                                                        2. 14

                                                                                                          Note that this is anecdotal hearsay. This person is saying their partner had to set things up… they may have misunderstood their partner or not realized their partner was exaggerating.

                                                                                                          Also, one might expect the amount of patience and general debugging skill necessary to transfer rather well to the domain of email configuration.

                                                                                                          It’s also possible that guiraldelli is assuming it was an excepted OpenBSD kernel patch, wheras we don’t know if the patch was accepted and we don’t know if it was a userland patch. It doesn’t take much skill to get a userland patch rejected.

                                                                                                        3. 30

                                                                                                          I am flabbergasted someone is able to send a patch to OpenBSD but is not able to set the web interface of GMail to sent plain-text emails.

                                                                                                          You can’t send patches, or indeed any formatted plain text emails through the gmail web interface. If you set gmail to plain text, gmail will mangle your emails. It will remove tabs and hardwrap the text. It’s hopeless.

                                                                                                          1. 26

                                                                                                            Think about impediments in terms of probabilities and ease of access.

                                                                                                            You wouldn’t believe how many people stop contributing because of tiny papercuts. There is a non-trivial amount of people who have the skills to make meaningful contributions, but lack the time.

                                                                                                            1. 18

                                                                                                              Lobsters requires an invite. That’s a barrier to entry, so those who make it are more likely to make good contributions, and have already extended effort to comply with the social norms.

                                                                                                              1. 8

                                                                                                                You may be willing to jump a barrier of entry for entertainment (that is lobsters) but not for free work for the benefit of others (because you have already done that work for yourself).

                                                                                                                1. 4

                                                                                                                  If you want to save yourself compiling and patching your kernel every time there’s an update, you might want to submit it to the project maintainer.

                                                                                                                  1. 18

                                                                                                                    If the project wants to have contributors in the future it might want to consider using modern, user friendly technologies for contribution.

                                                                                                                    Linux is considering it, a discussion is started, which is positive (for the future of the project) in my opinion. It is a sign or responsive project management, where its conservatism does not hinder progress, only slows it to not jump for quickly fading fads.

                                                                                                                    Other communities are closing their microverse on themselves, where it is not enough to format your contribution in N<80 column plain text email without attachments with your contribution added to it via come custom non-standard inline encoding (like pasting the patch to the end of the line), but you may even need to be ceremonially accepted for contribution by the elders. Also some such projects still don’t use CI, or automated QA. These communities will die soon, actually are already dead, especially as they don’t have large corporations backing them, as someone getting paid usually accepts quite a few papercuts a job demands. This is why Linux could actually allow itself to be more retrograde than the BSD projects for example, which I think are even more contributor-unfriendly (especially ergonomically).

                                                                                                                    1. 2

                                                                                                                      I tend to agree, however that discussion was started by someone with a very striking conflict of interest and therefore their opinion should be questioned and considered to be insincere at best and maleficent at worst.

                                                                                                                      1. 5

                                                                                                                        That doesn’t matter, the discussion can be done despite that. I think a forge-style solution is the key.

                                                                                                                        Despite Microsoft having 2 “forge” type solutions in its offering (both quite usable in my experience, github being a de-facto standard) I still cannot see a conflict of interest in this topic. There are other software forges available. The current process is simply outdated.A custom solution could also be developed, if deemed necessary, as it was the case with git.

                                                                                                              2. 14

                                                                                                                Pay attention my comment talks about maintainers, active contributors and superficial or ephemeral contributors.

                                                                                                                From the article:

                                                                                                                a problem recently raised by Linux kernel creator Linus Torvalds, that “it’s really hard to find maintainers.” Maintainers are the gatekeepers who determine what code ends up in the widely used open-source kernel, and who ensure its quality. It is part of a wider discussion about who will oversee Linux when the current team moves on.

                                                                                                                And later on…

                                                                                                                “We need to set up a better or a different or an additional way to view tools and the work that’s being done in the Linux project as we’re trying to bring in new contributors and maintain and sustain Linux in the future,” she told us.

                                                                                                                Picking her words carefully, she said work is being done towards “moving from a more text-based, email-based, or not even moving from, but having a text-based, email-based patch system that can then also be represented in a way that developers who have grown up in the last five or ten years are more familiar with.

                                                                                                                “Having that ability and that perspective into Linux is one of the ways we hope we can help bring newer developers into the kernel track.”

                                                                                                                So I understood Sarah Novotny is addressing the problem of new contributors, not the one Linus Torvalds see, of new maintainers.

                                                                                                                So, your comment that

                                                                                                                how many people stop contributing because of tiny papercuts. There is a non-trivial amount of people who have the skills to make meaningful contributions, but lack the time

                                                                                                                is not the problem the Linux Foundation has, but the one that Microsoft’s Sarah Novotny wants to solve. Those are two different problems, with two different solutions. On the surface, they might seem the same, but they are not. They might be correlated, but the solution for one problem does not necessarily mean it is the solution for the other.

                                                                                                                Therefore, my argument still stands:

                                                                                                                [having a plain-text email list as central communication and collaboration to the Linux’s kernel] might work as an efficient sieve to select the ones who want to be active contributors [i.e., maintainers] from those who are just superficial or ephemeral contributors.

                                                                                                                If we are going to address, though, that it might hinder new contributors, then I tend to agree with you. :)

                                                                                                                1. 21

                                                                                                                  Let’s not let Microsoft decide how Linux is developed.

                                                                                                                  The waning popularity of their own, proprietary kernel is no excuse for telling other projects how they need to be run.

                                                                                                                  1. 5

                                                                                                                    This is just my opinion but I think that while she’s totally missing the mark on finding Linux kernel module maintainers having anything at all to do with plain text email patch submission systems, the general issue she’s speaking to is one that has generated a lot of discussion recently and should probably not be ignored.

                                                                                                                    Also, in the spirit of the open source meritocracy, I’d prefer to let people’s actions and track records speak more loudly than which company they happen to work for, but then, lots of people consider my employer to be the new evil empire, so my objectivity in this matter could be suspect :)

                                                                                                                  2. 8

                                                                                                                    So I understood Sarah Novotny is addressing the problem of new contributors, not the one Linus Torvalds see, of new maintainers.

                                                                                                                    Bingo!

                                                                                                                    My first thought after reading this was “Does this person ACTUALLY think that having to use plaintext E-mail is even statistically relevant to the problem of finding module maintainers for the Linux kernel?”

                                                                                                                    In a general sense I believe that the open source community’s reliance on venerable tools that are widely rejected by younger potential contributors is a huge problem.

                                                                                                                    Martin Wimpress of Ubuntu Desktop fame has spoken about this a lot recently, and has been advocating a new perspective for open source to increase engagement by embracing the communications technologies where the people are not where we would like them to be.

                                                                                                                    So he streams his project sessions on Youtube and runs a Discord, despite the fact that these platforms are inherently proprietary, and has reported substantial success at attracting an entirely new audience that would otherwise never have engaged.

                                                                                                                    1. 5

                                                                                                                      is not the problem the Linux Foundation has, but the one that Microsoft’s Sarah Novotny wants to solve. Those are two different problems, with two different solutions. On the surface, they might seem the same, but they are not. They might be correlated, but the solution for one problem does not necessarily mean it is the solution for the other.

                                                                                                                      GKH even said that there are more than enough contributors in his last ama, so having contributors is “a priori” not a problem right now.

                                                                                                                      1. 2

                                                                                                                        So I understood Sarah Novotny is addressing the problem of new contributors, not the one Linus Torvalds see, of new maintainers.

                                                                                                                        Every maintainer was a new contributor at a point! Also better tooling would be beneficial for everyone.

                                                                                                                        1. 1

                                                                                                                          Every maintainer was a new contributor at a point!

                                                                                                                          That is true, but it is not the current problem: see /u/rmpr’s reply and his link to the Reddit’s AMA to verify that.

                                                                                                                          Also better tooling would be beneficial for everyone.

                                                                                                                          That is not necessarily true: the introduction of a tool requires a change of mindset and workflow of every single person already involved in the current process, as well as update of the current instructions and creation of new references.

                                                                                                                          I saw projects that took months to change simply a continuous integration tool (I have GHC in mind, except the ones in the companies I worked for). Here, we are talking about the workflow of many (hundreds for sure, but I estimate even thousands) maintainers and contributors.

                                                                                                                          As I already told before, RTFM [1] [2] does not take long for a single individual that wants to become a new contributor; in [1], there is even a session specially about the GMail’s Web GUI problem.

                                                                                                                          If the problem is retrievability of the manual or specific information, than I think that should be addressed first. But that topic is not brought up in the article.

                                                                                                                    2. 20

                                                                                                                      I am able to deal with email patches, but I hate doing it. I would not enjoying maintaining a project which accepts only email patches.

                                                                                                                      Actually, that might work as an efficient sieve to select the ones who want to be active contributors from those who are just superficial or ephemeral contributors.

                                                                                                                      Ephemeral contributors evolve to maintainers, but if that initial step is never made this doesn’t happen. I don’t know if this will increase the number of maintainers by 1%, 10%, or more, but I’m reasonably sure there will be some increase down the line.

                                                                                                                      Finding reliable maintainers is always hard by the way, for pretty much any project. I did some git author stats on various popular large open source projects, and almost all of them have quite a small group of regular maintainers and a long tail of ephemeral contributors. There’s nothing wrong with that as such, but if you’re really strapped for maintainers I think it’s a good idea to treat every contributor as a potential maintainer.

                                                                                                                      But like I said, just because you can deal with email patches doesn’t mean you like it.

                                                                                                                      1. 14

                                                                                                                        GMail, even in plain-text mode, tends to mangle inline patches since it has no way of only auto-wrapping some lines and not others.

                                                                                                                        Not a major issue as you can attach a diff, but from a review-perspective I’ve personally found folks more inclined to even look at my patches if they’re small enough to be inlined into the message. I say this as someone having both submitted patches to OpenBSD for userland components in base as well as having had those patches reviewed and accepted.

                                                                                                                        I personally gave up fighting the oddities of GMail and shifted to using Emacs for sending/receiving to the OpenBSD lists. I agree with Novotny’s statement that “GMail [is] the barrier.” The whole point of inlining patches like this is the email body or eml file or whatever can be direct input to patch(1)…if GMail wraps lines of the diff, it’s broken the patch. Text mode or not.

                                                                                                                        Obviously, this doesn’t mean if the Linux kernel maintainers want to change their process that I don’t think they shouldn’t. (Especially if they take a lot of funding from the Foundation…and as a result, Microsoft/GitHub.) OpenBSD is probably always going to take the stance that contributions need to be accessible to those using just the base system…which means even users of mail(1) in my interpretation.

                                                                                                                        1. 5

                                                                                                                          You can’t attach a diff – nearly all mailing lists remove attachments!

                                                                                                                          OpenBSD is probably always going to take the stance that contributions need to be accessible to those using just the base system

                                                                                                                          Heh, if you add arc to the base system, you can adopt Phabricator without violating that principle :)

                                                                                                                          1. 2

                                                                                                                            Good point…so basically GMail (the client) doesn’t really work with an email-only contribution system. It’s Gmail SMTP via mutt/emacs/thunderbird/etc. or bust.

                                                                                                                        2. 5

                                                                                                                          I have found several Linux subtle kernel bugs related to PCIe/sysfs/vfio/NVMe hotplug. I fixed them locally in our builds, which will ultimately get published somewhere. I don’t know where, but it’s unlikely to ever get pushed out to the real maintainers.

                                                                                                                          The reason I don’t try to push them out properly? The damn email process.

                                                                                                                          I did jump through the hoops many years ago to configure everything. I tried it with gmail, and eventually got some Linux email program working well enough that I was able to complete some tutorial. It took me some non-negligible amount of time and frustration. But that was on an older system and I don’t have things set up properly anymore.

                                                                                                                          I don’t want to go through that again. I have too many things on my plate to figure out how to reconfigure everything and relearn the process. And then I won’t use it again for several months to a year and I’ll have to go through it all again. Sometimes I can get a coworker to push things out for me - and those have been accepted into the kernel successfully. But that means getting him to jump through the hoops for me and takes time away from what he should be doing.

                                                                                                                          So for now, the patches/fixes get pulled along with our local branch until they’re no longer needed.

                                                                                                                          1. 4

                                                                                                                            I have no idea how dense they need to be to not be able to send text-only email from Apple Mail. It must be anecdotal, because I cannot believe that someone is smart enough to code anything, but dumb enough to be able to click Format > Change to text in the menu bar.

                                                                                                                          1. 1

                                                                                                                            Nitpick:

                                                                                                                            Widely considered to be the most secure operating system in the world

                                                                                                                            While true (the widely considered part), it definitely isn’t the most secure operating system in the world.

                                                                                                                            That label belongs to seL4, which has formal proof of correctness.

                                                                                                                            A better statement would be qualified, like: “most secure UNIX-like…”

                                                                                                                            1. 12

                                                                                                                              I’d argue that seL4 isn’t an operating system but merely a microkernel. Most of the insecurities in common systems these days happen outside its scope (where, for a long time, the focus was up the stack, but with all the CPU side channels, people are now looking down the stack, too) so whatever you add to seL4 to make it useful needs to be evaluated as well.

                                                                                                                              For example: Having an “unbreakable” seL4 is a nice basic component (and helps contain some damage) but is of little comfort if your file system server is thoroughly broken, giving attackers unbounded access to your data.

                                                                                                                              1. 10

                                                                                                                                I’d argue that seL4 isn’t an operating system but merely a microkernel.

                                                                                                                                I’d argue it’s not even that. For example, seL4 has no dynamic memory management in the kernel, rather, dynamic memory management is pushed to userspace. And it’s not that seL4 doesn’t use dynamic memory, in a real system that uses seL4 you have to implement it. seL4 is proven correct in the sense of not violating its invariants regardless of what the userspace is doing, but of course, the interesting part of a production system is in the userspace, not in the kernel.

                                                                                                                                If you want to formally prove your dynamic memory allocator, you are on your own, seL4 won’t help you.

                                                                                                                                Not to take anything away from the seL4 folks, what they did is both important and impressive, and I recommend anyone to read the papers, which are very approachable, but comparing components of a system with the system is a category error.

                                                                                                                                Similarly, comparing an embedded real-time system with an interactive timesharing system is also apple and oranges…

                                                                                                                                1. 5

                                                                                                                                  Similarly, comparing an embedded real-time system with an interactive timesharing system is also apple and oranges…

                                                                                                                                  seL4 does both. It implements mixed criticality. Critical realtime tasks and non-critical tasks can share the same system, without detriment to the properties the critical tasks need. The guarantees hold.

                                                                                                                                  in a real system that uses seL4 you have to implement it (dynamic memory).

                                                                                                                                  Not really. Many scenarios can and are in fact often implemented without dynamic memory; Components get what they need as they’re launched and that’s it. Non-deterministic behavior is avoided in critical systems.

                                                                                                                                  Furthermore, in a mixed criticality scenario, a subset of the non-critical tasks might actually get dynamic memory from a memory manager that has a limited pool of memory to begin with, doesn’t deal with non-critical tasks and thus does not need to be part of the TCB, nor formally proven.

                                                                                                                                  comparing components of a system with the system is a category error.

                                                                                                                                  This much is correct. Openbsd is more than the kernel or the base system. It’s the whole thing.

                                                                                                                                  SeL4, however, is a much better kernel than Openbsd’s is, on a fundamental level. But (unlike Minix3’s design) a system built around seL4 is not going to be UNIX-like. At most it will offer POSIX compatibility.

                                                                                                                                  Which is why I suggested “The most secure UNIX-like”. Because seL4 isn’t that, and Minix3 could be but isn’t quite there.

                                                                                                                                2. 5

                                                                                                                                  I’d argue that seL4 isn’t an operating system but merely a microkernel.

                                                                                                                                  Fair.

                                                                                                                                  but is of little comfort if your file system server is thoroughly broken

                                                                                                                                  Much emphasis must be put on seL4 being built around the concept of capabilities, making it better on a fundamental level, as it actually allows for security, which isn’t possible with UNIX’s ambient authority, due to the confused deputy problem.

                                                                                                                                  In this case, no matter how broken one instance of filesystem server is, it won’t affect processes that aren’t using it, nor other filesystem servers, nor any block devices that the filesystem server does not have a capability to. The affected fs server might not be part of the TCB. SeL4’s design allows for critical and non-critical (mixed criticality) tasks to run in the same system, with formal proof of enforcement of separation. It might be that the system is at the core of the hard realtime flight system of an helicopter full of people, keeping them safe despite this compromised fs server being stuck in an infinite loop.

                                                                                                                                  Remarkably, filesystems aren’t mounted in a global vfs as they are in UNIX, either. Whereas most processes, even those using files, do not have capabilities to filesystem access. To read a file, all that’s needed is a capability to it. There’s no need to e.g. know where it does reside.

                                                                                                                                  1. 4

                                                                                                                                    I’m well aware of how things are ideally done in a microkernel system. Thing is, people ask for (see sibling answer) and implement shortcuts.

                                                                                                                                    File system access is among the harder parts to fully redesign due to the many expectations users have when it comes to file management, so I picked that one as an example. See Android and iOS, both which tried to deemphasize files as a category but gave up at some point and ship file system browsers now.

                                                                                                                                    Thus I expect file system access to allow (not necessarily use, in all cases) broad access capabilities instead of the fine grained scheme advocated in microkernel/security kernel design literature (“user wants to open a file, so the editor process asks the file picker service. The file picker provides the UI to select files, but can’t read their contents. When the user has chosen, the file picker sends a capability to the editor process that provides the desired level of access to that file only, without even telling where it resides”)

                                                                                                                                    1. 1

                                                                                                                                      Thus I expect file system access to allow (not necessarily use, in all cases) broad access capabilities

                                                                                                                                      That’d be quite sad. If it helps be more optimistic, know that Genode’s dynamic general-purpose scenario, Sculpt, does not fall into that trap.

                                                                                                                                    2. 3

                                                                                                                                      This definitely feels like a good design. Are there intermediates possible? Such as limiting the effect of troubles in one subsystem onto the other, and bringing more features from microkernels to other operating systems?

                                                                                                                                      After all, MINIX3 is a distribution of NetBSD with the MINIX microkernel…

                                                                                                                                      1. 3

                                                                                                                                        After all, MINIX3 is a distribution of NetBSD with the MINIX microkernel…

                                                                                                                                        I wish Netbsd took Minix3 under its wing. Minix3 is a good idea, and I’m saddened to see it isn’t as active as I’d like it to be.

                                                                                                                                  2. 10

                                                                                                                                    I’m the author of these slides. I’ve changed the wording to say “most secure general purpose operating system in the world” in order to fix the issue you have with it. I’m a little disappointed the discussion has gone so far in this direction considering it’s meant as an introduction to OpenBSD for non-OpenBSD folk, and not a document of absolute truths.

                                                                                                                                    1. 5

                                                                                                                                      Don’t worry. With an increase in popularity, the amount of nitpickers increases as well. Thanks for creating and publishing the slides!

                                                                                                                                      1. 4

                                                                                                                                        I’m the author of these slides.

                                                                                                                                        Good work, I really liked them. I was also happy about the effort promoting Openbsd; There doesn’t seem to be anywhere near enough of that.

                                                                                                                                        I’ve changed the wording to say “most secure general purpose operating system in the world” in order to fix the issue you have with it.

                                                                                                                                        Well-intended change, but still, I stand by “most secure UNIX-like”.

                                                                                                                                        Some people seemingly believe it’s impossible to make a general-purpose operating system out of a microkernel. Like Genode (which implements Sculpt, a general-purpose dynamic scenario), I strongly object to this belief.

                                                                                                                                        I’m a little disappointed the discussion has gone so far in this direction

                                                                                                                                        As an Openbsd user (and fan) I’m also as disappointed that this thread got the discussion this far off track, when my nitpick was explicitly a nitpick.

                                                                                                                                        1. 2

                                                                                                                                          I am claiming “most secure” just like I’d claim “my dad’s the best dad”. Lightheartedly, but then checking out what OpenBSD does toward security I see there is something going on for real here.

                                                                                                                                          I do not care too much how many the 1 most secure operating systems there are around. I prefer “more secure everyday”, as you cannot reach this goal for real, feels funnier.

                                                                                                                                        2. 9

                                                                                                                                          Proof of correctness aren’t the same as proofs of security. It’s entirely possible to correctly specify that the wrong person is allowed to access my data – and, in most major security breaches I read about, it seems like that’s precisely what happens: Someone leaves an S3 bucket wide open, and an attacker looks at it. No component of the system has malfunctioned (other than the human in charge of configuring things, arguably).

                                                                                                                                          In my opinion, a secure system is one designed to not only be correct – that’s necessary, but not sufficient. It’s also one designed to restrict what the user-facing components are able to access by default, making it easy to reason who is able to access what, and focusing on reducing the number of places where interactions of correctly-functioning components can lead to unexpected behaviors.

                                                                                                                                          1. 5

                                                                                                                                            Proof of correctness aren’t the same as proofs of security.

                                                                                                                                            Yet seL4 is provably secure, as the proofs cover security enforcement. The kernel does guarantee confidentiality, integrity and availability. What you do with the kernel is up to you, but it does offer the tools. Building a secure system with it is thus possible. It also ensures safety of time-critical systems, by providing a sound analysis of worst-case execution time. More on this in the whitepaper.

                                                                                                                                            It’s also one designed to restrict what the user-facing components are able to access by default,

                                                                                                                                            Capabilities give explicit access to fine-grained resources, and thus they make POLA a possibility, whereas ambient authority runs contrary to POLA and does hinder efforts to attain security.

                                                                                                                                            Openbsd is no doubt the most secure UNIX-like at the present time, but UNIX isn’t the definitive of operating system design, nor anywhere near perfect by any means.

                                                                                                                                            Still, it doesn’t have the best possible overall design for a UNIX, from a security or reliability perspective. Minix3 is much closer to that, by going much further in privilege separation thanks to its pure microkernel, multiserver design.

                                                                                                                                          2. 4

                                                                                                                                            It feels like OpenBSD aims protecting the userland in addition to the kernel itself. With alternate implementations incorporating privsep features (the pledge(2) unveil(2) dance).

                                                                                                                                            So maybe OpenBSD userbase, designed to split responsibility and privilege across components communicating (imsg, pipes, unix sockets…) between each other, would be a good fit for something like SeL4 ? Maybe making use of CAmkES (that I still do not know) ?

                                                                                                                                            1. 3

                                                                                                                                              If I had the resources (time or excess money to buy developer time with), I’d personally give Minix3 (a stalled project) its much needed push.

                                                                                                                                              It’s still limited by the fact it implements a UNIX system, but thanks to its pure microkernel, multiserver architecture and its fault tolerance focus, it can do what Openbsd, limited by its monolith kernel, cannot.

                                                                                                                                              pledge/unveil

                                                                                                                                              Are really clever, and a feature I wish everybody else in the UNIX world adopted. I understand Dragonfly has made some steps into that, but to my knowledge nobody else has.

                                                                                                                                            2. 3

                                                                                                                                              Way to take the focus off of an already under-appreciated thing.

                                                                                                                                              1. 4

                                                                                                                                                I don’t know, OpenBSD gets a lot of love on lobste.rs.

                                                                                                                                                1. 3

                                                                                                                                                  It used to, but I think quite some OpenBSD devs have moved away from lobste.rs over the years.

                                                                                                                                                  1. 2

                                                                                                                                                    Selection bias. lobste.rs was originally founded by @jcs who is an OpenBSD developer, this naturally drew a lot of OpenBSD people towards the site. Quick grep hats for “openbsd.org” lists 20 committers with accounts on this site.

                                                                                                                                                    1. 1

                                                                                                                                                      fallacy of composition, sick burn, next

                                                                                                                                                      (please read this as me attempting humor, not me being insulting :) )

                                                                                                                                                2. 3

                                                                                                                                                  I think a better statement would have been: “Widely considered to be the most secure general purpose operating system in the world”

                                                                                                                                                  1. 2

                                                                                                                                                    I stand by UNIX-like, which is very different from general purpose.

                                                                                                                                                    There’s many ways to create general purpose operating systems without making a copy of UNIX. Many of them are fundamentally better equipped for security than UNIX is.

                                                                                                                                                    There’s also the quite unfortunate widespread yet baseless belief that microkernels aren’t suitable for the creation of general purpose operating systems. I’ll take steps to prevent supporting these beliefs, if they are easy and harmless to take. Restricting the claim to UNIX-like is one such step.

                                                                                                                                                  2. 2

                                                                                                                                                    no

                                                                                                                                                    no need to add qualifications to a true statement which aren’t relevant to the topic. at least here it feels like it would add confustion.

                                                                                                                                                    1. 1

                                                                                                                                                      no need to add qualifications to a true statement

                                                                                                                                                      That’s the issue. It isn’t true unless qualified. Openbsd isn’t the most secure operating system in the world.

                                                                                                                                                      I doubt the authors themselves do appreciate this sort of advertisement, either.

                                                                                                                                                      1. 3

                                                                                                                                                        Openbsd isn’t the most secure operating system in the world.

                                                                                                                                                        the article did not state that OpenBSD is the most secure operating system in the world

                                                                                                                                                        “openbsd is widely considered to be …” is the claim the author is making; the … is what the author is saying a lot of people believe. this is how english works.

                                                                                                                                                        1. 1

                                                                                                                                                          “openbsd is widely considered to be …” is the claim the author is making

                                                                                                                                                          And that’s the claim I’ve addressed. I quote myself (top level post) here:

                                                                                                                                                          While true (the widely considered part), it definitely isn’t the most secure operating system in the world.

                                                                                                                                                          The explicit nitpick was, very specifically, to ensure nobody falls for the “widely considered” trap and walks away with an unfortunately incorrect belief.

                                                                                                                                                          To achieve this, it’s necessary to point out why the belief is incorrect (a simple counterexample), and what small change would instead make it correct: The most secure UNIX-like.

                                                                                                                                                          So that’s what my post was meant to achieve, and how it was implemented to achieve the intended result.

                                                                                                                                                          What happened instead honestly baffles me.

                                                                                                                                                          1. 2

                                                                                                                                                            it’s subjective whether the statement is misleading, or whether a clarification is useful or needed. the response tells you what others think, which is important if you care about clear communication.

                                                                                                                                                            my last comment was responding to your statment that “it isn’t true unless qualified,” which is different from what it seems you are now saying, that people might misunderstand the “widely considered” part.

                                                                                                                                                            at any rate my subjective opinion is that the statement is fine, and we can expect tech readers not think the author is denying the existence of academic or niche projects that are technically more secure, whatever that means.

                                                                                                                                                        2. 1

                                                                                                                                                          The original statement (“widely considered”) only considers sentiment (and describes it reasonably well, I think). Your statement (”[being] the most secure operating system”) is much harder to deal with, not the least because it assumes that one could sort operating system by “security” - but that depends on the definition of security which is vast.

                                                                                                                                                          As mentioned somewhere else in the thread, seL4 still uses dynamic memory management in kernel, even though it defers that management to a userspace process. Muen (www.muen.sk) is designed to not need that at all, so there won’t be a kernel-needs-memory-but-can’t-get-it situation. Arguably that could be considered more secure than any seL4 configuration. Then again, the entire security model of Muen hinges on Intel VT-x and VT-d, so… maybe not?

                                                                                                                                                          1. 1

                                                                                                                                                            The original statement (“widely considered”) only considers sentiment (and describes it reasonably well, I think).

                                                                                                                                                            Sure, and I addressed that on my post at the top level. To be careful and not confuse the two. As a nitpick.

                                                                                                                                                            Then again, the entire security model of Muen hinges on Intel VT-x and VT-d, so… maybe not?

                                                                                                                                                            I like seL4’s approach to virtualization better anyway. Run VMM (which handles vm exits/exceptions) unprivileged. Since it doesn’t get more capabilities than it needs, an otherwise successful VM escape by attacking the VMM is fruitless.

                                                                                                                                                            so there won’t be a kernel-needs-memory-but-can’t-get-it situation.

                                                                                                                                                            No worries, seL4’s kernel can’t be in that situation.

                                                                                                                                                    1. 2
                                                                                                                                                      1. 10

                                                                                                                                                        I’m confused by “LLVM quietly changing its license”.

                                                                                                                                                        That was a multi-year effort ongoing since 2015, with multiple proposals, reports, a dedicated website and such.

                                                                                                                                                        http://llvm.org/foundation/relicensing/

                                                                                                                                                        1. 3

                                                                                                                                                          It literally means Why wasn’t I consulted?

                                                                                                                                                          1. 1

                                                                                                                                                            Nice rabbit hole :)

                                                                                                                                                        2. 2

                                                                                                                                                          I believe that modern Xorg requires LLVM 10, so they are forced to adopt LLVM 10 regardless of their feelings about the Apache license.

                                                                                                                                                          https://marc.info/?l=openbsd-misc&m=159662446424896&w=2/

                                                                                                                                                        1. 3

                                                                                                                                                          I am satisfied with Nextcloud, but this looks neat. It looks synchronous, though, and I wonder if that’d be a UX impediment to adoption.

                                                                                                                                                          1. 18

                                                                                                                                                            Yup. It is synchronous because it tries to establish a direct connection between the two peers. If, for example, they were on the same local network, the data would not travel out to some server only to be sent back again.

                                                                                                                                                            For asynchronous file transfer I love https://send.firefox.com. I also use https://upspin.io/ and a web gateway to it, but that requires some setup. I hope that changes one day.

                                                                                                                                                            Disclaimer: I’m author of https://webwormhole.io.

                                                                                                                                                            1. 4

                                                                                                                                                              I’d like to plug ffsend here as fully featured CLI tool for Firefox Send. It has been super useful ever since I built it.

                                                                                                                                                              1. 1

                                                                                                                                                                Pretty disappointed that upspin died out.

                                                                                                                                                                1. 1

                                                                                                                                                                  This is just ludicrously good. Thank you so so much.

                                                                                                                                                                  It’s also a great demo of RTCPeerConnection which is really helpful!

                                                                                                                                                                  The only one thing I can think of as feedback: the dictionary used for the keys has some difficult to spell words in it, so if the use case of “read it down the phone” is high on your priorities, it might be better to use a larger number of short words. Or maybe even present the same binary key two ways: would you rather read a phrase or a long number? Depending on language and medium, one might be easier than the other.

                                                                                                                                                                  1. 1

                                                                                                                                                                    Thanks for the feedback. I completely agree the dictionary has to change. We use the PGP word list but some of the word combinations are quite unsavoury. I think it would also be cool to have word lists for different languages.

                                                                                                                                                                    I’d also like to implement word completion at some point which I think might help with spelling.

                                                                                                                                                                    1. 3

                                                                                                                                                                      I have a similar need for a readable string encoding … I started off writing a response in this box but instead it is now a “shorter words list” blog post

                                                                                                                                                                      1. 1

                                                                                                                                                                        I really liked this blog, thanks for doing it :)

                                                                                                                                                                2. 4

                                                                                                                                                                  This is a very different use case than next cloud.

                                                                                                                                                                  1. 3

                                                                                                                                                                    Is it really so materially different? My use case: I want to send a file to someone [which is too big for email] in as non-technical a way as possible.

                                                                                                                                                                    1. 2

                                                                                                                                                                      The point of magic wormhole is easily useable e2e encryption. Nextcloud does not have e2e encryption.

                                                                                                                                                                      1. 1

                                                                                                                                                                        Ignoring the underlying technical details and coming at it from the viewpoint of a user who doesn’t care about security (it’s not part of their use-case spec):

                                                                                                                                                                        Wormhole means if I’m on the phone with you, I can transfer a file to you without setting anything up. We both just open the website, I tell you my code, you type it in, I drag in my file (or you drag in yours), done. Neither of us needs to have/sign up for an account or anything.

                                                                                                                                                                        (I only glanced at nextcloud but my understanding is you need to set it up. Apologies if that’s not right.)

                                                                                                                                                                        1. 1

                                                                                                                                                                          I concur that wormhole (web or CLI) serves a slightly different use case. That said, in fairness, Nextcloud allows you to provide single-URL (one-click) access to a file, and recipients don’t need a user account on the NC server. However, it’s true that you do need to install or set up NC on your server (or use a provider).

                                                                                                                                                                  1. 1

                                                                                                                                                                    It’s unbelievable to what extent people will go to avoid static linking.

                                                                                                                                                                    1. 4

                                                                                                                                                                      I love Acme (and loved reading about its ancestor help)…and I obviously love sam.

                                                                                                                                                                      (As an aside, the look and feel of Acme and Help are very heavily influenced by With’s Oberon system. In fact, Rob Pike’s most popular project, the Go programming language, owes a lot to Oberon too (and Oberon-Two) in terms of the type system, syntax, and so on…I have dead tree copies of a lot of the Oberon books, including the user manual. It is very truly the road not taken in operating systems and user interfaces.)

                                                                                                                                                                      Acme and Sam (and Oberon), however, absolutely require the use of a three-button mouse. Yes, technically, you can get away with a two-button mouse and chording and theoretically they’re usable with a touchpad but…well, I don’t really use my port of sam anymore because I often only have a touchpad and not a mouse (when I’ve got my laptop anywhere but my desk, basically).

                                                                                                                                                                      I’ve often toyed with the idea of reworking my sam port to have a console or GTK frontend that leaned more heavily on the keyboard, but the amount of time it would take is just not something I can give right now. Sam is “just” an editor, however, so it could theoretically be usable mostly from the keyboard…but for Help and Acme, I don’t know how they could meaningfully be made mouseless without changing them so fundamentally that they wouldn’t even be the same thing anymore.

                                                                                                                                                                      1. 2

                                                                                                                                                                        Oberon, in turn, was heavily influenced by Smalltalk-80, another three-button-mouse system on a road not taken. Acme reminds me of a unix-ified Smalltalk workspace.

                                                                                                                                                                        1. 2

                                                                                                                                                                          If anything, there might be more Cedar influence in Oberon. Wirth was there, after all.

                                                                                                                                                                          1. 2

                                                                                                                                                                            Wirth was at Xerox PARC in 1976-77, using an Alto workstation. Not sure exactly what it was running, but you’re quite right that it preceded Smalltalk 80. The Oberon language model isn’t very Smalltalk-like, but the UI seems quite similar in some ways, the reliance on a mouse being perhaps the most obvious.

                                                                                                                                                                        2. 1

                                                                                                                                                                          Acme and Sam (and Oberon), however, absolutely require the use of a three-button mouse.

                                                                                                                                                                          I’ve been using acme with the touchpad + control | option | command keys for years on the MacBook. It works better than using a mouse, because you don’t even need to move your hand anymore.

                                                                                                                                                                          1. 1

                                                                                                                                                                            Allow me to amend my statement: “I am completely incapable of using Acme without a mouse.” :)

                                                                                                                                                                            Seriously though, I did give it the old college try. Never on a Mac, though, always *nix. I would imagine the Mac’s multitouch support helps a lot.

                                                                                                                                                                            1. 1

                                                                                                                                                                              Same. I have a three button mouse on my desktop (AND a touchpad), but on my laptop I use acme with a touchpad just fine. I never bother carrying a mouse.

                                                                                                                                                                          1. 4

                                                                                                                                                                            As someone without any sysadmin type experience whose experience is with personal computers, and servers that have basically one or two users plus root, I think I’m missing some context.

                                                                                                                                                                            Under what circumstances do you want a user’s home directory to not be under /home/?

                                                                                                                                                                            1. 6

                                                                                                                                                                              This kind of workflow is common when you have an infrastructure with many users and many machines, and you want your users to have access to their homedir regardless of which machine they’re logged into.

                                                                                                                                                                              A typical solution to this is to store home directories on a central file server, and mount the home directories via some network filesystem on each machine. Especially if you have multiple file servers and share users across them, you probably want to mount these somewhere other than /home.

                                                                                                                                                                              This is a really common pattern on platforms like HPC compute clusters, where many users are running jobs on a shared set of machines.

                                                                                                                                                                              1. 2

                                                                                                                                                                                Thanks, that’s somewhat helpful, and it’s a world I’m completely unfamiliar with, but what’s the need to run Chromium as themselves on that machine?

                                                                                                                                                                                Admittedly, I have opened a web browser on a production machine, but I felt that represented that company’s “YOLO” approach to infrastructure.

                                                                                                                                                                                1. 6

                                                                                                                                                                                  but what’s the need to run Chromium as themselves on that machine?

                                                                                                                                                                                  …what?

                                                                                                                                                                                  These are the workstations the users use. Desktop systems where they do their work. Users usually want or need browsers.

                                                                                                                                                                                  1. 3

                                                                                                                                                                                    Funny: I literally did not process “log on” properly, I just assumed it was shorthand for connecting to a particular box with ssh or perhaps using an X11 environment.

                                                                                                                                                                                    I remember going to the computer lab in university and grad school, and logging on to a shared machine. But at the time, I didn’t study CS, and didn’t think about how that was implemented. I treated the machines as analogous to a thin client to my webmail, etc. The idea that I had a home directory there and might keep anything there (as opposed to on my laptop) was quite foreign.

                                                                                                                                                                                    Thanks for helping me to connect the dots.

                                                                                                                                                                                  2. 4

                                                                                                                                                                                    In the cluster usage model, I’ve seen a few different workflows that involved using a web browser to attach to a running job. If the cluster and the user’s desktop aren’t on the same network (or there’s a firewall, etc), then they might want to run a browser on a gateway node that mounts those filesystems. I personally think those workflows are pretty ugly, but they exist.

                                                                                                                                                                                    Another usage model I’ve seen, which came to mind a little later, is “we only have a few very powerful workstations, and many more users”. Think really expensive CAD workstations with multiple GPUs. If you have, say, three workstations and fifteen team members, you might “check out” a workstation to do your really intensive design work but otherwise work on a regular PC. In those cases, you’d also want your home directory when you went over to the workstation, rather than having to haul around USB drives etc.

                                                                                                                                                                                    1. 3

                                                                                                                                                                                      This is at the University of Toronto, I would assume. So picture thousands of COMPSCI students using Ubuntu workstations around campus to log in and do course work. As a student, you’re not always going to be on the same machine all the time in this environment.

                                                                                                                                                                                  3. 4

                                                                                                                                                                                    Under what circumstances do you want a user’s home directory to not be under /home/?

                                                                                                                                                                                    If an Unix user entity is used to host some service, not a real end user (say, MediaGoblin) one might want to have its home under /var/lib or even /data/media-goblin. Users aren’t just for end users.

                                                                                                                                                                                  1. 2

                                                                                                                                                                                    We have a blit(1) emulator in Plan 9 that can work with a vax emulator running v8 Unix.

                                                                                                                                                                                    Also an FPGA implementation.

                                                                                                                                                                                    And finally a javascript implementation.