1. 2

    When will FreeBSD switch to libressl? Version 13? The wiki says that maybe it could be version 11 but I think it is not updated.

    https://wiki.freebsd.org/LibreSSL#Base

    1.  

      Even HardenedBSD switched back to OpenSSL because of the maintenance burden. FreeBSD 12 has just upgraded to OpenSSL 1.1.1, so LibreSSL-in-base-and-as-default will come… probably never.

      Honestly, with OpenSSL 1.1, is there any reason to still prefer LibreSSL? LibreSSL has served as a great kick in the ass for OpenSSL, now OpenSSL is back in very active development, including deprecations and cleanups. And it’s up to date with modern TLS, while LibreSSL is still waiting for TLS 1.3 to “stabilize” or whatever ;)

      1.  

        On the other hand I think it would be nice for libtls instead of people hoping they just just the API correctly.

        As for support of libressl. HardenedBSD just like FreeBSD had a really bad time with for example Rust being required by Firefox and their SSL being slow to update for LibreSSL support. Most other projects were faster and often have test cases.

        I think that support for LibreSSL will improve over time. I also think that HardenedBSD has a harder time than OpenBSD and Linux distributions using LibreSSL per default because FreeBSD port maintainers defaulting to OpenSSL will update a package without a LibreSSL patch.

        Given the fact that OpenBSD will keep pushing LibreSSL and pushing stuff upstream I think it’s not giving up on LibreSSL just because HardenedBSD rightfully said that they were to early and that right now they have better stuff to put their ressources in.

        I think if FreeBSD had LibreSSL in base and used it per default, it would be in current for a while and people noticed LibreSSL compatibility problems early it would work out better especially for HardenedBSD because of shared workload.

        Another OpenBSD technology that also used to make problems in terms of compatibility was sndio. Only a while ago ones plan on replacing Pulseaudio with sndio was largely doomed to fail and one has all sorts of problems especially with ports that are not too famous. This has been fixed despite the hype being a lot smaller on sndio than on LibreSSL.

        I think that the never is out of place here. I think that all it takes is a few people maybe at some conference or so coming together and working out a plan on approaching it more effectively than last time.

        Last time a new version ruined it and it went in before broad support making patches or upstreamed code stop working. I really don’t think that’s an unsolvable problem. Also given the fact that the stuff that wasn’t updated quick enough were non-C projects so things where at least theoretically updating the stdlib would be enough to have it work for most code in a certain language.

        I’ think a bit of diversity and stuff like libtls could be good motivators for giving it another try. If involved/interested parties which do exist worked on it together I think the reasons for HardenedBSD switching back - as much as I am aware of them - could be made obsolete.

        However I do agree that the newer OpenSSL did also remove some incentives

        1.  

          Most other projects were faster and often have test cases

          Key word being “most”. Some projects refuse to support LibreSSL – hello Qt5…

          sndio

          That’s a different situation. Apps that play audio usually support multiple backends already, with a well defined and simple (massively simpler than OpenSSL) API for them.

          While LibreSSL tried to be a drop in replacement fork of OpenSSL, and turned out to not be completely drop-in, in a sort of almost “embrace-extend” way.

          libtls

          Can someone port it to OpenSSL 1.1 already? :)

          1.  

            For sndio I don’t think you can dismiss it so easily because your “usually” is my “most”. Yes it worked easily in most cases, but for example for various software around SDR it wasn’t such an easy thing. They ended up switching to things. In one particular example I remember Qt being the way to go.

            However I didn’t mean to say your arguments are wrong, just that I don’t think these problems are so big that it’s unrealistic to expect it happening let’s say for 13.0 if someone picks it up. After all stuff like opensmtpd uses it per default and I can imagine developers liking libtls enough.

            Of course if that goes into OpenSSL that incentive would go away.

            1.  

              Yeah, libtls is cool, but its only users in the ports tree are a couple OpenBSD projects, versus all the things that need libssl/libcrypto.

              As a project with limited resources, FreeBSD pretty much always leans toward things with less maintenance burden.

              I agree that most software works with LibreSSL – I used it as default from ports for over a year – but there are some Very Important things that don’t support it – like the aforementioned Qt.

              And if LibreSSL was the default, the Qt maintainers would have to go “oh damn do we wait for OpenBSD to pick this version up and patch it, or do we re-patch it ourselves” on every Qt release.

              Sure, even that is not the biggest problem, but the thing is, the benefits aren’t that important either.

            2.  

              libtls

              Can someone port it to OpenSSL 1.1 already? :)

              libtls uses a function to load certificates from memory that doesn’t exist in OpenSSL, where only loading from the file system works, but that’s bad for privilege separation and sandboxing.

              1.  

                SSL_CTX_use_certificate_ASN1() loads the ASN1 encoded certificate from the memory location d (with length len) into ctx
                SSL_CTX_use_PrivateKey_ASN1() adds the private key of type pk stored at memory location d (length len) to ctx

                Something exists, is this not it?

        2.  

          I believe FreeBSD also tried making ssl private in base, and I think some issues with libpam squashed it (reference).
          I also vaguely recall reading somewhere about interest in using bearssl (after it hits a stable release) at one point, but maybe that was just meetup discussion notes on the wiki?…

        1.  

          This also fixes tap top of screen to return to top, which breaks when content is scrolled inside an element. (Glaring at you, amp.)

          1.  

            Great point, yes, this also improves usability on iOS. :) I added a note to the bottom of the post.

          1.  

            All future exploit POCs should involve busy beavers.

            1. 2

              What’s wrong with current’s implementation written in ANSI C? Seems like a waste of human resources, in my opinion.

              1. 32

                What’s wrong with current’s implementation written in ANSI C?

                If you want to script a Go application, it’s a hell of a lot easier if the VM is native instead of trying to fight with C interop issues.

                Seems like a waste of human resources, in my opinion.

                Incredible that with zero insight into the business requirements or technical motivations behind a project, you still consider yourself qualified to judge the staffing allocations behind it

                1. 7

                  Incredible that with zero insight into the business requirements or technical motivations behind a project, you still consider yourself qualified to judge the staffing allocations behind it.

                  You are definitely right about this.

                  If you want to script a Go application, it’s a hell of a lot easier if the VM is native instead of trying to fight with C interop issues.

                  As @tedu commented, I understand the motivation behind it, thanks for pointing it out too.

                  1. 2

                    They give some further explanation of the technical requirements that led them not to pick up Shopify’s version here, which I found intersting: https://github.com/Azure/golua/issues/36.

                    The TL;DR is they’ve been scripting Helm for a while using another Lua-in-Go engine, but it isn’t updating to 5.3, which they very much want, along with closer conformance to the Lua spec, plus they have some design ideas that they feel would make debugging etc easier.

                    1. 3

                      To each their own, but I’m a little perplexed that strict 5.3 conformance would be a driver. I’m using gopher-lua which is like 5.1 but not exactly, but it’s just fine. What I need is some scripting language to embed, it’s close enough to lua it’s not really learning a whole new whatever, and it interfaces nicely with go (nicer than a strict port of the stack api would).

                      I don’t know jack about helm, but after seven seconds of research I can’t reverse engineer a requirement for lua 5.3 accept no substitutes.

                      1. 1

                        It’s possible that they have some need of true 64-bit integers, which only came to Lua in 5.3. That’s the only important change I can think of.

                2. 5

                  Writing memory-safe C is an artform and a craft.

                  Writing safe code with Go is something a trained orangutang can do.

                  1. 3

                    Lua bytecode is memory-unsafe in the reference implementation, for starters.

                    1. 2

                      Do you have a link or more details?

                      1. 1

                        My upload server was down, sorry I didn’t link this originally. Here’s a slide deck (pdf) of a presentation I gave at Houston Area Hackers with almost all the details you could want.

                        I still have to update this; there are quite a few more tricks you can pull. I ended up creating a pure-Lua dlsym() implementation for the project that spurred all this research.

                        1. 2

                          Hmmm… Do reimplementations like this one remove support for lightuserdata then? I’m having a hard time imagining a lua interpreter for arbitrary bytecode that supported lightuserdata but could nevertheless guarantee that those pointers were safe.

                          1. 1

                            Yeah. In gopher-lua, userdata actually looks kind of like lightuserdata, which doesn’t exist. When both host and lua state share an allocator, the difference becomes less meaningful.

                            1. 1

                              Looks like it’s unimplemented. lightuserdata is a pretty damn leaky Lua API. Bytecode itself is even more leaky in the Lua C implementation - arbitrary Lua bytecode is as expressive as C. No surprise that MS seemingly wanted to make an API/ABI compatible implementation that fixed some unnecessary leakiness.

                      2. 2

                        It’s annoying to interface with from go code.

                        1.  

                          seems like something that should be fixed in Go rather than reimplementing everything written in C.

                          1.  

                            For better or worse, it’s a result of deliberate design trade-offs in go, so it’s unlikely to change.

                      1. 6

                        Given that most bugs are transient, simply restarting processes back to a state known to be stable when encountering an error can be a surprisingly good strategy.

                        ~ Fred Hebert, Erlang in Anger

                        1. 3

                          This of course is how a great many bots find themselves on my blacklist. If a request returns 404, continuing to pound the same URL does not resolve the error.

                          1.  

                            Which is of course why error handling still needs some amount of contextual logic, e.g. for a 404 the resource is not there so stop trying. Or even a general-purpose retry logic like exponential backoff with a failure cutoff.

                        1. 2

                          It’s becoming increasingly difficult for me to determine who’s right and who’s just a whiner.

                          1. 4

                            Is there a race condition between the failed system call and checking for wake up? What if the time slice expires immediately after a system call failed for a different reason?

                            1. 5

                              Too much inneuendo and not enough fact for me to understand or learn much. I’m pretty sure I could write records keeping software that drops records even if it runs on unix.

                              1. 3

                                I suggest browsing through the thread (start here: https://mm.icann.org/pipermail/tz/2018-November/thread.html#27171) for more discussion, and interesting links.

                              1. 6

                                While it’s true that there are better languages to guarantee memory safety, I would argue it’s not a huge problem in terms of cost compared to other problems of the internet. I think the larger issue is energy cost of the internet. In fact likely most code running the web is NOT c/c++ and instead IS using memory safe languages.

                                The risk of climate change is so huge, and the costs so High (from 10 trillion to human extinction ) that security pales in comparison to the cost of global warming. I simply cannot justify sacrificing performance for security because of this. If using C/C++ can reduce your energy and server use, then it’s a superior choice over safety.

                                Rust is interesting here because it doesn’t sacrifice speed for safety. My biggest concern with it is that it’s just not as pleasant compared to c++. Maybe it will get better genetic programming in the future. But to me energy use is the elephant in the room.

                                1. 4

                                  The use of slower languages also favors “cloud vendors” (people spend more on compute nodes if their code is slow). Those same vendors often promote slower languages. Conflicting priorities for sure.

                                  1. 3

                                    Would be interesting if someone actually tried to measure the impact of scripting language usage on the power consumption of servers around the world in general.

                                    I guess it’s not that big. It’s mostly the small sites that are written in PHP/Python/Ruby/etc. The stuff serving most of the world’s traffic is very efficient. Netflix’s CDN uses in-kernel TLS crypto. Google Search is, I think, C++. Twitter famously migrated from Ruby to Scala and other JVM stuff. And so on.

                                    1. 6

                                      The problem is that most servers are on 24/7 and mostly idle. When people use slower languages, they tend to overprovision servers to handle load spikes. So instead of thinking about “efficiency per request” this about “machines per load”. If you can reduce the number of machines you use by 1/10th by using a faster language, that’s basically a 1/10th reduction in power. And it’s not ONLY a reduction in power but a huge reduction in physical waste.

                                      I think I read somewhere that 90% of machines on the internet are sitting idle.

                                      That’s why I love the idea of unikernels since they boot in miliseconds. If you can have a machine literally turn on and serve a request when a packet comes in, vs having it running idle, it could be a huge win for power. You simply can’t do that with a VM based language whichtypically has a cost to start the VM and requires a VM warmup to be fast.

                                        1. 1

                                          Nice! Didn’t know the name of that, thanks.

                                      1. 2

                                        Stuff like that exists. Hard to Google on mobile cuz lots of crap results. I did find a result that at a glance looks like kind of stuff I was thinking off. Hows this?

                                      2. 3

                                        “In fact likely most code running the web is NOT c/c++ and instead IS using memory safe languages.”

                                        Good point. The new ones are more efficient. That will help address this problem. Hardware people stay getting watts down. People really concerned should probably also use energy-optimized CPU’s like ARM makes for their systems with embedded or laptop peripherals since they’re low power. Such a rig wont have great performance per dollar ratio, though.

                                        1. 2

                                          The ‘energy cost’ to create and maintain a working programmer is pretty substantial, too - to what degree does that dictate ‘use the language that lets you get the most out of that expenditure’?

                                          1. 2

                                            It’s a false dichotomy. C++ is a high level language and modern c++ is very productive. You can make code that almost looks like python but with huge performance over python.

                                            Secondly, energy is the main cost of a data center, so let’s tackle that first. It hasn’t been true for a long time that programmers cost more than the cost of running their software.

                                            Third, we can produce less software in general. Productivity is the greatest source of waste anyway.

                                            1. 9

                                              Third, we can produce less software in general.

                                              Heh. There would be a lot fewer bugs if we fixed the software we have instead of churning out new bugs. But good luck getting developers to admit that.

                                              1. 1

                                                My cookie file has the following quote:

                                                Every program has at least one bug and can be shortened by at least one instruction—from which, by induction, one can deduce that every program can be reduced to one instruction which doesn’t work.

                                                And before you dismiss that as silly, there was a one-instruction program that had a bug.

                                        1. 33

                                          I’ve run into this mentality myself a couple of times, people claim in 2018 that they can write safe C/C++, it’s only those other people that can’t.

                                          1. 7

                                            I would claim that 2018 is the best time (yet) for writing memory-safe C++. We have tooling and paradigms now where we don’t need to rely on raw pointers and can make use of things like shared_ptr and unique_ptr. We can utilize the vector class which gives us OOB write protection (via the vector::at method) . We can even look to the design decisions made in Rust and see they draw their roots from modern C++ (think RIAA). All the protections of Rust already exist in C++, and are distinct enough that tooling can catch when developers don’t use them.

                                            1. 20

                                              I agree about it being best time to write safer C++. Ill add C, too, given all the analyzers. Empirical evidence Gaynor references show vast majority of C/C++ code fails to be safe anyway. So a safe-by-default option is better to use with unsafety turned on selectively when necessary.

                                              Also, the key aspects of Rust’s memory safety come from Cyclone language. It was a safer C with some temporal safety among other things. Clay was another which had linear and singleton types.

                                              1. 15

                                                All the protections of Rust already exist in C++

                                                Unless you are claiming C++ has a way to ensure data race freedom, this does not seem true.

                                                1. 10

                                                  Smart pointers have been around for more than a decade (in Boost before they got into std::), and STL has been around for ages of course. From the memory safety perspective, the tools have been around for a long time. Perhaps the issue is that none of these things are mandatory.

                                                  1. 4

                                                    Halfway there: C++ was built on unsafe-by-default foundation, C, with good protections added that also aren’t mandatory. It evolved from there in a lot of ways I didnt keep up with. I just know that plus its parsing/semantic headaches gave C++ a brutal start if goal was safety on average or high assurance.

                                                  2. 9

                                                    (via the vector::at method)

                                                    Which you have to remember to use instead of the far easier nicer looking []. I’ve seen v.at in a C++ codebase exactly once in my life. As usual, the default C++ is unsafe.

                                                    We can even look to the design decisions made in Rust and see they draw their roots from modern C++ (think RIAA)

                                                    True but not sure of the relevance.

                                                    All the protections of Rust already exist in C++

                                                    They most certainly do not.

                                                    tooling can catch when developers don’t use them

                                                    There are no tools that I’m aware of that guarantee an absence of bugs in a C++ codebase. I’ve used most of them. Bugs still happen. Bugs that don’t happen in safer languages.

                                                    1. 8

                                                      All the protections of Rust already exist in C++

                                                      I’m not sure that’s entirely true, but I could be wrong. I agree with what you are saying about RIAA, but a few other things pop to mind:

                                                      • Rust tries really hard to ensure you have either only immutable references, or a sole mutable reference. This prevents the old “iterate over a collection and mutate it” problem.

                                                      • It also forces you to think about sharing semantics across threads (i.e. Sync trait).

                                                      • In Rust you can design APIs which “consume” their parameters. i.e. you pass ownership to a callee, and it never hands it back. This is useful for scenarios where you don’t want the user to try and re-use a conceptually finalized data structure. Perhaps reuse of the structure could be unsafe for example.

                                                      I’m sure there will be other examples. And I’m sure the proponents of Rust will provide those in more comments ;)

                                                      Maybe you can do these kinds of things in modern C++?

                                                      1. 6

                                                        Problem with C & C++ is not only about memory safety. Are you 100% sure you have absolutely no chance of a signed int overflow anywhere in your codebase?

                                                        1. 15

                                                          Compile with -fwrapv to instantly unlock the same behavior as rust.

                                                          1. 5

                                                            To get the K&R C behavior not the house-of-horrors C standard.

                                                            1. 3

                                                              Or better, -ftrapv to trap even errors that Rust won’t catch. (I don’t understand why this wasn’t made the default behaviour in Rust, to be honest; the performance hit should ultimately be negligible, and they already make some choices for safety-over-performance).

                                                              1. 4

                                                                The reason it is not default in Rust is that performance hit was not negligible.

                                                                1. 0

                                                                  Got some numbers? Part of the problem is probably that the code generator (LLVM) doesn’t handle it very well; but basing a language decision on a current limitation of the code generator doesn’t seem wise. (Especially if it’s a choice between “fail to detect some potentially critical code issue” and somewhat reduced performance).

                                                                  1. 5

                                                                    I don’t have the reference for Rust handy, but NIST Technical Note 1860 is one good reference. To quote the abstract:

                                                                    We performed an experiment to measure the application-level performance impact of seven defensive code options… Of the seven options, only one yielded clear evidence of a significant reduction in performance; the main effects of the other six were either materially or statistically insignificant.

                                                                    That one option is -ftrapv.

                                                                    1. 1

                                                                      That report is specific to GCC, though, which has worse overflow checking even that LLVM.

                                                                      “In practice, this means that the GCC compiler generates calls to existing library functions rather than generating assembler instructions to perform these arithmetic operations on signed integers.”

                                                                      Clang/LLVM doesn’t do that. Needless to say, it explains much of the performance hit, but not how it would affect Rust. I’m curious about what the real numbers do look like.

                                                                    2.  

                                                                      (actually, having done some recent ad-hoc tests LLVM seems to do ok, mostly just inserting a jo instruction after operations that can overflow, which is probably about as good as you can do on x86-64. I’d still like to see some numbers of how this affects performance, though).

                                                              2. 2

                                                                I agree memory safety is not the only kind of security bug, but it is the focus of this article, which is why I focused on it in my comment.

                                                            2. 4

                                                              And how do you know they’re wrong?

                                                              We see evidence all the time. Easily preventable bugs in old software everybody used but nobody wanted to make good. Bugs that would’ve been obvious if people had looked at compiler warnings or run a static analyzer, but nobody did..

                                                              1. 23

                                                                And how do you know they’re wrong?

                                                                I don’t know about @szbalint, but I know they’re wrong because of ample experience with people who think they can write safe C code and can’t. I literally have never met or have worked with an engineer I’d trust to write C code, myself obviously included.

                                                                I’ve been compiling all C and C++ code with -Wall for at least the last 15 years, and the last 10 with -Wall -Wextra -Werror. I’ve used free and paid-for static analysers on production code. Tests, extensive code reviews, valgrind, clang sanitizers, the lot.

                                                                The end result were bugs, bugs, and more bugs, many of them security vulnerabilities. This problem isn’t solvable with developer discipline and tools. We’ve tried. We’ve failed.

                                                                1. 2

                                                                  Has your experience with D been much better? I’ve been able to segfault D because I didn’t realise that SomeClass c; generates a null pointer. I really wish the language were modified to make this impossible without some extra attributes like @trusted.

                                                                  That’s the only way I’ve managed to really “crash” D, though. The rest of the time I get pretty stack traces at runtime whenever I mess up, much more controlled and predictable error-handling.

                                                                  1. 1

                                                                    Has your experience with D been much better?

                                                                    Vastly so.

                                                                    I’ve been able to segfault D because I didn’t realise that SomeClass c; generates a null pointer.

                                                                    That’s not considered an issue in D. Variables are all default-initialised so you’ll get a crash at the point where you forgot to assign the reference to a new instance. Deterministic crashes in the face of bugs is a good thing.

                                                                  2. 2

                                                                    I don’t know what your use involves, but sometimes using e.g. Coverity is not much better than never using it. All those things have to be done in the context of a comprehensive build and test system and always done on every checkin. And there is no substitute for good programmers with solid education. I have met a number of really excellent C programmer who produce solid, trustworthy code.

                                                                    And, as always in such discussions, none of these assertions make sense without a “compared to”. People write unsafe C code, compared to which people who write what?

                                                                    1. 2

                                                                      using e.g. Coverity is not much better than never using it

                                                                      Not in my experience.

                                                                      And there is no substitute for good programmers with solid education.

                                                                      Few and far between.

                                                                      I have met a number of really excellent C programmer who produce solid, trustworthy code.

                                                                      I have not. And that includes meeting compiler writers and C++ luminaries.

                                                                      People write unsafe C code, compared to which people who write what?

                                                                      To people who write code in Ada, Rust, …

                                                                    2. 1

                                                                      The end result were bugs, bugs, and more bugs, many of them security vulnerabilities. This problem isn’t solvable with developer discipline and tools. We’ve tried. We’ve failed.

                                                                      Then why is it that when the bugs that relate to language level issues are dissected in the public, they nearly always seem like they’d be prevented by discipline & disciplined application of tooling?

                                                                      1. 4

                                                                        Even the best programmers have an error rate. Much lower than novice programmers, but it’s still there.

                                                                        1. 1

                                                                          You are stating the obvious here.

                                                                          The question is, why aren’t these errors being caught when it is so easy to do?

                                                                          1. 5

                                                                            The nature of the tooling seems like a prime candidate answer to that question. In C and C++ you often need to opt into constructs and static analyses in order to guard against memory safety bugs. Just look at all the tortured defenses in this thread alone. It’s some combination of “oh just use this particular subset of C++” all the way to “get gud.”

                                                                            In memory safe languages the tooling generally demands that you opt out in order to write code that is susceptible to memory safety bugs.

                                                                            It’s just another instance of “defaults matter.”

                                                                            1. 0

                                                                              I think we’re going in circles here. None of what you say is evidence that people can’t write safe code. You’re just saying they (most of them) won’t, which isn’t the contentious point.

                                                                              1. 5

                                                                                Seems like a distinction without a practical difference. You asked a question, “why aren’t these errors being caught when it is so easy to do,” and I answered with, essentially, “maybe it’s not as easy as you think.” I proposed a reason why it might not be easy.

                                                                            2. 4

                                                                              The question is, why aren’t these errors being caught when it is so easy to do?

                                                                              It isn’t easy. If it were, they’d be caught. The litany of security vulnerabilities everywhere shows us that we’re doing it wrong.

                                                                              1.  

                                                                                Right. Why not?

                                                                            3. 2

                                                                              Did I claim developer discipline wouldn’t prevent bugs? What I’m claiming is that it’s humanly impossible to manage that level of discipline in any large codebase. And I’m claiming that because we have decades of experience telling us that it’s the case.

                                                                          2. 3

                                                                            You raise a good point. I think anyone who thinks their own code is perfect is wrong, but I can’t prove it. I suppose, also, that it could be vacuously true in the sense that maybe their own code is perfect, but they haven’t written production systems entirely by themselves. Was that what you were suggesting?

                                                                            1. 1

                                                                              You raise a good point. I think anyone who thinks their own code is perfect is wrong, but I can’t prove it.

                                                                              Can you write one line of code that is perfect? Why not two? Why not ten? Why not a thousand?

                                                                              Even then, there’s a continuum between perfect and unmitigated disaster. I’ll grant you that I don’t really believe in large scale software ever being perfect in a literal sense of the word. Not even bug free. However, I believe there are individuals who can produce safe software in C. And if given enough control over the platform the code runs on, they can lock things down to virtually guarantee that language level issues are not ever going to turn into full compromise (RCE) or secret leaks.

                                                                              If you need something for a case study, why not take a look at qmail? The fun thing is that the papers and writeups about qmail’s security practices don’t really mention things such as extensive use of static analyzers, fuzzers, and formal verification. Despite that, it has an incredible track record. I think there is much more that could be done to raise the bar.

                                                                              I suppose, also, that it could be vacuously true in the sense that maybe their own code is perfect, but they haven’t written production systems entirely by themselves. Was that what you were suggesting?

                                                                              Security costs time and isn’t sexy. Worst of all, you can’t measure it like you can measure performance or the length of the list of features. So even if someone out there is producing “perfect” code, it’s likely that the project goes mostly unheard of. If one were to dig up and point out such a project, people would just call it a toy / academic exercise / “nobody uses it.” People might say they care about security but they really don’t, they just use whatever is convenient or popular. And when you point out the bugs in that, well, developers being developers, they just pat each other on the back. “Dude, bugs happen! Give ’em a break!”

                                                                              It’s especially bad in any commercial setting, which is why I think it is indeed the case that the people who are capable of writing secure software are, in the end, not doing that, because they don’t get to write an entire production system on their own terms. I don’t think it’s a coincidence that the project I just mentioned is essentially a solo project by one person.

                                                                              I’m in that boat too, sort of. At day job there’s so much code that’d get you kicked out if it were my personal project with a security guarantee. I’m not at liberty to rearchitect and rewrite the software. The markets opt out of security.

                                                                              1. 6

                                                                                Can you write one line of code that is perfect? Why not two? Why not ten? Why not a thousand?

                                                                                It depends on the line. Perfection isn’t just about the semantics the code has to the compiler, but about how a future reader will interpret it. Even the shortest snippet of code can have subtleties that may not be obvious later, no matter how clear they seem to me today. Concretely, in the 90s, “everyone knew” that system() was a huge security vulnerability because of how it adds to the attack surface, and it wasn’t a big deal because code that was thought of as needing to be secure was hardened against it. But those very same semantics came as a very unwelcome surprise to “everyone” when Shellshock was publicized in 2014.

                                                                                Lots of vulnerabilities can be traced to people misunderstanding single lines of code.

                                                                                I very much agree with your point about it being hard to sell security. I think that’s by far the biggest factor in how horrible the current state of affairs is.

                                                                                1. 3

                                                                                  Wasn’t the key shellshock problem that bash executed environment variables? 99% of these failures seem to come from parsing errors, eval, and convenience components. Why bash designers felt it was good to allow arbitrary functions to be passed in environment variables and then executed baffles me but probably came from feature creep. The same functionality could have been achieved more safely by include/load like mechanisms (not 100% safe either, but easier to lock down) or, better, running other programs.

                                                                                  BTW: Bash scripts are memory safe.

                                                                                  1. 2

                                                                                    It depends on what you see as the most unexpected part of it. The reason it was such an emergency to patch it was that bash was exposed to unauthenticated incoming network connections in the first place.

                                                                          3. -3

                                                                            Where are all the safe Rust based applications and operating systems?

                                                                              1. 6

                                                                                I mean, so what? The pool of talented systems programmers is vanishingly small, and also going to be dominated by people using C/C++, so its not that surprising that it hasn’t eradicated all the competition yet. And as a sibling commenter pointed out, its doing pretty well for being all of 3 years old.

                                                                                1. 0

                                                                                  There is a big difference between “has not eradicated the competition yet” and “still can’t point to 5 widely used and superior applications”.

                                                                                  1. 9

                                                                                    how many did python have within 3 years? Or ruby for that matter. How about clojure? None of those had “5 widely used and superior applications” within 3 1/2 years. It was probably a decade before python had 5, maybe a little less for ruby. And you didn’t respond at all to my point about the quantity of systems programmers. Rust is a kind of weird language, attempting to fill a very tight ecological niche with few potential adoptees in terms of actual programmers. I’m not in the least surprised that its still very much in an embryonic state in regards to a community and the works that would flow from that. And, fwiw, Servo and Fuscia aren’t exactly small scale applications, not to mention all of the companies using it internally for whom we have no data/reports other than job board postings.

                                                                                    1. 1

                                                                                      I’m not at all saying Rust is a failure - I don’t find it appealing, but that doesn’t mean anything. But it’s still in the experiment stage. You can declare C obsolete when you have that body of large successful, less buggy, products to point at.

                                                                                      1. 9

                                                                                        The whole point of the article is that literally decades of experience have shown that it is effectively impossible to write safe secure code in C/C++. The goal in fact is to to declare C/C++ a security nightmare. Does that make it obsolete? Maybe, maybe not. There are options out there that give you far more safety guarantees than C/C++ will ever be able to do.

                                                                                        Knowing that it is quite possibly flat our irresponsible to use it for any new project where security is important. Oh, and security is nearly always an important concern for a software project.

                                                                                        1. 1

                                                                                          Let me try again: To make that claim be anything more than marketing, you’d need some (a) data and (b) some indication that C programs were WORSE than some alternative. In the absence of both, it’s just weak marketing. The evidence seems to show more that it is very difficult to develop large software systems that are safe and secure and, at the same time, find an appreciative audience - no matter what the language. As I pointed out before, until there are significant examples of safe/secure/successful Rust programs to compare with, such claims are just blowing smoke.

                                                                                          1.  

                                                                                            While it’s not as strong a claim as you seem to want, I don’t think it’s incorrect to say that C and C++ in the hands of not superhuman developers tends to result in a whole class of bugs that Rust makes nearly impossible to create. Bugs that have appeared numerous times and continue to keep appearing in critical internet infrastructure.

                                                                                            It’s purely anecdotal but Rust has prevented multiple use after free, use before initialized, and buffer overrun errors in my own code multiple times just while playing around. It’s a bit disingenuous to suggest that C/C++ don’t have a problem and that Rust which provably prevents most of those problems isn’t a promising solution.

                                                                                            This isn’t blowing smoke it’s a recognition that there is a problem and possible solution to that problem that doesn’t involve cloning Dan Bernstein and firing all the other programmers.

                                                                                            1. 4

                                                                                              I agree the comment overstated by saying “effectively impossible.” Really hard with higher failure rates than safe languages would be more accurate. Yet, it seems misleading for you to use Bernstein and Hipp since most people are nowhere near as good at QA as them. Bernstein is also a security genius.

                                                                                              C defenders countering with the vulnerability results of security geniuses in minimalist apps instead of what average C coder achieves vs average user of safe language isnt a good comparison.

                                                                                              1. 0

                                                                                                Hence my quest for examples rather than handwaving. Examples of real applications that are not minimalist. Specific examples.

                                                                                                1.  

                                                                                                  Ripgrep? Firefox? The former is shipped and used for search in VS Code, the latter is pretty rapidly moving to using rust where it can to improve performance, and reduce security issues. Just two projects I can name off the top of my head as a non rust programmer.

                                                                                                  1.  

                                                                                                    The same two projects everyone names and I’m not dismissing either of them. I’m just pointing out that the triumphal declarations of the obsolescense of C and dawn of the Reign of Rust lack sufficient backing. If we just got enthusiastic reports about what people wrote in Rust and how well it worked that would be interesting and impressive. But this overblown marketing stuff is just irritating.

                                                                                      2. 5

                                                                                        Well, I guess ripgrep qualifies? Hopefully there will be more such applications.

                                                                                        1. 0

                                                                                          Ripgrep is interesting. Is it considerably less bug ridden than ag ? Is it considerably less bug ridden than grep?

                                                                                          https://www.cvedetails.com/product/23804/GNU-Grep.html?vendor_id=72 Well grep doesn’t seem so bad.

                                                                                          Come on, people need to do better than this.

                                                                                          1. 7

                                                                                            Is it considerably less bug ridden than ag ?

                                                                                            Yes, by a very very large margin across at least a couple different spectrums.

                                                                                            Is it considerably less bug ridden than grep?

                                                                                            Unlikely.

                                                                                            Come on, people need to do better than this.

                                                                                            Right. Nothing will ever be good enough. This is a classic Internet debate tactic. No matter what example anyone gives you, there will always be a reason to dismiss it. I don’t say that flippantly necessarily. Satisfying your standard (mentioned in another comment) to a high degree is nearly impossible. There will always be differences and variables that cannot reasonably be controlled for.

                                                                                            It’s totally fine to have high standards that are impossible to satisfy (it’s your choice), but at least state that up front explicitly and don’t be coy about it. And don’t accuse people who are trying to hit a lower standard of evidence as just “blowing smoke,” because that’s a bunch of bullshit and you know it.

                                                                                            1. -2

                                                                                              Oh come on: ripgrep is the example I’m given. Yet, grep, written in horrible pathetic C, has very few security bugs. Are there CVEs for ag even? I am ready to be persuaded and I don’t need ironclad proofs - I just want to see a number of examples of complex applications/systems written in XYZ that are significantly better than the standard C versions in terms of security. Until those are available it’s just marketing. Maybe Rust is a brilliant advance that will lead to the creation of highly reliable secure applications - maybe it is not. Without evidence, all this claim that C has been obsoleted by Rust or whatever is exactly that - just blowing smoke. Read software jewels by Parnas, this is not a new problem.

                                                                                              I’ll give you a good example: the OP makes a big deal about security issues in ImageMagick! Of all things. I cannot imagine that anyone involved in the development of ImageMagick worried about security at all - it was a tool people could use to manipulate images. Now it is being dragged into service as an online utility exposed to the open internet or used on images that come from anywhere and - lo and behold - because of C it’s insecure! Great, let me see a Rust program designed without any attention to security employed similarly and we’ll compare.

                                                                                    2. 1

                                                                                      Quite a few swift based applications out there though, so it isn’t like new languages aren’t being used for things.
                                                                                      Go is quite popular in certain spaces too, for what it’s worth.

                                                                                  1. 2

                                                                                    Congrats Drew! Looks really cool, will be nabbing a $2/month plan. Modest suggestion: if all the plans offer the same features then I think they should really be the same price, otherwise I think it violates some theory of economics, or at the very least is confusing.

                                                                                    1. 16

                                                                                      The idea is that, because sr.ht is not bootstrapped by investors, the users are the investors. You should choose the plan which matches your financial ability and investment in sr.ht, with the understanding that the money goes directly to supporting the site. It’s a novel financing model, we’ll see how well it works out!

                                                                                      1. 6

                                                                                        I see, so you’re saying that it’s not so much that I am your customer by purchasing a plan but instead I am your investor. Another confusion arises in that case since investors usually acquire some ownership in the company in return for their investment. It seems more like I’m a donor than either investor or customer. If that’s the case, and you’re committing to using the incoming revenue purely for supporting the site, I suggest setting up some sort of nonprofit entity (e.g. in US would be 501(c)(3) or 501(c)(6)) and using donation language instead of product/customer language in the billing section. it would be nice to be able to use this as a tax deduction as well!

                                                                                        1. 4

                                                                                          It’s entirely possible that I will share ownership with the users, or set up a nonprofit in the future. For now, this is a simpler approach. Thanks for the feedback!

                                                                                        2. 4

                                                                                          Reminds me of ‘name your price’ music on BandCamp. I hope it works out.

                                                                                        3. 9

                                                                                          It’s BYO price discrimination. :)

                                                                                          1. 1

                                                                                            Just signed up for the $5/month plan, because seriously, this is about 1.5 Lattes in a high end coffee shop, and for sure, I am spending more on coffee each month than that, … @SirCmpwn: I already love the taste of the site.

                                                                                          1. 2

                                                                                            The range check comments are reversed I think.

                                                                                            1. 1

                                                                                              Oh, that’s right. Thank you! I think the issue is in the README only though. Warnings are given in a proper format.

                                                                                            1. 3

                                                                                              Repeat after me the lesson from the founders of information security: every system, from individual components to their integration, is insecure until proven trustworthy by sufficient analysis. You have to also have a precise statement of what secure means to compare the system against. You then apply methods proven to work for various classes of problems. By 1992, they had found everything from kernel compromises to cache-based, timing channels using such methods. On this topic, every hardware and software component in every centralized or decentralized system has covert channels leaking your secrets. Now, if you’re concerned or want to be famous, there’s something you can do:

                                                                                              Shared Resource Matrix for Storage Channels (1983)

                                                                                              Wray’s Method for Timing Channel’s (1991)

                                                                                              Using such methods were mandatory under the first, security regulations called TCSEC. They found a lot of leaks. High-assurance, security researchers stay improving on this with some trying to build automated tools to find leaks in software and hardware. Buzzwords include “covert channels,” “side channels,” “non-interference proof or analysis,” “information flow analysis (or control),” and “information leaks.” There’s even programming languages designed to prevent accidental leaks in the app or do constant-time implementations for stuff like crypto. Here’s an old Pastebin I did with examples of those, too.

                                                                                              Go forth and apply covert-channel analysis and mitigation on all the FOSS things! :)

                                                                                              1. 4

                                                                                                Shared computers are shared. :)

                                                                                                1. 3

                                                                                                  I like that but oversimplifies it. My goto explanation, smaller than most, is Brian Snow’s from We Need Assurance (2005). I think he really nailed it with this excerpt with just enough info to frame most of the problem:

                                                                                                  “The problem is innately difficult because from the beginning (ENIAC, 1944), due to the high cost of components, computers were built to share resources (memory, processors, buses, etc.). If you look for a one-word synopsis of computer design philosophy, it was and is SHARING. In the security realm, the one word synopsis is SEPARATION: keeping the bad guys away from the good guys’ stuff!

                                                                                                  So today, making a computer secure requires imposing a “separation paradigm” on top of an architecture built to share. That is tough! Even when partially successful, the residual problem is going to be covert channels. We really need to focus on making a secure computer, not on making a computer secure – the point of view changes your beginning assumptions and requirements! “

                                                                                                  1. 2

                                                                                                    That is always a good paper.

                                                                                                    1. 1

                                                                                                      I still probably quote you in some situations. It’s got a nice “Ahhhh…” moment or face-palm trigger built into it depending on my audience.

                                                                                              1. 2

                                                                                                Some weird reverse-OCR bug turning “tt” and “fi” into “!” …?

                                                                                                1. 2

                                                                                                  pdf to text? They render as ligatures, but copy paste as !.

                                                                                                  1. 1

                                                                                                    I don’t know. I’ve seen two letters get turned into one symbol on at least two papers recently. On the other, it was multiple pairs of letters that did that. I fixed them before submit that time.

                                                                                                    I didn’t think they were OCR’d but I don’t know what that looks like now. I know there’s a lot of use of non-PDF formats or applications in academic circles that get exported to PDF. The exporter (translator) could be the source of the errors.

                                                                                                  1. 6

                                                                                                    At the processor level, all of this indirection is completely eliminated. One word structs get passed in registers, and the compiler will eliminate the redundant temporaries and conversions.

                                                                                                    Apparently this is not always the case. Rust has the #[repr(transparent)] annotation to guarantee that a one-member struct gets treated the same way as its member, since some ABIs (ARM64 being the motivating example) treat them differently.

                                                                                                    1. 4

                                                                                                      It seem even arm64 guarantees that small enough structs are passed via register. From Procedure Call Standard for the ARM 64-bit, section “5.4.2 Parameter Passing Rules”:

                                                                                                      If the argument type is a Composite Type whose size cannot be statically determined by both the caller and the callee, the argument is copied to memory and the argument is replaced by a pointer to the copy. (There are no such types in C/C++ but they exist in other languages or in language extensions).

                                                                                                      And:

                                                                                                      If the argument is a Composite Type and the size in double-words of the argument is not more than 8 minus NGRN, then the argument is copied into consecutive general-purpose registers, starting at x[NGRN]. The argument is passed as though it had been loaded into the registers from a double-word-aligned address with an appropriate sequence of LDR instructions loading consecutive registers from memory (the contents of any unused parts of the registers are unspecified by this standard). The NGRN is incremented by the number of registers used. The argument has now been allocated.

                                                                                                      More details obviously at the link :)

                                                                                                      1. 2

                                                                                                        Those excerpts you quote guarantee that a struct containing a double would be passed in a general-purpose register (“NGRN” is “next general-purpose register number”), but stage C.1 of that section says that a bare double of a struct would be passed in a floating-point register.

                                                                                                        So wrapping a primitive in a 1-member struct can introduce inefficiency, at least on ARM64 when that primitive is some kind of floating-point value.

                                                                                                        1. 1

                                                                                                          Good point, so callee will have to move that double from GPR to simd/fp register before doing any operations on it, right?

                                                                                                          It’s a shame that rfc you mentioned is so lacking in details in the motivation section :(

                                                                                                          1. 2

                                                                                                            Imagine an API like:

                                                                                                            typedef struct { double x; } Metres;
                                                                                                            typedef struct { double x; } SquareMetres;
                                                                                                            SquareMetres area(Metres width, Metres height);
                                                                                                            

                                                                                                            The caller has two dimensions as doubles, it has to move them from FPRs to GPRs to make the call, area() moves them from GPRs to FPRs to do the multiplication, then back to a GPR to return the new value, and the caller moves the result back to an FPR to use it.

                                                                                                            It is a shame the motivation isn’t more detailed, but I guess it’s really a document for building consensus about how the Rust language is going to react to a problem, not describing where the problem comes from.

                                                                                                      2. 2

                                                                                                        That’s interesting. I admit to only checking disassembly for amd64.

                                                                                                      1. 8

                                                                                                        One thing that seems like an oversight in C is the lack of struct comparison. You can assign

                                                                                                        struct s a = b;
                                                                                                        

                                                                                                        but you can’t compare them. memcmp() isn’t right because the content of padding bytes in structs is unspecifed. You’re left doing field by field comparisons, which breaks every time someone adds or removes a field.

                                                                                                        1. 3

                                                                                                          Is this a problem in practice? If I have a struct which I want to compare by equality, I will memset() it before initialization and then memcmp() it.

                                                                                                          I don’t want the compiler to generate code that looks at all the initialized fields and checks them one-by-one for equality. Isn’t that a lot slower?

                                                                                                          Ditto with hashing – memset() and then hash the raw bytes rather than look at each field. It seems worse to hash every field separately.

                                                                                                          1. 3

                                                                                                            The C Standard leaves padding as undefined to give leeway to the implementation to produce efficient code. Given the following:

                                                                                                            struct foo
                                                                                                            {
                                                                                                              char c;
                                                                                                              int  x;
                                                                                                              struct
                                                                                                              {
                                                                                                                unsigned int a : 1;
                                                                                                                unsigend int b : 1;
                                                                                                              } flags;
                                                                                                              int d;
                                                                                                             };
                                                                                                            
                                                                                                             struct foo f;
                                                                                                            
                                                                                                             memset(&g,0,sizeof(g));
                                                                                                             f.x = 12345;
                                                                                                             f.d = 32767;
                                                                                                             f.a = 1;
                                                                                                             f.b = 1;
                                                                                                             f.c = 33;
                                                                                                            

                                                                                                            While the padding around f.c might still be 0, it’s not guaranteed (although I’m not aware of any byte-addressable architecture that can’t write a single byte to memory). The unused bits in f.flags will probably not be 0 at all—I can see a compiler, with optimization, reusing a register used to initialize f.d and just set the low byte to 3 before writing to the entire field, thus leaving undefined padding in f.flags.

                                                                                                            That you do the memset() and memcmp() works for you is probably due to the compiler ensuring that. It doesn’t have to. And you might never encounter bad behavior with this if you don’t use bitfields and stick with byte-addressable architectures.

                                                                                                            1. 1

                                                                                                              memset(&g,0,sizeof(g));

                                                                                                              That should be f, not g, right?

                                                                                                              While the padding around f.c might still be 0, it’s not guaranteed

                                                                                                              That’s an interesting question – what about if you use a union there, like:

                                                                                                              struct foo {
                                                                                                                union {
                                                                                                                  char c;
                                                                                                                  char pad[4];
                                                                                                                }
                                                                                                              };
                                                                                                              

                                                                                                              Do you have a citation for the fact that storing c could change other bytes? I feel like C should not have things that work on 99% of architectures but fail on others… I have done some porting in the distant past and this isn’t one of the things that came up.

                                                                                                              That you do the memset() and memcmp() works for you is probably due to the compiler ensuring that. It doesn’t have to.

                                                                                                              Hm I wonder if this is something UBSan would catch? I am currently doing a lot of “punning” in my VM in C++. I am using unions though.

                                                                                                              1. 2

                                                                                                                That should be f, not g, right?

                                                                                                                Yes.

                                                                                                                Do you have a citation for the fact that storing c could change other bytes?

                                                                                                                From the C99 standard, section 6.2.6.1:

                                                                                                                1. When a value is stored in a member of an object of union type, the bytes of the object representation that do not correspond to that member but do correspond to other members take unspecified values.
                                                                                                                1. Where an operator is applied to a value that has more than one object representation, which object representation is used shall not affect the value of the result.43) Where a value is stored in an object using a type that has more than one object representation for that value, it is unspecified which representation is used, but a trap representation shall not be generated.

                                                                                                                footnote 43: It is possible for objects x and y with the same effective type T to have the same value when they are accessed as objects of type T, but to have different values in other contexts. In particular, if == is defined for type T, then x == y does not imply that memcmp(&x, &y, sizeof (T)) == 0. Furthermore, x == y does not necessarily imply that x and y have the same value; other operations on values of type T may distinguish between them.

                                                                                                                I don’t think it answers your direct question (does writing a char affect padding?) but the entire C standard is written in this tortured language to cover everything from 66-bit, 64-bit, 60-bit, 56-bit, 36-bit, 33-bit, 32-bit, 18-bit, 16-bit and 8-bit systems with sign magnitude, 1s-complement and 2s-complement, segmented memory, flat memory, separate I and D space, unified I and D space, with non-IEEE-754 and IEEE-754 floating point thrown in for a good time.

                                                                                                                Oh, and integer trap representations, can’t forget those (although in my 35+ years of programming I’ve never encountered one system with trapping integers, nor sign-magnitude or 1s-complement for that matter).

                                                                                                                Also be careful with type punning via unions. As stated in Appendix J.1 (Unspecified behavior):

                                                                                                                The value of a union member other than the last one stored into (6.2.6.1).

                                                                                                                But like duclare, I wouldn’t mind a struct compare, but that might be impossible to do correctly—do you run strcmp() over char *s? What do you do for other pointer types? Do you get back just equal/not equal, or will you get less than, equal, greater than? Or is just a memcmp()?

                                                                                                                1. 1

                                                                                                                  OK interesting. I just ran my nascent VM under ubsan and it found that my hash function left-shifts large signed integers, but nothing else.

                                                                                                                  I guess I will have to write a rigorous torture test for the parts of the code that do type punning, and run it on every platform. Trying to interpret the standard does seem a bit difficult (and even experts can disagree).

                                                                                                                  It makes me glad that I wrote most of Oil in a high-level language… I’m hoping there will be 5K-10K lines of native code, not 130-160K like bash … The fewer parts of the program that are subject to these portability issues, the better!

                                                                                                                  FWIW I did run into a crash bug with a union used for alignment in CPython under gcc 8! When distros started using gcc 8 instead of 7, Oil would would segfault on startup in CPython code. A point release for Python 2.7 fixed it but I was still on the old one. So yes this area is pretty fraught :-/

                                                                                                                  1. 1

                                                                                                                    But like duclare, I wouldn’t mind a struct compare, but that might be impossible to do correctly—do you run strcmp() over char *s? What do you do for other pointer types? Do you get back just equal/not equal, or will you get less than, equal, greater than? Or is just a memcmp()?

                                                                                                                    Pointer comparison is already a part of C and it does not involve calls to strcmp(), so I don’t see why pointers as members of structs should need any fancier treatment. In practice, doing the equivalent of memcmp() ignoring padding ought to be the simplest way to spec this. You still need to tighten the semantics around unions.

                                                                                                                    1. 1

                                                                                                                      Given the following code:

                                                                                                                      struct foo
                                                                                                                      {
                                                                                                                        int x;
                                                                                                                        char *s;
                                                                                                                      };
                                                                                                                      
                                                                                                                      struct foo a;      
                                                                                                                      struct foo b;
                                                                                                                      
                                                                                                                      a.x = 5;
                                                                                                                      b.x = 5;
                                                                                                                      a.s = malloc(5); strcpy(a.s,"one");
                                                                                                                      b.s = malloc(5); strcpy(b.s,"one");
                                                                                                                      
                                                                                                                      if (a == b) 
                                                                                                                        printf("true\n");
                                                                                                                      else
                                                                                                                        printf("false\n");
                                                                                                                      

                                                                                                                      does it print “true” or “false”? a.s and b.s have different pointer values, but what they point to is semantically the same.

                                                                                                                      1. 2

                                                                                                                        It prints false. This seems obvious. Nobody is asking for deep pointer chasing comparison. They want the == equivalent to memcmp, just as = is equivalent to memcpy.

                                                                                                                        1. 1

                                                                                                                          What tedu said.

                                                                                                                          Why should pointers in a struct be compared any different than how pointers outside a struct are compared? I see no justification. Therefore, this is how it should work:

                                                                                                                          char *a = malloc(5); /* strcpy(a, "one"); */
                                                                                                                          char *b = malloc(5); /* strcpy(b, "one"); */
                                                                                                                          printf(a == b ? "true\n" : "false\n");
                                                                                                                          

                                                                                                                          Besides, it simply makes no sense to make the assumption that two char pointers should compare equal if they point to strings that compare equal. That kind of assumption would break a ton of software. There are legitimate reasons to have equal strings in different locations. That’s even before you consider the possibility of having a char* that does not point to a string at all.

                                                                                                            1. 5

                                                                                                              I clicked on a few of the links that looked interesting to me, and the progress report from the C Memory Object Model Study Group was particularly interesting. The Q&A at the end with regard to effective types and TBAA was pretty interesting to read.

                                                                                                              The “unitialized reads” section was also interesting. In particular, this part stood out to me (emphasis mine):

                                                                                                              Despite the above, several WG14 members said that the intent of the standard was to make all reading of uninitialised values (perhaps except at character type) undefined behaviour.

                                                                                                              Why is it that reading unitialized data at the char level might not necessarily be UB? Or are they saying that it is UB today, but maybe it shouldn’t be?

                                                                                                              1. 4

                                                                                                                It’s unclear if the standard today actually says UB or not. As in, if there aren’t any trap values, then maybe it’s not UB?

                                                                                                                More reading: https://queue.acm.org/detail.cfm?id=3041020

                                                                                                                1. 1

                                                                                                                  Interesting! Towards the end, it seems to suggest that reading from uninitialized chars is indeed UB:

                                                                                                                  Consider the following code:

                                                                                                                  void f(void) { 
                                                                                                                    unsigned char x[1]; /* intentionally uninitialized */ 
                                                                                                                    x[0] ^= x[0]; 
                                                                                                                    printf("%d\n", x[0]); 
                                                                                                                    printf("%d\n", x[0]); 
                                                                                                                    return; 
                                                                                                                  }
                                                                                                                  

                                                                                                                  In this example, the unsigned char array x is intentionally uninitialized but cannot contain a trap representation because it has a character type. Consequently, the value is both indeterminate and an unspecified value. The bitwise exclusive OR operation, which would produce a zero on an initialized value, will produce an indeterminate result, which may or may not be zero. An optimizing compiler has the license to remove this code because it has undefined behavior. The two printf calls exhibit undefined behavior and, consequently, might do anything, including printing two different values for x[0].

                                                                                                                  1. 2

                                                                                                                    Right. So the counter argument was that an indeterminate value is still one value. But this might interfere with a program where, for instance, x is in register r1 in one block and r2 in another block. The compiler must ensure it propagates the value when initialized, but may omit the move if not.

                                                                                                                2. 4

                                                                                                                  So, just for one example, it is impossible to write a driver for a device using memory mapped control registers. I wonder how mmap works with this bizarre proposal?

                                                                                                                  1. 1

                                                                                                                    Why is it that reading unitialized data at the char level might not necessarily be UB?

                                                                                                                    Because the standard tries to reconcile a crappy version of Pascal types with C practice by using a bunch of hacky special cases. They needed the char * exception so that, in principle, it was possible to implement memcpy in C despite the lvalue rules that otherwise forbid it.

                                                                                                                    1. 1

                                                                                                                      Hmm. I guess I still have gaps in my understanding. Why does memcpy need this? It needs to only read from one region and only write to another, right? So as long as you assume the region you read from is initialized, then aren’t you okay since you only need to write to the other region, which could be uninitialized? Or is there some case I’m missing?

                                                                                                                      1. 1

                                                                                                                        You can’t copy a structure that may contain uninitialized data - and even if you initialize all fields, you may have uninitialized padding. If you implement memcpy to do this, it has to use the char * hack.

                                                                                                                        https://shape-of-code.coding-guidelines.com/2017/06/18/how-indeterminate-is-an-indeterminate-value/

                                                                                                                        There are several other places this pops up. For example, when one asks how to do a standard network function like checksum a packet structure, the usual answer from people who find the Standard appealing is that you memcpy your packed packet structure to an int array and checksum that array (packed is another problem, but … ). In order for this hack to work, memcpy has to be able to copy between different types.

                                                                                                                        Of course, once you allow char * to ignore the type rules, you might as well not have them at all, but …

                                                                                                                        1. 1

                                                                                                                          Ah gotya. Thanks. I was forgetting about padding!

                                                                                                                  1. 2

                                                                                                                    My thinkpad didn’t allow installing openbsd either, but after paying the $500000 DMCA fine everything works fine.

                                                                                                                    1. 7

                                                                                                                      Millions of US Government employees succeed at encrypted email every day.

                                                                                                                      1. 6

                                                                                                                        … as long as they are only emailing each other. True, but completely beside the point.

                                                                                                                        1. 1

                                                                                                                          Is it true? Is one in 300 or so Americans working for the government? In a capacity with mandatory encrypted email? Could be, but I gotta ask.

                                                                                                                          1. 2

                                                                                                                            Definitely not 1 in 300. Most intra-gov emails are definitely unencrypted. I had interpreted the comment as “some government employees…”

                                                                                                                            1. 1

                                                                                                                              There are close to 3 million civilian federal employees. I have no idea how well they deal with PGP though.

                                                                                                                              1. 7

                                                                                                                                Not at all. The majority of federal agencies use Exchange, and ~all federal agencies use ID cards with X.509 certs, so many use S/MIME for signed/encrypted emails.

                                                                                                                                There’s a few important caveats:

                                                                                                                                a) It’s actually relatively rare for people to use this; it’s not on by default. People really only use encryption for sending PII. I’m not sure I ever saw anyone (besides myself) use signing besides that. (Not true, now that I think about, at DoD signing was pretty common. Nowhere else though.)

                                                                                                                                b) There’s 0 support for cross-agency encrypted emails. Key management is handled through Exchange’s GAL, so there’s no way for someone at the State Department to send someone at the VA an encrypted email on the unclassified networks, period. Cross agency signature verification sometimes works, depending on the vagueries of path building.

                                                                                                                            2. 1

                                                                                                                              ZixCorp lets you email others outside the system. They get a link to a secure portal, authenticate with out-of-band credentials, and then get the message. Maybe combine that with Keybase for OOB stuff.

                                                                                                                            3. 5

                                                                                                                              Joe Q. Average’s kid sister does not work for the US government, mind you.

                                                                                                                              1. 3

                                                                                                                                We had encrypted e-mail when I was a staff worker at a University. It involved getting a Comodo client side SSL cert and loading it in to the Active Directory GAL. My director accidentally revoked his cold cert by requesting a new one. Someone at a conference the next week talked about getting multiple client SSL certs and I said, “You can’t. It revkos your old ones and you can no longer decrypt your old mail.” Other people chimed in experiencing the same thing and we discovered it was a bug together.

                                                                                                                                Other than at that University and one small open source shop that e-mailed us password with PGP encrypted emails, I’ve only used encrypted e-mails with one other person.

                                                                                                                                1. 1

                                                                                                                                  The trick is using mail guards/gateways and putting crypto in usable proxy in front of legacy clients with similar workflow. Then, IT/ITSEC depts manage it for them. Proven model that isnt followed by FOSS alternatives.

                                                                                                                                1. 3

                                                                                                                                  Perhaps it’s too obvious, but they don’t list wired networks as a defense. Everybody has wifi and a phone, but none of the stuff in my house uses wifi unless it’s actually mobile.

                                                                                                                                  It sounds like the attack would fail if my phone is moving around, since they assume fixed anchors. And if my phone is moving around, that should be even easier to follow through walls.

                                                                                                                                  1. 1

                                                                                                                                    If you read the paper pre-print ( https://arxiv.org/pdf/1810.10109.pdf ) you can see how they handle moving targets.

                                                                                                                                    1. 2

                                                                                                                                      By ignoring them?