1. 48
  1.  

  2. 18

    I have nothing but admiration for your work and the dedication it took to educate yourself well enough to produce what is apparently a good working crypto library. I also think you are literally one in a million for not only being able to do that, but investing the time and effort to actually do it. You are nowhere near the target for the advice to not roll your own crypto.

    In other words, replacing the advice “don’t roll your own crypto“ with “don’t roll your own crypto unless you can spend years learning how to do it properly” is not a significant change to the advice.

    Not to mention that even if you do know what you’re doing, at any moment you might find out you don’t know what you’re doing. Remember when everyone seemed to suddenly realize constant timing was important? Cryptography is a continuous study, not like a data structure you learn once.

    1. 10

      One fear I have is that some people might be discouraged to even start because of the negativity. Though my bigger fear is that the wrong kind of people end up charging ahead.

      1. 11

        I’m just not sure discouraging people is a bad thing. Cryptographic code is extremely unusual, perhaps even unique, in that it can fail catastrophically and silently, years after a perfect test suite passes perfectly. The gap between amateur and professional quality is huge, yet nearly invisible by normal code quality standards. Even the normal tools don’t work: I mean, have you ever seen a unit test framework that automatically tests timing attacks in addition to inputs and outputs? So if you’re going to propagate a one-liner meme about “rolling your own”, there are very good reasons to make it a discouraging one.

        1. 3

          How you discourage people is important in my opinion. One way to do it is to show the gap between casual and professional. Takes more than a one liner, though.

      2. 5

        suddenly realize constant timing was important?

        Suddenly? There were papers at least as early as 1996 (Kocher’s Timing Attacks on Implementations of Diffie-Hellman, RSA, DSS, and Other Systems that appeared in CRYPTO 1996). AES was standardized in 2001. The writing was on the wall if anyone had cared to look.

        1. 11

          That’s why I said people “suddenly realized it was important”, not “it was suddenly important”. It was always important, but the news seemed to take a while to reach a lot of implementors who had thought they knew what they were doing (and probably continued to think that).

          1. 7

            Yep. As late as 2017 (maybe later, that was when I checked), the most popular Java bcrypt library did not compare the hash in constant time when verifying passwords.

      3. 12

        Post quantum cryptography I know even less than Jon Snow. Can’t help you, sorry.

        PQC is also not magic. We achieve this by making algorithms which rely on primitives that aren’t broken by known quantum algorithms. For example, Pete Shor’s algorithm factors numbers and can be applied to elliptic curve point multiplications as well, so RSA and ECC are out.

        However, the best we have for e.g. hashes would be Grover’s algorithm, which (very roughly) makes search problems more efficient, and so might reduce work from exponential to the square root of that exponential. So as long as we have large hashes (256 bits -> 2^256 -> sqrt = 128 bits of security) our constructions can be secure. Lamport signatures, for example, are based on hash functions.

        1. 6

          PQC is also not magic.

          I trust it’s not. It’s just that I only know “Lattices have promises, but the literature doesn’t look stable”, and “Hash based algorithms work well, but they’re slow, and key/signatures are big”. And even then I’m not sure. I figured a humorous quip would be better than ignoring the subject altogether (which at worst could be misleading).

        2. 12

          “You don’t know what you’re doing” is not an insult, it’s a give-or-take statement of fact. People need to stop taking things so personally. Your essay ends with “Make it bug free”. I would guess the industry average is something like 10 bugs for every 1,000 lines of code… Most people thinking of rolling their own crypto, history shows, do not know what they’re doing, lest it wouldn’t be an oft-repeated warning.

          1. 9

            “You don’t know what you’re doing” is not an insult, it’s a give-or-take statement of fact. People need to stop taking things so personally.

            I despise the current culture of deeming everything as possible offensive as much as the other guy. But that is not the issue. Who asserts who gets to say that to whom? Because if someone asks a simple question on stackoverflow they must absolutely be a noob… (/sarcasm). Perhaps the number one expert in information theory in the world never bothered with cryptography particularly and just feels like throwing a simple question. Or perhaps the question is just decisively more complicated than a bunch of one line replies.

            The noob that committed the heresy of pushing their own implementation of rc4 to github must absolutely be pointed out that “thou shall not roll their own crypto”! Ever! Light that holly fire!

            Seriously. The problem with such condescending “statements of fact” is not that they are insulting. The problem is that they are gratuitous assertions of own expertise, 99% of the times with no leg to stand on.

            Furthermore, the well established libraries out there were written by someone. How dare they rolled their own cryptos? Oh.. they are the experts…. ok… how did they become the experts? By writing them?

            When someone states that something “is difficult to understand”, it is more likely that they are the ones that don’t understand it, rather than the person they are talking to.

            1. 3

              The noob that committed the heresy of pushing their own implementation of rc4 to github must absolutely be pointed out that “thou shall not roll their own crypto”! Ever! Light that holly fire!

              Are they using that implementation of RC4 in a piece of software used by a community or paying customers? The crypto being broken or buggy doesn’t matter in a vacuum, it’s also about where and/or why the cryptography is being deployed.

              Seriously. The problem with such condescending “statements of fact” is not that they are insulting. The problem is that they are gratuitous assertions of own expertise, 99% of the times with no leg to stand on.

              I disagree. If I say to someone “don’t try to build a nuclear reactor with americium stolen from fire alarms”, I’m not saying “I’m an expert in radiation and nuclear reactor construction”, I’m saying “from what I’ve seen and heard from the experts, that is a terrible idea”.

              how did they become the experts? By writing them?

              By learning enough to feel competent in writing libraries. Keep in mind, “well established” libraries are the crème de la crème of the n-hundred libraries you will find on Github or wherever, they’re not just random libraries someone decided to put together one day but well-designed, thought out pieces on software developed from a position of deep understanding.

              When someone states that something “is difficult to understand”, it is more likely that they are the ones that don’t understand it, rather than the person they are talking to.

              Maybe. Other situations include: One does understand it, they both understand it, or neither of them under it.

              1. 3

                Seriously. The problem with such condescending “statements of fact” is not that they are insulting. The problem is that they are gratuitous assertions of own expertise, 99% of the times with no leg to stand on.

                It’s true some people are doing this. But mostly what is happening is locals repeating a cultural warning. The locals have no special expertise and don’t think they do. But they know a lot of people have been killed at that beach over there – they may not know why, exactly – and they’re passing the information along.

                This seems like a very good thing to me.

                People who are truly interested in digging into the matter deeper, like the OP, can do so.

              2. 9

                Statements of fact can be insulting, but that can still be effective. Shaming someone into stopping what they are doing can and does work to some extent. Make them look like an idiot, and that would at least deter others from using their work, if they don’t stop altogether.

                People who don’t know what they are doing are generally not even aware of what they should know. Like someone inventing their own cipher without even thinking of authentication. Why? Because they are treating cryptography like a soft magic system. They think they can get away with whatever they come up with because they don’t know how rigid the rules actually are.

                Modern cryptographic primitives and constructions are actually one of the easiest thing to make bug free. Constant time code have extremely predictable code paths, that only depend on the length of its parameters. It’s stupidly easy to test. There are some caveats of course (modular arithmetic, being clever with elliptic curves), but if you do the simple thing and imitate existing designs, there isn’t much room for error. “Bug free” is quite possible.

              3. 5

                Your post starts with the advice that you claim is problematic, but then proceeds to prove it correct. If this were a research paper I’d thank you for publishing it despite finding the opposite that you’ve expected :)

                More seriously though, learning that cryptography is advanced technology and not magic takes years of study, and until then they are indistinguishable and best treated with a healthy distance.

                There are technologies that exist at the overlap of many different areas and that makes them particularly difficult. Applied cryptography is one of them. The need for mathematical formalism, high-stakes security threat modeling, detailed hardware and compiler knowledge are rare combinations.

                1. 2

                  Good essay, thanks! To my (limited) knowledge, it’s mostly at the higher levels that people make mistakes. I haven’t heard that much about buggy implementations of ciphers or hashes, but there are so many cases where people reused symmetric keys, or forgot to initialize nonces, or used the wrong block cipher mode…

                  (One of the things I like about NaCl/libSodium is that it provides higher level operations like CryptoBox that do common things the correct way and are easier for non-expert users to reason about. Those don’t seem to be present in Monocypher; or did I miss something?)

                  One primitive you didn’t address is the RNG, which has been the cause of multiple vulnerabilities. This is of course hard to implement from scratch since you need a good source of entropy, which is very platform dependent. And pretty much every platform already has a RNG.

                  1. 4

                    One of the things I like about NaCl/libSodium is that it provides higher level operations like CryptoBox that do common things the correct way and are easier for non-expert users to reason about. Those don’t seem to be present in Monocypher; or did I miss something?

                    Two things. The first is that crypto_box() is not ideal. There’s no forward secrecy for instance. Pretty hard to do with this API. The second is that Monocypher provides the equivalent detached interface, with crypto_key_exchange() and crypto_lock(). I could have added crypto_box() on top, but to be honest we don’t need it: combining the two functions Monocypher provides is not error prone. Only implementing AEAD would be.

                    One primitive you didn’t address is the RNG, which has been the cause of multiple vulnerabilities. This is of course hard to implement from scratch since you need a good source of entropy, which is very platform dependent.

                    RNG is less hard than you might think. All you need is a stream cipher and a 32 byte random seed. Build fast key erasure on top for forward secrecy, and you’re all set. You don’t even need further entropy. The hard part are maintaining the seed across reboot, or gathering entropy when you boot up; thread safety; fork safety; mitigating temporary accesses to the RNG state. One big reason user space RNG is a nightmare is because users fail to securely split their RNG when they spawn a new process (leading to repeated numbers), or get data races when doing multi threading (repeated numbers again).