But as long as Zig is written in C++, it will never replace C, simply because for those platforms where there is only a C compiler and no LLVM support, Zig cannot replace C.
or it can generate code directly for all of those platforms,
This is also well underway, with swift progress being made for the following areas, by various contributors:
Jakub Konka - Mach-O linking for aarch64 and x86_64, and DWARF
Joachim Schmidt - aarch64 and arm backend codegen
Alexandros Naskos - linking for COFF/PE
Andrew Kelley (me) - ELF & DWARF linking, x86_64 codegen, C backend demo
Noam Preil - C backend
Robin Voetter - SPIR-V
Luuk de Gram & Isaac Freund - WebAssembly
Right now, as far as I know, there is no official language spec for Zig,
we have broken ground on this. We also have Martin Wickham paid part-time by Zig Software Foundation to create the first draft of the language spec. He’s doing great work and we’re having weekly meetings to iron out the details of the language.
First, their argument for Rust (and against C) because of memory safety implies that they have not done due diligence in finding and fixing such bugs.
[…]
And with my bc, I did my due diligence with memory safety. I fuzzed my bc and eliminated all of the bugs.
This seems like such a short-sighted, limited view. Software is not bug-free. And ruling out a class of bugs by choice of technology as a measure of improving overall robustness won’t fix everything, but at the very least it’s a trade-off that deserves more thorough analysis than this empty dismissal.
I think he is probably right (after all, he wrote it) when he says rewriting his bc in Rust would make it more buggy. I disagree this is an empty dismissal, since it is backed by his personal experience.
For the same reason, I think cryptography developers are probably right (after all, they wrote it) when they say rewriting their software in Rust would make it less buggy. So the author is wrong about this. His argument is not convincing why he knows better than developers.
I think there’s a big difference between programs and libraries with stable requirements and those that evolve here. The bc utility is basically doing the same thing that it did 20 years ago. It has a spec defined by POSIX and a few extensions. There is little need to modify it other than to fix bugs. It occasionally gets new features, but they’re small incremental changes.
Any decision to rewrite a project is a trade off between the benefits from fixing the accumulated technical debt and the cost of doing and validating the rewrite. For something stable with little need of future changes, that trade is easy to see: the cost of the rewrite is high, the benefit is low. In terms of rewriting in a memory-safe language, there’s an additional trade between the cost of a memory safety vulnerability and the cost of the rewrite. The cost of Heartbleed in OpenSSL was phenomenal, significantly higher than the cost of rewriting the crypto library. In the cast of bc, the cost of a memory safety bug is pretty negligible.
Data from Microsoft’s Security Response Center and Google’s Project Zero agree that around 70-75% of vulnerabilities are caused by memory safety bugs. Choosing a language that avoids those by construction means that you can focus your attention on the remaining 25-30% of security-related bugs. The author talks about fuzzing, address sanitiser, and so on. These are great tools. They’re also completely unnecessary in a memory-safe language because they try to find classes of bugs that you cannot introduce in the first place in a memory-safe language (and they do so probabilistically, never guaranteeing that they’ve found them all).
If you’re starting a new project, then you need a really good reason to start it in C and pay the cost of all of that fuzzing.
Data from Microsoft’s Security Response Center and Google’s Project Zero agree that around 70-75% of vulnerabilities are caused by memory safety bugs. Choosing a language that avoids those by construction means that you can focus your attention on the remaining 25-30% of security-related bugs.
There’s an implied assumption here that if a language is memory safe, those memory safe bugs will simply go away. In my experience, that is not quite true. Sometimes those memory safety bugs will turn into logic bugs.
Not to pick on Rust here, but in Rust it is very common to put values into an array and use array indices instead of pointers when you have some kind of self-referential data structure that’s impossible to express otherwise using rust’s move semantics. If you simply do such a naive transformation of your C algorithm, your code will be memory safe, but all your bugs, use after free, etc, will still be there. You just lifted them to logical bugs.
Rust has no good abstractions to deal with this problem, there are some attempts but they all have various practical problems.
Other languages like ATS and F* have abstractions to help with this problem directly, as well as other problems of logical soundness.
Right - but in lifting these from memory bugs to logic bugs, you get a runtime panic/abort instead of a jump to a (likely-attacker-controllable) address. That’s a very different kind of impact!
You don’t get a panic if you access the “wrong” array index. The index is still a valid index for the array. Its meaning (allocated slot, free slot, etc), is lost to the type system, though in a more advanced language it need not be. This later leads to data corruption, etc, just like in C.
It leads to a much safer variant of data corruption though. Instead of corrupting arbitrary memory in c or c++ (like a function pointer, vtable, or return address), you are only corrupting a single variable’s value in allocated and valid and aligned memory (like a single int).
You would get a panic in rust for every memory corruption bug that could cause arbitrary code execution, which is what matters.
This later leads to data corruption, etc, just like in C.
Can you expand on this? I had expected the behaviour in Rust to be significantly safer than C here. In C, the data corruption caused by use-after free often allows an attacker to execute arbitrary code.
I totally see your point about logical corruption (including things like exposing critical secrets), but I don’t follow that all the way to “just like C”. How would an array index error be exploited in Rust to execute arbitrary code?
I have once written a bytecode interpreter in C++, for a garbage collected scripting language. I implemented my own two-space garbage collector. For performance reasons, I didn’t use malloc() directly, but instead allocated a big enough byte array to host all my things. If I overflew that array, Valgrind could see it. But if I messed up it’s internal structure, no dice. That heap of mine was full of indices and sizes, and I made many mistakes that caused them to be corrupted, or somehow not quite right. And I had no way to tell.
I solved this by writing my own custom heap analyser, that examined the byte array and tell me what’s in there. If I see all my “allocated” objects in order, all was well. Often, I would see something was amiss, and I could go and fix the bug. Had I written it in Rust instead, I would have had to write the exact same custom heap analyser. Because Rust wouldn’t have prevented me from putting the wrong values inside my array. It’s perfectly “safe” after all, to write gibberish in that array as long as I don’t overflow it.
Now could this particular bug lead to arbitrary code execution? Well, not quite. It would generate wrong results, but it would only execute what my C++/Rust program would normally execute. In this case however, I was implementing a freaking scripting language. The code an attacker could execute wasn’t quite arbitrary, but it came pretty damn close.
The effects of data corruption depend on what the code does with the data. This often means arbitrary code execution, but not always. It’s not a property of C, it’s a property of the code. This doesn’t change when you change the implementation language.
Fundamentally there is no semantic difference between a pointer in a C heap and an array index into a Rust array. In fact some sophisticated blog authors that explain this array technique often point out they compile to the exact same assembly code. It’s what the code does with the data that leads to exploitation (or not).
Of course Rust has many additional safety advantages compared to C, buffer overflows don’t smash the stack, etc, and using references in Rust if you can is safe. And when using references, there’s a great deal of correlation between Rust’s notion of memory safety and true logic safety. This is good! But many people don’t realise that this safety is predicated on the lack of aliasing. The borrow checker is only a mechanism to enforce this invariant, it’s not an operative abstraction. It’s the lack of aliasing that gets you the safety, not the borrow checker itself. When you give up aliasing, you lose a lot of what Rust can do for you. Virtually everybody understands that if you introduce unsafe pointers, they give up safety, but less people seem to understand that introducing aliasing via otherwise safe mechanism has the same effect. Of course, the program continues to be memory safe in Rust terms, but you lose the strong correlation between memory safety and logic safety that you used to have.
Not that there’s anything wrong with this, mind you, it’s just something people need to be aware of, just as they are already aware of the tradeoffs that they make when using unsafe. It does make a projection for the number of bugs that Rust can prevent in practice more difficult, though.
I think this is incorrect. Arbitrary code execution does not mean “can execute an arbitrary part of my program due to a logic bug”, it means “can execute arbitrary code on the host, beyond the code in my program”. Even a rust alias logic bug dies not open up this kind of arbitrary code execution exposure because you can’t alias an int with a function pointer or a vtable or a return address on the stack, like you can in c or c++. You can only alias an int with an int in safe rust, which is an order of magnitude safer and really does eliminate an entire class of vulnerabilities.
I think this is incorrect. Arbitrary code execution does not mean “can execute an arbitrary part of my program due to a logic bug”, it means “can execute arbitrary code on the host, beyond the code in my program”.
In the security research world, we usually treat control of the program counter (the aptly named rip on x86-64) as “arbitrary code execution.” You can do a surprising of programming using code that’s already in a process without sending any byte code of your own with return-oriented programming.
But does Rust let you do that here? What does a snippet of Rust code look like that allows attacker-controlled indexing into an array escalate to controlling the program counter?
Surely you agree that “variables changing underfoot” implies “programs flow becomes different from what I expect”. That’s why we use variables, to hold the Turing machine state which influences the next state.
A logical use after free means “variables changing underfoot”. You don’t expect a free array slot’s value (perhaps now reallocated) to change based on some remote code, but it does.
Right, but “program flow becomes different from what I expect, but it still must flow only to instruction sequences that the original program encoded” is much much safer than “program flow can be pointed at arbitrary memory, which might not even contain instructions, or might contain user-supplied data”.
Maybe not directly within the native code of the program itself, but I think (at least part of) 4ad’s point is that that’s not the only level of abstraction that matters (the memory bug vs. logic bug distinction).
As an example, consider a CPU emulator written entirely in safe Rust that indexes into a u8 array to perform its emulated memory accesses. If you compile an unsafe program to whatever ISA you’re emulating and execute it on your emulator, a bad input could still lead to arbitrary code execution – it’s at the next semantic level up and not at the level of your program itself, but how much does that ultimately matter? (It’s not really terribly different than ROP – attacker-controlled inputs determining what parts of your program get executed.)
That’s admittedly a somewhat “extreme” case, but I don’t think the distinction between programs that do fall into that category and those that don’t is terribly clear. Nearly any program can, if you squint a bit, be viewed essentially as a specialized interpreter for the language of its config file (or command-line flags or whatever else).
There’s no distinction here. If your program implements a cpu emulator then your program can execute with no arbitrary code execution at all and still emulate arbitrary code execution on the virtual cpu. If you want the virtual program executing to not possibly execute arbitrary virtual instructions, you need to generate the virtual program’s instructions using a safe language too.
In most cases, though, arbitrary virtual code execution is less dangerous than arbitrary native code execution, though that’s beside the point.
So…we agree? My point was basically that attacker-controlled arbitrary code execution can happen at multiple semantic levels – in the emulator or in the emulated program (in my example), and writing the emulator in a safe language only protects against the former, while the latter can really be just as bad.
Though I realize now my example was poorly chosen, so a hopefully better one: even if both the emulator and the emulated program are written in memory-safe languages, if the emulator has a bug due to an array-index use-after-free that causes it to misbehave and incorrectly change the value of some byte of emulated memory, that destroys the safety guarantees of the emulated program and we’re back in arbitrary-badness-land.
Sure but this is just as meaningful as talking about a cpu hardware bug that might cause a native safe program to run amok. Technically true but not very useful when evaluating the safe programming language
Yeah exactly, this is the part I thought @4ad was arguing was possible. Eg. in C, use-after-free often allows me to make the program start interpreting attacker-provided data as machine code. I thought this is what 4ad was saying was also possible in Rust, but I don’t think that’s what they are claiming now.
To me, that’s a big difference. Restricting the possible actions of a program to only those APIs and activities the original code includes, vs C where any machine code can be injected in this same scenario, is a major reduction in attack surface, to me.
One thing to note is that code is data and data is code, in a true, hard-mathematical sense.
The set of
the possible actions of a program to only those APIs and activities the original code includes,
and
C where any machine code can be injected in this same scenario
is exactly the same (unbounded!). Of course it is much easier in practice to effect desired behavior when you can inject shell code into programs, but that’s hardly required. You don’t need to inject code with ROP either (of course ROP itself is not possible in Rust because of other mitigations, this is just an example).
Please note that in no way I am suggesting that Rust is doing anything bad here. Rust is raising the bar, which is great. I want the bar raised even higher, and we know for a fact that this is possible today both in theory and practice. Until we raise the bar, I want people to understand why we need to raise the bar.
At the end of a day you either are type safe or you aren’t. Of course the specifics of what happens when you aren’t type safe depend on the language!
PS: arrays can contain other things than integers, e.g. they can contain function pointers. Of course you can’t confuse an int with a function pointer, but using the wrong function pointer is pretty catastrophic.
In my mind, there are two differences between this code in Rust and similar code in C:
This only allows the user to launch nuclear missiles; it does not allow them to, say, write to the harddrive or make network calls (unless one of the actions contained code that did that ofc); in C, I’d likely be able to make something like this call any system function I wanted to, whether machine code to do that was present in the original binary or not.
In Rust, this doesn’t allow arbitrary control flow, I can’t make this jump to any function in the binary, I can only trick it into running the wrong Action; in C, I can call run on any arbitrary object anywhere in the heap.
ie. in C, this would let me execute anything in the binary, while in Rust it still has to abide by the control flow of the original program, I thought was the case, anyway.
I think you’re saying this is wrong, can you explain how/why and maybe show a code example if you can spare the time?
Good analysis. You didn’t use the words, but this is a great description of the distinction between stocks and flows: https://en.wikipedia.org/wiki/Stock_and_flow. I wish more people talking about software paid attention to it.
I would also argue that crypto should not change often, like bc. You might add ciphers, or deprecate old ones, but once a cipher is written and tested, there should be very little need for it to change. In my opinion.
For the same reason, I think cryptography developers are probably right (after all, they wrote it) when they say rewriting their software in Rust would make it less buggy.
Have they actually rewrote anything? Or have they instead selected a different crypto library they trust better than the previous one? On the one hand, Rust has no advantage over C in this particular context. On the other hand, they may have other reasons to trust the Rust library better than the C one. Maybe it’s better tested, or more widely used, or audited by more reputable companies.
If I take your word for it however, I have to disagree. Rewriting a cryptographic library in Rust is more likely to introduce new bugs, than it is to fix bugs that haven’t already been found and fixed in the C code. I do think however that the risk is slim, if they take care to also port the entire test suite as well.
In the Cryptography case isn’t the Rust addition some ASN.1 parsing code? This is cryptography adjacent but very much not the kind of different that your point about cryptography code makes. Parsing code unless it is very trivial and maybe not even then tends to be some of the more dangerous code you can write. In this particular case Rust is likely a large improvement in both ergonomics for the parsing as well as safety.
You’ve got a point. I can weaken it somewhat, but not entirely eliminate it.
I don’t consider ASN.1 “modern”. It’s over complicated for no good reason. Certificates can be much, much simpler than that: at each level, you have a public key, ID & expiration date, a certificate of the CA, and a signature from the CA. Just put them all in binary blobs, and the only thing left to parse are the ID & expiration date, which can be left to the application. And if the ID is an URL, and the expiration date is an 64-bit int representing seconds from epoch, there won’t be much parsing to do… Simply put, parsing certificate can be “very trivial”.
Another angle is that if you need ASN.1 certificates, then you are almost certainly using TLS, so you’re probably in a context where you can afford the reduced portability of a safer language. Do use the safer language in this case.
Yet another angle is that in practice, we can separate the parsing code from the rest of the cryptographic library. In my opinion, parsing of certificate formats do not belong to a low-level cryptographic library. In general, I believe the whole thing should be organised in tiers:
At the lowest level, you have the implementation of the cryptographic primitives.
Just above that, you have constructions: authenticated encryption, authenticated key exchange, PAKE…
Higher up still, you have file format, network packet formats, and certificates. They can (and should) still be trivial enough that even C can be trusted with them. They can still be implemented with zero dependencies, so C’s portability can still be a win. Though at that level, you probably have an idea of the target platforms, making portability less of a problem.
Higher up still is interfacing with the actual system: getting random numbers, talking to the file system, actually sending & receiving network packets… At that level, you definitely know which set of platforms you are targetting, and memory management & concurrency start becoming real issues. At that point you should seriously consider switching to a non-C, safer language.
At the highest level (the application), you should have switched away from C in almost all cases.
For the same reason, I think cryptography developers are probably right (after all, they wrote it) when they say rewriting their software in Rust would make it less buggy. So the author is wrong about this. His argument is not convincing why he knows better than developers.
This is a fair point. When it comes down to it, whether I am right or wrong about it will only be seen in the consequences of the decision that they made.
Here’s the more thorough analysis you’re asking for: this is cryptographic code we’re talking about. Many assumptions that would be reasonable for application code simply does not apply here:
Cryptographic code is pathologically straight-line, with very few branches.
Cryptographic code has pathologically simple allocation patterns. It often avoids heap allocation altogether.
Cryptographic code is pathogenically easy to test, because it is generally constant time: we can test all code paths by covering all possible input & output lengths. If it passes the sanitizers & valgrind under those conditions, it is almost certainly correct (with very few exceptions).
I wrote a crypto library, and the worst bug it ever had wasn’t caused by C, but by a logic error that would have happened even in Haskell. What little undefined behaviour it did have didn’t have any visible effect on the generated code.
Assuming you have a proper test suite (that tests all input & output lengths), and run that test suite with sanitisers & Valgrind, the kind of bug Rust fixes won’t occur in your cryptographic C code to begin with. There is therefore no practical advantage, in this particular case to using Rust over C. Especially when the target language is Python: you have to write bindings anyway, so you can’t really take advantage of Rust’s better APIs.
These bugs still occur in critical software frequently. It is more difficult and time consuming to do all of the things you proposed than it is to use a safer language (in my opinion), and the safer language guarantees more than your suggestions would. And there’s also no risk of someone forgetting to run those things.
These bugs still occur in critical software frequently.
Yes they do. I was specifically talking about one particular kind of critical software: cryptographic code. It’s a very narrow niche.
It is more difficult and time consuming to do all of the things you proposed than it is to use a safer language (in my opinion)
In my 4 years of first hand experience writing cryptographic code, it’s really not. Rust needs the same test suite as C does, and turning on the sanitizers (or Valgrind) on this test suite is a command line away. The real advantage of Rust lies in its safer API (where you can give bounded buffers instead of raw pointers). Also, the rest of the application will almost certainly be much safer if it’s written in Rust instead of C.
And there’s also no risk of someone forgetting to run those things.
Someone who might forget those things has no business writing cryptographic code at all yet, be it in C or in Rust. (Note: when I started out, I had no business writing cryptographic code either. It took over 6 months of people findings bugs and me learning to write a better test suite before I could reasonably say my code was “production worthy”.)
Rusts advantage goes much further than at the api boundary, but again the discussion should be around how to get safer languages more widely used (ergonomics, platform support) and not around “super careful programmers who have perfect test suites and flawless build pipelines don’t need safer languages”. To me it is like saying “super careful contractors with perfect tools don’t need safety gear”, except if you make a mistake in crypto code, you hurt more than just yourself. Why leave that up to human fallability?
Rusts advantage goes much further than at the api boundary
Yes it does. In almost all domains. I’m talking about modern cryptographic code.
again the discussion should be around how to get safer languages more widely used (ergonomics, platform support)
Write a spec. A formal one if possible. Then implement that spec for more platforms. Convincing projects to Rewrite It In Rust may work as a way to coerce people into supporting more platforms, but it also antagonises users who just get non-working software; such a strategy may not be optimal.
not around “super careful programmers who have perfect test suites and flawless build pipelines don’t need safer languages”.
You’re not hearing me. I’m not talking in general, I’m talking about the specific case of cryptographic code (I know, I’m repeating myself.)
In this specific case, the amount of care required to write correct C code is the same as the amount of care required to write Rust code.
In this specific case, Rust is not safer.
In this specific case, you need that perfect test suite. In either language.
In this specific case, you can write that perfect test suite. In either language.
except if you make a mistake in crypto code, you hurt more than just yourself. Why leave that up to human fallability?
I really don’t. I root out potential mistakes by expanding my test suite as soon as I learn about a new class of bugs. And as it happens, I am painfully aware of the mistakes I made. One of them was even a critical vulnerability. And you know what? Rust wouldn’t have saved me.
Here are the bugs that Rust would have prevented:
An integer overflow that makes elliptic curves unusable on 16-bit platforms. Inconvenient, but (i) it’s not a vulnerability, and (ii) Monocypher’s elliptic curve code is poorly suited to 16-bit platforms (where I recommend C25519 instead).
An instance of undefined behaviour the sanitizers didn’t catch, that generated correct code on the compilers I could test. (Note that TweetNaCl itself also have a couple instances of undefined behaviour, which to my knowledge never caused anyone any problem so far. Undefined behaviour is unclean, but it’s not always a death sentence.)
A failure to compile code that relied on conditional compilation. I expect Rust has better ways than #ifdef, though I don’t actually know.
Here are the bugs that Rust would not have prevented:
Failure to wipe internal buffers (a “best effort” attempt to erase secrets from the computer’s RAM).
A critical vulnerability where fake signatures are accepted as if they were genuine.
Lesson learned: in this specific case, Rust would have prevented the unimportant bugs, and would have let the important ones slip through the cracks.
In this discussion, I think it is important to remind that cryptography developers are explicitly and intentionally not writing modern cryptographic code. One thing they want to use Rust on is ASN.1 parsing. Modern cryptographic practice is that you shouldn’t use ASN.1 and they are right. Implementing ASN.1 in Rust is also right.
In this specific case, the amount of care required to write correct C code is the same as the amount of care required to write Rust code.
I disagree.
In this specific case, Rust is not safer.
I disagree here too.
In this specific case, you need that perfect test suite. In either language.
I partially agree. There is no such thing as a perfect test suite. A good crypto implementation should have a comprehensive test suite, of course, no matter the language. But that still isn’t as good as preventing these classes of bugs at compile time.
Rust wouldn’t have saved me.
Not really the point. Regardless of how lucky or skilled you are that there are no known critical vulnerabilities in these categories in your code, that disregards both unknown vulnerabilities in your code, and vulnerabilities in other people’s code as well. A safe language catches all three and scales; your method catches only one and doesn’t scale.
Note that I did go the extra mile and went a bit further than Valgrind & the sanitisers. I also happen to run Monocypher’s test suite under the TIS interpreter, and more recently TIS-CI (from TrustInSoft). Those things guarantee that they’ll catch any and all undefined behaviour, and they found a couple bugs the sanitisers didn’t.
that disregards both unknown vulnerabilities in your code
Sure there is. Other people write cryptographic code too. Unless you are here just arguing against safe languages for only this single project? Because it seemed like a broader statement originally.
I went out of my way not to scale.
I mean scale as in other developers also writing cryptographic software, not scale as in your software scaling up.
Sure there is. Other people write cryptographic code too. Unless you are here just arguing against safe languages for only this single project
I was talking about Monocypher specifically. Other projects do have dependencies, and any project that would use Monocypher almost certainly has dependencies, starting with system calls.
I mean scale as in other developers also writing cryptographic software, not scale as in your software scaling up.
Fair enough. I was thinking from the project’s point of view: a given project only need one crypto library. A greenfield project can ditch backward compatibility and use a modern crypto library, which can be very small (or formally verified).
Yes, other people write cryptographic code. I myself added my own to this ever growing pile because I was unsatisfied with what we had (not even Libsodium was enough for me: too big, not easy to deploy). And the number of bugs in Monocypher + Libsodium is certainly higher than the number of bugs in Libsodium alone. No doubt about that.
Another reason why crypto libraries written in unsafe languages don’t scale, is the reputation game: it doesn’t matter how rigorously tested or verified my library is, if you don’t know it. And know it you cannot, unless you’re more knowledgeable than I am, and bother to audit my work yourself, which is prohibitively expensive. So in practice, you have to fall back to reputation and external signs: what other people say, the state of documentation, the security track record, issues from the bug tracker…
Why make a choice which prevents an entire class of bugs when you could simply put in extra time and effort to make sure you catch and fix them all?
Why lock your doors when you can simply stand guard in front of them all night with a baseball bat?
While personally would back the cryptography devs’ decision here, I think there is a legitimate discussion to be had around whether breaking compatibility for some long-standing users is the right thing to do. This post isn’t contributing well to that discussion.
Oh, imagine if Mozilla, Microsoft, and Google knew they can just fuzz their C code to get rid of all the bugs! These foolish companies have spent so much money and engineering time on their security programs, patching CVE after CVE, of which ~70% were caused by memory unsafety. If only they knew…
With C it’s always “all these vulnerabilities don’t count”, because you should have fuzzed more, you should have got better sanitizers, you should have got better programmers — for 40 years. The real futility is in “just don’t write bugs” approach.
The safer languages, on top of built-in safety guarantees, also support fuzzing and memory sanitizers, so you can perform your usual due diligence there too.
“Battle-tested” is a weak property. It only applies to software that’s unmaintained. As soon as you change the code, even a single line, that code isn’t proven to be reliable any more (remember when Debian broke SSH key generation to silence a compiler warning?).
Being battle-tested is merely an effect of software getting shipped. So ship the new stuff, and it’ll get battle-tested too.
This is cryptographic code we’re talking about. That fuzzing test suite you need in C? You also need it in Rust. And now the only difference between C and Rust is the fact that with C, you need to run your existing test suite under Valgrind and the sanitizers. Push a button, wait for a minute at most, and you’ll get almost all the safety Rust guarantees.
Because unlike most application code, cryptographic code, thanks to being constant time, is stupidly easy to test. And test it you must, considering the stakes. It is one of the few cases where “just don’t write bugs” is both mandatory and possible.
“Battle-tested” is a weak property.
I agree. What you really want is a kickass test suite.
Test suites are not enough. This is proven time and time again. Any class of bugs you can move from “has to be caught by a test or fuzzer” to “can’t occur in the first place” is a huge win. Cryptographic code should desire these properties even more because it is usually used when security and correctness is paramount.
I would love to go even further and be able to prove constant time algorithms or higher level mathematical relationships statically, etc. Perhaps with theorem provers.
The discussion of the tradeoffs should be around language ergonomics and platform support, not around “but c basically has the same guarantees if you squint”. It absolutely does not.
Cryptographic code really is different. Yes, you want correctness, safety and whatnot. More than for pretty much anything else, save real time avionics and lethal medical devices. Thing is, Rust does not help you there.
The discussion of the tradeoffs should be around language ergonomics and platform support, not around “but c basically has the same guarantees if you squint”. It absolutely does not.
Oh but I am discussing tradeoffs there. C has (as of early 2020) substantially better platform support, significantly worse API, and in the specific case of cryptographic code, a negligible safety disadvantage.
Yes, for most domains, the safety advantage of Rust is freaking huge. I’m only talking about cryptographic code, that I have first hand experience with. Yes, you still need to have some kickass test suite, but Rust would need the exact same test suite anyway. Yes, you need to use sanitizers & Valgrind, but that’s often trivial for cryptographic code, since it tends to have zero dependencies to begin with.
I’m not squinting here, I’m just looking at the application domain at hand. In this particular case, C actually provides basically the same guarantees as Rust. I wouldn’t have expected this 4 years ago when I started working on Monocypher (I actually considered Rust, but favoured C’s portability), but now I can confidently take it for granted: libraries that:
do not allocate on the heap;
have code paths independent from the content of their inputs;
have no dependencies;
are only marginally safer when implemented in Rust, provided they have a good test suite. Modern cryptographic code fits that bill. It’s probably the only niche that does.
That said, cryptographic code doesn’t stand in a vacuum. It must be used somehow. C APIs are naturally unsafe, and the rest of the application will likely hugely benefit from Rust’s guarantees. The crypto library must then have a Rust interface. We could write bindings, but it’s a significant hassle. I would never chose C over Rust for new cryptographic code for a Rust project. For old cryptographic code, I wouldn’t mind considering a reputable C library, though I’d likely favour a Rust library instead.
Would you trust a C crypto lib more than HACL*, a lib written in F* (a dependently-typed language) and formally verified for memory safety, correctness, and resistance to side-channel attacks?
I wouldn’t. Monocypher’s comparative advantage are its simplicity, it’s ease of use, its ease of deployment, and its portability (part of its portability comes from its relatively small code size). I did sacrifice confidence a bit to get there.
I respect your opinion but boy I don’t understand it. I would take a formally verified library any day, assuming it is an option. And I don’t get why someone in the field wouldn’t. Maybe some day I will become enlightened.
I don’t think Daniel Bernstein got a lot of flak when he failed to make NaCl and TweetNaCl formally verified. He’s Daniel Freaking Bernstein after all, world renowned cryptographer who successfully invented primitives. Or maybe because this was 2008, and formal verification wasn’t such a thing at the time?
I don’t think Frank Denis got a lot of flak when he failed to make Libsodium formally verified. After all, it started out as a repackaging of NaCl, itself not formally verified, and expanded since. Not long ago, security advocates were still chanting use Libsodium, use Libsodium, use Libsodium. On the supported platforms, it is indeed a very strong alternative, used by high profile projects and audited by one of the most reputable security companies out there.
Even I didn’t get a lot of flak for failing to make Monocypher formally verified, when I started out over 4 years ago. I did get a lot of flack for daring to write a crypto library without first building a reputation for myself (which is backwards: achievements build reputation, not the other way around). Though that was almost exclusively from people who didn’t even look at my work (as evidenced by their criticism).
Now there is no question that a formally verified crypto library maximises confidence. We can’t beat machine checked proofs. But we can approach them. Here’s how I did it:
Monocypher, like TweetNaCl is small. 2K SLOC. I also made sure I adhere to some subjective coding standard instead of packing everything as tightly as I could, to make sure this metric isn’t entirely bogus.
Monocypher has zero dependencies. Not even Libc. The only code I have to test is my own.
Monocypher, like the entire NaCl family, is constant time (with the exception of signature verification). This makes it much easier for the test suite to have “true” 100% coverage (that is, it covers not only all the code, but all the code paths).
Monocypher is dumb. No clever trick of any kind. The only one I tried cost me a critical vulnerability, and I have since reverted to bog standard implementation techniques that I fully understand.
Monocypher is rigorously tested. The test suite compares with standard test vectors (including Wycheproof, which was designed to catch subtle bugs), tests every possible input & output lengths, uses property based testing to asses consistency, and runs all that under all sanitisers under the sun, Valgrind, and even the TIS interpreter (and now TIS-CI).
Monocypher is partially proven correct. I didn’t wait for proof assistants to make sure its more delicate parts were free of errors. I have written a full proof for Poly1305, the source code for 2^255 - 19 arithmetic contains proof of absence of overflow. I also instrumented the elliptic curve code in an alternate branch to check that the invariants of the modular arithmetic code are respected by the elliptic curve code that uses it.
Monocypher underwent a successful third party security audit.
Is that as good as a formal, machine checked proof of correctness of the generated binary code? Certainly not. But it damn sure approaches it, especially if your compiler is formally verified like ccert. So when I say I sacrificed confidence, that’s how little. While I would hesitate to trust it with the nuke launch codes, I would trust it with my payment card information.
Moreover a project using a crypto library must also consider other sources of errors. If the rest of the program isn’t formally verified, we’re kinda back to square one, where the whole thing must be thoroughly tested to make sure we don’t have some vulnerability or error lurking there. For instance, have we made sure that the network packets can’t be adversarially reordered without us noticing? Using authenticated encryption is not enough, we need message numbers or timestamps for this. So unless you formally prove the whole thing, that vaunted confidence will be significantly lowered, and the difference between a formally verified crypto library and a “merely” rigorously tested one is even slimmer.
As for the advantages, well: Monocypher is a single file library. One source file, one header file. One more of each if you really really need compatibility with Ed25519 (by default, Monocypher uses Blake2b instead of SHA-512 for signatures). Putting it in a C/C++ project is trivial: just copy & past the files in. Language bindings are easy to write, and people wrote several without me even needing to ask. The size of the binary is very small, and further increases Monocypher’s portability. Though it wasn’t the initial goal, it turns out Monocypher is very well suited to small 32-bit microcontrollers: doesn’t take much space, and it is fast.
Now I’m not saying a formally verified library can’t also be extremely portable and very easy to use. Strictly speaking, Monocypher itself could be formally verified (assuming the C standard can be at all formalised). But if it turns out the currently formally verified libraries are harder to deploy, or less well documented (Monocypher has exquisite documentation, thanks to a couple contributors), then the time you take learning how to use it is time you don’t take writing or testing your own program. By being so cheap to integrate, Monocypher can also contribute to the overall confidence of the projects that use it. Or at least lower their costs.
That being said, sometimes the cost of using formally verified cryptography is very low. Jason Donenfeld briefly used Monocypher in the Windows Wireguard installer (to check that the fetched binaries were the right ones), then at some point switched to formally verified code in very little time. As much as I’d like to deny it, this was the right move.
If those people come into a forum where I’m discussing the advantages of safe programming languages and formal verification and they argue what you argued, I would respond the same way. This isn’t a personal attack…
And all of the other text in your post, to me, further strengthens the case for safe languages (assuming ergonomics, portability, etc are on par, as we’ve discussed). Having lots of arbitrary application code to secure, having lots of programmers who would otherwise have to be extremely diligent with tooling, etc - it just isn’t going to happen.
I may have been more salty than I have any right to be, sorry about that. Rest assured, I perceived no such attack. The advantages of formally verified code are real, and I reckon I have a vested interest in ignoring them.
Speaking of which, a more recent post taught me that the first thing they wrote in Rust was the ASN.1 parsing. That kinda changes everything: the advantages of a safe language in this case is substantial. Chacha20 is easy to implement safely in C, but recursive parsing code is a whole ‘nother can of worms. I’ve written parsers, and C is never my first choice there. The compatibility problem sucks, but if they need ASN.1, that was probably the right move. (Modulo a slap on the wrist for failing to use semver.)
(assuming ergonomics, portability, etc are on par, as we’ve discussed)
Incidentally, I believe safe languages have the potential to be even more portable than C currently is. Once you have a formal specification and a formally verified reference implementation, you can make guarantees far beyond C could ever provide. And if you find a way to compile to standard C itself (in a way that doesn’t rely on undefined/unspecified/implementation defined behaviour), then you instantly exceeds its portability, though talking to existing C code may still be a hassle.
Having lots of arbitrary application code to secure, having lots of programmers who would otherwise have to be extremely diligent with tooling, etc - it just isn’t going to happen.
I agree, even in the case of Monocypher: while I have personally been very diligent about its implementation, and can confidently claim it is solid, its interface is definitely unsafe. Because C, and because no dependency: it’s not just the buffer overflows, there’s also the lack of an RNG. Users can’t just generate a key pair, they have to provide the random private key themselves, and I can just pray they stumbled upon one of the many warnings in the manual about how they should use the OS’s RNG. Same problem with authenticated encryption: users have to provide the nonce themselves, and the onus is on them to avoid catastrophic nonce reuse.
Because of this, I strongly believe that a low level crypto library such as NaCl, Libsodium or Monocypher is not enough (they’re higher-level than the low-level operations of OpenSSL, but my work on protocols convinced me they’re still fairly low level). At the very least, I strongly encourage language bindings to provide a higher-level API where users no longer have to worry about providing clean random numbers.
I’m also working on higher-level protocols. Key exchange first, then I’ll tackle a file format. Those can still be free of I/O, small, simple, and in C (with the usual extreme required diligence, which I’ve become better at the last few years). The end game, I think, will be a high-level network library. I’ll have to tackle actual I/O this time, so extreme portability is not possible. Pure C is really not looking good with this one. A Rust or Zig implementation (both of which support async), with a C interface, are probably much better. I’m not sure yet, we’ll see.
I don’t think the original author of the article realizes that cryptographic code is different. His arguments might be right for cryptographic code, but wrong for code in general, which was what he intended. Probably a Gettier problem here.
I am aware of many reasons (though not all of course) where crypto code is different from bc. However, what I do know is that crypto code should be small, thoroughly tested, and frozen. My argument in the post was “I did this amount of due diligence for bc, which is not ‘critical’ code, so have the cryptography authors done that much due diligence?”
For some reason this article made me quite angry. It is the epitome of “better things aren’t possible”, which means it is the year 2400 and we get regularly owned by buffer overflows because we keep on using a rather average language from 1978 since this is the best there is and just someone think about 32 bit SPARC machines that were last sold somewhere in 2000.
This is exactly the reason why obviously good ideas like GC, ADTs etc, stuff invented in the seventies is only now becoming mainstream, slowly being dragged into the mainstream by Java and TypeScript while still being worse alternatives than non-mainstream things we had a long time ago. How much money will we continue to waste on completely preventable issues in software just because people are hesitant? And with our modern life becoming increasingly dependent on software, how many lives will be lost because somewhere there is a bug in software that a compiler from the 1990ies could’ve flagged?
I am not even going to go into the part where the author is misinformed about Zig or the responsibility of open source developers, that can has been opened here already.
I acknowledge that I was misinformed about Zig. I have updated my post to make that clear.
For the record, if we could get everyone to move off of bad tech to good tech, I would be ecstatic. My point is that it’s not going to happen, not easily.
I guess you could say that the post is about pragmatism vs idealism. I think pragmatism will give us faster progress towards those better things than idealism.
I think it’s a fairly big assumption to say that your post contains no idealism, and the approach of those you disagree with contains no pragmatism. I’d argue that’s an incorrect assumption.
“Because I went through all that hardship, everyone else should as well” appears to be most of the argument
Firstly, cryptography is taking the good approach to a Rust rewrite: one little bit at a time. If they just rewrote everything in one go, sure it’d be buggy. Doing a chunk at a time means you can use the non-rewritten bits as extra checks.
Secondly, Rust means that tools like Valgrind, etc are much less needed. I’ve used them, they’re great tools, but they’re tools of desperation, of “this is all I’ve got, so let’s use this”, v.s. actually fixing the underlying issues, which we just can’t in C.
If the author wants to ignore all that for his own tools, that’s his choice and he can do it. I’m not going to stop him expending all that extra effort, but don’t force others to do so.
My crypto library takes 30 second to test under Valgrind. Less than a minute if we add all the sanitisers as well. And believe me, the test suite is very comprehensive. The real tools of desperation here are the FramaC and the TIS interpreter.
Saying that users paid for this software via exposure and thus are entitled to paid-level support is ridiculous. So gas-lighty
“
One of the most vocal (in favor of Rust) developers of cryptography works for Red Hat Security Engineering (if I read his GitHub profile right). I don’t know if his work on cryptography helped him get that job, but it might have.
Another of the most vocal developers has a computer security company. I would bet money that his work on cryptography gives his company relevance.
So they did not get “nothing” from users. Quite the opposite, in fact.
“
The developers don’t owe anyone jack shit. They could nuke it tomorrow, like that one left-pad dude did, and walk away with a clean conscience. These projects are massive amounts of work, for 0 pay. And no, recognition of your own hard work is not pay.
I have to disagree on this one. The authors wrote this stuff publicly, with the full expectation that they would influence other people’s behaviour over this. Specifically, they expected they’d have users. They have set expectations, made implicit promises.
More generally, being a decent human being means making sure you’re not doing more harm than good. At the very least, you owe it to your users not to waste their time.
In this particular case, rewriting the code in Rust has dubious benefits, and a very clear downside: instant breakage (or at least major inconvenience) on all platforms that don’t enjoy 1st tier Rust support. Shockingly, we still have many of those. They tried to make things better, but in the process broke their implicit promise that their stuff would continue working as long as they maintain it. They lost many people quite a bit of time. Though unwittingly, they very likely did more harm than good.
Now if Rust was supported everywhere (maybe by generating portable C code?), that would be another story. It isn’t, though. Not yet.
I’m sorry I still have to disagree on the fact that FOSS projects have some sort of implicit responsibility to coddle the 10% of their users using esoteric platforms from the 90’s. Your comments on not wasting user time is accurate though, and choosing Rust is a subjective opinion IMO, someone could have a long and thoughtful discussion on the pros and cons of such a decision.
But.
“but in the process broke their implicit promise that their stuff would continue working as long as they maintain it”
This is backwards to every single FOSS package I have ever used. If they don’t explicitly communicate long term support of legacy platforms as a project goal, there is no reason to assume that is a goal! Maintaining legacy platforms is a massive pill, and I can only count on 1 or two hands the amount of projects I have utilized that have dictated that as a goal.
As for the users whose CI’s automatically broke, that’s just poor engineering on their part. Dependencies should be pinned, and extra work should be expected when upgrading packages. It’s an upgrade, the maintainer doesn’t have the responsibility to make it seamless.
As for the users whose CI’s automatically broke, that’s just poor engineering on their part.
My intel suggests otherwise: the maintainers don’t follow semantic versioning to begin with. This update was treated as a minor update, despite the existence of a breaking change. Of course automated CI tools started to break.
Now I don’t (and shouldn’t) have a say about which direction a project I’m not even participating in should take. They want to sacrifice some portability for a bit of safety, that’s their prerogative. I won’t even ask them to continue support of the old versions. As nice as it would be, it still costs time. My problem here is the failure to mark that change as “breaking”, and therefore at least bump the major version number. Even if it was an honest mistake (I reckon it probably was), they should have been a bit more careful.
And when users started to complain left and right, they should have reverted the change, bumped the major version number, then applied the change again. (Maybe they already have, I haven’t checked.) Insisting that a breaking change is not worthy of being marked as such would definitely not be okay.
They have set expectations, made implicit promises. … broke their implicit promise that their stuff would continue working as long as they maintain it.
Please go ahead and read the licenses of all these projects where they make completely explicit the terms of use, and make it clear that there should be no other expectations beyond those terms.
I am continuously puzzled by this trend in a certain section of open source users to conjure out of thin air new terms and promises of support from these projects that they use for free, with no basis in fact, and coincidentally with effects that would benefit these users. But perhaps I should stop being puzzled by this. /r/ChoosingBeggars exists, after all.
Licences are legal disclaimers. The README (of pretty much any project that went to version 1.0 and beyond, including mine), still say that this product is suitable for a range of purposes.
I have to disagree on this one. The authors wrote this stuff publicly, with the full expectation that they would influence other people’s behaviour over this. Specifically, they expected they’d have users. They have set expectations, made implicit promises.
I’d agree with this if the license didn’t set explicit expectations that override the implicit.
(Although I think there are reasonable limits, just as there are limits to what’s enforceable in any contract.)
I’d agree if the README (or other documentation) didn’t explicitly contradict the licence by describing what the project is suitable for (at least for projects that are beyond 1.0).
Cryptography is wrong. They would be right in the vast majority of cases, but cryptographic code really is different. While a Rust rewrite isn’t very risky, it is fairly pointless: if the test suite for the C code is any good (and it’d better be, considering the stakes of cryptographic code), the Rust rewrite is very unlikely to fix bugs that weren’t already found and fixed long ago in the C code.
Now if Rust really was as portable as C (first tier support in all relevant platforms that currently have a worthy C compiler), I wouldn’t object. But it’s not. Not yet.
I remember lots of such posts even in 2000s. Not exactly “futility of replacing assemblers”, but “C, C++ won but we should go back to assembler, because of unnecessary abstractions causing bloat”. Even “Windows suxx because it’s written in C”. With the same arguments as today: “high-level languages are for those with poor skills, skilled programmers can write everything in assembler”.
Oh look another “I’m competent so I can write safe C” blog post. How many more of these are we going to write before we accept that it isn’t possible. If we decide that platforms like the Amiga or DEC Alpha are important enough to hold the rest of us back from real security improvements, I don’t know what to say. Sorry, we aren’t supporting your sun SPARC workstation anymore?
Both points about Zig are refuted (strong, intentional effort to spec underway; “trying” to make it self-hosted is some really interesting language). It’s explicitly designed to compete with C at C’s level, which differentiates its goals from Rust’s.
If a software project actively goes out and gets users, which just about
any project with a serious number of users has done, then yes, they have
an obligation to those users. The reason is that they sold users on the
idea of using their software. In other words, they were marketing their
software, which means making promises.
This argument is regularly contradicted by free software licenses themselves. Here is GPLv3, a copyleft license:
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
And here’s a permissive MIT license:
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
When does this supposed entitlement to a volunteer’s time end? When they leave the project? When they die?
Paid free software development muddies the waters, because people employed to work on a free software project aren’t usually volunteers. Regardless, there is still a lack of warranty.
Literal interpretation of licenses will conclude that there was nothing wrong with event-stream version 3.3.6, which after all, was released under MIT license. event-stream’s maintainership was taken over and code was added to harvest private keys of Bitcoin accounts. Is that okay because the maintainer was a volunteer and code was provided WIHTOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED? I reject such interpretation.
Let’s be clear–what happened with event-stream and what may happen with other attacks on open source software should not be taken as an excuse for users to get free support. Using a giant dependency chain while taking absolutely no responsibility for locking it down and doing vulnerability checks (which are both supported by the npm ecosystem) is super irresponsible, perhaps even rising to the level of malpractice. And just to be clear, we shouldn’t place the blame solely on lazy or nefarious devs; this is a systemic problem in organizations that use open source software, starting from the top down.
There’s a reasonable argument that the event-stream update is piracy/unlawful intrusion in someone else’s computer. That goes beyond just breaking your code and directly into illegality. So, not the same thing at all.
I liked the article except for the “they have obligation part”. No way. Each piece of FOSS is just a precious gift. What if I release something and then die the next day. Do I have an obligation to not die?
Yeah the bits about lib authors wanting users to hop onto their stuff sounded right. But then going on about an obligation to support the same set of features (platforms,languages..) for eternity is just plain wrong. Also it’s not like their working non-rust version is going away. People really could fork it and continue with that. There should be enough developers capable of doing that.
I’d argue it’s worse than that. There’s this huge culture in the Linux and FreeBSD space which has been clearly demonstrated by this saga where distribution packagers/ports maintainers will build software in completely untested/unsupported configurations and then rouse a mob when the authors break these (despite never committing to support them in the first place).
When you make a “minor patch” to your software, you kinda promise there won’t be any breaking change. Such as, no longer working on a bunch of relevant platforms.
Deprecation is okay. Just mark it as such, and do change the major version number when making breaking changes.
All changes in open source software are made on a best-effort basis. No ‘promises’ are made, ‘kinda’ or otherwise. Publishing open source software does not entitle users to expect free support or any other expectations from the software. Let’s be really clear on this.
Oh, yeah, I didn’t allow for the possibility of an honest mistake. My bad. Of course any change can introduce bugs, so there’s that.
What would not be okay would be insisting that this change is not breaking, even after being told it is. The correct response from the feedback they got, I believe, would be to revert the change, bump the major version number, then apply the change back. The old versions don’t even have to be maintained.
My broader point is that not all FOSS is a pure gift, but an exchange. We can definitely disagree on that, but your opinion is just as valid as mine. In fact, I fully acknowledge that I may be wrong. It’s just that my opinion is what it is based on my current experience.
Using the licence’s legal disclaimers to absolve yourself of all actual responsibility is dishonest. I use the same legal disclaimers, and I would never dare.
When does this supposed entitlement to a volunteer’s time end? When they leave the project? When they die?
This is not a case of developers stopping to work on their product. This is a case of developers actively making the life of some of their users more difficult. They failed to do no harm.
This is a case of developers actively making the life of some of their users more difficult.
I’d argue at the point that a distribution starts repackaging software in ways the authors haven’t tested, or shipping it on platforms the original authors never tested or supported, the users of that software are no longer the author’s responsibility but the distribution maintainer’s. Just because the authors didn’t list that the software definitely won’t work on a 30 year old DEC Alpha processor they are not suddenly owed to make it work there.
Well, I also happen to disagree with the very notion of Linux distribution. We shouldn’t need those middle men either.
Nevertheless, depending on Rust is a breaking change. That would have warranted at least a major version bump. That’s how much effort I would have asked of them: increment a number. That would be enough to make it right in my book.
It’s obviously not a breaking change in supported platforms, because it didn’t break those platforms. And breaking changes in unsupported platforms are, by definition, not supported.
That’s something important I didn’t check: do they have an explicit description, or list, of supported platforms? If they did, that’s all good, and shame on distro maintainers for distributing them on unsupported platforms.
If they did not… then the list of supported platforms is “wherever it builds and works out of the box”. Whether the maintainers like it or not.
There’s a world of difference between what the python-cryptography developers did and what happened with eventstream. The latter was intentionally malicious; the former was not. Legal language doesn’t absolve one of responsibility for malicious behavior.
People are justified in being pissed off about the python-cryptography change. They’re justified in saying that “python-cryptography has broken trust with us”. They’re justified in applying Freedom 3 from GNU’s Four Freedoms and forking the project.
They aren’t justified in feeling and voicing a sense of entitlement. That is all I’m saying.
I use the same legal disclaimers, and I would never dare.
So if someone sued you, you would tell your lawyer to ignore the legal disclaimer that would protect you? Why even use it at all, just use public domain.
But there are a lot of them that LLVM, Rust’s backend, cannot generate
code for. In fact, there are a lot of them that C++ cannot run on.
Make no mistake; embedded software is still running the majority of
devices in the world. And C is the king of embedded software.
If C++ cannot run on them, what are the odds that python-cryptography does,
even when it isn’t using Rust? Phrased differently, how big, really,
is the intersection of the set of systems that will run python-cryptography
with C primitives and the set of systems that don’t support C++?
I think I smell a strawman but I’m not an embedded systems person.
Yes python-cryptography depending on Rust is a problem. It broke architectures
that are supported by Gentoo, because LLVM – and hence Rust – aren’t available there.
I think the real problem is that Rust is not a mature language,
but I wish the Rust folks well.
Flag-waving for C isn’t the answer. Insisting on “portable” C is how
we get things like glib.
As an aside: if you want C++, by all means use it. Please don’t try and reimplement huge chunks of it in C.
There’s obviously a need and a desire for safe languages, and playing ostrich won’t make it go away.
I’ve been taking a closer look at Nim lately. It compiles to C and
interoperates well with C. It looks like it could be wildly portable and
a really good option.
glib seems to work? The reason glib is written in C and not C++ is unrelated to portability. It is related to the fact that C++ is much harder to bind to other languages than C.
Glib works. Lots of things work. That doesn’t mean they are necessarily good ideas.
As for the difficulty in making bindings for C++ libraries, is that due to the language itself, or due to the external interface of the library? I think it’s probably due to the external interface of the library. You can make a C API for a C++ library.
mrustc can compile Rust to C. So it doesn’t depend on LLVM (with some caveats). Also fuzzing cannot match static guarantees. The former is: “I did a search through all code paths I could find and didn’t find a bug”, whereas the latter is “there may be no bug, no matter what code path is taken”.
Case in point: I once had a bug in a string handling library that was extensively fuzzed, but all that fuzzing didn’t find the UTF-8 short space character that led to a byte index landing within a UTF8 character.
While the average Rust code is much safer than the average C code, even in the hand of very competent developers, we must keep in mind that cryptographic code is not average. Especially modern cryptographic code, designed after NaCl.
It is pathologically straight-line, with very few branches.
It has pathologically simple allocation patterns. It often avoids heap allocation altogether.
It is pathologically easy to test, because it is generally constant time: we can test all code paths by covering all possible input & output lengths. If it passes the sanitizers & Valgrind under those conditions, it is almost certainly correct (with very few exceptions).
You thought writing cryptographic code would be harder to write than application code? I mean, don’t everyone tell you not to roll your own? Well, the answer is two-fold: the logic of cryptography is often very delicate. Rust won’t save you here. But the system aspects of it are often very easy. Rust will help you there, but we hardly need that help to begin with.
I have to agree with the author here. There was no need to rewrite it in Rust, and the decision to do so anyway is most probably not worth the trouble. Especially damning is treating this major breaking change as if it was a mere patch. Affected users should probably leave and use something else.
Speaking of which, Shameless plug time: Monocypher is an extremely simple crypto library, written in portable C99. It has Python bindings, written by a user who needed it to update the firmware of the embedded device they sell. If you were using this crypto library and your platform doesn’t support Rust, take a look at Monocypher. It’s not a drop-in replacement, but it might be a worthy alternative.
Or, instead of using a non audited library written by a single person, look for actual improvements like project Everest where absence of memory errors, and validity of the code, are proven formally, not just trusted because “fuzzer says it’s ok”.
Now Project Everest sounds very nice, if you can get it to build on your particular platform, with little enough effort. By comparison, Monocypher is basically a single file C library, extremely easy to deploy for C/C++ project, and easy to write bindings for (though it already has bindings for Python, .NET, WASM, Rust, Go, and more. I think that count as an “actual improvement” over TweetNaCl and Libsodium in some niches.
Ah, good about the successful recent audit, I wrote that based on memories from earlier :-). That changes my view a bit on your library, and maybe some day, if it gets OCaml bindings, I might use it.
And now I regret the salt in my earlier comment. :-)
maybe some day, if it gets OCaml bindings, I might use it.
You can find the list of language bindings here. Someone did the Ocaml bindings there. You may want to take a look, perhaps get in touch with the author if you feel like contributing.
Right now, as far as I know, there is no official language spec for Zig, like there is for Go. This means that this new compiler may not match what the official compiler does.
They’re actively working on a spec for Zig btw. Here’s Andrew talking about it, first thing as part of the roadmap video: https://youtu.be/pacsngNYXI0
The author seems to be very short sighted in my opinion. Even though I don’t like that cryptography library is moving to Rust, I’d rather they created a new crypto library in rust and donated the old C one to new maintainers. It is in the end their call, and Rust will provide a ton of safety features out-of-the-box.
I think that it all boils down to which platforms the cryptography library committed to support. If they want to support all platforms that Python runs, then RIIR is the wrong option. If they’re going to make it clear that it supports just a subset of those, then it is quite OK. People who are in the orphaned platforms can fork the last known working version and move ahead with a new team.
also SIGH but the first post linked on his sidebar is a rant where he explains why he refuses to wear a mask and calls business who force him to wear a mask, tyrants…
Is your concern that they are keeping the “brand” for cryprography? There’s certainly nothing preventing new maintainers from taking over the C version…
not the brand from a marketing perspective, but that build systems and other automated tools point-of-view. Those who are pulling their sources will depend on a whole new toolchain to build it, that affects package maintainers and products for a whole range of architectures.
IMO, drastic changes like that should be done in new libraries so that it doesn’t break other systems. They could’ve declared that library finished, created cryptography2 or some other name and proceed to RIIR. The people using their library would have the option to stay with the unmaintained system and keep their stuff working, or go through the trouble of upgrading to the new library.
They could’ve declared that library finished, created cryptography2 or some other name and proceed to RIIR
That’s a strange inversion of responsibility. It’s also not viable since they’re doing an incremental rewrite, not a wholesale one. A wholesale rewrite might warrant a new library, but incremental rewrites are reliant on the full existing code and history to do properly.
The responsibility for maintaining a library falls on the library maintainers. The responsibility for maintaining a distro falls on the distro maintainers. If a library stops supporting an environment, then the distro has to choose: also stop supporting the environment, or fork the library which they have license to do. “Tell the library maintainers to maintain their library differently” is not one of the options. Or, at least, not an option that will work.
Rust will provide a ton of safety features out-of-the-box.
Those safety features are almost irrelevant in the specific case of cryptographic code. Platform support aside, it won’t hurt, but it won’t do much good either. Don’t get me wrong, I would love that Rust be supported everywhere. Until then, C is still the better choice for portable cryptographic code.
My biggest qualm about this change is that they didn’t mark it as “breaking change, let’s bump the major version number”.
They’ve changed their mind, and from now on their project will use a semver-compatible system. The version they released that caused the issues was on their old (weird) scheme, tho.
Kudos to them, then. While ideally they should probably have reverted the change, switch to semver, then apply the change back, the switch to semver shows that they are listening to feedback.
To add, most embedded isn’t self-hosting, so it’s an odd debate anyways. Designs newer languages can’t target tend to either be rising (like RISC-V) or on their way out. Do you think i.e Microchip would invest in a Rust compiler for an MCU they no longer support?
Note: ADA spawned a child with a formal specification!! That’s better than even C, and in the long run potentially even more portable (that with the potential to writing formally verified compilers for new and old platforms).
This calls for some discussion about the value of accumulated user testing. How illegible does code need to be or how much accidental complexity does it need to contain in order to warrant refactoring and thus invalidating the hardening that happened in production?
To be fair, author’s extremist views on christianity, society, medicine or politics doesn’t make him wrong on another unrelated subject. “You should first fix bugs instead of switching to a less portable language” is not a wrong thing to say. “You should not break other people’s operating systems” is not a wrong thing to say. Rust may be superior in terms of end product quality, but it is not the default choice for every situation.
“don’t write bugs” doesn’t work despite 40 years of industry experience with unsafe languages. see other comments.
the piece is littered with opinions about what others should do (with their spare time even), while at the same time maintaining poor people shouldn’t be allowed to vote, and holding other dehumanising views. imo, that is very relevant background: it tells me that whatever this person thinks that others should do is completely irrelevant.
yes you are entitled to your opinion, so is the author else we could not have a dialog about these subjects. The view in the piece is “fix your bugs” and “fuzz your bugs” and “be responsible” not “don’t write bugs”.
And Rust is about “preempt a class of bugs” well so is any language that consider nulls a design flaw for instance. So are languages that prevent side effects, that enforce exhaustive conditionals, that enforce exception handling, and so on. So it isn’t the ultimate language.
“don’t write bugs” doesn’t work despite 40 years of industry experience with unsafe languages.
Except in cryptography. In this particular niche, “don’t write bug” is both mandatory and possible. In this particular niche, Rust is of very little help.
That said, the author may not be aware of the specifics of cryptography. He may be right for the wrong reasons.
“You should not break other people’s operating systems” is not a wrong thing to say.
Very reasonable, but not what’s happening, nor what he’s saying.
What’s happening is “making a change to one software project”, that is provided under a license that does not guarantee support.
What he’s saying is “changing what platforms you support is unacceptable”. He’s also saying “switching to Rust is a dereliction of duty”. Which implies “you have a duty that is not listed in your software license, and which I won’t elaborate on”.
In other words, what he’s saying is, ultimately, “nobody should make changes to their software that I disapprove of”. Which is wildly unreasonable.
Unsurprisingly, his political views skew towards “the only people who are allowed to be part of society are people that I approve of” (he approves only of a certain subset of Christian faithful, who must also be straight and cisgendered, believe that certain parts of the Constitution are invalid, and be in the top fraction of earners).
In other words, his poor political thinking mirrors his thinking on the politics of open source development in ways that make it relevant to the conversation.
What’s happening is “making a change to one software project”, that is provided under a license that does not guarantee support.
What’s happening is “making a breaking change” without marking it as such (deprecation, major version number bump, that kind of thing). That’s not okay. I don’t care it was a labour of love provided for free. The rules should be:
Don’t. Break. Users. Pretty please.
If you break rule (1), bump the major version number.
If rule (1) was broken, keep the old versions around for a while. Bonus point if you can actually support them (fix bugs etc).
It’s not a breaking change if it breaks on platforms that were never explicitly supported (not even by python). (I’m not entirely sure if they used to support these explicitly, but it’s not obvious).
I agree if there’s an explicit description of supported platforms. If not… the only reasonable fallback would be any platform supported by Python where it builds & run out of the box. That can mean a lot of platforms.
If you break rule (1), bump the major version number
They did, at least for certain values of “major version number”. Rust was introduced in 3.4. The x.y releases are when backwards-incompatible changes are introduced. They’re changing their versioning scheme to make it more explicity: next major version will be 35 (instead of 3.5).
Honestly, a quick look at their changelog shows backwards-incompatible changes at 3.3, 3.1, 3.0, 2.9, … In other words, adding rust when they did was entirely in line with their release cycle.
I commend them for listening to feedback (or is it cave in to mass anger?), and switch to semantic versioning. That’s cogent evidence of trustworthiness in my book.
Everyone is full of crud on some level. However, interacting with people nicely on the Internet is a good way for me to get rid of that and to become less of a terrible person.
Again and again, languages that compile to C win. So at this point, it doesn’t matter how C is generated, “compiled” from another language or written directly, undef behavior is the main issue.
Compilers can guarantee that the code they generate is free of undefined behaviour. Or at least of certain classes of undefined behaviour. Same way Rust compilers can guarantee the x86-64 instruction they generate are safe, even though using x86-64 assembly directly definitely isn’t.
Without getting into the argument, I just want to make sure this information is out there:
Zig’s New Relationship with LLVM
zig is written mostly in zig already, with c++ code only for interfacing with (optional) LLVM extensions
the plan to bootstrap via pure C
the self-hosted C backend is well underway
This is also well underway, with swift progress being made for the following areas, by various contributors:
we have broken ground on this. We also have Martin Wickham paid part-time by Zig Software Foundation to create the first draft of the language spec. He’s doing great work and we’re having weekly meetings to iron out the details of the language.
Author here.
Wish I could upvote your post more to make everyone aware of these links.
I mean… it’s your article. You can always update it to be more representative of the facts, even if your opinion hasn’t changed.
Edit: I now see that you have updated the article. Nice!
This seems like such a short-sighted, limited view. Software is not bug-free. And ruling out a class of bugs by choice of technology as a measure of improving overall robustness won’t fix everything, but at the very least it’s a trade-off that deserves more thorough analysis than this empty dismissal.
I think he is probably right (after all, he wrote it) when he says rewriting his bc in Rust would make it more buggy. I disagree this is an empty dismissal, since it is backed by his personal experience.
For the same reason, I think cryptography developers are probably right (after all, they wrote it) when they say rewriting their software in Rust would make it less buggy. So the author is wrong about this. His argument is not convincing why he knows better than developers.
I think there’s a big difference between programs and libraries with stable requirements and those that evolve here. The bc utility is basically doing the same thing that it did 20 years ago. It has a spec defined by POSIX and a few extensions. There is little need to modify it other than to fix bugs. It occasionally gets new features, but they’re small incremental changes.
Any decision to rewrite a project is a trade off between the benefits from fixing the accumulated technical debt and the cost of doing and validating the rewrite. For something stable with little need of future changes, that trade is easy to see: the cost of the rewrite is high, the benefit is low. In terms of rewriting in a memory-safe language, there’s an additional trade between the cost of a memory safety vulnerability and the cost of the rewrite. The cost of Heartbleed in OpenSSL was phenomenal, significantly higher than the cost of rewriting the crypto library. In the cast of
bc, the cost of a memory safety bug is pretty negligible.Data from Microsoft’s Security Response Center and Google’s Project Zero agree that around 70-75% of vulnerabilities are caused by memory safety bugs. Choosing a language that avoids those by construction means that you can focus your attention on the remaining 25-30% of security-related bugs. The author talks about fuzzing, address sanitiser, and so on. These are great tools. They’re also completely unnecessary in a memory-safe language because they try to find classes of bugs that you cannot introduce in the first place in a memory-safe language (and they do so probabilistically, never guaranteeing that they’ve found them all).
If you’re starting a new project, then you need a really good reason to start it in C and pay the cost of all of that fuzzing.
There’s an implied assumption here that if a language is memory safe, those memory safe bugs will simply go away. In my experience, that is not quite true. Sometimes those memory safety bugs will turn into logic bugs.
Not to pick on Rust here, but in Rust it is very common to put values into an array and use array indices instead of pointers when you have some kind of self-referential data structure that’s impossible to express otherwise using rust’s move semantics. If you simply do such a naive transformation of your C algorithm, your code will be memory safe, but all your bugs, use after free, etc, will still be there. You just lifted them to logical bugs.
Rust has no good abstractions to deal with this problem, there are some attempts but they all have various practical problems.
Other languages like ATS and F* have abstractions to help with this problem directly, as well as other problems of logical soundness.
Right - but in lifting these from memory bugs to logic bugs, you get a runtime panic/abort instead of a jump to a (likely-attacker-controllable) address. That’s a very different kind of impact!
You don’t get a panic if you access the “wrong” array index. The index is still a valid index for the array. Its meaning (allocated slot, free slot, etc), is lost to the type system, though in a more advanced language it need not be. This later leads to data corruption, etc, just like in C.
It leads to a much safer variant of data corruption though. Instead of corrupting arbitrary memory in c or c++ (like a function pointer, vtable, or return address), you are only corrupting a single variable’s value in allocated and valid and aligned memory (like a single int).
You would get a panic in rust for every memory corruption bug that could cause arbitrary code execution, which is what matters.
Can you expand on this? I had expected the behaviour in Rust to be significantly safer than C here. In C, the data corruption caused by use-after free often allows an attacker to execute arbitrary code.
I totally see your point about logical corruption (including things like exposing critical secrets), but I don’t follow that all the way to “just like C”. How would an array index error be exploited in Rust to execute arbitrary code?
I have once written a bytecode interpreter in C++, for a garbage collected scripting language. I implemented my own two-space garbage collector. For performance reasons, I didn’t use
malloc()directly, but instead allocated a big enough byte array to host all my things. If I overflew that array, Valgrind could see it. But if I messed up it’s internal structure, no dice. That heap of mine was full of indices and sizes, and I made many mistakes that caused them to be corrupted, or somehow not quite right. And I had no way to tell.I solved this by writing my own custom heap analyser, that examined the byte array and tell me what’s in there. If I see all my “allocated” objects in order, all was well. Often, I would see something was amiss, and I could go and fix the bug. Had I written it in Rust instead, I would have had to write the exact same custom heap analyser. Because Rust wouldn’t have prevented me from putting the wrong values inside my array. It’s perfectly “safe” after all, to write gibberish in that array as long as I don’t overflow it.
Now could this particular bug lead to arbitrary code execution? Well, not quite. It would generate wrong results, but it would only execute what my C++/Rust program would normally execute. In this case however, I was implementing a freaking scripting language. The code an attacker could execute wasn’t quite arbitrary, but it came pretty damn close.
The effects of data corruption depend on what the code does with the data. This often means arbitrary code execution, but not always. It’s not a property of C, it’s a property of the code. This doesn’t change when you change the implementation language.
Fundamentally there is no semantic difference between a pointer in a C heap and an array index into a Rust array. In fact some sophisticated blog authors that explain this array technique often point out they compile to the exact same assembly code. It’s what the code does with the data that leads to exploitation (or not).
Of course Rust has many additional safety advantages compared to C, buffer overflows don’t smash the stack, etc, and using references in Rust if you can is safe. And when using references, there’s a great deal of correlation between Rust’s notion of memory safety and true logic safety. This is good! But many people don’t realise that this safety is predicated on the lack of aliasing. The borrow checker is only a mechanism to enforce this invariant, it’s not an operative abstraction. It’s the lack of aliasing that gets you the safety, not the borrow checker itself. When you give up aliasing, you lose a lot of what Rust can do for you. Virtually everybody understands that if you introduce unsafe pointers, they give up safety, but less people seem to understand that introducing aliasing via otherwise safe mechanism has the same effect. Of course, the program continues to be memory safe in Rust terms, but you lose the strong correlation between memory safety and logic safety that you used to have.
Not that there’s anything wrong with this, mind you, it’s just something people need to be aware of, just as they are already aware of the tradeoffs that they make when using unsafe. It does make a projection for the number of bugs that Rust can prevent in practice more difficult, though.
I think this is incorrect. Arbitrary code execution does not mean “can execute an arbitrary part of my program due to a logic bug”, it means “can execute arbitrary code on the host, beyond the code in my program”. Even a rust alias logic bug dies not open up this kind of arbitrary code execution exposure because you can’t alias an int with a function pointer or a vtable or a return address on the stack, like you can in c or c++. You can only alias an int with an int in safe rust, which is an order of magnitude safer and really does eliminate an entire class of vulnerabilities.
In the security research world, we usually treat control of the program counter (the aptly named
ripon x86-64) as “arbitrary code execution.” You can do a surprising of programming using code that’s already in a process without sending any byte code of your own with return-oriented programming.But does Rust let you do that here? What does a snippet of Rust code look like that allows attacker-controlled indexing into an array escalate to controlling the program counter?
Surely you agree that “variables changing underfoot” implies “programs flow becomes different from what I expect”. That’s why we use variables, to hold the Turing machine state which influences the next state. A logical use after free means “variables changing underfoot”. You don’t expect a free array slot’s value (perhaps now reallocated) to change based on some remote code, but it does.
Right, but “program flow becomes different from what I expect, but it still must flow only to instruction sequences that the original program encoded” is much much safer than “program flow can be pointed at arbitrary memory, which might not even contain instructions, or might contain user-supplied data”.
With ROP, the program flow only goes through “instruction sequences that the original program encoded”, and yet ROP is pretty much fatal.
ROP is not possible when you index an array wrong in rust, what is your point?
And you can’t do rop in safe rust.
Maybe not directly within the native code of the program itself, but I think (at least part of) 4ad’s point is that that’s not the only level of abstraction that matters (the memory bug vs. logic bug distinction).
As an example, consider a CPU emulator written entirely in safe Rust that indexes into a u8 array to perform its emulated memory accesses. If you compile an unsafe program to whatever ISA you’re emulating and execute it on your emulator, a bad input could still lead to arbitrary code execution – it’s at the next semantic level up and not at the level of your program itself, but how much does that ultimately matter? (It’s not really terribly different than ROP – attacker-controlled inputs determining what parts of your program get executed.)
That’s admittedly a somewhat “extreme” case, but I don’t think the distinction between programs that do fall into that category and those that don’t is terribly clear. Nearly any program can, if you squint a bit, be viewed essentially as a specialized interpreter for the language of its config file (or command-line flags or whatever else).
There’s no distinction here. If your program implements a cpu emulator then your program can execute with no arbitrary code execution at all and still emulate arbitrary code execution on the virtual cpu. If you want the virtual program executing to not possibly execute arbitrary virtual instructions, you need to generate the virtual program’s instructions using a safe language too.
In most cases, though, arbitrary virtual code execution is less dangerous than arbitrary native code execution, though that’s beside the point.
So…we agree? My point was basically that attacker-controlled arbitrary code execution can happen at multiple semantic levels – in the emulator or in the emulated program (in my example), and writing the emulator in a safe language only protects against the former, while the latter can really be just as bad.
Though I realize now my example was poorly chosen, so a hopefully better one: even if both the emulator and the emulated program are written in memory-safe languages, if the emulator has a bug due to an array-index use-after-free that causes it to misbehave and incorrectly change the value of some byte of emulated memory, that destroys the safety guarantees of the emulated program and we’re back in arbitrary-badness-land.
Sure but this is just as meaningful as talking about a cpu hardware bug that might cause a native safe program to run amok. Technically true but not very useful when evaluating the safe programming language
Right, I agree, and safe rust aliasing that the GP described is not possible to control the program counter arbitrarily.
Yeah exactly, this is the part I thought @4ad was arguing was possible. Eg. in C, use-after-free often allows me to make the program start interpreting attacker-provided data as machine code. I thought this is what 4ad was saying was also possible in Rust, but I don’t think that’s what they are claiming now.
To me, that’s a big difference. Restricting the possible actions of a program to only those APIs and activities the original code includes, vs C where any machine code can be injected in this same scenario, is a major reduction in attack surface, to me.
One thing to note is that code is data and data is code, in a true, hard-mathematical sense.
The set of
and
is exactly the same (unbounded!). Of course it is much easier in practice to effect desired behavior when you can inject shell code into programs, but that’s hardly required. You don’t need to inject code with ROP either (of course ROP itself is not possible in Rust because of other mitigations, this is just an example).
Please note that in no way I am suggesting that Rust is doing anything bad here. Rust is raising the bar, which is great. I want the bar raised even higher, and we know for a fact that this is possible today both in theory and practice. Until we raise the bar, I want people to understand why we need to raise the bar.
At the end of a day you either are type safe or you aren’t. Of course the specifics of what happens when you aren’t type safe depend on the language!
PS: arrays can contain other things than integers, e.g. they can contain function pointers. Of course you can’t confuse an int with a function pointer, but using the wrong function pointer is pretty catastrophic.
I guess this is what I don’t understand, sorry for being dense. Can you show a concrete code example?
In my mind I see a program like this:
In my mind, there are two differences between this code in Rust and similar code in C:
This only allows the user to launch nuclear missiles; it does not allow them to, say, write to the harddrive or make network calls (unless one of the actions contained code that did that ofc); in C, I’d likely be able to make something like this call any system function I wanted to, whether machine code to do that was present in the original binary or not.
In Rust, this doesn’t allow arbitrary control flow, I can’t make this jump to any function in the binary, I can only trick it into running the wrong
Action; in C, I can callrunon any arbitrary object anywhere in the heap.ie. in C, this would let me execute anything in the binary, while in Rust it still has to abide by the control flow of the original program, I thought was the case, anyway.
I think you’re saying this is wrong, can you explain how/why and maybe show a code example if you can spare the time?
This is correct and 4ad is mistaken. I’m not sure why 4ad believes the two are equivalent; they aren’t.
“is exactly the same”
It simply isn’t, and I’m not sure why you think it is.
Do you have any links on this that you recommend?
Good analysis. You didn’t use the words, but this is a great description of the distinction between stocks and flows: https://en.wikipedia.org/wiki/Stock_and_flow. I wish more people talking about software paid attention to it.
Author here.
I would also argue that crypto should not change often, like
bc. You might add ciphers, or deprecate old ones, but once a cipher is written and tested, there should be very little need for it to change. In my opinion.Have they actually rewrote anything? Or have they instead selected a different crypto library they trust better than the previous one? On the one hand, Rust has no advantage over C in this particular context. On the other hand, they may have other reasons to trust the Rust library better than the C one. Maybe it’s better tested, or more widely used, or audited by more reputable companies.
If I take your word for it however, I have to disagree. Rewriting a cryptographic library in Rust is more likely to introduce new bugs, than it is to fix bugs that haven’t already been found and fixed in the C code. I do think however that the risk is slim, if they take care to also port the entire test suite as well.
In the Cryptography case isn’t the Rust addition some ASN.1 parsing code? This is cryptography adjacent but very much not the kind of different that your point about cryptography code makes. Parsing code unless it is very trivial and maybe not even then tends to be some of the more dangerous code you can write. In this particular case Rust is likely a large improvement in both ergonomics for the parsing as well as safety.
You’ve got a point. I can weaken it somewhat, but not entirely eliminate it.
I don’t consider ASN.1 “modern”. It’s over complicated for no good reason. Certificates can be much, much simpler than that: at each level, you have a public key, ID & expiration date, a certificate of the CA, and a signature from the CA. Just put them all in binary blobs, and the only thing left to parse are the ID & expiration date, which can be left to the application. And if the ID is an URL, and the expiration date is an 64-bit int representing seconds from epoch, there won’t be much parsing to do… Simply put, parsing certificate can be “very trivial”.
Another angle is that if you need ASN.1 certificates, then you are almost certainly using TLS, so you’re probably in a context where you can afford the reduced portability of a safer language. Do use the safer language in this case.
Yet another angle is that in practice, we can separate the parsing code from the rest of the cryptographic library. In my opinion, parsing of certificate formats do not belong to a low-level cryptographic library. In general, I believe the whole thing should be organised in tiers:
This is a fair point. When it comes down to it, whether I am right or wrong about it will only be seen in the consequences of the decision that they made.
Here’s the more thorough analysis you’re asking for: this is cryptographic code we’re talking about. Many assumptions that would be reasonable for application code simply does not apply here:
I wrote a crypto library, and the worst bug it ever had wasn’t caused by C, but by a logic error that would have happened even in Haskell. What little undefined behaviour it did have didn’t have any visible effect on the generated code.
Assuming you have a proper test suite (that tests all input & output lengths), and run that test suite with sanitisers & Valgrind, the kind of bug Rust fixes won’t occur in your cryptographic C code to begin with. There is therefore no practical advantage, in this particular case to using Rust over C. Especially when the target language is Python: you have to write bindings anyway, so you can’t really take advantage of Rust’s better APIs.
These bugs still occur in critical software frequently. It is more difficult and time consuming to do all of the things you proposed than it is to use a safer language (in my opinion), and the safer language guarantees more than your suggestions would. And there’s also no risk of someone forgetting to run those things.
Yes they do. I was specifically talking about one particular kind of critical software: cryptographic code. It’s a very narrow niche.
In my 4 years of first hand experience writing cryptographic code, it’s really not. Rust needs the same test suite as C does, and turning on the sanitizers (or Valgrind) on this test suite is a command line away. The real advantage of Rust lies in its safer API (where you can give bounded buffers instead of raw pointers). Also, the rest of the application will almost certainly be much safer if it’s written in Rust instead of C.
Someone who might forget those things has no business writing cryptographic code at all yet, be it in C or in Rust. (Note: when I started out, I had no business writing cryptographic code either. It took over 6 months of people findings bugs and me learning to write a better test suite before I could reasonably say my code was “production worthy”.)
Rusts advantage goes much further than at the api boundary, but again the discussion should be around how to get safer languages more widely used (ergonomics, platform support) and not around “super careful programmers who have perfect test suites and flawless build pipelines don’t need safer languages”. To me it is like saying “super careful contractors with perfect tools don’t need safety gear”, except if you make a mistake in crypto code, you hurt more than just yourself. Why leave that up to human fallability?
Yes it does. In almost all domains. I’m talking about modern cryptographic code.
Write a spec. A formal one if possible. Then implement that spec for more platforms. Convincing projects to Rewrite It In Rust may work as a way to coerce people into supporting more platforms, but it also antagonises users who just get non-working software; such a strategy may not be optimal.
You’re not hearing me. I’m not talking in general, I’m talking about the specific case of cryptographic code (I know, I’m repeating myself.)
I really don’t. I root out potential mistakes by expanding my test suite as soon as I learn about a new class of bugs. And as it happens, I am painfully aware of the mistakes I made. One of them was even a critical vulnerability. And you know what? Rust wouldn’t have saved me.
Here are the bugs that Rust would have prevented:
#ifdef, though I don’t actually know.Here are the bugs that Rust would not have prevented:
Lesson learned: in this specific case, Rust would have prevented the unimportant bugs, and would have let the important ones slip through the cracks.
In this discussion, I think it is important to remind that cryptography developers are explicitly and intentionally not writing modern cryptographic code. One thing they want to use Rust on is ASN.1 parsing. Modern cryptographic practice is that you shouldn’t use ASN.1 and they are right. Implementing ASN.1 in Rust is also right.
So am I.
I disagree.
I disagree here too.
I partially agree. There is no such thing as a perfect test suite. A good crypto implementation should have a comprehensive test suite, of course, no matter the language. But that still isn’t as good as preventing these classes of bugs at compile time.
Not really the point. Regardless of how lucky or skilled you are that there are no known critical vulnerabilities in these categories in your code, that disregards both unknown vulnerabilities in your code, and vulnerabilities in other people’s code as well. A safe language catches all three and scales; your method catches only one and doesn’t scale.
Note that I did go the extra mile and went a bit further than Valgrind & the sanitisers. I also happen to run Monocypher’s test suite under the TIS interpreter, and more recently TIS-CI (from TrustInSoft). Those things guarantee that they’ll catch any and all undefined behaviour, and they found a couple bugs the sanitisers didn’t.
After that level of testing and a successful third party audit, I am confident there are none left.
There is no such code. I have zero dependencies. Not even the standard library. The only thing I have to fear now is a compiler bug.
I went out of my way not to scale. Yet another peculiarity of modern cryptographic code, is that I don’t have to scale.
Sure there is. Other people write cryptographic code too. Unless you are here just arguing against safe languages for only this single project? Because it seemed like a broader statement originally.
I mean scale as in other developers also writing cryptographic software, not scale as in your software scaling up.
I was talking about Monocypher specifically. Other projects do have dependencies, and any project that would use Monocypher almost certainly has dependencies, starting with system calls.
Fair enough. I was thinking from the project’s point of view: a given project only need one crypto library. A greenfield project can ditch backward compatibility and use a modern crypto library, which can be very small (or formally verified).
Yes, other people write cryptographic code. I myself added my own to this ever growing pile because I was unsatisfied with what we had (not even Libsodium was enough for me: too big, not easy to deploy). And the number of bugs in Monocypher + Libsodium is certainly higher than the number of bugs in Libsodium alone. No doubt about that.
Another reason why crypto libraries written in unsafe languages don’t scale, is the reputation game: it doesn’t matter how rigorously tested or verified my library is, if you don’t know it. And know it you cannot, unless you’re more knowledgeable than I am, and bother to audit my work yourself, which is prohibitively expensive. So in practice, you have to fall back to reputation and external signs: what other people say, the state of documentation, the security track record, issues from the bug tracker…
This made me twitch!
Why make a choice which prevents an entire class of bugs when you could simply put in extra time and effort to make sure you catch and fix them all?
Why lock your doors when you can simply stand guard in front of them all night with a baseball bat?
While personally would back the
cryptographydevs’ decision here, I think there is a legitimate discussion to be had around whether breaking compatibility for some long-standing users is the right thing to do. This post isn’t contributing well to that discussion.Oh, imagine if Mozilla, Microsoft, and Google knew they can just fuzz their C code to get rid of all the bugs! These foolish companies have spent so much money and engineering time on their security programs, patching CVE after CVE, of which ~70% were caused by memory unsafety. If only they knew…
With C it’s always “all these vulnerabilities don’t count”, because you should have fuzzed more, you should have got better sanitizers, you should have got better programmers — for 40 years. The real futility is in “just don’t write bugs” approach.
The safer languages, on top of built-in safety guarantees, also support fuzzing and memory sanitizers, so you can perform your usual due diligence there too.
“Battle-tested” is a weak property. It only applies to software that’s unmaintained. As soon as you change the code, even a single line, that code isn’t proven to be reliable any more (remember when Debian broke SSH key generation to silence a compiler warning?). Being battle-tested is merely an effect of software getting shipped. So ship the new stuff, and it’ll get battle-tested too.
This is cryptographic code we’re talking about. That fuzzing test suite you need in C? You also need it in Rust. And now the only difference between C and Rust is the fact that with C, you need to run your existing test suite under Valgrind and the sanitizers. Push a button, wait for a minute at most, and you’ll get almost all the safety Rust guarantees.
Because unlike most application code, cryptographic code, thanks to being constant time, is stupidly easy to test. And test it you must, considering the stakes. It is one of the few cases where “just don’t write bugs” is both mandatory and possible.
I agree. What you really want is a kickass test suite.
Test suites are not enough. This is proven time and time again. Any class of bugs you can move from “has to be caught by a test or fuzzer” to “can’t occur in the first place” is a huge win. Cryptographic code should desire these properties even more because it is usually used when security and correctness is paramount.
I would love to go even further and be able to prove constant time algorithms or higher level mathematical relationships statically, etc. Perhaps with theorem provers.
The discussion of the tradeoffs should be around language ergonomics and platform support, not around “but c basically has the same guarantees if you squint”. It absolutely does not.
Cryptographic code really is different. Yes, you want correctness, safety and whatnot. More than for pretty much anything else, save real time avionics and lethal medical devices. Thing is, Rust does not help you there.
Oh but I am discussing tradeoffs there. C has (as of early 2020) substantially better platform support, significantly worse API, and in the specific case of cryptographic code, a negligible safety disadvantage.
Yes, for most domains, the safety advantage of Rust is freaking huge. I’m only talking about cryptographic code, that I have first hand experience with. Yes, you still need to have some kickass test suite, but Rust would need the exact same test suite anyway. Yes, you need to use sanitizers & Valgrind, but that’s often trivial for cryptographic code, since it tends to have zero dependencies to begin with.
I’m not squinting here, I’m just looking at the application domain at hand. In this particular case, C actually provides basically the same guarantees as Rust. I wouldn’t have expected this 4 years ago when I started working on Monocypher (I actually considered Rust, but favoured C’s portability), but now I can confidently take it for granted: libraries that:
are only marginally safer when implemented in Rust, provided they have a good test suite. Modern cryptographic code fits that bill. It’s probably the only niche that does.
That said, cryptographic code doesn’t stand in a vacuum. It must be used somehow. C APIs are naturally unsafe, and the rest of the application will likely hugely benefit from Rust’s guarantees. The crypto library must then have a Rust interface. We could write bindings, but it’s a significant hassle. I would never chose C over Rust for new cryptographic code for a Rust project. For old cryptographic code, I wouldn’t mind considering a reputable C library, though I’d likely favour a Rust library instead.
Would you trust a C crypto lib more than HACL*, a lib written in F* (a dependently-typed language) and formally verified for memory safety, correctness, and resistance to side-channel attacks?
I wouldn’t. Monocypher’s comparative advantage are its simplicity, it’s ease of use, its ease of deployment, and its portability (part of its portability comes from its relatively small code size). I did sacrifice confidence a bit to get there.
I respect your opinion but boy I don’t understand it. I would take a formally verified library any day, assuming it is an option. And I don’t get why someone in the field wouldn’t. Maybe some day I will become enlightened.
I don’t think Daniel Bernstein got a lot of flak when he failed to make NaCl and TweetNaCl formally verified. He’s Daniel Freaking Bernstein after all, world renowned cryptographer who successfully invented primitives. Or maybe because this was 2008, and formal verification wasn’t such a thing at the time?
I don’t think Frank Denis got a lot of flak when he failed to make Libsodium formally verified. After all, it started out as a repackaging of NaCl, itself not formally verified, and expanded since. Not long ago, security advocates were still chanting use Libsodium, use Libsodium, use Libsodium. On the supported platforms, it is indeed a very strong alternative, used by high profile projects and audited by one of the most reputable security companies out there.
Even I didn’t get a lot of flak for failing to make Monocypher formally verified, when I started out over 4 years ago. I did get a lot of flack for daring to write a crypto library without first building a reputation for myself (which is backwards: achievements build reputation, not the other way around). Though that was almost exclusively from people who didn’t even look at my work (as evidenced by their criticism).
Now there is no question that a formally verified crypto library maximises confidence. We can’t beat machine checked proofs. But we can approach them. Here’s how I did it:
Is that as good as a formal, machine checked proof of correctness of the generated binary code? Certainly not. But it damn sure approaches it, especially if your compiler is formally verified like ccert. So when I say I sacrificed confidence, that’s how little. While I would hesitate to trust it with the nuke launch codes, I would trust it with my payment card information.
Moreover a project using a crypto library must also consider other sources of errors. If the rest of the program isn’t formally verified, we’re kinda back to square one, where the whole thing must be thoroughly tested to make sure we don’t have some vulnerability or error lurking there. For instance, have we made sure that the network packets can’t be adversarially reordered without us noticing? Using authenticated encryption is not enough, we need message numbers or timestamps for this. So unless you formally prove the whole thing, that vaunted confidence will be significantly lowered, and the difference between a formally verified crypto library and a “merely” rigorously tested one is even slimmer.
As for the advantages, well: Monocypher is a single file library. One source file, one header file. One more of each if you really really need compatibility with Ed25519 (by default, Monocypher uses Blake2b instead of SHA-512 for signatures). Putting it in a C/C++ project is trivial: just copy & past the files in. Language bindings are easy to write, and people wrote several without me even needing to ask. The size of the binary is very small, and further increases Monocypher’s portability. Though it wasn’t the initial goal, it turns out Monocypher is very well suited to small 32-bit microcontrollers: doesn’t take much space, and it is fast.
Now I’m not saying a formally verified library can’t also be extremely portable and very easy to use. Strictly speaking, Monocypher itself could be formally verified (assuming the C standard can be at all formalised). But if it turns out the currently formally verified libraries are harder to deploy, or less well documented (Monocypher has exquisite documentation, thanks to a couple contributors), then the time you take learning how to use it is time you don’t take writing or testing your own program. By being so cheap to integrate, Monocypher can also contribute to the overall confidence of the projects that use it. Or at least lower their costs.
That being said, sometimes the cost of using formally verified cryptography is very low. Jason Donenfeld briefly used Monocypher in the Windows Wireguard installer (to check that the fetched binaries were the right ones), then at some point switched to formally verified code in very little time. As much as I’d like to deny it, this was the right move.
If those people come into a forum where I’m discussing the advantages of safe programming languages and formal verification and they argue what you argued, I would respond the same way. This isn’t a personal attack…
And all of the other text in your post, to me, further strengthens the case for safe languages (assuming ergonomics, portability, etc are on par, as we’ve discussed). Having lots of arbitrary application code to secure, having lots of programmers who would otherwise have to be extremely diligent with tooling, etc - it just isn’t going to happen.
I may have been more salty than I have any right to be, sorry about that. Rest assured, I perceived no such attack. The advantages of formally verified code are real, and I reckon I have a vested interest in ignoring them.
Speaking of which, a more recent post taught me that the first thing they wrote in Rust was the ASN.1 parsing. That kinda changes everything: the advantages of a safe language in this case is substantial. Chacha20 is easy to implement safely in C, but recursive parsing code is a whole ‘nother can of worms. I’ve written parsers, and C is never my first choice there. The compatibility problem sucks, but if they need ASN.1, that was probably the right move. (Modulo a slap on the wrist for failing to use semver.)
Incidentally, I believe safe languages have the potential to be even more portable than C currently is. Once you have a formal specification and a formally verified reference implementation, you can make guarantees far beyond C could ever provide. And if you find a way to compile to standard C itself (in a way that doesn’t rely on undefined/unspecified/implementation defined behaviour), then you instantly exceeds its portability, though talking to existing C code may still be a hassle.
I agree, even in the case of Monocypher: while I have personally been very diligent about its implementation, and can confidently claim it is solid, its interface is definitely unsafe. Because C, and because no dependency: it’s not just the buffer overflows, there’s also the lack of an RNG. Users can’t just generate a key pair, they have to provide the random private key themselves, and I can just pray they stumbled upon one of the many warnings in the manual about how they should use the OS’s RNG. Same problem with authenticated encryption: users have to provide the nonce themselves, and the onus is on them to avoid catastrophic nonce reuse.
Because of this, I strongly believe that a low level crypto library such as NaCl, Libsodium or Monocypher is not enough (they’re higher-level than the low-level operations of OpenSSL, but my work on protocols convinced me they’re still fairly low level). At the very least, I strongly encourage language bindings to provide a higher-level API where users no longer have to worry about providing clean random numbers.
I’m also working on higher-level protocols. Key exchange first, then I’ll tackle a file format. Those can still be free of I/O, small, simple, and in C (with the usual extreme required diligence, which I’ve become better at the last few years). The end game, I think, will be a high-level network library. I’ll have to tackle actual I/O this time, so extreme portability is not possible. Pure C is really not looking good with this one. A Rust or Zig implementation (both of which support async), with a C interface, are probably much better. I’m not sure yet, we’ll see.
I don’t think the original author of the article realizes that cryptographic code is different. His arguments might be right for cryptographic code, but wrong for code in general, which was what he intended. Probably a Gettier problem here.
Agreed, I also suspect he’s being right for the wrong reasons.
Author here.
You are probably right. I should go write some crypto code. (And not publish it, of course.)
The road to publishing your own crypto code is hard, but not impossible. If you see the need and have the motivation, there is a path.
I’ve actually read a lot of your articles already, for years. I’m a fan. :)
Sitting down to read that article now.
Author here.
I am aware of many reasons (though not all of course) where crypto code is different from
bc. However, what I do know is that crypto code should be small, thoroughly tested, and frozen. My argument in the post was “I did this amount of due diligence forbc, which is not ‘critical’ code, so have thecryptographyauthors done that much due diligence?”For some reason this article made me quite angry. It is the epitome of “better things aren’t possible”, which means it is the year 2400 and we get regularly owned by buffer overflows because we keep on using a rather average language from 1978 since this is the best there is and just someone think about 32 bit SPARC machines that were last sold somewhere in 2000.
This is exactly the reason why obviously good ideas like GC, ADTs etc, stuff invented in the seventies is only now becoming mainstream, slowly being dragged into the mainstream by Java and TypeScript while still being worse alternatives than non-mainstream things we had a long time ago. How much money will we continue to waste on completely preventable issues in software just because people are hesitant? And with our modern life becoming increasingly dependent on software, how many lives will be lost because somewhere there is a bug in software that a compiler from the 1990ies could’ve flagged?
I am not even going to go into the part where the author is misinformed about Zig or the responsibility of open source developers, that can has been opened here already.
Author here.
I acknowledge that I was misinformed about Zig. I have updated my post to make that clear.
For the record, if we could get everyone to move off of bad tech to good tech, I would be ecstatic. My point is that it’s not going to happen, not easily.
I guess you could say that the post is about pragmatism vs idealism. I think pragmatism will give us faster progress towards those better things than idealism.
I think it’s a fairly big assumption to say that your post contains no idealism, and the approach of those you disagree with contains no pragmatism. I’d argue that’s an incorrect assumption.
I think that’s a fair assessment.
“Because I went through all that hardship, everyone else should as well” appears to be most of the argument
Firstly, cryptography is taking the good approach to a Rust rewrite: one little bit at a time. If they just rewrote everything in one go, sure it’d be buggy. Doing a chunk at a time means you can use the non-rewritten bits as extra checks.
Secondly, Rust means that tools like Valgrind, etc are much less needed. I’ve used them, they’re great tools, but they’re tools of desperation, of “this is all I’ve got, so let’s use this”, v.s. actually fixing the underlying issues, which we just can’t in C.
If the author wants to ignore all that for his own tools, that’s his choice and he can do it. I’m not going to stop him expending all that extra effort, but don’t force others to do so.
My crypto library takes 30 second to test under Valgrind. Less than a minute if we add all the sanitisers as well. And believe me, the test suite is very comprehensive. The real tools of desperation here are the FramaC and the TIS interpreter.
Yes, Rust removes the need for those tools. The difference, in the particular case of crypto libraries, is very small in practice.
Saying that users paid for this software via exposure and thus are entitled to paid-level support is ridiculous. So gas-lighty
“ One of the most vocal (in favor of Rust) developers of cryptography works for Red Hat Security Engineering (if I read his GitHub profile right). I don’t know if his work on cryptography helped him get that job, but it might have.
Another of the most vocal developers has a computer security company. I would bet money that his work on cryptography gives his company relevance.
So they did not get “nothing” from users. Quite the opposite, in fact. “
The developers don’t owe anyone jack shit. They could nuke it tomorrow, like that one left-pad dude did, and walk away with a clean conscience. These projects are massive amounts of work, for 0 pay. And no, recognition of your own hard work is not pay.
I have to disagree on this one. The authors wrote this stuff publicly, with the full expectation that they would influence other people’s behaviour over this. Specifically, they expected they’d have users. They have set expectations, made implicit promises.
More generally, being a decent human being means making sure you’re not doing more harm than good. At the very least, you owe it to your users not to waste their time.
In this particular case, rewriting the code in Rust has dubious benefits, and a very clear downside: instant breakage (or at least major inconvenience) on all platforms that don’t enjoy 1st tier Rust support. Shockingly, we still have many of those. They tried to make things better, but in the process broke their implicit promise that their stuff would continue working as long as they maintain it. They lost many people quite a bit of time. Though unwittingly, they very likely did more harm than good.
Now if Rust was supported everywhere (maybe by generating portable C code?), that would be another story. It isn’t, though. Not yet.
I’m sorry I still have to disagree on the fact that FOSS projects have some sort of implicit responsibility to coddle the 10% of their users using esoteric platforms from the 90’s. Your comments on not wasting user time is accurate though, and choosing Rust is a subjective opinion IMO, someone could have a long and thoughtful discussion on the pros and cons of such a decision.
But.
“but in the process broke their implicit promise that their stuff would continue working as long as they maintain it”
This is backwards to every single FOSS package I have ever used. If they don’t explicitly communicate long term support of legacy platforms as a project goal, there is no reason to assume that is a goal! Maintaining legacy platforms is a massive pill, and I can only count on 1 or two hands the amount of projects I have utilized that have dictated that as a goal.
As for the users whose CI’s automatically broke, that’s just poor engineering on their part. Dependencies should be pinned, and extra work should be expected when upgrading packages. It’s an upgrade, the maintainer doesn’t have the responsibility to make it seamless.
My intel suggests otherwise: the maintainers don’t follow semantic versioning to begin with. This update was treated as a minor update, despite the existence of a breaking change. Of course automated CI tools started to break.
Now I don’t (and shouldn’t) have a say about which direction a project I’m not even participating in should take. They want to sacrifice some portability for a bit of safety, that’s their prerogative. I won’t even ask them to continue support of the old versions. As nice as it would be, it still costs time. My problem here is the failure to mark that change as “breaking”, and therefore at least bump the major version number. Even if it was an honest mistake (I reckon it probably was), they should have been a bit more careful.
And when users started to complain left and right, they should have reverted the change, bumped the major version number, then applied the change again. (Maybe they already have, I haven’t checked.) Insisting that a breaking change is not worthy of being marked as such would definitely not be okay.
Please go ahead and read the licenses of all these projects where they make completely explicit the terms of use, and make it clear that there should be no other expectations beyond those terms.
I am continuously puzzled by this trend in a certain section of open source users to conjure out of thin air new terms and promises of support from these projects that they use for free, with no basis in fact, and coincidentally with effects that would benefit these users. But perhaps I should stop being puzzled by this. /r/ChoosingBeggars exists, after all.
Licences are legal disclaimers. The README (of pretty much any project that went to version 1.0 and beyond, including mine), still say that this product is suitable for a range of purposes.
I’d agree with this if the license didn’t set explicit expectations that override the implicit.
(Although I think there are reasonable limits, just as there are limits to what’s enforceable in any contract.)
I’d agree if the README (or other documentation) didn’t explicitly contradict the licence by describing what the project is suitable for (at least for projects that are beyond 1.0).
This article is not a good source of facts. Lots of opinion, truisms, appeal to convention, past, hierarchy and status quo.
Cryptography is right. “Battle tested” doesn’t mean squat. And if you aren’t puttying your C in a Wasm env, you are doing it wrong.
It isn’t futile, it is the right thing to do.
Cryptography is wrong. They would be right in the vast majority of cases, but cryptographic code really is different. While a Rust rewrite isn’t very risky, it is fairly pointless: if the test suite for the C code is any good (and it’d better be, considering the stakes of cryptographic code), the Rust rewrite is very unlikely to fix bugs that weren’t already found and fixed long ago in the C code.
Now if Rust really was as portable as C (first tier support in all relevant platforms that currently have a worthy C compiler), I wouldn’t object. But it’s not. Not yet.
I’m pretty sure people printed out and circulated “C, Algol, and the futility of replacing assemblers” posts in the 70’s. ;)
I remember lots of such posts even in 2000s. Not exactly “futility of replacing assemblers”, but “C, C++ won but we should go back to assembler, because of unnecessary abstractions causing bloat”. Even “Windows suxx because it’s written in C”. With the same arguments as today: “high-level languages are for those with poor skills, skilled programmers can write everything in assembler”.
Oh look another “I’m competent so I can write safe C” blog post. How many more of these are we going to write before we accept that it isn’t possible. If we decide that platforms like the Amiga or DEC Alpha are important enough to hold the rest of us back from real security improvements, I don’t know what to say. Sorry, we aren’t supporting your sun SPARC workstation anymore?
That’s not quite the argument; he says that this is why he’s writing his own language that compiles to C …
Both points about Zig are refuted (strong, intentional effort to spec underway; “trying” to make it self-hosted is some really interesting language). It’s explicitly designed to compete with C at C’s level, which differentiates its goals from Rust’s.
Zig also interfaces amazingly well with C, and has the best cross compilation workflow I’ve ever encountered.
Author here.
I have added links about Zig in the post that refute my points.
Thank you! <3
This argument is regularly contradicted by free software licenses themselves. Here is GPLv3, a copyleft license:
And here’s a permissive MIT license:
When does this supposed entitlement to a volunteer’s time end? When they leave the project? When they die?
Paid free software development muddies the waters, because people employed to work on a free software project aren’t usually volunteers. Regardless, there is still a lack of warranty.
Literal interpretation of licenses will conclude that there was nothing wrong with event-stream version 3.3.6, which after all, was released under MIT license. event-stream’s maintainership was taken over and code was added to harvest private keys of Bitcoin accounts. Is that okay because the maintainer was a volunteer and code was provided WIHTOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED? I reject such interpretation.
https://blog.npmjs.org/post/180565383195/details-about-the-event-stream-incident.html
Let’s be clear–what happened with event-stream and what may happen with other attacks on open source software should not be taken as an excuse for users to get free support. Using a giant dependency chain while taking absolutely no responsibility for locking it down and doing vulnerability checks (which are both supported by the npm ecosystem) is super irresponsible, perhaps even rising to the level of malpractice. And just to be clear, we shouldn’t place the blame solely on lazy or nefarious devs; this is a systemic problem in organizations that use open source software, starting from the top down.
Agreed. As is often the case, what the legal document says is rather at odds with reality. There’s multiple layers in that:
There’s a reasonable argument that the event-stream update is piracy/unlawful intrusion in someone else’s computer. That goes beyond just breaking your code and directly into illegality. So, not the same thing at all.
You are right.
I liked the article except for the “they have obligation part”. No way. Each piece of FOSS is just a precious gift. What if I release something and then die the next day. Do I have an obligation to not die?
Yeah the bits about lib authors wanting users to hop onto their stuff sounded right. But then going on about an obligation to support the same set of features (platforms,languages..) for eternity is just plain wrong. Also it’s not like their working non-rust version is going away. People really could fork it and continue with that. There should be enough developers capable of doing that.
I’d argue it’s worse than that. There’s this huge culture in the Linux and FreeBSD space which has been clearly demonstrated by this saga where distribution packagers/ports maintainers will build software in completely untested/unsupported configurations and then rouse a mob when the authors break these (despite never committing to support them in the first place).
When you make a “minor patch” to your software, you kinda promise there won’t be any breaking change. Such as, no longer working on a bunch of relevant platforms.
Deprecation is okay. Just mark it as such, and do change the major version number when making breaking changes.
All changes in open source software are made on a best-effort basis. No ‘promises’ are made, ‘kinda’ or otherwise. Publishing open source software does not entitle users to expect free support or any other expectations from the software. Let’s be really clear on this.
Oh, yeah, I didn’t allow for the possibility of an honest mistake. My bad. Of course any change can introduce bugs, so there’s that.
What would not be okay would be insisting that this change is not breaking, even after being told it is. The correct response from the feedback they got, I believe, would be to revert the change, bump the major version number, then apply the change back. The old versions don’t even have to be maintained.
Author here.
No, you don’t have an obligation to not die. :)
My broader point is that not all FOSS is a pure gift, but an exchange. We can definitely disagree on that, but your opinion is just as valid as mine. In fact, I fully acknowledge that I may be wrong. It’s just that my opinion is what it is based on my current experience.
Your opinion is incorrect based on the actual license terms that you agree to when you use the software.
Legally, I agree with you. Socially? That is still debatable.
As far as I know, open source projects don’t agree to any ‘social’ terms of service with users.
EDIT: also, try saying that commercial software projects have a ‘social’ obligation to support you for free :-)
Your edit is a good counterpoint.
I think you and I have clarified where we disagree. I think it is best to leave the conversation there, with your leave. :)
Using the licence’s legal disclaimers to absolve yourself of all actual responsibility is dishonest. I use the same legal disclaimers, and I would never dare.
This is not a case of developers stopping to work on their product. This is a case of developers actively making the life of some of their users more difficult. They failed to do no harm.
I’d argue at the point that a distribution starts repackaging software in ways the authors haven’t tested, or shipping it on platforms the original authors never tested or supported, the users of that software are no longer the author’s responsibility but the distribution maintainer’s. Just because the authors didn’t list that the software definitely won’t work on a 30 year old DEC Alpha processor they are not suddenly owed to make it work there.
Well, I also happen to disagree with the very notion of Linux distribution. We shouldn’t need those middle men either.
Nevertheless, depending on Rust is a breaking change. That would have warranted at least a major version bump. That’s how much effort I would have asked of them: increment a number. That would be enough to make it right in my book.
It’s obviously not a breaking change in supported platforms, because it didn’t break those platforms. And breaking changes in unsupported platforms are, by definition, not supported.
That’s something important I didn’t check: do they have an explicit description, or list, of supported platforms? If they did, that’s all good, and shame on distro maintainers for distributing them on unsupported platforms.
If they did not… then the list of supported platforms is “wherever it builds and works out of the box”. Whether the maintainers like it or not.
There’s a world of difference between what the python-cryptography developers did and what happened with eventstream. The latter was intentionally malicious; the former was not. Legal language doesn’t absolve one of responsibility for malicious behavior.
People are justified in being pissed off about the python-cryptography change. They’re justified in saying that “python-cryptography has broken trust with us”. They’re justified in applying Freedom 3 from GNU’s Four Freedoms and forking the project.
They aren’t justified in feeling and voicing a sense of entitlement. That is all I’m saying.
s/feeling and //Right? People can feel all kinds of weird and unfounded and unjust feelings and that’s OK.
Excellent correction, thank you. It was the voicing and the language that I had a problem with.
So if someone sued you, you would tell your lawyer to ignore the legal disclaimer that would protect you? Why even use it at all, just use public domain.
If C++ cannot run on them, what are the odds that python-cryptography does, even when it isn’t using Rust? Phrased differently, how big, really, is the intersection of the set of systems that will run python-cryptography with C primitives and the set of systems that don’t support C++? I think I smell a strawman but I’m not an embedded systems person.
Yes python-cryptography depending on Rust is a problem. It broke architectures that are supported by Gentoo, because LLVM – and hence Rust – aren’t available there. I think the real problem is that Rust is not a mature language, but I wish the Rust folks well.
Flag-waving for C isn’t the answer. Insisting on “portable” C is how we get things like glib. As an aside: if you want C++, by all means use it. Please don’t try and reimplement huge chunks of it in C. There’s obviously a need and a desire for safe languages, and playing ostrich won’t make it go away.
I’ve been taking a closer look at Nim lately. It compiles to C and interoperates well with C. It looks like it could be wildly portable and a really good option.
glib seems to work? The reason glib is written in C and not C++ is unrelated to portability. It is related to the fact that C++ is much harder to bind to other languages than C.
Glib works. Lots of things work. That doesn’t mean they are necessarily good ideas.
As for the difficulty in making bindings for C++ libraries, is that due to the language itself, or due to the external interface of the library? I think it’s probably due to the external interface of the library. You can make a C API for a C++ library.
Yes, having a C API for a C++ library is a good idea. In fact, this is how LLVM provides its stable API.
mrustc can compile Rust to C. So it doesn’t depend on LLVM (with some caveats). Also fuzzing cannot match static guarantees. The former is: “I did a search through all code paths I could find and didn’t find a bug”, whereas the latter is “there may be no bug, no matter what code path is taken”.
Case in point: I once had a bug in a string handling library that was extensively fuzzed, but all that fuzzing didn’t find the UTF-8 short space character that led to a byte index landing within a UTF8 character.
While the average Rust code is much safer than the average C code, even in the hand of very competent developers, we must keep in mind that cryptographic code is not average. Especially modern cryptographic code, designed after NaCl.
You thought writing cryptographic code would be harder to write than application code? I mean, don’t everyone tell you not to roll your own? Well, the answer is two-fold: the logic of cryptography is often very delicate. Rust won’t save you here. But the system aspects of it are often very easy. Rust will help you there, but we hardly need that help to begin with.
I have to agree with the author here. There was no need to rewrite it in Rust, and the decision to do so anyway is most probably not worth the trouble. Especially damning is treating this major breaking change as if it was a mere patch. Affected users should probably leave and use something else.
Speaking of which, Shameless plug time: Monocypher is an extremely simple crypto library, written in portable C99. It has Python bindings, written by a user who needed it to update the firmware of the embedded device they sell. If you were using this crypto library and your platform doesn’t support Rust, take a look at Monocypher. It’s not a drop-in replacement, but it might be a worthy alternative.
Or, instead of using a non audited library written by a single person, look for actual improvements like project Everest where absence of memory errors, and validity of the code, are proven formally, not just trusted because “fuzzer says it’s ok”.
Two things you should probably have checked before writing this:
Now Project Everest sounds very nice, if you can get it to build on your particular platform, with little enough effort. By comparison, Monocypher is basically a single file C library, extremely easy to deploy for C/C++ project, and easy to write bindings for (though it already has bindings for Python, .NET, WASM, Rust, Go, and more. I think that count as an “actual improvement” over TweetNaCl and Libsodium in some niches.
Ah, good about the successful recent audit, I wrote that based on memories from earlier :-). That changes my view a bit on your library, and maybe some day, if it gets OCaml bindings, I might use it.
And now I regret the salt in my earlier comment. :-)
You can find the list of language bindings here. Someone did the Ocaml bindings there. You may want to take a look, perhaps get in touch with the author if you feel like contributing.
They’re actively working on a spec for Zig btw. Here’s Andrew talking about it, first thing as part of the roadmap video: https://youtu.be/pacsngNYXI0
The author seems to be very short sighted in my opinion. Even though I don’t like that cryptography library is moving to Rust, I’d rather they created a new crypto library in rust and donated the old C one to new maintainers. It is in the end their call, and Rust will provide a ton of safety features out-of-the-box.
I think that it all boils down to which platforms the cryptography library committed to support. If they want to support all platforms that Python runs, then RIIR is the wrong option. If they’re going to make it clear that it supports just a subset of those, then it is quite OK. People who are in the orphaned platforms can fork the last known working version and move ahead with a new team.
also SIGH but the first post linked on his sidebar is a rant where he explains why he refuses to wear a mask and calls business who force him to wear a mask, tyrants…
Is your concern that they are keeping the “brand” for cryprography? There’s certainly nothing preventing new maintainers from taking over the C version…
not the brand from a marketing perspective, but that build systems and other automated tools point-of-view. Those who are pulling their sources will depend on a whole new toolchain to build it, that affects package maintainers and products for a whole range of architectures.
IMO, drastic changes like that should be done in new libraries so that it doesn’t break other systems. They could’ve declared that library finished, created cryptography2 or some other name and proceed to RIIR. The people using their library would have the option to stay with the unmaintained system and keep their stuff working, or go through the trouble of upgrading to the new library.
That’s a strange inversion of responsibility. It’s also not viable since they’re doing an incremental rewrite, not a wholesale one. A wholesale rewrite might warrant a new library, but incremental rewrites are reliant on the full existing code and history to do properly.
The responsibility for maintaining a library falls on the library maintainers. The responsibility for maintaining a distro falls on the distro maintainers. If a library stops supporting an environment, then the distro has to choose: also stop supporting the environment, or fork the library which they have license to do. “Tell the library maintainers to maintain their library differently” is not one of the options. Or, at least, not an option that will work.
I personally would be okay with just a major version number bump or equivalent. Keep the brand, say the newer version is now the canonical one. Just:
Those safety features are almost irrelevant in the specific case of cryptographic code. Platform support aside, it won’t hurt, but it won’t do much good either. Don’t get me wrong, I would love that Rust be supported everywhere. Until then, C is still the better choice for portable cryptographic code.
My biggest qualm about this change is that they didn’t mark it as “breaking change, let’s bump the major version number”.
“My biggest qualm about this change is that they didn’t mark it as “breaking change, let’s bump the major version number”.”
This was misunderstood by others as well. This project does not use semantic versioning, and the users were warned beforehand :
https://cryptography.io/en/latest/api-stability.html#versioning
https://github.com/pyca/cryptography/issues/5771#issuecomment-775038889
https://github.com/pyca/cryptography/issues/5771#issuecomment-775041677
They’ve changed their mind, and from now on their project will use a semver-compatible system. The version they released that caused the issues was on their old (weird) scheme, tho.
Kudos to them, then. While ideally they should probably have reverted the change, switch to semver, then apply the change back, the switch to semver shows that they are listening to feedback.
it’s significantly worse than that; see my other response here. this is simply a terrible person.
Why don’t we stick to discussing the article, and leave the person out of it?
I guess C is futile too, because of all the 6502s that can’t handle a decent C compiler.
Well, there’s cc65, but it has many limitations. Of course you knew that, so the real question is how “decent” you think it is. :P
Substitute 6502 for 8052 then.
To add, most embedded isn’t self-hosting, so it’s an odd debate anyways. Designs newer languages can’t target tend to either be rising (like RISC-V) or on their way out. Do you think i.e Microchip would invest in a Rust compiler for an MCU they no longer support?
ada is ignored :(
Note: ADA spawned a child with a formal specification!! That’s better than even C, and in the long run potentially even more portable (that with the potential to writing formally verified compilers for new and old platforms).
Author here.
I know about Ada and SPARK, but unfortunately, it seems to have most of the same portability problem as Rust.
For the record, Yao is going to steal a lot from Ada/SPARK in order to make creating quality software easy.
Author here.
Now that I have an account, you are welcome to ask me questions. :)
Also, I have made some updates to the post based on some comments from here, on Reddit, and Hacker News.
The biggest change is that I have added links to the post refuting my claims about Zig. I have also clarified some arguments.
This calls for some discussion about the value of accumulated user testing. How illegible does code need to be or how much accidental complexity does it need to contain in order to warrant refactoring and thus invalidating the hardening that happened in production?
Author here.
That is a good question, one I do not know the answer to.
a terrible take from a person who is full of 💩 https://campaign.gavinhoward.com/platform/
the author is too shortsighted to see how shortsighted they are 🤷
https://twitter.com/wbolster/status/1365983352835231744?s=19
To be fair, author’s extremist views on christianity, society, medicine or politics doesn’t make him wrong on another unrelated subject. “You should first fix bugs instead of switching to a less portable language” is not a wrong thing to say. “You should not break other people’s operating systems” is not a wrong thing to say. Rust may be superior in terms of end product quality, but it is not the default choice for every situation.
“don’t write bugs” doesn’t work despite 40 years of industry experience with unsafe languages. see other comments.
the piece is littered with opinions about what others should do (with their spare time even), while at the same time maintaining poor people shouldn’t be allowed to vote, and holding other dehumanising views. imo, that is very relevant background: it tells me that whatever this person thinks that others should do is completely irrelevant.
yes you are entitled to your opinion, so is the author else we could not have a dialog about these subjects. The view in the piece is “fix your bugs” and “fuzz your bugs” and “be responsible” not “don’t write bugs”.
And Rust is about “preempt a class of bugs” well so is any language that consider nulls a design flaw for instance. So are languages that prevent side effects, that enforce exhaustive conditionals, that enforce exception handling, and so on. So it isn’t the ultimate language.
Except in cryptography. In this particular niche, “don’t write bug” is both mandatory and possible. In this particular niche, Rust is of very little help.
That said, the author may not be aware of the specifics of cryptography. He may be right for the wrong reasons.
I think “consistently being wrong” is a valuable insight, and can span unrelated subjects.
Very reasonable, but not what’s happening, nor what he’s saying.
What’s happening is “making a change to one software project”, that is provided under a license that does not guarantee support.
What he’s saying is “changing what platforms you support is unacceptable”. He’s also saying “switching to Rust is a dereliction of duty”. Which implies “you have a duty that is not listed in your software license, and which I won’t elaborate on”.
In other words, what he’s saying is, ultimately, “nobody should make changes to their software that I disapprove of”. Which is wildly unreasonable.
Unsurprisingly, his political views skew towards “the only people who are allowed to be part of society are people that I approve of” (he approves only of a certain subset of Christian faithful, who must also be straight and cisgendered, believe that certain parts of the Constitution are invalid, and be in the top fraction of earners).
In other words, his poor political thinking mirrors his thinking on the politics of open source development in ways that make it relevant to the conversation.
What’s happening is “making a breaking change” without marking it as such (deprecation, major version number bump, that kind of thing). That’s not okay. I don’t care it was a labour of love provided for free. The rules should be:
It’s not a breaking change if it breaks on platforms that were never explicitly supported (not even by python). (I’m not entirely sure if they used to support these explicitly, but it’s not obvious).
I agree if there’s an explicit description of supported platforms. If not… the only reasonable fallback would be any platform supported by Python where it builds & run out of the box. That can mean a lot of platforms.
They did, at least for certain values of “major version number”. Rust was introduced in 3.4. The x.y releases are when backwards-incompatible changes are introduced. They’re changing their versioning scheme to make it more explicity: next major version will be 35 (instead of 3.5).
Honestly, a quick look at their changelog shows backwards-incompatible changes at 3.3, 3.1, 3.0, 2.9, … In other words, adding rust when they did was entirely in line with their release cycle.
I commend them for listening to feedback (or is it cave in to mass anger?), and switch to semantic versioning. That’s cogent evidence of trustworthiness in my book.
Author here.
Everyone is full of crud on some level. However, interacting with people nicely on the Internet is a good way for me to get rid of that and to become less of a terrible person.
On that note, hi! I’m Gavin. Nice to meet you. :)
Do you have a moment for our lord and savior Zig ;) (I like it, like rust, just in case..)
Author here.
I like Zig too, and I have updated my post with Andrew Kelley’s links to refutations of my points.
Interesting perspective on responsibility. Quite a compelling case, in my opinion.
Again and again, languages that compile to C win. So at this point, it doesn’t matter how C is generated, “compiled” from another language or written directly, undef behavior is the main issue.
Compilers can guarantee that the code they generate is free of undefined behaviour. Or at least of certain classes of undefined behaviour. Same way Rust compilers can guarantee the x86-64 instruction they generate are safe, even though using x86-64 assembly directly definitely isn’t.