I think at this point rustls can use multiple cryptographic backends but I could be wrong. Last time I was doing crypto stuff with it I had to explicitly choose to use ring for some stuff iirc
This causes a bit of headache for me. I doubled down on ring as the default backend for rustls when releasing ureq 3.0 just a couple of months ago. But this might mean I should switch to aws-lc-rs. Hopefully that doesn’t upset too many ureq users :/
There’s been some positive momentum on the GitHub discussion since you posted. Namely the crates.io ownership has been transferred to the rustls people and 2 of them have explicitly said they’ll maintain it. They need to make a new release to reflect the change and then will be able to resolve the advisory.
My dream for ureq is a Rust native library without C underpinnings. The very early releases of rustls made noises I interpreted to be that too, even though they never explicitly stated it being a goal (and ring certainly isn’t Rust native like that). rustls picked ring, and ureq 1.x and 2.x used rust/ring.
As I was working on ureq 3.x, rustls advertised they were switching their default to aws-lc-rs. However the build requirements for aws-lc-rs were terrible – like requiring users on Windows to install nasm (this has since been fixed).
One of ureq’s top priorities has always been to “just work”, especially for users new to Rust. I don’t want new users to face questions about which TLS backend to chose. Hence I stuck with rustls/ring for ureq 3.x.
aws-lc-rs has improved, but it is still the case that ring has a higher chance to compile on more platforms. RISCV is the one I keep hearing about.
Wait does that mean the Rust ecosystem is moving towards relying on Amazon and AWS for its cryptography? That doesn’t sound great. Not that I believe Amazon would add backdoors or anything like that, but I expect them to maintain aws-lac and aws-lc-rs to suit their own needs rather than the needs of the community. It makes me lose some confidence in Rust for these purposes to be honest.
I expect them to maintain aws-lac and aws-lc-rs to suit their own needs rather than the needs of the community
What do you see as the conflict here, i.e. where would the needs differ for crypto libraries?
I’d expect a corporate funded crypto project to be more likely to get paid audits, do compliance work (FIPS etc), and add performance changes for the hardware they use (AWS graviton processors I guess), but none of that seems necessarily bad to me.
Things like maintaining API stability, keeping around ciphers and cryptographic primitives which AWS happens to not need, accepting contributions to add features which AWS doesn’t need or fix bugs that doesn’t affect AWS, and improving performance on platforms which AWS doesn’t use are all things that I wouldn’t trust Amazon for.
This is another reminder of how much we often take for granted. On a positive note, the project has demonstrated value and now the need is better known. The transparency is refreshing because there are a lot of other projects with commercial backing with very limited community response.
Slightly off-topic, but I have been very keen to see the rustcrypto project evolve, (and hopefully one day see formal verification).
Their use of strong traits (e.g. aead) make it very easy to enforce safety constraints directly in the type system. For example, these traits have already been used to wrap ring itself to allow easy swap-outs of cryptographic backends.
I often find libraries that will only accept, say, use of a file-backed private key. Instead, having libraries accept a generic signer trait implementation allows agility between cryptographic backends, such as different hardware devices or crypto libraries.
With cryptographic libraries in Rust being far from a settled debate, I strong believe there’s massive value in settling on standardized trait implementations for cryptographic libraries in the crate ecosystem to make these kinds of switches easier going forward.
OTOH, I strongly dislike the traits. They make usage of basic primitives more difficult. Their interface is spread over multiple crates, and everything goes through layers of indirection. It’s super annoying if I just need sha256, but need to import traits and convert from a GenericArray.
They also make semver-major changes too often for the ecosystem to keep up. A Signer trait is more trouble than it’s worth when you end up with a v0.9 implementation not working with a v0.11 interface.
For example, the yubikey.rs crate exposes a signer trait implementation for keys backed by the Yubikey PIV interface. This means that my code for Kerberos signing doesn’t need to concern itself with how the key is stored; all it needs is a struct that implements signer.
I think this sort of thing, beyond anything technical, is the biggest argument for growing the standard library (or, at least, those official ordained and maintained by the rust-lang org). I can argue about the technical merits of Rust to my Rust-sceptic colleagues all day, but it’s difficult to deny that a worrying number of important dependencies in the ecosystem have a low bus factor.
I don’t understand how the standard library is any different?
Python, as one example, has several part of its standard library that are unmaintained and falling backward. Same for erlang. How is being part of the standard library helping?
Indeed, being in the standard library doesn’t guarantee active maintenance.
There aren’t enough people in the core team to work on everything. There still has to be someone with expertise and dedication to improve the code.
There aren’t that many people available for reviewing pull requests either. The standard library is focused on preserving backward compatibility and portability, so changes take more time and it’s harder to contribute more than an occasional drive-by tweak.
In Rust, mpsc channels were an example of this: they were in the standard library, technically maintained, but nobody actively worked on improving the implementation. Eventually the std code got replaced with an implementation developed outside of the standard library.
This just puts a bit of rusty paint on the problem. The Rust project had very little capacity for library maintenance, the bus factor would not differ substantially. Indeed, for that reason, the project has more moved libraries out rather than in.
Coming from another language ecosystem, I think this doesn’t acknowledge an extremely common scenario: library maintainer is AWOL, and people show up with patches, and it’s just that there is nobody there to hit merge and do a release.
There are huge piles of projects where there are a bunch of people who are capable, reviewing each other’s changes, and saying “yes this works, I have a fork and we have it working” etc etc etc, but the release can’t happen because one person holds the keys.
Having a large team who holds release keys is generally something that can work out! For every “the maintainer is the only one who could actually make a call on this”-style issue there are hundreds of “maintainer just is done with the project” situations.
I’m not calling to have eminent domain, but having lang teams be able to adopt packages (with maintainer consent) and allowing for a light touch can totally work, especially in the age of Github & co.
Bus factor and capacity are different things. I’m not expecting development to go faster or anything: but when something really, really critical pops up that needs fixing with haste, a larger pool of people ready to merge a fix and deploy a release matters. Ossification and glacial development is fine, not having an official patch for a CVE because the one dev is (quite understandably!) on holiday, less so.
The standard library only releases every six weeks. The more stuff that’s in the standard library, the more stuff that would maybe call for an emergency release, which is far heavier weight and far more disruptive to the ecosystem. People already think Rust releases too often.
Also, a crypto vulnerability is likely to have a high priority. If have to urgently release/deploy, I’d much rather update a single crate than the whole compiler and stdlib.
Not trying to be snarky, but is the complaint about too many releases (and operational churn)? Or too much breakage (from stuff like the “partial ordering for sorts can trigger panics now” saga)?
Like one, you do land on “we should bundle up releases”, but the other is a different flavor of thing, right? Though I mention the sorting panic thing only because it’s the only case I have even heard of.
Saying all that, while I like fat standard libraries, it does feel like the weaker version of “it’s in the rust-lang org but still packaged separately” gets things 95% of the way there for these kinds of issues, IMO.
Operational and “reputational” churn. The time crate is the only breakage I’ve experienced personally in a long time. The process for releasing rustc is involved, the process of releasing a crate is “cargo publish.”
I already regularly hear complaints from very conservative type folks who look at the six week cadence and complain about how often releases happen. Adding more to the mix regularly increases the perception that Rust is still mega unstable, when that’s just not the case.
Can defeinitely see rustc’s released process being involved vs cargo. Having multiple trusted people be able to run cargo publish is the valuable thing here, moreso than it being in the standard library itself.
I do think it’s important to dig into what the reason for the cadence complaints really are though. I feel like the biggest lesson to be taken from Chrome showing up and doing releases all the time is that smaller releases (-> more release) are easier to manage in theory. It would be a shame if those complaints are downstream of other issues that are, in a word, “more” fixable?
This is all context free of course. At least in Python land complaints of things moving too fast that I hear tend to be broken down into “too much new stuff to learn” (a reason I guess, but one I don’t care about) and “old code is not working anymore too often” combined with “package maintainers disappearing” (which are realer issues, and can be handled).
It’s just systems people who are used to C89 and so they see backwards compatible but new features as “breaking” because they want to stay on old compilers but compile new code.
OK I see the case now. The maximally generous version of this is “well there is some issue X that is stopping my upgrade but everything around me is moving so fast that it throws me forward” (in Node this is common because packages agressively upper bound node versions even when they don’t need to, leading to a bunch of incidental things making upgrades hard), but tbh I feel like Rust isn’t doing it that much. Especially nowadays where people seem pretty serious about not using stuff from nightly etc.
The maximally generous version of this is “well there is some issue X that is stopping my upgrade but everything around me is moving so fast that it throws me forward”
I agree, but the “issue x” is mostly “i want to use a dependency and it uses some rust language feature that is newer than my current stable version” and the “stopping my upgrade” is simply “I don’t want to” not “I tried it and it didn’t work.” That is, it’s perceived instability, rather than actual instability. For these folks, “stable” means “unchanging” not “backwards compatible.”
The shift from RSA to ECDSA is, relatively speaking, quite recent and we just had Microsoft announce their crazy qubit capabilities meaning quantum resistant cryptos are maybe sooner required than we previously thought. I think crypto is exactly the kind of thing that needs to be kept out of a standard library to avoid churn/deprecation.
I’ll leave a top-level comment on it, but aside from having a single “blessed” library, I think consensus on cryptographic trait implementations for backends would be fantastic.
High-level libraries that can be built to consume those traits, low-level implementations that implement them; would lead to easier agility when the “blessed” library is no longer the go-to.
The idea that absorbing something like ring into stdlib would automatically give it new maintainers feels like wishful thinking. Getting maintenance done is a combination of having the required skill (and for crypto, the bar is high) and accepting that it’s not someone else’s problem.
The “someone else’s problem” feeling can be pretty high for code in a stdlib. Whereas here, in the crate ecosystem, when the ring maintainer gave up we got an advisory that prompted new maintainers to step in. Looks like FOSS is working.
Yeah, kind of a big problem for something so huge and important.
It does appear that a few volunteers are stepping forward to handle patching.
For something important the README is quite lacking.
It seems to be a dependency for rustls.
I think at this point rustls can use multiple cryptographic backends but I could be wrong. Last time I was doing crypto stuff with it I had to explicitly choose to use ring for some stuff iirc
In what way?
[Comment removed by author]
It doesn’t say what it is. It says it has code from BoringSSL but not an outright fork?
fair. The one-liner is at the top of the docs though at least: “Safe, fast, small crypto using Rust with BoringSSL’s cryptography primitives.”
This causes a bit of headache for me. I doubled down on ring as the default backend for rustls when releasing ureq 3.0 just a couple of months ago. But this might mean I should switch to aws-lc-rs. Hopefully that doesn’t upset too many ureq users :/
There’s been some positive momentum on the GitHub discussion since you posted. Namely the crates.io ownership has been transferred to the rustls people and 2 of them have explicitly said they’ll maintain it. They need to make a new release to reflect the change and then will be able to resolve the advisory.
That does buy some time. It’s the same people stepping up who are writing/maintaining rustls, which makes me happy.
Out of interest, what factors did you consider when choosing between aws-lc-rs and ring?
My dream for ureq is a Rust native library without C underpinnings. The very early releases of rustls made noises I interpreted to be that too, even though they never explicitly stated it being a goal (and ring certainly isn’t Rust native like that). rustls picked ring, and ureq 1.x and 2.x used rust/ring.
As I was working on ureq 3.x, rustls advertised they were switching their default to aws-lc-rs. However the build requirements for aws-lc-rs were terrible – like requiring users on Windows to install nasm (this has since been fixed).
One of ureq’s top priorities has always been to “just work”, especially for users new to Rust. I don’t want new users to face questions about which TLS backend to chose. Hence I stuck with rustls/ring for ureq 3.x.
aws-lc-rs has improved, but it is still the case that ring has a higher chance to compile on more platforms. RISCV is the one I keep hearing about.
Wait does that mean the Rust ecosystem is moving towards relying on Amazon and AWS for its cryptography? That doesn’t sound great. Not that I believe Amazon would add backdoors or anything like that, but I expect them to maintain aws-lac and aws-lc-rs to suit their own needs rather than the needs of the community. It makes me lose some confidence in Rust for these purposes to be honest.
What do you see as the conflict here, i.e. where would the needs differ for crypto libraries?
I’d expect a corporate funded crypto project to be more likely to get paid audits, do compliance work (FIPS etc), and add performance changes for the hardware they use (AWS graviton processors I guess), but none of that seems necessarily bad to me.
Things like maintaining API stability, keeping around ciphers and cryptographic primitives which AWS happens to not need, accepting contributions to add features which AWS doesn’t need or fix bugs that doesn’t affect AWS, and improving performance on platforms which AWS doesn’t use are all things that I wouldn’t trust Amazon for.
Yeah. To AWS advantage is quantum resistant cryptos and FIPS. In a comment below I found there is another initiative “graviola” that seems promising.
There is also the RustCrypto set of projects.
There is, but AIUI AWS-LC uses assembly code and thus can provide timing-safety, whereas RustCrypto is “written in pure Rust”.
TIL there’s a project called graviola which is by the same guy who did rustls. Maybe that’s the future post ring and aws-lc-rs?
The listed limitations would rule it out as a general replacement.
Yeah, but they got to start somewhere. Hopefully it’s not too bad adding the other arch later.
This is another reminder of how much we often take for granted. On a positive note, the project has demonstrated value and now the need is better known. The transparency is refreshing because there are a lot of other projects with commercial backing with very limited community response.
Slightly off-topic, but I have been very keen to see the rustcrypto project evolve, (and hopefully one day see formal verification).
Their use of strong traits (e.g. aead) make it very easy to enforce safety constraints directly in the type system. For example, these traits have already been used to wrap
ringitself to allow easy swap-outs of cryptographic backends.I often find libraries that will only accept, say, use of a file-backed private key. Instead, having libraries accept a generic
signertrait implementation allows agility between cryptographic backends, such as different hardware devices or crypto libraries.With cryptographic libraries in Rust being far from a settled debate, I strong believe there’s massive value in settling on standardized trait implementations for cryptographic libraries in the crate ecosystem to make these kinds of switches easier going forward.
OTOH, I strongly dislike the traits. They make usage of basic primitives more difficult. Their interface is spread over multiple crates, and everything goes through layers of indirection. It’s super annoying if I just need sha256, but need to import traits and convert from a
GenericArray.They also make semver-major changes too often for the ecosystem to keep up. A
Signertrait is more trouble than it’s worth when you end up with a v0.9 implementation not working with a v0.11 interface.That’s fair, I can’t argue with that - I’ve faced those same issues myself.
For example, the
yubikey.rscrate exposes asignertrait implementation for keys backed by the Yubikey PIV interface. This means that my code for Kerberos signing doesn’t need to concern itself with how the key is stored; all it needs is a struct that implementssigner.I think this sort of thing, beyond anything technical, is the biggest argument for growing the standard library (or, at least, those official ordained and maintained by the
rust-langorg). I can argue about the technical merits of Rust to my Rust-sceptic colleagues all day, but it’s difficult to deny that a worrying number of important dependencies in the ecosystem have a low bus factor.I don’t understand how the standard library is any different?
Python, as one example, has several part of its standard library that are unmaintained and falling backward. Same for erlang. How is being part of the standard library helping?
Indeed, being in the standard library doesn’t guarantee active maintenance.
There aren’t enough people in the core team to work on everything. There still has to be someone with expertise and dedication to improve the code.
There aren’t that many people available for reviewing pull requests either. The standard library is focused on preserving backward compatibility and portability, so changes take more time and it’s harder to contribute more than an occasional drive-by tweak.
In Rust, mpsc channels were an example of this: they were in the standard library, technically maintained, but nobody actively worked on improving the implementation. Eventually the std code got replaced with an implementation developed outside of the standard library.
This just puts a bit of rusty paint on the problem. The Rust project had very little capacity for library maintenance, the bus factor would not differ substantially. Indeed, for that reason, the project has more moved libraries out rather than in.
Coming from another language ecosystem, I think this doesn’t acknowledge an extremely common scenario: library maintainer is AWOL, and people show up with patches, and it’s just that there is nobody there to hit merge and do a release.
There are huge piles of projects where there are a bunch of people who are capable, reviewing each other’s changes, and saying “yes this works, I have a fork and we have it working” etc etc etc, but the release can’t happen because one person holds the keys.
Having a large team who holds release keys is generally something that can work out! For every “the maintainer is the only one who could actually make a call on this”-style issue there are hundreds of “maintainer just is done with the project” situations.
I’m not calling to have eminent domain, but having lang teams be able to adopt packages (with maintainer consent) and allowing for a light touch can totally work, especially in the age of Github & co.
Bus factor and capacity are different things. I’m not expecting development to go faster or anything: but when something really, really critical pops up that needs fixing with haste, a larger pool of people ready to merge a fix and deploy a release matters. Ossification and glacial development is fine, not having an official patch for a CVE because the one dev is (quite understandably!) on holiday, less so.
The standard library only releases every six weeks. The more stuff that’s in the standard library, the more stuff that would maybe call for an emergency release, which is far heavier weight and far more disruptive to the ecosystem. People already think Rust releases too often.
Also, a crypto vulnerability is likely to have a high priority. If have to urgently release/deploy, I’d much rather update a single crate than the whole compiler and stdlib.
Not trying to be snarky, but is the complaint about too many releases (and operational churn)? Or too much breakage (from stuff like the “partial ordering for sorts can trigger panics now” saga)?
Like one, you do land on “we should bundle up releases”, but the other is a different flavor of thing, right? Though I mention the sorting panic thing only because it’s the only case I have even heard of.
Saying all that, while I like fat standard libraries, it does feel like the weaker version of “it’s in the
rust-langorg but still packaged separately” gets things 95% of the way there for these kinds of issues, IMO.Operational and “reputational” churn. The time crate is the only breakage I’ve experienced personally in a long time. The process for releasing rustc is involved, the process of releasing a crate is “cargo publish.”
I already regularly hear complaints from very conservative type folks who look at the six week cadence and complain about how often releases happen. Adding more to the mix regularly increases the perception that Rust is still mega unstable, when that’s just not the case.
Can defeinitely see
rustc’s released process being involved vs cargo. Having multiple trusted people be able to runcargo publishis the valuable thing here, moreso than it being in the standard library itself.I do think it’s important to dig into what the reason for the cadence complaints really are though. I feel like the biggest lesson to be taken from Chrome showing up and doing releases all the time is that smaller releases (-> more release) are easier to manage in theory. It would be a shame if those complaints are downstream of other issues that are, in a word, “more” fixable?
This is all context free of course. At least in Python land complaints of things moving too fast that I hear tend to be broken down into “too much new stuff to learn” (a reason I guess, but one I don’t care about) and “old code is not working anymore too often” combined with “package maintainers disappearing” (which are realer issues, and can be handled).
It’s just systems people who are used to C89 and so they see backwards compatible but new features as “breaking” because they want to stay on old compilers but compile new code.
OK I see the case now. The maximally generous version of this is “well there is some issue X that is stopping my upgrade but everything around me is moving so fast that it throws me forward” (in Node this is common because packages agressively upper bound node versions even when they don’t need to, leading to a bunch of incidental things making upgrades hard), but tbh I feel like Rust isn’t doing it that much. Especially nowadays where people seem pretty serious about not using stuff from
nightlyetc.I agree, but the “issue x” is mostly “i want to use a dependency and it uses some rust language feature that is newer than my current stable version” and the “stopping my upgrade” is simply “I don’t want to” not “I tried it and it didn’t work.” That is, it’s perceived instability, rather than actual instability. For these folks, “stable” means “unchanging” not “backwards compatible.”
The shift from RSA to ECDSA is, relatively speaking, quite recent and we just had Microsoft announce their crazy qubit capabilities meaning quantum resistant cryptos are maybe sooner required than we previously thought. I think crypto is exactly the kind of thing that needs to be kept out of a standard library to avoid churn/deprecation.
Some officially blessed crate, sure.
I’ll leave a top-level comment on it, but aside from having a single “blessed” library, I think consensus on cryptographic trait implementations for backends would be fantastic.
High-level libraries that can be built to consume those traits, low-level implementations that implement them; would lead to easier agility when the “blessed” library is no longer the go-to.
The idea that absorbing something like ring into stdlib would automatically give it new maintainers feels like wishful thinking. Getting maintenance done is a combination of having the required skill (and for crypto, the bar is high) and accepting that it’s not someone else’s problem.
The “someone else’s problem” feeling can be pretty high for code in a stdlib. Whereas here, in the crate ecosystem, when the ring maintainer gave up we got an advisory that prompted new maintainers to step in. Looks like FOSS is working.