It’s not just OpenSSL. High-performance servers use KTLS, which keeps the TLS session negotiation in userspace but moves all of the fast-path crypto into the kernel (and, ideally, all the way down to the NIC) so that you need a single copy to go from the buffer cache to the NIC without hardware TLS offload and zero copies with. There isn’t yet a mechanism for doing the same with QUIC, which means you need one more copy to go from disk to network with QUIC than with TCP+TLS.
I cannot find a clear stated reason why this approach was rejected. I read through that PR, and the communication from the OpenSSL people throughout has been, ehm, concise.
Nothing is posted in the PR at all except a link to this mailing list message, which simply states “PR#8797 will not be merged and compatibility with the APIs proposed in that PR is a non-goal.” That’s all I can find other than the message from a year and a half ago expressing concerns about API stability. A valid concern as such, but it’s not entirely clear to me this actually is an unstable API. And even if it was: wouldn’t starting from an existing tested implementation be better?
Back in 2013/2014, when I was working on what became HardenedBSD’s ASLR implementation, my own goal was to get ASLR merged upstream into FreeBSD. Indeed, I felt so entitled that I thought FreeBSD should be grateful for my free work. Ultimately, it was not the implementation that FreeBSD chose, and that’s okay. I’ve learned countless lessons through that process, and one which I want to talk about is the feeling of entitlement–a feeling that pervades every part of tech.
After my years of attempting to work with an upstream, I’ve simply decided that having differing opinions and implementations is just fine. In fact, getting rid of software monocultures makes the world a safter place. I’ve learned not to try to enforce my own ideas on other projects. If upstream wants to do things differently, that’s fine. Unless I’m a committer of the project, I have no right to influence over the project. I can only submit patches and hope they match upstream’s wishes.
So if OpenSSL won’t take quictls’ patches, then so what? We have more implementations to choose from. More ways to rid ourselves of the software monocultures that plague the security of our industry.
I’ve also learned the importance of writing code that favors modularity. Any code I write that deals with a crypto library is written in a way that I can swap out the crypto library, just updating a shim. I try to take the same “lightweight shim” approach to any of my code that interfaces with third-party projects I don’t own or control.
The shim approach also provides buffer against API/ABI changes from impacting the larger codebase. If the larger codebase references the shim and only the shim, then chasing upstream changes is easaier, too.
Not a good look for openssl, just a few years after all their other issues that led to several forks at the time (libressl, boringssl). Now there is more craziness, and a new fork: quictls (collab betweek microsoft and akamai, apparently).
For context, MSFT uses Akamai as their CDN (that means Windows Update, Xbox content delivery etc), Akamai is also a supplier for Facebook.
Politically that fork carries tremendous leverage.
Adding a new toolchain and language to the system seems far more complex than simply static-linking a different TLS library in the same language. BoringSSL comes to mind; it’s used in Cloudflare’s Quiche, nginx-quic, Chromium, macOS/iOS/Android, and the Ring crate (which is used by Rustls). I think PicoTLS also has support.
If you’re already using Rust, then Rustls is a good choice. The TLS impl. in the Go stdlib is also solid. But if your software is already in C, I think that adding a Rust library as a core dependency would negate most of the benefits of C: you’d be left with the pitfalls of both C and Rust.
Rusty Ring borrowing lowest-level implementation details from BoringSSL is IMHO quite a sensible approach. It uses C literally as a portable assembly (plus actual assembly) where constant-time implementations matter. Rust API on top of that still helps with correct memory management and logic errors.
QUIC is a non-trivial protocol with a complex dynamic state, custom compression schemes, etc. A lot of room for mistakes, so it’s a good candidate for oxidation.
How much does a TLS library need to know about the QUIC protocol? Normally with TCP it just acts as a stream plugin: you feed the socket’s input stream through it and feed your output through it to the socket.
But I know QUIC runs over UDP not TCP. It doesn’t just use DTLS?
Without any serious research, if QUIC provides the sequencing, checksumming, and retransmit in user space (instead of wherever it’s implemented in TCP, the kernel or hardware?) the TLS library would need to be able to handle that level of normal network existence.
QUIC has TLS bundled into the protocol. So TLS and QUIC are really the same layer if you’re using QUIC. In that sense, a TLS library probably doesn’t need to know much about QUIC. But a QUIC library needs to know everything about TLS.
It’s not just OpenSSL. High-performance servers use KTLS, which keeps the TLS session negotiation in userspace but moves all of the fast-path crypto into the kernel (and, ideally, all the way down to the NIC) so that you need a single copy to go from the buffer cache to the NIC without hardware TLS offload and zero copies with. There isn’t yet a mechanism for doing the same with QUIC, which means you need one more copy to go from disk to network with QUIC than with TCP+TLS.
I cannot find a clear stated reason why this approach was rejected. I read through that PR, and the communication from the OpenSSL people throughout has been, ehm, concise.
Nothing is posted in the PR at all except a link to this mailing list message, which simply states “PR#8797 will not be merged and compatibility with the APIs proposed in that PR is a non-goal.” That’s all I can find other than the message from a year and a half ago expressing concerns about API stability. A valid concern as such, but it’s not entirely clear to me this actually is an unstable API. And even if it was: wouldn’t starting from an existing tested implementation be better?
In short: it seems NIH strikes again.
Back in 2013/2014, when I was working on what became HardenedBSD’s ASLR implementation, my own goal was to get ASLR merged upstream into FreeBSD. Indeed, I felt so entitled that I thought FreeBSD should be grateful for my free work. Ultimately, it was not the implementation that FreeBSD chose, and that’s okay. I’ve learned countless lessons through that process, and one which I want to talk about is the feeling of entitlement–a feeling that pervades every part of tech.
After my years of attempting to work with an upstream, I’ve simply decided that having differing opinions and implementations is just fine. In fact, getting rid of software monocultures makes the world a safter place. I’ve learned not to try to enforce my own ideas on other projects. If upstream wants to do things differently, that’s fine. Unless I’m a committer of the project, I have no right to influence over the project. I can only submit patches and hope they match upstream’s wishes.
So if OpenSSL won’t take quictls’ patches, then so what? We have more implementations to choose from. More ways to rid ourselves of the software monocultures that plague the security of our industry.
I’ve also learned the importance of writing code that favors modularity. Any code I write that deals with a crypto library is written in a way that I can swap out the crypto library, just updating a shim. I try to take the same “lightweight shim” approach to any of my code that interfaces with third-party projects I don’t own or control.
The shim approach also provides buffer against API/ABI changes from impacting the larger codebase. If the larger codebase references the shim and only the shim, then chasing upstream changes is easaier, too.
I just ran across this on the orange site too.
https://mta.openssl.org/pipermail/openssl-project/2021-October/002777.html
Not a good look for openssl, just a few years after all their other issues that led to several forks at the time (libressl, boringssl). Now there is more craziness, and a new fork: quictls (collab betweek microsoft and akamai, apparently).
For context, MSFT uses Akamai as their CDN (that means Windows Update, Xbox content delivery etc), Akamai is also a supplier for Facebook.
Politically that fork carries tremendous leverage.
Maybe the introduction of QUIC should be a good reason for many things to switch to.. something written in a better language?
Adding a new toolchain and language to the system seems far more complex than simply static-linking a different TLS library in the same language. BoringSSL comes to mind; it’s used in Cloudflare’s Quiche, nginx-quic, Chromium, macOS/iOS/Android, and the Ring crate (which is used by Rustls). I think PicoTLS also has support.
If you’re already using Rust, then Rustls is a good choice. The TLS impl. in the Go stdlib is also solid. But if your software is already in C, I think that adding a Rust library as a core dependency would negate most of the benefits of C: you’d be left with the pitfalls of both C and Rust.
Rusty Ring borrowing lowest-level implementation details from BoringSSL is IMHO quite a sensible approach. It uses C literally as a portable assembly (plus actual assembly) where constant-time implementations matter. Rust API on top of that still helps with correct memory management and logic errors.
QUIC is a non-trivial protocol with a complex dynamic state, custom compression schemes, etc. A lot of room for mistakes, so it’s a good candidate for oxidation.
How much does a TLS library need to know about the QUIC protocol? Normally with TCP it just acts as a stream plugin: you feed the socket’s input stream through it and feed your output through it to the socket.
But I know QUIC runs over UDP not TCP. It doesn’t just use DTLS?
Without any serious research, if QUIC provides the sequencing, checksumming, and retransmit in user space (instead of wherever it’s implemented in TCP, the kernel or hardware?) the TLS library would need to be able to handle that level of normal network existence.
QUIC has TLS bundled into the protocol. So TLS and QUIC are really the same layer if you’re using QUIC. In that sense, a TLS library probably doesn’t need to know much about QUIC. But a QUIC library needs to know everything about TLS.
https://www.rfc-editor.org/rfc/rfc9001.html