Perhaps this is partially the result of a new generation of security researchers gaining prominence, but progressive insight from the infosec industry has produced a lot of U-turns. STARTTLS was obviously the way forward, until it wasn’t, and now it’s always been a stupid idea. Never roll your own crypto, use reliable implementations like OpenSSL! Oh wait, it turns out OpenSSL is a train wreck, ha ha why did people ever use this crap?
As someone who is not in the infosec community but needs to take their advice seriously, it makes me a bit more wary about these kinds of edicts.
Getting rid of STARTTLS will be a multi-year project for some ISPs, first fixing all clients until they push implicit TLS (and handle the case when a server doesn’t offer implicit TLS yet), then moving all the email servers forward.
Regarding OpenSSL I think you got some bad messaging. The message is not “don’t use OpenSSL”. The real message was “all crypto libraries are train wrecks and need more funding and security auditing”. But luckily OpenSSL has improved a lot, and you should still use a well-tested implementation and not roll your own crypto and OpenSSL is not the worst choice.
Regarding STARTTLS I think what we’re seeing here is that there was a time when crypto standards valued flexibility over everything else. We also see this in TLS itself where TLS 1.2 was like “we offer the insecure option and the secure option, you choose”, while TLS 1.3 was all about “we’re gonna remove the insecure options”. The idea that has gained a lot of traction is that complexity breeds insecurity and should be avoided, but that wasn’t a popular idea 20-30 years ago when many of these standards were written.
The message is not “don’t use OpenSSL”. The real message was “all crypto libraries are train wrecks and need more funding and security auditing”. But luckily OpenSSL has improved a lot, and you should still use a well-tested implementation and not roll your own crypto and OpenSSL is not the worst choice.
100%
I prefer libsodium over OpenSSL where possible, but some organizations can only use NIST-approved algos.
Your hindsight may be 20/20 but STARTTLS was born in an era where almost nothing on the Internet was encrypted. At that time, 99% of websites only used HTTPS on pages that accepted credit card numbers. (It was considered not worth the administrative and computing burden to encrypt a whole site that was open to the public to view anyway.)
STARTTLS was a clever hack to allow opportunistic encryption of mail over the wire. When it was introduced, getting the various implementations and deployments of SMTP servers (either open source or commercial) even to work together in an RFC-compliant manner was an uphill battle on its own. STARTTLS allowed mail administrators to encrypt the SMTP exchange where they could while (mostly) not breaking existing clients and servers, nor requiring the coordination of large ISPs and universities around the world to upgrade their systems and open new firewall ports.
Some encryption was better than no encryption, and that’s still true today.
That being said, I run my own mail server and I only allow users to send outgoing mail on port 465 (TLS). But for mail coming in from the Internet, I still have to allow plaintext SMTP (and hence STARTTLS support) on port 25 or my users and I would miss a lot a messages. I look forward to the day that I can shut off port 25 altogether, if it ever happens.
Your hindsight may be 20/20 but STARTTLS was born in an era where almost nothing on the Internet was encrypted.
I largely got involved with computer security/cryptography in the late 2000’s, when we suspected a lot of the things Snowden revealed to be true, so “encrypt every packet securely” was my guiding principle. I recognize that wasn’t always a goal for the early Internet, but I was too young to be heavily involved then.
Some encryption was better than no encryption, and that’s still true today.
Defense against passive attackers have value, but in the face of active attackers, opportunistic encryption is merely security theater.
I look forward to the day that I can shut off port 25 altogether, if it ever happens.
Defense against passive attackers have value, but in the face of active attackers, opportunistic encryption is merely security theater.
That’s not quite true, it still provides an audit trail. The goal of STARTTLS, as I understand it, is to avoid trying to connect to a TLS port, potentially having to wait for some arbitrary timeout if a firewall somewhere is set to drop packets rather than reject connections, and then retry on the unencrypted path. Instead, you connect to the port that you know will be there and then try to do the encryption. At this point, a passive attacker can’t do anything, an active attacker can strip out the server’s notification that STARTTLS is available and leave the connection in plaintext mode. This kind of injection is tamper-evident. The sender (at least for mail servers doing relaying) will typically log whether a particular message was sent with or without STARTTLS. This logging lets you detect which messages were potentially leaked / tampered with at a later date. You can also often enforce policies that say things like ‘if STARTTLS has ever been supported by this server, refuse if it isn’t this time’.
Now that TLS support is pretty-much table stakes, it is probably worth revisiting this and defaulting to connecting on the TLS port. This is especially true now that most mail servers use some kind of asynchronous programming model so trying to connect on port 465 and waiting for a timeout doesn’t tie up too many resources. It’s not clear what the failure mode should do though. If an attacker can tamper with port 25 traffic, they can also trivially drop everything destined for port 465, so trying 465 and retrying on 25 if that fails is no better than STARTTLS (actually worse - rewriting packets is harder than dropping packets, one can be done by inspecting the header the other requires deep-packet inspection). Is there a DNS record that can tell connecting mail servers to not try port 25? Just turning off port 25 doesn’t help because an attacker doing DPI can intercept packets for port 25 and forward them over a TLS connection that it establishes to 465.
How is SNI a terrible hack? How is direct usage of TLS “legacy”? How would SMTP and IMAP ever need SNI if they just, uhhh, don’t have anything like the Host header? i.e. there’s no “virtual hosting” in email.
Well, that’s a different problem, that’s because no SMTP server expects the hostname of the cert to match the hostname of the email addresses. For historical reasons this will probably never get added, so in SMTP land SNI might never be needed.
the hostname will never match for anyone using hosted mail for custom domains
It could, though, and if we wanted authenticity on the link it needs to (if using TLS for the auth and not DNSSEC, of course), but there’s just so much already-deployed the other way it’ll never get changed.
That you have to type https to get it. And we’ve layers of hacks (redirects, hsts, https everywhere) to try to compensate for this. If it was starttls on the main port you would always get it when available, like other protocols do.
Port 465 isn’t legacy, and you don’t need to care about it unless you’re running mail server or writing a mail client. STARTTLS was a hack to allow for opportunistic encryption, and that’s about it. Any competent mailserver admin should be publishing SRV records for this stuff, and any properly written mail client should be looking for them.
SNI isn’t a hack either. It was rather daft that TLS went so long without a way to pass vhost information along. And it turns out that it’s really useful is you’re running a service mesh regardless of what the underlying application protocol is.
Yes STARTTLS can be downgraded, but how do most clients react when you simply block ports 465, 993 and 995? The client will often try to connect on port 25, 110 or 143 instead.
Those might be closed server-side since you don’t want to allow clients to connect insecurely, but a hacker that was determined enough to inject data in a STARTTLS session can just as easy set up an stunnel MITM on the insecure ports.
STARTTLS always struck me as a terrible idea. TLS everywhere should be the goal. Great work.
Perhaps this is partially the result of a new generation of security researchers gaining prominence, but progressive insight from the infosec industry has produced a lot of U-turns. STARTTLS was obviously the way forward, until it wasn’t, and now it’s always been a stupid idea. Never roll your own crypto, use reliable implementations like OpenSSL! Oh wait, it turns out OpenSSL is a train wreck, ha ha why did people ever use this crap?
As someone who is not in the infosec community but needs to take their advice seriously, it makes me a bit more wary about these kinds of edicts.
Getting rid of STARTTLS will be a multi-year project for some ISPs, first fixing all clients until they push implicit TLS (and handle the case when a server doesn’t offer implicit TLS yet), then moving all the email servers forward.
Introducing STARTTLS had no big up-front costs …
Regarding OpenSSL I think you got some bad messaging. The message is not “don’t use OpenSSL”. The real message was “all crypto libraries are train wrecks and need more funding and security auditing”. But luckily OpenSSL has improved a lot, and you should still use a well-tested implementation and not roll your own crypto and OpenSSL is not the worst choice.
Regarding STARTTLS I think what we’re seeing here is that there was a time when crypto standards valued flexibility over everything else. We also see this in TLS itself where TLS 1.2 was like “we offer the insecure option and the secure option, you choose”, while TLS 1.3 was all about “we’re gonna remove the insecure options”. The idea that has gained a lot of traction is that complexity breeds insecurity and should be avoided, but that wasn’t a popular idea 20-30 years ago when many of these standards were written.
100%
I prefer libsodium over OpenSSL where possible, but some organizations can only use NIST-approved algos.
Agreed. It always felt like a band-aid as opposed to a well thought out option. Good stuff @hanno.
Your hindsight may be 20/20 but STARTTLS was born in an era where almost nothing on the Internet was encrypted. At that time, 99% of websites only used HTTPS on pages that accepted credit card numbers. (It was considered not worth the administrative and computing burden to encrypt a whole site that was open to the public to view anyway.)
STARTTLS was a clever hack to allow opportunistic encryption of mail over the wire. When it was introduced, getting the various implementations and deployments of SMTP servers (either open source or commercial) even to work together in an RFC-compliant manner was an uphill battle on its own. STARTTLS allowed mail administrators to encrypt the SMTP exchange where they could while (mostly) not breaking existing clients and servers, nor requiring the coordination of large ISPs and universities around the world to upgrade their systems and open new firewall ports.
Some encryption was better than no encryption, and that’s still true today.
That being said, I run my own mail server and I only allow users to send outgoing mail on port 465 (TLS). But for mail coming in from the Internet, I still have to allow plaintext SMTP (and hence STARTTLS support) on port 25 or my users and I would miss a lot a messages. I look forward to the day that I can shut off port 25 altogether, if it ever happens.
I largely got involved with computer security/cryptography in the late 2000’s, when we suspected a lot of the things Snowden revealed to be true, so “encrypt every packet securely” was my guiding principle. I recognize that wasn’t always a goal for the early Internet, but I was too young to be heavily involved then.
Defense against passive attackers have value, but in the face of active attackers, opportunistic encryption is merely security theater.
Hear hear!
That’s not quite true, it still provides an audit trail. The goal of STARTTLS, as I understand it, is to avoid trying to connect to a TLS port, potentially having to wait for some arbitrary timeout if a firewall somewhere is set to drop packets rather than reject connections, and then retry on the unencrypted path. Instead, you connect to the port that you know will be there and then try to do the encryption. At this point, a passive attacker can’t do anything, an active attacker can strip out the server’s notification that STARTTLS is available and leave the connection in plaintext mode. This kind of injection is tamper-evident. The sender (at least for mail servers doing relaying) will typically log whether a particular message was sent with or without STARTTLS. This logging lets you detect which messages were potentially leaked / tampered with at a later date. You can also often enforce policies that say things like ‘if STARTTLS has ever been supported by this server, refuse if it isn’t this time’.
Now that TLS support is pretty-much table stakes, it is probably worth revisiting this and defaulting to connecting on the TLS port. This is especially true now that most mail servers use some kind of asynchronous programming model so trying to connect on port 465 and waiting for a timeout doesn’t tie up too many resources. It’s not clear what the failure mode should do though. If an attacker can tamper with port 25 traffic, they can also trivially drop everything destined for port 465, so trying 465 and retrying on 25 if that fails is no better than STARTTLS (actually worse - rewriting packets is harder than dropping packets, one can be done by inspecting the header the other requires deep-packet inspection). Is there a DNS record that can tell connecting mail servers to not try port 25? Just turning off port 25 doesn’t help because an attacker doing DPI can intercept packets for port 25 and forward them over a TLS connection that it establishes to 465.
Great, let’s switch every protocol to legacy SSL mode instead so it can suck as much as HTTPS and rely on terrible hacks like SNI…
How is SNI a terrible hack? How is direct usage of TLS “legacy”? How would SMTP and IMAP ever need SNI if they just, uhhh, don’t have anything like the Host header? i.e. there’s no “virtual hosting” in email.
Well, that’s a different problem, that’s because no SMTP server expects the hostname of the cert to match the hostname of the email addresses. For historical reasons this will probably never get added, so in SMTP land SNI might never be needed.
Not just “for historical reasons”, it’s all by design – the hostname will never match for anyone using hosted mail for custom domains.
It could, though, and if we wanted authenticity on the link it needs to (if using TLS for the auth and not DNSSEC, of course), but there’s just so much already-deployed the other way it’ll never get changed.
What sucks most about HTTPS in your opinion?
That you have to type https to get it. And we’ve layers of hacks (redirects, hsts, https everywhere) to try to compensate for this. If it was starttls on the main port you would always get it when available, like other protocols do.
…until an active attacker comes along! Opportunistic encryption reduces to plaintext.
Port 465 isn’t legacy, and you don’t need to care about it unless you’re running mail server or writing a mail client. STARTTLS was a hack to allow for opportunistic encryption, and that’s about it. Any competent mailserver admin should be publishing SRV records for this stuff, and any properly written mail client should be looking for them.
SNI isn’t a hack either. It was rather daft that TLS went so long without a way to pass vhost information along. And it turns out that it’s really useful is you’re running a service mesh regardless of what the underlying application protocol is.
Yes STARTTLS can be downgraded, but how do most clients react when you simply block ports 465, 993 and 995? The client will often try to connect on port 25, 110 or 143 instead.
Those might be closed server-side since you don’t want to allow clients to connect insecurely, but a hacker that was determined enough to inject data in a STARTTLS session can just as easy set up an stunnel MITM on the insecure ports.
I haven’t seen such behavior during our tests and I would definitely consider it a security vulnerability.
Can you name a specific client that will connect through plaintext ports if TLS ports are blocked?