It’s also not clear what you’d do in the case where the DMARC policy is not sufficiently strong for you to be able to guarantee that the email is not spoofed. If someone who runs their own mail server hits this, they can fix it. If someone who doesn’t run their own server hits it then they’re stuck. Note that, in addition to the existing constraints, the server should also require DNSSEC. This isn’t a requirement for DMARC normally but this is changing the threat model. Someone who can tamper with a DNS response can tamper with the DMARC keys. Normally, this just causes emails to be rejected or some spam to get through, but with this model a DNS spoofing attack can compromise an account.
It’s a huge shame that XMPP never caught on. Almost 20 years ago, I built a proof-of-concept that used XMPP for login, where you’d get an XMPP message containing a URL and clicking on the URL would log you in. The message was sent as a normal message but, because XMPP messages can contain arbitrary XML, also included metadata saying that this is what it is. I modified a client to allow you to automatically fetch the URL for senders that you put on an allow list, so you could automatically log in while your client was running. The biggest downside of this was that it gave the server your JID, which could be used for cross-site tracking (it’s much easier with email to generate throw-away addresses than it is with XMPP).
Right, which is why the scheme proposed in the post doesn’t use DMARC – it essentially ignores DMARC entirely and mandates both SPF and DKIM. Those in turn could be misconfigured, but you could do additional normative filtering from there (e.g. rejecting messages from hosts who allow any IP to send mail).
This admittedly makes the scheme much, much less generally useful :-)
Because we need it, unfortunately – DKIM only protects the email body (meaning headers and message), not the envelope. A spoofer can lie in the RCPT TO SMTP command, which DKIM won’t catch.
I meant MAIL FROM, sorry. It matters because you can use it to cause backscatter traffic, which can then be used to get a host blackballed for conveying spam.
I agree that it makes assumption (1), but I don’t think it assumes (2) in a meaningful sense – email verification as a whole is based on the latent assumption that the user has an email account and knows how to use it, meaning that the scheme only requires them to understand that they need to click the “send” button.
but it also doesn’t resolve your issue of “I have a work, personal, and secret e-mail account/browser/whatever”. I can have gmail in chrome, work email in outlook and all my other mailboxes in thunderbird.
For “muggles” the regular process is the same anyway, they will usually not be logged in two-three sessions per browsers, two-three browsers per desktop session. For e.g. devs like us, the problem is similar just reversed - when I click on a mailto link, what does it get opened in?
For most websites, the firemost thing the website’s operator needs of your email account is that you have to be able to receive emails there. So verifying that you can send emails from that address is neither necessary nor sufficient.
FWIW, if it wasn’t for the fact that email authentication is a torrential shitshow, you could flip around the flow for password reset emails too in exactly the same fashion.
I’m fairly certain this would be a bad idea. Just from a user’s perspective, it’d mean you were limited to supporting a limited set of email providers, for reasons that are far, far below an average user’s threshold of caring.
And I’d be wary of this from a technical perspective–from what I recall, DKIM are mostly focussed on provider to provider trust, rather than identifying individuals. It might be that it can do user to provider trust, but that’s likely to be more of a happy accident than anything else.
It might be useful if you’ve got a sophisticated security setup, where you can rely on an ensemble of signals to determine user trust, but even then, I suspect just the overall UX wierdness would disqualify it.
Having run mail servers (at personal, hosting provider, and ISP scales) for over 2 decades now I can confidently say this is not a good idea.
Technologically it is full of pitfalls. Market penetration of DKIM/SPF/DMARC is way below what it should be, and most sites wanting to verify your email don’t actually care. Verifying them is a much higher technological barrier for the server end that just sending out a valid message and hence would frequently be improperly implemented. It would be even easier for phony anonymous providers to setup a system to send valid emails from disposable addresses than it already is to setup receive addresses.
Pragmatically is is verifying the wrong thing. For many sites the goal is not just to verify an ID it is usually to verify that you have a way to reach the user. Landing in the spam folder isn’t just bad for the signup process, it is bad far the next time you want to send the user a notification or something like that. Avoiding the problem (by not checking the user can get messages) means any problems go unfixed and you don’t have a way to reach the user.
DKIM only guarantees that the email was sent from an authorised server, it doesn’t ensure that the headers or body are genuine. I have sent email from my Gmail account with a From: header of larry.page@gmail.com. I don’t know if they still allow that, but I would be unsurprised if other large mail servers allow it.
This scheme also relies on correct use of DKIM by mail servers, which is fine for some of the major players, but there is a long tail of institutional email servers, etc where this will not be true.
Finally, the scheme requires a fair bit more complexity from the implementor because receiving and validating incoming email is quite a bit more complicated than sending mail.
I might be misunderstanding, but I think you’re mixing up SPF and DKIM: DKIM ensures that the body and a subset of the headers are authentic; SPF ensures (non-cryptographically) that the message envelope is authentic. The scheme I proposed in the post requires both.
I agree about the “long tail,” however! I have no idea how practical this actually is; it just popped into my head.
SPF and DKIM don’t allow authenticating the sender, they allow authenticating the sender’s mail server. If the mail server is properly configured then it will prevent anyone from sending mail as an account that is not theirs but some do.
There’s a bigger problem though: DKIM signs the envelope from, which is set by the server, but this often doesn’t match the email address that the user sets (for example, before they moved to M365, my Cambridge email address had an envelope from of {my unique ID}@hermes.cam.ac.uk, whereas my email address was {my name}@cl.cam.ac.uk. If I’d entered the latter on your web site, you’d have received a confirmation email that would have looked fake. You’d need to do an additional round trip to ensure that I could receive an email from the sent account.
You might be able to use the normal flow for the initial account creation step and this for normal login though: If I receive an email and, rather than clicking on a link, reply to it, then you will know the from header and the envelope from field that are associated with the account and so can then perform your reversed flow for a subsequent login.
Note that, for this to be responsive and reliable, the mail server processing the inbound login emails can’t use any kind of DNSRBL or greytraping. This means that it will end up processing a pretty huge volume of spam. That’s probably not a huge problem (especially if you only care about the subject line in the emails that you receive and can discard the rest).
You’re right that I was mixing them up, tho DKIM doesn’t do enough to guarantee what you want: DKIM specifies some metadata and signs it plus the body and some headers of the mail. That can optionally include the “agent identity” and/or the “From:” header. The sending server may choose not to include either, or may choose to sign them but only validate the domain part (the DMARC “domain alignment” policy only looks at the domain part); or it may choose to sign without validating at all.
Can’t sent email addresses be spoofed pretty well?
Yes and no: With DMARC / DKIM, you can ensure that the message came from the server that it claims to. The server might be incorrectly configured. Without, this is far harder to verify.
Note that you also need DNSSEC on the DKIM records, otherwise an attacker who can intercept and spoof DNS can give the server a fake public key and then send a spoofed message with the corresponding private key.
I’m that one luser who doesn’t send click “mailto” links because that opens the mac email client, which I don’t use.
This is an interesting use case. One of the problems with mailto links is that they’ll usually open with the default mail client (no idea why you’ve set the default to the one that you don’t use), but a lot of people they have separate work and personal accounts and use separate clients (especially if they’re web-based ones). It’s quite likely that they’d open with one that wasn’t the one that they wanted to log in with and then they’d wonder why logging in didn’t work.
The waterfall of multi-letter acronyms I’ve been reading in all these discussions suggests to me this will be trouble.
no idea why you’ve set the default to the one that you don’t use
I think the simplest explanation here is the correct one: I didn’t bother to change the default because I don’t use mailto: links and I use webmail.
My computer is a tool, not a home. I don’t spend much time decorating it just right. If the default isn’t right for me, I’m not going to waste time discovering what the right setting is.
I have a sneaking suspicion this is the most common user persona.
I think the unspoken assumption that if an user can READ mail to an address he/she also can SEND from the same address.
That is typically my case. I use uniq addresses every where with wildcarding to my one of my primary addresses.
I can not easily send from the same address as I must configure that address as an sending address.
And the main goal I guess is still this same, that the service can send password resets and news like breaches (or god forbid spam) to the correct account holder.
And the main goal I guess is still this same, that the service can send password resets and news like breaches (or god forbid spam) to the correct account holder.
And birthday greetings! I never feel as appreciated as during my birthday when all the forums I’ve signed up for during the years send me email congratulating me.
Relies on mailto: links actually working. Given that webmail is probably a high majority for personal accounts nowadays, it’s now common that a mailto: link won’t actually directly open your MUA. Heck, I don’t even have it configured myself, and as a developer, I’m not a “regular” user. And “regular” users, I know don’t have this configured either. Heck, just going through GMail’s interface right now, I can’t see how to enable that…
Actually we already do part of this reversal for mailing lists, when you start the subscription by sending an email and confirm by clicking a link. e.g. for mailman
I am not convinced it completely avoids the spam box, the response can still land in there, but yes it reduces the probability if your email server auto allows emails in your contact list.
Concerning the challenge I think some mailing lists use unique subscription email addresses as an alternative to any token in the subscription email, or they do two round trips to place a unique token in the reply text. The second round trip is also an alternative if you do not have DKIM and so on.
There is probably a deeper question about whether we need email verification at all, but that is besides the point :D
This is a neat idea but it would confuse people as it’s not normal - ux is all about figuring out what people consider “normal” and trying to fit around that.
And with that I still agree with all the criticisms of email, they’re ancient and insecure theoretically (I’ve never actually seen any of the criticisms affect anyone or any company I know however)
At the risk of sounding like a crypto bro (and I am definitely a huge crypto-skeptic) this is one nice thing it kind of solves: logging in with metamask is such a nice flow compared to email or phone verification! Though it isn’t a suitable replacement, there are some valuable ideas.
I agree with the general terribleness of email verification, and I’m still not sure what sites are actually trying to do beyond getting your email address to spam you. It could be “make sure you’re in control of the account”, but I generally only see this for account creation/user id, not for a generically associated email address.
The other points are also largely correct, the generally html “rich” emails can easily trigger spam filters, try to use remote resources which are blocked, if the remote resources are loaded it violates the privacy of the recipient especially if they didn’t submit their email address in the first place, and finally the rich text is often a train wreck of poorly accessible/readable layout for screen readers, occasionally with a link to go to the site if the email is broken.
Ignoring the user experience as envisioned here, I think it would be tough for any organization to rely on incoming email to be their primary point of entry. It’s had a reputation for being unreliable and when something goes wrong (e.g. bring mistakenly blocked) you have very little direct control. Similarly, while there are methods of authenticating incoming email, the alternative contact mechanisms (e.g. cell phone) don’t have a similar level of authentication.
I’d propose a middle ground. Instead of signing up with a bunch of details and then verifying that email address, I think it would be an improvement just to ask for account details after verifying the email address. That would eliminate the biggest issue with the verification step: it lets you down after you’ve already emotionally committed to using a service.
This 1. Assumes that the integration with the email client works 2. A user can follow instructions beyond clicking a link.
It’s also not clear what you’d do in the case where the DMARC policy is not sufficiently strong for you to be able to guarantee that the email is not spoofed. If someone who runs their own mail server hits this, they can fix it. If someone who doesn’t run their own server hits it then they’re stuck. Note that, in addition to the existing constraints, the server should also require DNSSEC. This isn’t a requirement for DMARC normally but this is changing the threat model. Someone who can tamper with a DNS response can tamper with the DMARC keys. Normally, this just causes emails to be rejected or some spam to get through, but with this model a DNS spoofing attack can compromise an account.
It’s a huge shame that XMPP never caught on. Almost 20 years ago, I built a proof-of-concept that used XMPP for login, where you’d get an XMPP message containing a URL and clicking on the URL would log you in. The message was sent as a normal message but, because XMPP messages can contain arbitrary XML, also included metadata saying that this is what it is. I modified a client to allow you to automatically fetch the URL for senders that you put on an allow list, so you could automatically log in while your client was running. The biggest downside of this was that it gave the server your JID, which could be used for cross-site tracking (it’s much easier with email to generate throw-away addresses than it is with XMPP).
Right, which is why the scheme proposed in the post doesn’t use DMARC – it essentially ignores DMARC entirely and mandates both SPF and DKIM. Those in turn could be misconfigured, but you could do additional normative filtering from there (e.g. rejecting messages from hosts who allow any IP to send mail).
This admittedly makes the scheme much, much less generally useful :-)
Why bother with something broken like SPF when DKIM exists?
Because we need it, unfortunately – DKIM only protects the email body (meaning headers and message), not the envelope. A spoofer can lie in the
RCPT TOSMTP command, which DKIM won’t catch.You mean RCPT FROM? Why does that matter? That value is invisible to most mail software anyway.
I meant
MAIL FROM, sorry. It matters because you can use it to cause backscatter traffic, which can then be used to get a host blackballed for conveying spam.I agree that it makes assumption (1), but I don’t think it assumes (2) in a meaningful sense – email verification as a whole is based on the latent assumption that the user has an email account and knows how to use it, meaning that the scheme only requires them to understand that they need to click the “send” button.
but it also doesn’t resolve your issue of “I have a work, personal, and secret e-mail account/browser/whatever”. I can have gmail in chrome, work email in outlook and all my other mailboxes in thunderbird.
For “muggles” the regular process is the same anyway, they will usually not be logged in two-three sessions per browsers, two-three browsers per desktop session. For e.g. devs like us, the problem is similar just reversed - when I click on a mailto link, what does it get opened in?
For most websites, the firemost thing the website’s operator needs of your email account is that you have to be able to receive emails there. So verifying that you can send emails from that address is neither necessary nor sufficient.
FWIW, if it wasn’t for the fact that email authentication is a torrential shitshow, you could flip around the flow for password reset emails too in exactly the same fashion.
I’m fairly certain this would be a bad idea. Just from a user’s perspective, it’d mean you were limited to supporting a limited set of email providers, for reasons that are far, far below an average user’s threshold of caring.
And I’d be wary of this from a technical perspective–from what I recall, DKIM are mostly focussed on provider to provider trust, rather than identifying individuals. It might be that it can do user to provider trust, but that’s likely to be more of a happy accident than anything else.
It might be useful if you’ve got a sophisticated security setup, where you can rely on an ensemble of signals to determine user trust, but even then, I suspect just the overall UX wierdness would disqualify it.
Having run mail servers (at personal, hosting provider, and ISP scales) for over 2 decades now I can confidently say this is not a good idea.
Technologically it is full of pitfalls. Market penetration of DKIM/SPF/DMARC is way below what it should be, and most sites wanting to verify your email don’t actually care. Verifying them is a much higher technological barrier for the server end that just sending out a valid message and hence would frequently be improperly implemented. It would be even easier for phony anonymous providers to setup a system to send valid emails from disposable addresses than it already is to setup receive addresses.
Pragmatically is is verifying the wrong thing. For many sites the goal is not just to verify an ID it is usually to verify that you have a way to reach the user. Landing in the spam folder isn’t just bad for the signup process, it is bad far the next time you want to send the user a notification or something like that. Avoiding the problem (by not checking the user can get messages) means any problems go unfixed and you don’t have a way to reach the user.
DKIM only guarantees that the email was sent from an authorised server, it doesn’t ensure that the headers or body are genuine. I have sent email from my Gmail account with a From: header of larry.page@gmail.com. I don’t know if they still allow that, but I would be unsurprised if other large mail servers allow it.
This scheme also relies on correct use of DKIM by mail servers, which is fine for some of the major players, but there is a long tail of institutional email servers, etc where this will not be true.
Finally, the scheme requires a fair bit more complexity from the implementor because receiving and validating incoming email is quite a bit more complicated than sending mail.
I might be misunderstanding, but I think you’re mixing up SPF and DKIM: DKIM ensures that the body and a subset of the headers are authentic; SPF ensures (non-cryptographically) that the message envelope is authentic. The scheme I proposed in the post requires both.
I agree about the “long tail,” however! I have no idea how practical this actually is; it just popped into my head.
SPF and DKIM don’t allow authenticating the sender, they allow authenticating the sender’s mail server. If the mail server is properly configured then it will prevent anyone from sending mail as an account that is not theirs but some do.
There’s a bigger problem though: DKIM signs the envelope from, which is set by the server, but this often doesn’t match the email address that the user sets (for example, before they moved to M365, my Cambridge email address had an envelope from of {my unique ID}@hermes.cam.ac.uk, whereas my email address was {my name}@cl.cam.ac.uk. If I’d entered the latter on your web site, you’d have received a confirmation email that would have looked fake. You’d need to do an additional round trip to ensure that I could receive an email from the sent account.
You might be able to use the normal flow for the initial account creation step and this for normal login though: If I receive an email and, rather than clicking on a link, reply to it, then you will know the from header and the envelope from field that are associated with the account and so can then perform your reversed flow for a subsequent login.
Note that, for this to be responsive and reliable, the mail server processing the inbound login emails can’t use any kind of DNSRBL or greytraping. This means that it will end up processing a pretty huge volume of spam. That’s probably not a huge problem (especially if you only care about the subject line in the emails that you receive and can discard the rest).
You’re right that I was mixing them up, tho DKIM doesn’t do enough to guarantee what you want: DKIM specifies some metadata and signs it plus the body and some headers of the mail. That can optionally include the “agent identity” and/or the “From:” header. The sending server may choose not to include either, or may choose to sign them but only validate the domain part (the DMARC “domain alignment” policy only looks at the domain part); or it may choose to sign without validating at all.
Yes and no: With DMARC / DKIM, you can ensure that the message came from the server that it claims to. The server might be incorrectly configured. Without, this is far harder to verify.
Note that you also need DNSSEC on the DKIM records, otherwise an attacker who can intercept and spoof DNS can give the server a fake public key and then send a spoofed message with the corresponding private key.
This is an interesting use case. One of the problems with mailto links is that they’ll usually open with the default mail client (no idea why you’ve set the default to the one that you don’t use), but a lot of people they have separate work and personal accounts and use separate clients (especially if they’re web-based ones). It’s quite likely that they’d open with one that wasn’t the one that they wanted to log in with and then they’d wonder why logging in didn’t work.
The waterfall of multi-letter acronyms I’ve been reading in all these discussions suggests to me this will be trouble.
I think the simplest explanation here is the correct one: I didn’t bother to change the default because I don’t use
mailto:links and I use webmail.My computer is a tool, not a home. I don’t spend much time decorating it just right. If the default isn’t right for me, I’m not going to waste time discovering what the right setting is.
I have a sneaking suspicion this is the most common user persona.
I think the unspoken assumption that if an user can READ mail to an address he/she also can SEND from the same address.
That is typically my case. I use uniq addresses every where with wildcarding to my one of my primary addresses.
I can not easily send from the same address as I must configure that address as an sending address.
And the main goal I guess is still this same, that the service can send password resets and news like breaches (or god forbid spam) to the correct account holder.
And birthday greetings! I never feel as appreciated as during my birthday when all the forums I’ve signed up for during the years send me email congratulating me.
Relies on
mailto:links actually working. Given that webmail is probably a high majority for personal accounts nowadays, it’s now common that amailto:link won’t actually directly open your MUA. Heck, I don’t even have it configured myself, and as a developer, I’m not a “regular” user. And “regular” users, I know don’t have this configured either. Heck, just going through GMail’s interface right now, I can’t see how to enable that…Yes, 100% this. I have exactly 2 machines/accounts where I have this configured. Thunderbird on my main workstation at home and my phone.
Not even on my work laptop (I use Outlook webmail there, but only in one profile in one of two browsers), and on none of my laptops/accounts.
So even as not-really webmail user… won’t work.
Actually we already do part of this reversal for mailing lists, when you start the subscription by sending an email and confirm by clicking a link. e.g. for mailman
https://www.list.org/mailman-member/node13.html
I am not convinced it completely avoids the spam box, the response can still land in there, but yes it reduces the probability if your email server auto allows emails in your contact list.
Concerning the challenge I think some mailing lists use unique subscription email addresses as an alternative to any token in the subscription email, or they do two round trips to place a unique token in the reply text. The second round trip is also an alternative if you do not have DKIM and so on.
There is probably a deeper question about whether we need email verification at all, but that is besides the point :D
This is a neat idea but it would confuse people as it’s not normal - ux is all about figuring out what people consider “normal” and trying to fit around that.
And with that I still agree with all the criticisms of email, they’re ancient and insecure theoretically (I’ve never actually seen any of the criticisms affect anyone or any company I know however)
At the risk of sounding like a crypto bro (and I am definitely a huge crypto-skeptic) this is one nice thing it kind of solves: logging in with metamask is such a nice flow compared to email or phone verification! Though it isn’t a suitable replacement, there are some valuable ideas.
I agree with the general terribleness of email verification, and I’m still not sure what sites are actually trying to do beyond getting your email address to spam you. It could be “make sure you’re in control of the account”, but I generally only see this for account creation/user id, not for a generically associated email address.
The other points are also largely correct, the generally html “rich” emails can easily trigger spam filters, try to use remote resources which are blocked, if the remote resources are loaded it violates the privacy of the recipient especially if they didn’t submit their email address in the first place, and finally the rich text is often a train wreck of poorly accessible/readable layout for screen readers, occasionally with a link to go to the site if the email is broken.
Probably because it’s easier to spoof sending an email than receiving it.
Ignoring the user experience as envisioned here, I think it would be tough for any organization to rely on incoming email to be their primary point of entry. It’s had a reputation for being unreliable and when something goes wrong (e.g. bring mistakenly blocked) you have very little direct control. Similarly, while there are methods of authenticating incoming email, the alternative contact mechanisms (e.g. cell phone) don’t have a similar level of authentication.
I’d propose a middle ground. Instead of signing up with a bunch of details and then verifying that email address, I think it would be an improvement just to ask for account details after verifying the email address. That would eliminate the biggest issue with the verification step: it lets you down after you’ve already emotionally committed to using a service.