After some experimentation, I’ve discovered that it’s not just with large integers that this happens. Even with a lowly floating-point number it’ll fall back to sequential scan behaviour. i.e, if you do SELECT * FROM table_name WHERE id = 0.5 it’ll read the 0.5 as NUMERIC. Note that this even happens for integral values in numeric syntax like 2.0. But even if you explicitly pass in a type of double, it’ll fall back to sequentially scanning the table.
This should be fun in any JSON API, even if the implementation language doesn’t support bignums…
Use PQexecParams / PQexecPrepared (or the ‘extended query’ protocol they rely on). They take a datatype for each parameter, which means you can be explicit about what those types should be, and not have one of your values implicitly upcast to something expensive. The alternative is to use PQexec (the ‘simple query’ protocol) with parameter-binding handled on the client side as an operation that just outputs an SQL string.
I figured maybe that’s what they meant, but that depends a lot on the specifics of how that’s done. It’ll only work if you explicitly pass in the type, or the driver does not specify types at all (in which case they’ll default to unspecified, which is a bit like, but not exactly the same, as putting it in a string literal). Some drivers will automatically detect the type to use from the value you pass in, which leads back to the original problem.
For instance, the official Postgres JDBC driver’s .setObject method (which adds arguments to a prepared statement) will detect the input value’s class and if it’s a number other than Long, Int or Double, it will call the .setNumber method which helpfully tells Postgres that the argument type is NUMERIC.
In Clojure (and probably also in JRuby), this means that if the input is a bignum it’ll behave the same way as if you plugged the large number in the query as a literal. I.e., (jdbc/query db ["SELECT pg_typeof(?)" 9223372036854775808]) returns numeric.
If you’re using the JDBC driver more directly (eg from Java or Kotlin), you have to call .setLong() for example explicitly, not .setObject(), otherwise you may be affected as well. Although there you more usually have a more explicit type anyway and will not end up with random bignums so easily.
Thanks for the information. I was not aware of the JDBC behavior with setObject. I only use JDBC for testing Sequel, I don’t have production experience with it.
Sequel’s pg_auto_parameterize extension always forcible casts integers to integer or bigint based on the value of the integer, so it shouldn’t be subject to this issue. Sequel’s JDBC adapter always uses setLong for integer bound variables, so it is also not subject to this issue. I also checked Sequel’s postgres adapter when using bound variables and it is also not subject to this issue. If you don’t forcibly cast when using Sequel’s postgres adapter, PostgreSQL will implicitly cast, or raise an error if no implicit cast is possible. For example, SELECT pg_typeof($1) will result in an error because the type cannot be determined.
Here’s how we worked around it in our Clojure codebase. We reject bigints (of both kinds! sometimes you may get a Java BigInteger, like when deserializing JSON) and provide an escape hatch so that one can still decide to serialize bigints when it’s really needed on a case-by-case basis.
(defn bignum->pgobject
"Wrap a BigInt (or BigDecimal) instance in a PGObject so that it can
be passed on properly as a query argument. This is necessary
because we reject regular BigInts for safety reasons, and we may
decide to do the same for BigDecimals later on too."
[num]
(doto (PGobject.)
(.setType "numeric")
(.setValue (str num))))
(extend-protocol jdbc/ISQLParameter
clojure.lang.BigInt
(set-parameter [v ^PreparedStatement _ ^long _]
(throw (ex-info "Serializing bigints in queries has been disabled to avoid forced sequential scans by malicious user input; if you know it's safe, use bignum->pgobject" {:value v})))
java.math.BigInteger
(set-parameter [v ^PreparedStatement _ ^long _]
(throw (ex-info "Serializing bigints in queries has been disabled avoid forced sequential scans by malicious user input; if you know it's safe, use bignum->pgobject" {:value v}))))
Yeah, I think you’re right. What it really means is “use this interface, and don’t put anything clever between it and the user that could possibly make a bad guess in the same way that the SQL parser does”.
I recently started running my own email server for receiving email, and sending email through Sendgrid. Sendgrid has a free plan that allows for 100 emails per day. Here’s the related Lobsters post in case you are interested: https://lobste.rs/s/s10jr0/running_my_own_email_server
I’m nervous about things like SendGrid. If I send a mail from my server to yours, then I can read it and so can anyone with access to your server. If I send through sendgrid, they’re able to see the plain text of every email that I send. I find it quite hard to believe that they’d offer this as a free service if they weren’t data mining that.
They’re offering a free service because it costs them very little and is useful for getting people to buy their commercial offering. Test for free is an excellent method.
As for security:
if you’re sending to a mailing list, you weren’t going to encrypt
if you’re sending private email, you need to encrypt the payload end-to-end
if you’re sending really secret messages, you want to avoid traffic analysis, too, so email is not for you
The concerning thing about SendGrid et al is that they continue to devalue individual email servers and make it easier for the NSA or agency of your choice to do mass surveillance.
I just started using SendGrid, and I’m a little wary, but I think they offer the service as a loss leader, more than a source of revenue. They did send a nag email every day for a couple of weeks after I signed up, but that seems to have stopped.
That being said, I’ve always considered email definitely not secure, and open to be read by anyone, so them being able to see the plain-text is not really a concern for me.
My email server is similar in a lot of ways (OpenBSD, OpenSMTPD, Vultr) but also different.
The biggest difference is how the emails actually make their way to my LAN. I wanted to avoid a 5‐minute cronjob because I want some mails, like login codes, to show up as soon as they come in. Instead of using rsync, my Vultr server’s smtpd is configured as a relay. Incoming mail is sent over a WireGuard tunnel to a small always‐on machine on my LAN. Helpfully, if the LAN server is down, the relay will queue the mail and automatically retry once the connection is restored. The LAN server’s smtpd delivers incoming mail to a maildir, and Dovecot serves IMAP to my LAN with the maildir as the backing store.
I also have httpd serving a MTA-STS policy for my domain. Several big providers (Google, Microsoft, and Comcast at least) will use it to send email to me via TLS rather than negotiating STARTTLS (which can be MITM’d).
It shouldn’t be able to be MITM’d unless you’re using an implementation with a nasty bug. If you require STARTTLS and validate the certificate correctly, it provides exactly the same MITM protection as a TLS MTA. An MITM on STARTTLS would either not validate or not be able to see the traffic it is MITM-ing.
MTA-STS is new to me, though, and it looks like I should enable it. Thanks for pointing it out.
If you require STARTTLS and validate the certificate correctly
You can’t require STARTTLS. It’s designed to fall back to plaintext if the destination server doesn’t support encryption. Thus it can be exploited by attackers—and has been, rather famously by the Thai government, and also by Cricket.
It’s analogous to a plaintext HTTP URL that immediately 30x redirects to HTTPS: better than nothing, but still vulnerable, because an active attacker can simply strip the redirect to keep the entire session in plaintext.
If the sender simply refused to send mail without a successful STARTTLS negotiation, you’d be right that it would provide as much protection as a full TLS session. But STARTTLS has an insecure fallback baked in. MTA-STS was designed to solve that problem by providing a means for receiving servers to advertise a desire for TLS‐only mail.
You damn sure can, on a client. I have personally written a client that refuses to proceed if STARTTLS doesn’t work, and it wasn’t anything exotic.
It’s designed to fall back to plaintext if the destination server doesn’t support encryption.
This isn’t accurate at all. While I’d dispute that STARTTLS was “designed” to begin with, it certainly isn’t meant to fall back to cleartext. STARTTLS is a means of upgrading an insecure channel to a secure one. Failing to proceed if that upgrade doesn’t work is the only sensible behavior, IMO; otherwise why would you write your client to issue the STARTTLS command in the first place?
If the sender simply refused to send mail without a successful STARTTLS negotiation, you’d be right that it would provide as much protection as a full TLS session.
This is literally the only reasonable way to write a client that uses STARTTLS. If a client isn’t going to do that, why bother with STARTTLS at all? TLS isn’t something that’s just supposed to work if everyone is being nice. It’s a means of secure communication over an insecure, hostile network. Giving up and falling back to insecure communication in response to an error defeats the purpose of TLS. If the developer of an endpoint is going to do that, they may as well save themselves the time and labor and leave TLS support out altogether.
Sure, from the server’s perspective, requiring STARTTLS would be weird. But on a client, requiring it is the only sane thing to do if the client’s operator indicates that STARTTLS should be used. Anyone who writes a client that issues the STARTTLS command and doesn’t require it to succeed opens their users to the rather famous exploits you yourself point out.
The context here isn’t sanely written clients. The context is a public‐facing MX that has no control over whether incoming mailservers continue to send mail upon failure to negotiate STARTTLS and cannot reject plaintext mail.
While I’d dispute that STARTTLS was “designed” to begin with, it certainly isn’t meant to fall back to cleartext.
I refer you to RFC 3207:
A publicly-referenced SMTP server MUST NOT require use of the
STARTTLS extension in order to deliver mail locally. This rule
prevents the STARTTLS extension from damaging the interoperability of
the Internet's SMTP infrastructure. A publicly-referenced SMTP
server is an SMTP server which runs on port 25 of an Internet host
listed in the MX record (or A record if an MX record is not present)
for the domain name on the right hand side of an Internet mail
address.
TLS isn’t something that’s just supposed to work if everyone is being nice. It’s a means of secure communication over an insecure, hostile network. Giving up and falling back to insecure communication in response to an error defeats the purpose of TLS.
I agree entirely, which is precisely why I suggest using MTA-STS, because STARTTLS behaves insecurely on a hostile network, exactly the way you decry.
The context here isn’t sanely written clients. The context is a public‐facing MX that has no control over whether incoming mailservers fall back to plaintext or not.
Then we’re having a stupid discussion, and I apologize for that. In the case of either STARTTLS or a plain TLS server on a public-facing MX, we’re still only talking about “server authenticated” TLS. That means that the server presents a certificate for the client to validate, but not vice versa. A server would never be able to detect an MITM in either scenario. They are equivalent in that regard.
Without a context of sanely written clients, server-auth TLS is useless. You need mutual auth to harden against badly behaved clients in this regard.
Your edit pointed out a MUST NOT in the RFC that I absolutely hadn’t been thinking about. So thanks, even if I replied too fast to catch it the first time. I’ve spilled a lot of words to mostly agree with you, I think.
After a pass over RFC 8461, though, I still don’t think there’s a meaningful difference even for a public facing MX unless you can turn off cleartext SMTP completely.
Section 4 explicitly calls out that STARTTLS is supported:
4. Policy Validation
When sending to an MX at a domain for which the sender has a valid
and non-expired MTA-STS Policy, a Sending MTA honoring MTA-STS MUST
check whether:
1. At least one of the policy's "mx" patterns matches the selected
MX host, as described in Section 4.1, "MX Host Validation".
2. The recipient mail server supports STARTTLS and offers a PKIX-
based TLS certificate, during TLS handshake, which is valid for
that host, as described in Section 4.2, "Recipient MTA
Certificate Validation".
If you can’t turn the non-TLS MTA off, I don’t see the difference.
I do still think the STS record is a win. But I can’t see a meaningful difference between an MTA that requires TLS from the handshake versus STARTTLS.
In the context of SMTP server‐to‐server transactions, if STARTTLS negotation fails, the mail sender is likely to continue in plaintext rather than drop the mail. If STARTTLS succeeds but the TLS certificate is invalid, the mail sender is likely to continue with the invalid certificate rather than drop the mail. Not all senders continue in these cases, but many do, because STARTTLS‐capable servers with valid certificates have historically been far from universal (and still aren’t, probably, even after Let’s Encrypt). Naturally, this situation is ripe for MITM.
Turning off cleartext SMTP on your MX doesn’t help. Senders will still default to port 25, which an attacker can spoof a cleartext server on without you in the loop at all.
MTA-STS policies provide a way for an MX to require secure transport. In “enforce” mode, STARTTLS is mandatory, and certificates must be valid. Senders can’t continue without STARTTLS, and they can’t accept an invalid certificate; such mail must be rejected. So MTA-STS prevents MITM in a way that was not possible before.
This helps only when the sender checks for MTA-STS. Even so, you’re no worse off than before, and slightly better, because many of the big providers like Google and Microsoft do check. If you’ve configured your policy properly, incoming mail can’t be MITM’d between you and any MTA-STS–compliant sender.
I guess there’s the rub. If the MTA-STS spec lets servers consider STARTTLS as mandatory, though, I’m still not seeing how STARTTLS is more vulnerable to MITM than TLS at the handshake.
After all this, I see a few options for a server operator:
Cleartext SMTP listener with STARTTLS. Can be MITMd if it doesn’t require STARTLS, and might be MITM-able even if it does due to poor “client” (by this I mean othethatr servers, principally) implementations.
Cleartext SMTP listener with no STARTTLS. Obviously MITM-able.
TLS at the handshake. MITM could just block the port and cause “clients” to fall back to 25 anyway. Also, it’s unlikely that those clients who don’t validate certs for STARTTLS suddenly start doing so at the handshake.
Turn off port 25. Still MITM-able because non MTA-STS “clients” will try it first and proceed with no STARTTLS anyway.
MTA-STS is obviously a win insofar as it induces a few big implementations to be better clients. I wasn’t doubting that, though. I’m not seeing where sending via TLS at the handshake buys anything over STARTTLS. Am I missing a spot where it does?
That’s a pretty good way to set things up, definitely better than my hacky cron job. I didn’t have experience with wireguard, and it’s usually not a big deal for me to wait 5 minutes. If I’m expecting an email and don’t want to wait, I run the cron job manually. But maybe I’ll look into using wireguard.
Serving MTA-STS also sounds interesting, I may look into doing that as well.
It’s a shame Zoho was discounted quickly. I’m paying them for the email only and love the service. I’ve got multiple domains on it and use catch-all addresses, so I don’t have to do any extra work for the per-service aliases.
I probably should have spent more time researching Zoho. I just went to their website and was overwhelmed with all of the options.
I’ve considered using a catch-all, as it saves work setting up aliases. However, I prefer to only allow a specific addresses, so that if someone has to know a valid email address for me to accept the email.
I’m on my third year of Zoho for one of my domains, and I can’t say I love it. I’ve had some small, occasional mail delivery issues both receiving and sending, and the response from their support desk has been underwhelming. It hasn’t been enough of a problem for me to move that domain to another service, but it’s been annoying enough that I’m not moving any more of my mail to Zoho either, even now that tuffmail is going to force me to move a big chunk of it somewhere.
I’m the author of the post. I hadn’t heard of Migadu before, but it almost looks like it would work. The only issue is their Micro plan ($19/year) only allows 200 inbound emails per day. I guess that may not be an issue most of the time, but there are days where I receive more than 200 emails. The inability to control how much inbound email you are receiving makes me hesitant to use such a service.
I also use migadu for a family account. The thing I really like about it is that you can add as many accounts as you like for your domain: this@my.domain, that@my.domain, theother@my.domain, it’s so nice.
But yes, the 200-in limit has been my concern too. I subscribe to a few mailing lists and have a worry that one day some big heated mailing list conversation will put me over. This FAQ answer suggests they are lenient and that emails won’t actually be lost (assuming senders follow correct email practices!), but my tentative plan has been to wait and see if I ever get a warning email, and upgrade to the next tier if it becomes an issue. It hasn’t so far, after a year or so.
For what it’s worth, it’s not a hard limit. They won’t block the 201st email — if it’s a recurring thing, they’ll ask you to upgrade. This is mentioned in their docs, somewhere. cc @jeremyevans
I checked and it is in their documentation. So maybe that would have been a simpler option. I might have switched to Migadu instead of using a VM if I had known about it first. I think the only issue is the next level up from the $19/year plan is the $90/year plan, which is a pretty significant jump. But for someone who isn’t likely to go over their limits, Migadu looks like a really nice option.
Which email client(s) do you use? Last time I checked, Thunderbird doesn’t put design thought toward this use case. As such it is clunky to use for sending emails from different addresses.
I’m on Evolution now, but always looking for better options.
I primarily use mutt, which I have configured with 4 different email accounts: 1 work, 1 gmail, 2 migadu. So I don’t actually send from different addresses exactly (although I think that is easy to do in mutt), but have commands which switch me completely to a different account and inbox.
But what I meant about migadu is not that they give you multiple email addresses to send to and from within your domain, but that they let you add as many accounts as you like within that domain. So my daughters get their own email addresses and passwords and can sign into them on whatever mail client they like. And I can give these out to as many of my family as I like (the domain is a play on our surname), as long as I don’t hit the 200/20 limit.
Thanks for posting your setup. I’ve been sniffing at things adjacent to this for a while, looking at some other providers for SMTP. mailroute was the one that had looked most promising, but their definition of a “user” would have had me paying heavily for all the aliases I use, so I had not made the jump yet. Tuffmail’s EOL is going to force my hand.
Right now, I’m deciding between Migadu and a setup similar to what you’ve done. I had almost given up on the self hosted setup. Sendgrid could work for me, though. My only heartburn about it is, if they decide to kill off their free plan, it’s a huge jump up to $15/mo while I work out an alternative. Where I’d be flirting with the 200 in/day limit on Migadu, the jump up to the next tier isn’t as nasty if I need to do that.
Someone on the orange site (which I really don’t know why I keep going to, habit I guess) was upset saying “they’re spread too thin”, which seems entirely wrong. It just means that Microsoft is investing heavily. That is only good. Heck, I’d consider a job if they’d set up an office in Santa Cruz :)
But I do wish that they’d invest in GitHub Issues and code review. I don’t think anyone can say with a straight face that they like them. There has to be a better way, perhaps a V2 that runs alongside using the same database but allowing a V1 UI for the people that can’t let it go. Gerrit has a better code review system, I don’t know about GitLab. There are certainly examples that can be cribbed from.
EDIT: I’d also like to see them expand the remit of their CLI project to try and rethink the Git terminal workflow. Again, as above, I don’t think many people like it. It’s grown organically and a new way of interacting with Git knowing what we know now could be huge.
I really like GitHub Issues. My company forced my team to switch to Jira a few months ago, and it’s miserable. I can’t believe that in 2020 there are still major commercial web-apps as shitty as Jira. They can’t even get their textareas to work right; half the time when I hit Submit I watch it mangle my markup.
Heck, I’d consider a job if they’d set up an office in Santa Cruz :)
Besides the unusual circumstances of everyone working from home, GitHub is very remote-friendly. You should apply! (If you meant Microsoft, I can’t help there…)
All I can say about the other things is stay tuned. These were some impactful releases today, but we’re not done yet.
I also like Github issues. I’ve used issue trackers for something like 18 years, literally every day at points, and I can’t think of one that’s clearly better than Github. I’ve been using it regularly for at least 4 years.
Like all software, it can be improved more (e.g. latency), but I can find about 100 other pieces of software to complain about first.
The above command will hang if the host is vulnerable. After running syspatch (no reboot), the command fails with the following error:
debug1: Next authentication method: keyboard-interactive
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: No more authentication methods to try.
-sresponse:passwd@bsd66.domain.tld: Permission denied (publickey,password,keyboard-interactive).
I have a 6.4 host that is still vulnerable after running syspatch though.
This game is great. The computer destroyed me though (predictably since it will play perfectly every time through a brute force search). I imagine it would be much better to play against another (less than perfect) human.
I wrote a multiplayer version of Quinto about 6 years ago, with a couple of rewrites since. An online demo is available at http://quinto-demo.jeremyevans.net/ if you can find another person to play with. Source code is at https://github.com/jeremyevans/quinto if you want to run your own server.
That’s amazing! Small world. I’d love to see your CoffeScript source code, but couldn’t find it in the repo - is it intentionally kept secret, or is that just an accident? Either way, great game choice, and awesome project!
The code was originally written in CoffeeScript+Node. The server was rewritten in Go, and then later rewritten in Ruby. At some point, I stopped using CoffeeScript on the front end and just started editing the resulting Javascript file directly. All of the information is in the repository if you look in the history: https://github.com/jeremyevans/quinto/tree/7ad48e43f76c1a9a847d5a677a8f11c69c9fa5bc
There’s potential for beating a computer that plays the highest scoring move each time. You can play to avoid setting up long parallel plays, and it may be worth saving 5s and 0s since those can extend words of length greater than 1.
After some experimentation, I’ve discovered that it’s not just with large integers that this happens. Even with a lowly floating-point number it’ll fall back to sequential scan behaviour. i.e, if you do
SELECT * FROM table_name WHERE id = 0.5it’ll read the0.5asNUMERIC. Note that this even happens for integral values in numeric syntax like2.0. But even if you explicitly pass in a type ofdouble, it’ll fall back to sequentially scanning the table.This should be fun in any JSON API, even if the implementation language doesn’t support bignums…
This is covered in the post under the “Behavior of Other Numeric Type Comparisons on PostgreSQL” heading. :)
I should really work on my reading skills ;)
Under mitigations, you say
What do you mean by this exactly?
Use PQexecParams / PQexecPrepared (or the ‘extended query’ protocol they rely on). They take a datatype for each parameter, which means you can be explicit about what those types should be, and not have one of your values implicitly upcast to something expensive. The alternative is to use PQexec (the ‘simple query’ protocol) with parameter-binding handled on the client side as an operation that just outputs an SQL string.
I figured maybe that’s what they meant, but that depends a lot on the specifics of how that’s done. It’ll only work if you explicitly pass in the type, or the driver does not specify types at all (in which case they’ll default to
unspecified, which is a bit like, but not exactly the same, as putting it in a string literal). Some drivers will automatically detect the type to use from the value you pass in, which leads back to the original problem.For instance, the official Postgres JDBC driver’s
.setObjectmethod (which adds arguments to a prepared statement) will detect the input value’s class and if it’s a number other thanLong,IntorDouble, it will call the.setNumbermethod which helpfully tells Postgres that the argument type isNUMERIC.In Clojure (and probably also in JRuby), this means that if the input is a bignum it’ll behave the same way as if you plugged the large number in the query as a literal. I.e.,
(jdbc/query db ["SELECT pg_typeof(?)" 9223372036854775808])returnsnumeric.If you’re using the JDBC driver more directly (eg from Java or Kotlin), you have to call
.setLong()for example explicitly, not.setObject(), otherwise you may be affected as well. Although there you more usually have a more explicit type anyway and will not end up with random bignums so easily.Thanks for the information. I was not aware of the JDBC behavior with
setObject. I only use JDBC for testing Sequel, I don’t have production experience with it.Sequel’s pg_auto_parameterize extension always forcible casts integers to integer or bigint based on the value of the integer, so it shouldn’t be subject to this issue. Sequel’s JDBC adapter always uses
setLongfor integer bound variables, so it is also not subject to this issue. I also checked Sequel’s postgres adapter when using bound variables and it is also not subject to this issue. If you don’t forcibly cast when using Sequel’s postgres adapter, PostgreSQL will implicitly cast, or raise an error if no implicit cast is possible. For example,SELECT pg_typeof($1)will result in an error because the type cannot be determined.Here’s how we worked around it in our Clojure codebase. We reject bigints (of both kinds! sometimes you may get a Java BigInteger, like when deserializing JSON) and provide an escape hatch so that one can still decide to serialize bigints when it’s really needed on a case-by-case basis.
Yeah, I think you’re right. What it really means is “use this interface, and don’t put anything clever between it and the user that could possibly make a bad guess in the same way that the SQL parser does”.
Does anybody know what the “y” in yjit stands for? Maybe my google-fu has gone stale, but I couldn’t find anything to answer this.
“Yet Another Ruby” https://github.com/ruby/ruby/blob/master/doc/yjit/yjit.md
I recently started running my own email server for receiving email, and sending email through Sendgrid. Sendgrid has a free plan that allows for 100 emails per day. Here’s the related Lobsters post in case you are interested: https://lobste.rs/s/s10jr0/running_my_own_email_server
How good a sender reputation does Sendgrid have? Have you had any instances where mail bounced or didn’t get delivered?
I haven’t had any problems. Sendgrid has a fairly good sender reputation as far as I know. It’s one of the larger email handling companies.
In my experience if you’re on free tier, you get an IP with a bad reputation. You have to pay to get a good IP.
I’m nervous about things like SendGrid. If I send a mail from my server to yours, then I can read it and so can anyone with access to your server. If I send through sendgrid, they’re able to see the plain text of every email that I send. I find it quite hard to believe that they’d offer this as a free service if they weren’t data mining that.
They’re offering a free service because it costs them very little and is useful for getting people to buy their commercial offering. Test for free is an excellent method.
As for security:
if you’re sending to a mailing list, you weren’t going to encrypt
if you’re sending private email, you need to encrypt the payload end-to-end
if you’re sending really secret messages, you want to avoid traffic analysis, too, so email is not for you
The concerning thing about SendGrid et al is that they continue to devalue individual email servers and make it easier for the NSA or agency of your choice to do mass surveillance.
Yes, I also consider E-mail “wide open,” so deliverability matters more to me than security, even though both are nice.
I just started using SendGrid, and I’m a little wary, but I think they offer the service as a loss leader, more than a source of revenue. They did send a nag email every day for a couple of weeks after I signed up, but that seems to have stopped.
That being said, I’ve always considered email definitely not secure, and open to be read by anyone, so them being able to see the plain-text is not really a concern for me.
My email server is similar in a lot of ways (OpenBSD, OpenSMTPD, Vultr) but also different.
The biggest difference is how the emails actually make their way to my LAN. I wanted to avoid a 5‐minute cronjob because I want some mails, like login codes, to show up as soon as they come in. Instead of using rsync, my Vultr server’s smtpd is configured as a relay. Incoming mail is sent over a WireGuard tunnel to a small always‐on machine on my LAN. Helpfully, if the LAN server is down, the relay will queue the mail and automatically retry once the connection is restored. The LAN server’s smtpd delivers incoming mail to a maildir, and Dovecot serves IMAP to my LAN with the maildir as the backing store.
I also have httpd serving a MTA-STS policy for my domain. Several big providers (Google, Microsoft, and Comcast at least) will use it to send email to me via TLS rather than negotiating STARTTLS (which can be MITM’d).
It shouldn’t be able to be MITM’d unless you’re using an implementation with a nasty bug. If you require STARTTLS and validate the certificate correctly, it provides exactly the same MITM protection as a TLS MTA. An MITM on STARTTLS would either not validate or not be able to see the traffic it is MITM-ing.
MTA-STS is new to me, though, and it looks like I should enable it. Thanks for pointing it out.
You can’t require STARTTLS. It’s designed to fall back to plaintext if the destination server doesn’t support encryption. Thus it can be exploited by attackers—and has been, rather famously by the Thai government, and also by Cricket.
It’s analogous to a plaintext HTTP URL that immediately 30x redirects to HTTPS: better than nothing, but still vulnerable, because an active attacker can simply strip the redirect to keep the entire session in plaintext.
If the sender simply refused to send mail without a successful STARTTLS negotiation, you’d be right that it would provide as much protection as a full TLS session. But STARTTLS has an insecure fallback baked in. MTA-STS was designed to solve that problem by providing a means for receiving servers to advertise a desire for TLS‐only mail.
You damn sure can, on a client. I have personally written a client that refuses to proceed if STARTTLS doesn’t work, and it wasn’t anything exotic.
This isn’t accurate at all. While I’d dispute that STARTTLS was “designed” to begin with, it certainly isn’t meant to fall back to cleartext. STARTTLS is a means of upgrading an insecure channel to a secure one. Failing to proceed if that upgrade doesn’t work is the only sensible behavior, IMO; otherwise why would you write your client to issue the STARTTLS command in the first place?
This is literally the only reasonable way to write a client that uses STARTTLS. If a client isn’t going to do that, why bother with STARTTLS at all? TLS isn’t something that’s just supposed to work if everyone is being nice. It’s a means of secure communication over an insecure, hostile network. Giving up and falling back to insecure communication in response to an error defeats the purpose of TLS. If the developer of an endpoint is going to do that, they may as well save themselves the time and labor and leave TLS support out altogether.
Sure, from the server’s perspective, requiring STARTTLS would be weird. But on a client, requiring it is the only sane thing to do if the client’s operator indicates that STARTTLS should be used. Anyone who writes a client that issues the STARTTLS command and doesn’t require it to succeed opens their users to the rather famous exploits you yourself point out.
The context here isn’t sanely written clients. The context is a public‐facing MX that has no control over whether incoming mailservers continue to send mail upon failure to negotiate STARTTLS and cannot reject plaintext mail.
I refer you to RFC 3207:
I agree entirely, which is precisely why I suggest using MTA-STS, because STARTTLS behaves insecurely on a hostile network, exactly the way you decry.
Then we’re having a stupid discussion, and I apologize for that. In the case of either STARTTLS or a plain TLS server on a public-facing MX, we’re still only talking about “server authenticated” TLS. That means that the server presents a certificate for the client to validate, but not vice versa. A server would never be able to detect an MITM in either scenario. They are equivalent in that regard.
Without a context of sanely written clients, server-auth TLS is useless. You need mutual auth to harden against badly behaved clients in this regard.
Sorry for editing my comment. I didn’t notice you’d replied already.
Your edit pointed out a
MUST NOTin the RFC that I absolutely hadn’t been thinking about. So thanks, even if I replied too fast to catch it the first time. I’ve spilled a lot of words to mostly agree with you, I think.After a pass over RFC 8461, though, I still don’t think there’s a meaningful difference even for a public facing MX unless you can turn off cleartext SMTP completely.
Section 4 explicitly calls out that STARTTLS is supported:
If you can’t turn the non-TLS MTA off, I don’t see the difference.
I do still think the STS record is a win. But I can’t see a meaningful difference between an MTA that requires TLS from the handshake versus STARTTLS.
In the context of SMTP server‐to‐server transactions, if STARTTLS negotation fails, the mail sender is likely to continue in plaintext rather than drop the mail. If STARTTLS succeeds but the TLS certificate is invalid, the mail sender is likely to continue with the invalid certificate rather than drop the mail. Not all senders continue in these cases, but many do, because STARTTLS‐capable servers with valid certificates have historically been far from universal (and still aren’t, probably, even after Let’s Encrypt). Naturally, this situation is ripe for MITM.
Turning off cleartext SMTP on your MX doesn’t help. Senders will still default to port 25, which an attacker can spoof a cleartext server on without you in the loop at all.
MTA-STS policies provide a way for an MX to require secure transport. In “enforce” mode, STARTTLS is mandatory, and certificates must be valid. Senders can’t continue without STARTTLS, and they can’t accept an invalid certificate; such mail must be rejected. So MTA-STS prevents MITM in a way that was not possible before.
This helps only when the sender checks for MTA-STS. Even so, you’re no worse off than before, and slightly better, because many of the big providers like Google and Microsoft do check. If you’ve configured your policy properly, incoming mail can’t be MITM’d between you and any MTA-STS–compliant sender.
I guess there’s the rub. If the MTA-STS spec lets servers consider STARTTLS as mandatory, though, I’m still not seeing how STARTTLS is more vulnerable to MITM than TLS at the handshake.
After all this, I see a few options for a server operator:
Cleartext SMTP listener with STARTTLS. Can be MITMd if it doesn’t require STARTLS, and might be MITM-able even if it does due to poor “client” (by this I mean othethatr servers, principally) implementations.
Cleartext SMTP listener with no STARTTLS. Obviously MITM-able.
TLS at the handshake. MITM could just block the port and cause “clients” to fall back to 25 anyway. Also, it’s unlikely that those clients who don’t validate certs for STARTTLS suddenly start doing so at the handshake.
Turn off port 25. Still MITM-able because non MTA-STS “clients” will try it first and proceed with no STARTTLS anyway.
MTA-STS is obviously a win insofar as it induces a few big implementations to be better clients. I wasn’t doubting that, though. I’m not seeing where sending via TLS at the handshake buys anything over STARTTLS. Am I missing a spot where it does?
That’s a pretty good way to set things up, definitely better than my hacky cron job. I didn’t have experience with wireguard, and it’s usually not a big deal for me to wait 5 minutes. If I’m expecting an email and don’t want to wait, I run the cron job manually. But maybe I’ll look into using wireguard.
Serving MTA-STS also sounds interesting, I may look into doing that as well.
Thanks for the ideas!
It’s a shame Zoho was discounted quickly. I’m paying them for the email only and love the service. I’ve got multiple domains on it and use catch-all addresses, so I don’t have to do any extra work for the per-service aliases.
I probably should have spent more time researching Zoho. I just went to their website and was overwhelmed with all of the options.
I’ve considered using a catch-all, as it saves work setting up aliases. However, I prefer to only allow a specific addresses, so that if someone has to know a valid email address for me to accept the email.
I’m on my third year of Zoho for one of my domains, and I can’t say I love it. I’ve had some small, occasional mail delivery issues both receiving and sending, and the response from their support desk has been underwhelming. It hasn’t been enough of a problem for me to move that domain to another service, but it’s been annoying enough that I’m not moving any more of my mail to Zoho either, even now that tuffmail is going to force me to move a big chunk of it somewhere.
FWIW, I’ve personally enjoyed email hosting by Migadu.
Same here. Switched after self-hosting mail for a couple years. Can absolutely recommend Migadu.
I’m the author of the post. I hadn’t heard of Migadu before, but it almost looks like it would work. The only issue is their Micro plan ($19/year) only allows 200 inbound emails per day. I guess that may not be an issue most of the time, but there are days where I receive more than 200 emails. The inability to control how much inbound email you are receiving makes me hesitant to use such a service.
I also use migadu for a family account. The thing I really like about it is that you can add as many accounts as you like for your domain: this@my.domain, that@my.domain, theother@my.domain, it’s so nice.
But yes, the 200-in limit has been my concern too. I subscribe to a few mailing lists and have a worry that one day some big heated mailing list conversation will put me over. This FAQ answer suggests they are lenient and that emails won’t actually be lost (assuming senders follow correct email practices!), but my tentative plan has been to wait and see if I ever get a warning email, and upgrade to the next tier if it becomes an issue. It hasn’t so far, after a year or so.
For what it’s worth, it’s not a hard limit. They won’t block the 201st email — if it’s a recurring thing, they’ll ask you to upgrade. This is mentioned in their docs, somewhere. cc @jeremyevans
I checked and it is in their documentation. So maybe that would have been a simpler option. I might have switched to Migadu instead of using a VM if I had known about it first. I think the only issue is the next level up from the $19/year plan is the $90/year plan, which is a pretty significant jump. But for someone who isn’t likely to go over their limits, Migadu looks like a really nice option.
It’s mentioned in the FAQ answer I linked to
Ah, didn’t notice you’d done that.
Re: using multiple addresses at the same domain:
Which email client(s) do you use? Last time I checked, Thunderbird doesn’t put design thought toward this use case. As such it is clunky to use for sending emails from different addresses.
I’m on Evolution now, but always looking for better options.
I primarily use mutt, which I have configured with 4 different email accounts: 1 work, 1 gmail, 2 migadu. So I don’t actually send from different addresses exactly (although I think that is easy to do in mutt), but have commands which switch me completely to a different account and inbox.
But what I meant about migadu is not that they give you multiple email addresses to send to and from within your domain, but that they let you add as many accounts as you like within that domain. So my daughters get their own email addresses and passwords and can sign into them on whatever mail client they like. And I can give these out to as many of my family as I like (the domain is a play on our surname), as long as I don’t hit the 200/20 limit.
Thanks for posting your setup. I’ve been sniffing at things adjacent to this for a while, looking at some other providers for SMTP. mailroute was the one that had looked most promising, but their definition of a “user” would have had me paying heavily for all the aliases I use, so I had not made the jump yet. Tuffmail’s EOL is going to force my hand.
Right now, I’m deciding between Migadu and a setup similar to what you’ve done. I had almost given up on the self hosted setup. Sendgrid could work for me, though. My only heartburn about it is, if they decide to kill off their free plan, it’s a huge jump up to $15/mo while I work out an alternative. Where I’d be flirting with the 200 in/day limit on Migadu, the jump up to the next tier isn’t as nasty if I need to do that.
Really sad to hear about Tuffmail. They were truly the best option.
I think this looks great.
Someone on the orange site (which I really don’t know why I keep going to, habit I guess) was upset saying “they’re spread too thin”, which seems entirely wrong. It just means that Microsoft is investing heavily. That is only good. Heck, I’d consider a job if they’d set up an office in Santa Cruz :)
But I do wish that they’d invest in GitHub Issues and code review. I don’t think anyone can say with a straight face that they like them. There has to be a better way, perhaps a V2 that runs alongside using the same database but allowing a V1 UI for the people that can’t let it go. Gerrit has a better code review system, I don’t know about GitLab. There are certainly examples that can be cribbed from.
EDIT: I’d also like to see them expand the remit of their CLI project to try and rethink the Git terminal workflow. Again, as above, I don’t think many people like it. It’s grown organically and a new way of interacting with Git knowing what we know now could be huge.
I really like GitHub Issues. My company forced my team to switch to Jira a few months ago, and it’s miserable. I can’t believe that in 2020 there are still major commercial web-apps as shitty as Jira. They can’t even get their textareas to work right; half the time when I hit Submit I watch it mangle my markup.
I’ve used GitHub Issues for over 10 years and like it. Of the 10 or so issue tracking tools I’ve used, it’s my favorite.
Besides the unusual circumstances of everyone working from home, GitHub is very remote-friendly. You should apply! (If you meant Microsoft, I can’t help there…)
All I can say about the other things is stay tuned. These were some impactful releases today, but we’re not done yet.
I also like Github issues. I’ve used issue trackers for something like 18 years, literally every day at points, and I can’t think of one that’s clearly better than Github. I’ve been using it regularly for at least 4 years.
Like all software, it can be improved more (e.g. latency), but I can find about 100 other pieces of software to complain about first.
Most importantly, running
syspatch(8)will fix the issue on openbsd 6.6. (patch 010_libcauth).As specified in the article, the vulnerability can be tested using the following command:
The above command will hang if the host is vulnerable. After running syspatch (no reboot), the command fails with the following error:
I have a 6.4 host that is still vulnerable after running syspatch though.
Awesome bug report and quick fix !
OpenBSD only produces syspatches for the 2 most recent releases (6.5 and 6.6 currently). Upgrade from 6.4 to 6.6. :)
This game is great. The computer destroyed me though (predictably since it will play perfectly every time through a brute force search). I imagine it would be much better to play against another (less than perfect) human.
I wrote a multiplayer version of Quinto about 6 years ago, with a couple of rewrites since. An online demo is available at http://quinto-demo.jeremyevans.net/ if you can find another person to play with. Source code is at https://github.com/jeremyevans/quinto if you want to run your own server.
That’s amazing! Small world. I’d love to see your CoffeScript source code, but couldn’t find it in the repo - is it intentionally kept secret, or is that just an accident? Either way, great game choice, and awesome project!
The code was originally written in CoffeeScript+Node. The server was rewritten in Go, and then later rewritten in Ruby. At some point, I stopped using CoffeeScript on the front end and just started editing the resulting Javascript file directly. All of the information is in the repository if you look in the history: https://github.com/jeremyevans/quinto/tree/7ad48e43f76c1a9a847d5a677a8f11c69c9fa5bc
There’s potential for beating a computer that plays the highest scoring move each time. You can play to avoid setting up long parallel plays, and it may be worth saving 5s and 0s since those can extend words of length greater than 1.