1. 72
  1.  

  2. 32

    I wonder how long it will take GMail users to realize that it’s not other people who use unreliable mail servers, it’s them using a service that rejects legitimate messages.

    1. 28

      Here’s the problem, though; I just ran the host(1) command against a number of popular domains for some of the biggest companies around:

      twitter.com mail is handled by 10 aspmx.l.google.com.
      reddit.com mail is handled by 1 aspmx.l.google.com.
      a16z.com mail is handled by 10 aspmx.l.google.com.
      ycombinator.com mail is handled by 10 aspmx.l.google.com.
      arista.com mail is handled by 1 aspmx.l.google.com.
      wework.com mail is handled by 1 aspmx.l.google.com.
      stackoverflow.com mail is handled by 1 aspmx.l.google.com.
      netflix.com mail is handled by 1 aspmx.l.google.com.
      xvideos.com mail is handled by 1 aspmx.l.google.com.
      

      So, it’s not a matter of who’s unreliable, but who has the biggest customer base, and who’s the monopoly.

      Sadly, Gmail and G Suite prevail with their monoculture.

    2. 11

      This is exactly why I haven’t hosted my own email in years. Even one rejection to a GMail address is just not worth the risk. I think Fastmail is the least bad option, so that’s what I’ve been sticking with.

      1. 8

        To avoid any misunderstanding: in the last few weeks, I can 100% reproduce identical emails being either accepted (if my secondary “new” domain is used in From and MAIL FROM) or 550-5.7.1 rejected (if the old primary one with the cron-acquired “low reputation” is used) at the SMTP stage; e.g., you get a bounce right away in a few seconds at most. Note that in the case of them being accepted, they don’t show up in Spam folder, either, but appear straight in the Inbox of my own Gmail (so, I assume G Suite would be the same).

        So, as above, there’s zero reasons to conclude that even one rejection is not worth the risk, as you can simply resend.

        However, to extrapolate your conclusion — you appear to assume that using a third-party email service is a sureway for 100% acceptance, but, due to the way Bayesian filters work, that’s 100% not the case. Setting up SPF and DMARC for your own server is trivial; and that’s exactly the control parts of the equation. Your email may still end up in the Spam folder even if you do use a hosted provider. Some of these companied like SendGrid can’t even go through Greylisting, something that any homemade email server would have zero trouble going through, so, the assumption that these people know what they’re doing is not even correct to start with.

        1. 9

          I know how to setup my own spam filters, SPF, DKIM, DMARC, SMTP, IMAP, sieve, etc etc. it just hasnt been worth the hassle since … 2013?

          I used to run email hosting at an ISP for thousands of local businesses. Never again. GMail ruined my day every day. People screaming because no matter what we do their mail to customers, clients of lawyers, etc at gmail addresses are getting bounced at random.

          I stated using Fastmail because my own employer’s mail hosting was unreliable for me communicating with devs of open source projects with GMail addresses. So I’ve been with Fastmail for about 7 years and be never had a message bounced or marked as spam, nor have I had an inbound mail lost due to greylisting, spam, or rejection. (Fastmail’s greylisting was extremely advanced back in 2012)

          1. 1

            I stated using Fastmail because my own employer’s mail hosting was unreliable for me communicating with devs of open source projects with GMail addresses.

            In my experience, OSS people are actually one of the few folk that still run their own mail servers, so, communicating with them is rarely an issue; in fact, I’ve had issues communicating with some OpenBSD folk through Gmail, because of Gmail not being too friendly with greylisting (using a diff IP on each attempt).

            So, given your ISP experience, what do you think is the magic thing that Fastmail does here? Do they send some sort of magic bytes with their mail? Different byte alignment? Or is it just a matter of them being whitelisted by Gmail as another major player, to not get the emails rejected? TBH, I find your absolutes a little hard to believe — have you moved the customers of the ISP to Fastmail, and did they stop complaining about Gmail, or what? You can’t even send inline patchsets through Gmail without it mangling the whitespace, don’t really see how it could be used in OSS too effectively.

            1. 2

              In my experience, OSS people are actually one of the few folk that still run their own mail servers

              Depends a lot. Nodejs and webdev people are not going to run their own mailservers. People writing C code and working on OSes will.

              I’ve had issues communicating with some OpenBSD folk through Gmail, because of Gmail not being too friendly with greylisting (using a diff IP on each attempt).

              OpenBSD’s spamd needs to a) support IPv6, which I think it still doesn’t and b) needs to have some database of all the known IP addresses a domain can use for sending.

              So, given your ISP experience, what do you think is the magic thing that Fastmail does here?

              Have a good reputation from sending a lot of mail, have a great spam filter, and their greylisting system is clustered, knows all the possible sending IPs for domains, and automatically skips greylisting in several situations. e.g., if the sender is in your address book, which most greylisting systems won’t have access to because they’re not that tightly integrated into the actual email service. More details here: https://www.fastmail.com/help/technical/smtpchecks.html#greylising

              TBH, I find your absolutes a little hard to believe — have you moved the customers of the ISP to Fastmail, and did they stop complaining about Gmail, or what?

              I pushed hard to move our email hosting to Fastmail, but the owner of the company felt that ISPs are worthless if they don’t provide their own DNS, email, and webhosting. That was understandable until the mid 2000s, but then the competition got so big it is impossible as a small company to do all of these things in-house and do them well.

              I did get several thousand domains moved to an external DNS hosting provider after screaming about the dangers for years, and then finally we had a DDoS that was so bad it caused customers to have a full outage because their primary webhosting was with us and their DNS was with us. Their backup webhosting was elsewhere, but our DNS was down because our AS was unreachable…

              Owner never budged on email, and to this day they still do their own email hosting on servers that are 10 years old with no reliable backups running end of life open source software from around 2009. It’s bad bad bad all the way around, but hey – not my company, not my money.

              edit: fastmail actually provides nice reselling features where we could have hosted everything including the webUI on our own domains, with own branding, etc and nobody would have ever knows it was Fastmail underneath. Contacts and calendars support, which [the ISP] still does not offer. It would be cheaper to pay Fastmail than what it was costing us to maintain our infrastructure, AND Fastmail would automate the entire migration process for each domain for you: ingestion of mail, DNS cutovers, etc. Seamless.

              1. 7

                I’m a web developer, I don’t write C code, and I run my own mailservers on OpenBSD and I’ve never had issues with my emails getting through. I’m not really a fan of this whole ‘web developers aren’t real developers’ stereotype.

                1. 2

                  Yeah that’s a dumb stereotype, I’m glad I didn’t use it.

            2. 1

              I’m also on Fastmail, but I have configured it so that I can use my domain on all the messages. However, Google still blocks my messages (and that is quite annoying when I’ve sent out celebration invites).

              I did set SPF and DKIM correctly. What do people recommend, how to set it up so that chances of Gmail marking all messages as spam are minimized?

              1. 4

                What do people recommend, how to set it up so that chances of Gmail marking all messages as spam are minimized?

                I think the moral of the story is that it’s not us, it’s them. They’re at Gmail are the unreliable email provider.

        2. 5
          My own email troubleshooting story

          I had a similar problem with my Gmail account a few months ago. All email sent (by others) to my personal email on my own domain is forwarded to a Gmail address, from where I read it. At some point, without me having changed my email habits or anything, I noticed that my inbox was missing emails that should have been sent. The emails weren’t in my spam folder – they were just missing. After a few weeks of mild confusion, I checked the mail forwarding logs on my web host and saw that Google was rejecting the forwarded mail.

          I went through the same troubleshooting journey as in the post – finding troubleshooting steps only for G Suite owners, and signing up for Postmaster Tools only to be told that they can’t tell me anything about my domain because I’m not a mass email sender.

          Then, somewhere, I read that Gmail should accept the incoming emails if you verify ownership of that custom-domain address such that you can send email “From” that address. I already had such an authorization from long ago, but I tried deleting it and adding it again, once again verifying my ownership of that address.

          A week or two later, I checked my inbox, and I had received all the emails that I should have received in that period. A week after that, I checked the logs, and found no more bounces. The bounce problem went away as silently as it had arrived. I don’t know if that last re-verification step helped fix the problem or if Google merely improved their spam detection algorithm, but re-verification is worth trying if you haven’t.

          On switching email providers

          I considered switching email providers after that, but eventually decided to stick with Gmail. My main reason was its large amount of free storage for attachments. Another significant reason was that many of the alternative providers (including Fastmail) did not support categorizing emails with (multiple) labels, only with (single) folders. I’m surprised that support for labels is still so rare this long after Gmail popularized the concept.

          I’ve stopped looking for alternative providers for now, but ProtonMail sounds like a good option if you’re willing to pay at least €48.00 / yr for email. It supports labels, its web client is open source, and it has a free plan on which you can try the service first.

          1. 0

            I went through the same troubleshooting journey as in the post – finding troubleshooting steps only for G Suite owners, and signing up for Postmaster Tools only to be told that they can’t tell me anything about my domain because I’m not a mass email and sender.

            This so much. It’s no mistake that not assuming that I’m a G Suite customer is one of the points in my post. Every other commenter on Reddit somehow assumed the very same thing, even though it’s mentioned in my post right away in the second paragraph!


            Then, somewhere, I read that Gmail should accept the incoming emails if you verify ownership of that custom-domain address such that you can send email “From” that address. I already had such an authorization from long ago, but I tried deleting it and adding it again, once again verifying my ownership of that address.

            This is a very bad advice, because they don’t let you add any From email addresses any longer. If you delete yours, you won’t be able to add it back, without giving authentication credentials. And I’m just a bit too lazy to setup separate account just for Google here; and obviously won’t be giving them access to the main ones.

            1. 2

              This is a very bad advice, because they don’t let you add any From email addresses any longer. If you delete yours, you won’t be able to add it back, without giving authentication credentials.

              I’m not sure what you mean by this. I deleted and re-added my custom domain’s email in the middle of June, and Google authenticated my ownership by sending me an email “Gmail Confirmation - Send Mail as roryokane@example.com”. The email contained a numeric confirmation code that I could paste into the appropriate section of Gmail’s settings. I don’t remember having to deal with anything I would call “authentication credentials”. Has the process changed in the last four months?

              1. 2

                Google Mail has started requiring SMTP credentials for any new From addresses since a few years ago or so. It’s not possible to add a new From address without providing the SMTP credentials anymore. It’s quite a bummer, really, because some alias addresses from certain services (like the various OSS projects) don’t even come with outgoing SMTP access.

          2. 4

            Assuming your domain is constantine.su, I wonder whether the issue might be a combination of:

            1. 4

              Hetzner

              Oh boy. Possibly related, possibly unrelated, but at work recently we had to block an entire IP range from Hetzner due to misbehaving crawlers that were not respecting various robots.txt rules and nofollow on internal links. There is a chance that there are probably some legitimate IPs in that range, but not worth the BS we were getting from those crawlers.

              Also seconding your recommendation of rDNS. It has been essential for many, many years now.

              1. 9

                Well in that case you won’t get my mails, or be able to interact with any of my services, or update Quasseldroid.

                Hetzner is one of the few hosters offering dedicated hosting powered with fully renewable energy, and one of the few hosters actually handing abuse reports correctly (as in, not terminating service from any abuse report, but only from court orders, which is useful behavior if you’re getting SWATed by internet trolls, who’ve also found they can use abuse reports for the same purpose)

                1. 4

                  +1 for Hetzner. Their support and service is great! I’m using them as well because of their use of renewable energy. Changed from Linode a while back.

                  1. 3

                    They also aren’t crooks like some of their competitors. I’ve had Scaleway (Online SAS) increase prices for old dedicated servers without much advance notice, either; which is really a shame, because the only reason I bought the server was a low price (one of them I didn’t even have powered on, apparently). OVH appears to have played similar games as well. Hetzner does the opposite for long-term customers.

                  2. 2

                    Not to worry, I will still get your mail and all the rest!

                    AFAIK it the block was various front-end web services. I do not think it even applies to API instances, just those serving up full web pages. So you couldn’t access the various websites from a script that is deployed to Hetzner. And I suppose if you did mail a web instance, it wouldn’t receive them, but the IP block wouldn’t be the only reason for that.

                    Also good to hear another anecdote on Hetzner as a host. Aside from your comment, my only exposure to them is as the host of a hive of over-aggressive and poorly-configured crawlers over the last year.

                    I shared my anecdote because it might be relevant to the article’s main concern: If we had to block one of their IP ranges for web traffic, it is conceivable that other entities have blocked them for email.

                  3. 1

                    Oh that’s unfortunate. They’re a good host. I only moved off them because they finally stopped offering the VPS I was on after seven years.

                  4. 5

                    No, I’ve never used that domain for mail; it’s too long.

                    • Note that this is not a TLD issue, either, because only one of my domains is affected by “low reputation”, the other ones in the very same TLD are not. This has been 100% reproducible over the last few weeks.

                    • Hetzner IP space is not involved here, either — none of these rejects or accepts were over Hetzner IP space. Regardless, you’re ignoring the fact that Google has blacklisted a specific domain name, not the IP address which I’m using, because the very same IP address with the very same email body and the very same TLD, just a different (rarely-used) domain itself in From and MAIL FROM, gets accepted by Gmail, and doesn’t even end up in the Spam folder, either — goes straight to Inbox. Again, this has been reproducible 100% in the last few weeks. And just because some users report issues with their newly purchased servers at a huge provider like Hetzner doesn’t mean that it’s something that’s not supported or isn’t supposed to work. Of course, with enough volume and enough churn, some individual IPs may come blacklisted, which doesn’t mean that it’s representative for the whole space.

                    • And let’s not get all McCarthyism here on Lobsters, shall we? All those stories from 2013 about .su being used for spam and scam have zero credence, and are built around some scammer from abuse.ch shopping the very same story across multiple venues, going as far as Fox News (reprinting AP, I guess). Their suggestion on their own blog at the time was to completely block .su. (I don’t recall ever communicated with anyone from .ch. Should I maybe block .ch? Why don’t we all just block and blacklist each other?) And even if you disregard the potential bias of these databases and unclear methodologies, .su is still one of the cleanest TLDs out there, especially for how many domain name registrations that it has. Your own Spamhaus link reports .us at 33% bad (ouch!), .biz at 24%, .cn at 18,4%, so, .su at 11,5% bad comes out pretty clean in comparison (.com and .net are between 4 and 5%, which is hardly very clean, either, especially given the absolute numbers). This is even if you disregard the potential bias of their methodologies in the first place.

                    1. 2

                      I just re-read your email and it looks like the sequence of events is this:

                      • you configured your server to forward mail from your primary domain to your free GMail account
                      • GMail began thinking a significant portion of emails from your domain were malicious
                      • after a few months of this happening, GMail began blocking emails from your domain

                      I can see how this situation suggests that there should an easy way to get your domain unblocked. I also can see why Google doesn’t make it easy for actual malicious actors.

                      I ran my own email server (on a VPS provider with as many reputation issues as Hetzner) for more than a decade. I stopped not because my emails were being sent to spam or were being rejected, but because running your own email server correctly is hard. I think I can assume you weren’t running an open relay and had SPF and DKIM set up correctly, but without knowing the domain (which you didn’t mention in your original email and haven’t mentioned here) or the contents of the messages you were forwarding to GMail, it’s impossible for anyone to state that Google is overreaching by not accepting email from your domain.

                      1. 2
                        • The server has been forwarding the mail and running cron jobs for many years. Same domain, same IP, same recipient Gmail account. It’s not actually a free Gmail, BTW, because I was duped into believing that the mailbox size is infinite, whereas it has stopped growing at 15GB; so, due to all the mailing list archives, I now have to pay 1,99 USD/mo to be able to continue to receive new mail.

                        • In a newly added cron job a couple of months back, I’ve started sending myself a list of a few dozen domain names which I don’t control over to my Gmail. This has been done exclusively to my own Gmail address. How could you possibly classify a few dozen of plaintext domain names as malicious in a clean room?

                        • You make it a point that I’ve been sending these “malicious” emails for a “few months”, but you’re ignoring the fact that they aren’t actually malicious, nor were these the only emails that were being sent. How was I even supposed to know that one or two of these emails daily, in the presence of dozens of emails not so marked, would turn my domain name into having a persistent “low reputation”?

                        BTW, I do not actually use DKIM, but do use SPF and DMARC; note that these rejected emails do pass both SPF and DMARC; DMARC requires either SPF or DKIM to pass with domain alignment in order to generate a DMARC pass. My forwarding doesn’t appear to mangle existing DKIM signatures, but it would seem that even those emails are rejected, too. (However, emails from my own secondary domains without DKIM but with an SPF pass do get through.)

                    2. 1

                      Just as a semi-relevant data point, I send bulk mail from a server hosted at Hetzner and Gmail doesn’t block that. Gmail blocked that mail at the start and so did several others, because the server’s IPv4 address had been used for all kinds of evil things (the previous customer ran an unpatched wordpress site and was 0wned). But then I

                      • investigated each and every 4xx and 5xx SMTP response, and took care of every problem
                      • signed everything with DKIM and added an explicit SPF yes
                      • made the hostnames match, even ones that shouldn’t need to

                      It took a month or two for the old reputation to age away, and investigating every SMTP transaction for bulk mail was tedious, but the mail has been flowing smootly since. I don’t know what OP is doing, but “being hosted at Hetzner” isn’t a problem in itself, even if you start with your IPv4 address on a half-dozen blacklists.

                      1. 1

                        It took a month or two for the old reputation to age away

                        You don’t really have to do that, BTW. I think it’s pretty standard practice for providers to exchange the IP address in case you get one that’s burned and where it’s an issue for you (it might as well not be for their next customer).

                        1. 1

                          It’s not much time, anyway, and mostly overlapped with the time to investigate other possible problems. Noone had checked the recipient list, for a start.

                    3. 2

                      anecdata… an acquaintance runs a modest volume mailman list (~1500 messages per month, ~150 senders, no idea on receivers… I’m not an admin… 400? 500?) and has since the late 90’s. The host is a Rackspace VPS for the last seven years. Starting around April 2019 several regular long-time participants using GMail and GSuite have seen themselves unsubscribed because Google, from their perspective, silently bounced messages. Since there’s a high concentration of technical people for a non-technical list (nanog regulars, [gx]ooglers, etc.) the theory is that google screens for content and forwarded DKIM signed messages (sort of thing right out of rfc6377) and busted header/signature that are typical of spam but also happens on lists. No one has a solution.

                      1. 2

                        Exodus incoming

                        1. 1

                          I don’t known if you’re joking or not, but I’m also looking at my Spam folder, and there’s plenty of mailing list posts there, too; including from people who are the core developers in the projects the mailing lists are for. And I’m actually even paying Google for all of this — 1,99 USD each month because they tricked me into the infinite storage only to stop growing it past 15GB at one point. (Yandex Mail still seems to offer infinite storage, BTW.)

                          1. 1

                            Haven’t used gmail for over 10 years now. I hosted a few projects by creating their own domain email servers on Digital Ocean. https://github.com/0xboz/digitalocean_email_server I gotta say it is hard, esp. the domain reputation part. It seriously reminds me of US credit scores system monopolized by only 3 agencies. In google’s case, ugh, even worse.