1. 42
  1. 28

    It’s not black and white. You don’t have to choose between bending the knee for Google and Microsoft on the one side and running mail on a Raspberry Pi in your cupboard on the other. You can pay a normal hosting provider like Hetzner and get decent space, a domain and mail for like 1,50€ a month. If you really care more for federalisation get a managed server which gives you an own server and IP. The provider will take care of software updates, IP reputation, etc. while you still can set it up however you like.

    The central goal should be that people start questioning free services, because they sell your data. Mail is indeed hard for non-enthusiasts, but there are options. It’s not too much to ask to pay a bit of money for the peace of mind that there’s no business incentive for the company to sell your data.

    And even this little step keeps e-mail federation alive.

    1. 8

      You cannot reliably send from Hetzner IP to the world. Or at least I could not from 2018 to 2020. And thus I used Amazon Simple Email service for machines I used to run in Hetzner’s IP space that needed to send email out.

      For similar reasons my RPi at home sends from Mailgun.

      Maybe your experience was better.

      1. 14

        I’ve had no issues with that for over a year, even to Google and Microsoft. You have to set up SPF, DKIM, DMARC, MTA-STS, etc. though.

        One of my customers sends out thousands of bulk mails per week via a managed Hetzner server I set up a year ago without issues.

        1. 5

          My experience with Google has been that if you just have SPF+DKIM+DMARC they’ll still treat you as suspicious… until a Gmail account sends some mail to your domain.

          1. 3

            I agree. I was constantly running into the spam trap when running on VMs on Vultr. But since I moved my e-mail from openbsd to a container running on Hetzner, I’ve had way fewer delivery problems. I feel like sending from that server is even more reliable than Mailgun.

            The only downside is there are certain people I cannot e-mail from a German server, specifically US based mortgage/loan officers and real estate agents. Most use services that restrict connections to their websites, listings and even e-mail to US. The DNS people who use Google mail doesn’t even resolve; and if I pump the DNS traffic to a US server, it won’t connect!

            Other than that, it works pretty well. I will say calmav and spamassassin will let more spam in, which is not good in a corporate professional environment for people who don’t know how to deal with it (going to the author’s/article’s point about business use cases). But for personal use, it works really well.

            1. 6

              Hetzner cloud has a new presence in Ashburn, Virginia. Maybe that helps with the US based recipients

              1. 2

                I bought one of their dedicated servers (It’s under 60 euro a month for an i6700k, 64GB of ram, 1TB nvme, 2TB hdd). It was a worthwhile investment, giving me more power for less money over 4 Vultr VMs.

                It doesn’t seem like the have dedicated servers in Virginia yet. If they ever started offering them, it might be worth looking into.

            2. 3

              One day you’ll get someone (or a service) hitting a spam trap by accident and some destination using spamcop results strictly - and your mail is going to get blocked. Not a huge issue or very common, but it happens. More often with a shared endpoint rather than a dedicated one.

              Alternatively your hezner neighbour can get on https://www.invaluement.com/#sip24 or some other ip-range list.

            3. 13

              No idea about Hertzner, but Vultr requires you to send them an email to unblock outbound port 25. As a result, their IP ranges manage to avoid any of the major block lists and you can get a small VM that can send email from $3.50/month.

              1. 3

                Even with that Vultr deterrent, I found Hetzner IP ranges way more reliable; moved my e-mail to them a few months back (see comment above)

              2. 4

                I’ve had a good experience sending from a Linode VPS. And that’s sending bulk commercial marketing, which you’d expect to have the most trouble with.

                1. 2

                  Are you are it was the IP, not missing reverse IP, DKIM, etc.?

                  I’ve heard that argument a couple of tubes, but so far haven’t seen it myself. Maybe reused IP address that previously was used for shady stuff?

                  Many email systems (Google’s) tend to provide a reason why something isn’t working. Usually it’s missing one of DKIM, SPF or reverse DNS. All easily fixable.

                  1. 2

                    I am pretty sure I know my business when it is related to email. If you lurk to mailing lists like mailop there have been a few flames about such issues. I am not passing blame on anyone here mind you. I solved my problem by relaying to Amazon’s Simple Email Service and everyone was happy.

                  2. 1

                    You cannot reliably send from Hetzner IP to the world.

                    It’s not impossible since I’ve been doing for years, with multiple IPs. Though we did end up on a preemptive blacklist so it does take some vigilance to delist promptly when you initially get an IP.

                    It works everywhere except to get mail on Microsoft’s servers, but judging by other people’s experiences, that’s probably not a problem on our end but just Microsoft’s standard monopolistic practices.

                    1. 3

                      I will agree with you that it is not impossible. But on the other hand mail routing has to be your primary business in order to be proactive and not reactive because you got included in blocklist X or some other service. If mail routing is not your primary business, negotiating with a big mail service operator is time consuming and distracting from what you are really paid to do.

                      Let me also note, that I’m not pointing fingers to Hetzner. They do a fantastic job at providing a service that does not cost an arm and a leg. s/Hetzner/any other cloud provider that does not have a relay service/g

                      1. 1

                        I agree with you somewhat, but I do want to point out I run my email server together with a friend. We use the server for business purposes and personal matters, so email routing is not a primary business. This works pretty well for us! I can’t say we’ve had significant problems with it nor that it is particularly time consuming.

                        The only destination we regularly have problems with are Microsoft-owned domains. Emails sent there regularly end up in spam for no apparent reason at all.

                        So from my perspective, if Microsoft would stop being negligent, or a bully, personal email servers would be perfectly smooth and viable.

                2. 20

                  Once upon a time I wanted to be the best sendmail.cf person in the world. I gave up in 2014 (not a sendmail fanatic, but the expression surely underlines the drive). You either go and work for an email provider if you want to be that person, or you find something else to be the best at.

                  I used to run a system for 100K users. sendmail, MIMEDefang, clamAV, SPF, graylisting and some homegrown software to detect spam (especially outgoing spam from leaked accounts since the spammers had even figured out our shifts) and the like.

                  What were the two most common complaints?

                  • SquirrelMail is not as beautiful as GMail (which is free)
                  • You are not giving us as much disk space as GMail (which is free). We even showed them how to use GMail as their “webmail” of sorts if they wanted.

                  I tried to outsource the service to two of the big providers with presence in my country. The price tag was outrageous. The service is still running with minor changes from the current operators, and the complaints are roughly the same.

                  1. 3

                    I was always confused about squirrel mail. I had that back in school, so a bit less than 20 years ago and even back then it was ugly, inconvenient, and even as a teenager I knew of nicer alternatives.

                    Never really understood the choice. It’s a bit like Docker Swarm to me and saying container orchestration isn’t nice.

                    There also were already better alternatives than sendmail.

                    So maybe that could have been at least part of it?

                    1. 5

                      When you run large email systems like the one I used to, or the one the original poster talks about, it happens that you run a large mail hub (100K users in my case) while in fact you are not in the email hosting / routing business. As such, by definition you are running the service understaffed (let’s not go into why you run the service in the first place, the service is there, it got big, you have to deal with it now within the economic freedom that your management provides).

                      Such services have too many moving parts that you need to keep in order with less staff than it should be there (like running a 24x7 service with two persons). Moving parts include the servers for relaying, for POP3/IMAP, the storage system, the user directory, the backup, the legal framework within you operate, the webmail, antispam, antivirus, other antiabuse specific to a certain user etc. All these change daily. Compromises are in order if you want to keep it running.

                      One such compromise is SquirrelMail. It is ugly, but it is simple, dead simple to maintain (at least until 2014) and it works. Sure there are better looking alternatives, but you need to factor in the operation cost for an organisation, like your school, that is not selling email services. That’s how you get stuck with SquirrelMail.

                      There also were already better alternatives than sendmail.

                      I seriously do not want to go into this debate. I have run every opensource email server that was in use until 2014. I’ve even run Postfix in production when it was called VMailer. Exim, Qmail also. I even tried MeTA1. You want to tell me about security? I will tell you OK, I’ll have a Qmail or Postfix in front that relays to firewalled sendmails. For decades there was nothing that could beat sendmail’s DSL for doing stuff. And if you understood sendmail, you could understand any email system. And when you did, you missed the DSL (that looked like modem noise) to do stuff. The only thing that came to beat this and was equally usable with sendmail and Postfix was the MIMEDefang milter that allowed you to write complex stuff in Perl instead of sendmail.cf.

                      So maybe that could have been at least part of it?

                      So no, it is not sendmail, SquirrelMail, UW-IMAP, Cyrus, Courier, Dovecot, Perdition or what not. It is all that the original poster described. Which basically boils down to running an email service when it is not your primary job and you have to talk with systems that it is their primary job.

                      It becomes exhausting.

                      1. 3

                        I know it’s a bit off topic, but I’m curious. How does one end up with 100k mail users when it’s not a email/routing business?

                        1. 2

                          By working for an Engineering Association that has as members all the Professional Engineers in their country.

                          • Here’s an idea: The IEEE provides email forwarding service to its members. We should do the same, only better (~1999).

                          Interestingly, sometime after 2014 (when I left the Technical Chamber of Greece) the IEEE outsourced their email forwarding service to GMail in order to provide full email service to their members.

                  2. 15

                    A lot of the issues in the article and in this thread are about web mail. On the IMAP side things are pretty great ♥

                    1. 13

                      And there’s a lot less need for webmail now than ten years ago. Most folks seem to check mail on mobile device (where a native app is available) rather than want to use a web browser on an untrusted machine.

                      1. 2

                        I think that’s where the “artisanal” comes in, for some definition of artisanal. Most folks just want to use Webmail. I’m okay with just using IMAP again because I have a smartphone but before I had a smartphone I’d often be running around using different shared computers and Webmail was almost an absolute must. I am pretty happy that I can use IMAP everywhere these days though.

                        1. 2

                          I’ve even heard home cooking described as “only artisanal”. But call it what you like, I’m more interested in hearing about problems and solutions for email administration than using semantics to dismiss it. “It’s artisanal and artisanal things are not widely worthwhile, only niche” is kind of a backwards, harder-to-argue with way to say “It’s not widely worthwhile, only niche, sorta like artisanal things are”. That’d be a lot easier to confront since it either is worthwhile already (I’m pretty happy with my dovecot/postfix setup) or it needs to become worthwhile (I’ve seen a few attempts like mailcow or mail-in-a-box or maddy, but I don’t have any experience with them).

                          Every single individual absolutely should not run their own instances, that’d be super wasteful, but, us handing over the future of digital infrastructure to be operated and controlled by private interests rather than democratically run organizations isn’t a great future either. I’m pretty happy with my handful of users, that’s pretty lagom for me.

                          1. 2

                            I don’t see “artisanal” as a good or bad thing, just something with a different set of tradeoffs. If anything, I just think we need to be honest about the tradeoffs. Even in the email space, there’s a lot of tradeoffs. You can pay for outbound SMTP access, but still run your own MTA and/or IMAP server if you’d rather not deal with the configuration necessary to keep your mailer’s IP in the clear. You could use something like Fastmail and just use POP3 to download your mail elsewhere and administer that. There’s lots of tradeoffs.

                            I certainly don’t think there’s anything bad about hosting email, just that we need to be honest about the effort needed. I ran my own mailserver for years, even when I needed my own Webmail all the time. I still do a bunch of overly complicated Email stuff for my own personal use though just let Fastmail do the spam checking and outbound reputation maintenance for me for the most part.

                            1. 2

                              I got the impression that the article was using artisanal as a shorthand for “don’t bother with this”.

                              Running email is difficult (it’s not continuously difficult: there are problems, we solve them, then it keeps ticking. But it’s difficult in the beginning) and yes, we need to be honest about that and I’m so grateful that there are efforts to change that. I don’t think we should thow in the towel to Gmail and Hotmail, is what I’m saying.

                              You’re right that there are a lot of intermediate solutions, but that’s a great thing. It’s great that there’s this spectrum between home server and full on outsourcing it. For example, I use a VPS so I don’t have to try to get reverse DNS at home. The shell, /etc, dovecot, and postfix is all ours but the machine and its hardware is abstracted from us. (Sometimes I disparagingly describe VPS as “all the downsides of cloud computing plus you need to do all the work”, but the upside is that we could move it. We are less locked-in. I have a home server too and sometimes I’m like “why amn’t I running all my stuff from here instead? It’s got pretty much the same uptime.”)

                        2. 2

                          Some webmail packages I’ve heard of: squirrelmail, roundcube, and alps. I’ve tried the first two but it was 15 years ago. I prefer IMAP fun. ♥

                          I use a combination of notmuch (for longer messages, for scripts/automated tagging and sorting and acting, for indexing, for years of history) and Delta Chat (for quick three.sentenc.es style replies). If I needed web access to email (in some extremely implausible hypothetical when I do have access to trusted browsers but I don’t have access to my own device) I’d set up shellinabox or similar and use notmuch that way. I was carrying a USB stick with a windows SSH client for a few years in the early 00s. Not that that’s a easy-to-learn options for the masses, it’s not, and I’d be happy to see work being done on easy clients for web or mobile.

                        3. 6

                          You can run a little artisanal one and feel happy about it, but it will not at all measure up to the quality of systems run by eg Google and Microsoft.

                          This does apply to basically every techno out there: Web servers, DNS, container platforms, online storage, backups, visioconference, … you name it.

                          The “artisanal self-hosted” versus “huge, company-owned” software stack has been the case for many years now, it has never been “increasing” IMO.

                          1. 3

                            Yeah, I wanted to say something similar. I think the discussion here applies to everything but in different ways and I don’t know why people seem to mostly discuss it for email.

                            IM is way harder than email. You basically have XMPP which I use a lot but it’s basically shite or …nothing. You can do IRC which works okay but without most of the features or you use one of the big services.

                            Calendar sharing, Nextcloud sort of works. But only sort of.

                            Web servers are pretty doable but also way harder to set up than should really be necessary and let’s be honest, who even needs a webserver anymore?

                            The list goes on. Maybe this post would be more interesting with a list of cheapish well managed services for people who at least want to move away from the “I am the product”-space.

                            1. 6

                              Web servers are pretty doable but also way harder to set up than should really be necessary and let’s be honest, who even needs a webserver anymore?

                              lost me here, I thought loads of people were still running their own little webservers.

                              1. 1

                                Yes, I do too but mostly for other services: e.g. webmail, gittea, nextcloud. For just hosting a personal website it’s kind of overkill.

                              2. 5

                                You basically have XMPP which I use a lot but it’s basically shite or …nothing.

                                There’s also Matrix these days, in case you missed it.

                                1. 2

                                  True! It’s not better though.

                                  1. 4

                                    I have to disagree on that account. I was previously a heavy XMPP user and the smoothness of Matrix enables usages several classes above of what I was able to achieve with XMPP.

                                    Which aspects to you consider lacking?

                            2. 5

                              I switched from gmail to simple-nixos-mailserver running on Vultr a couple years back and I have had zero maintenance issues. I spend near zero amount of time thinking about operating it.

                              1. 5

                                Strong agree; in my experience, there just isn’t any good FOSS email software that enabled me to be productive, especially when it came to things like managing email rules competently.

                                Fastmail lets me hit “create rule from message” and sort my mail far more effectively, especially since I can do it on my phone instead of having to SSH in and write a Sieve script…

                                1. 2

                                  I prefer hosting my own e-mail, but I do agree if you’re going to use hosted e-mail; go with Fastmail or Protonmail or some other service that you pay for, making you the customer and not the cattle.

                                  There is no increased privacy using those services of course. E-mail is a shit show when it comes to privacy or confidentiality, but I do trust a paid provide to not censor inbound e-mail. I don’t trust Google or Microsoft (LinkedIn is already silently deleting peoples’ posts without notification)

                                  1. 2

                                    This won’t fit your mobility requirement, but Emacs + mu4e (the client) + isync (to fetch mail from the server) is pretty good. You can write your filters in lisp which is at least slightly better than Sieve, and your user experience is only limited by your imagination and lisp skills.

                                    But it’s also a niche solution :D

                                    1. 2

                                      Dovecot and Thunderbird (with an extension, as with everything else useful in Thunderbird) support the ManageSieve extension, which allows you to edit Sieve scripts directly from the mail client. This is a bit better than editing over SSH (the server can be asked to validate the script before you save the rules) but I’ve not found anything for writing Sieve scripts that’s better than writing them by hand. Outlook, for all its faults, is quite good at suggesting rules from a message and I’d love to see that kind of functionality in ManageSieve clients.

                                    2. 5

                                      I ran my own mail server for more than a decade from the mid-2000s to the mid-2010s. It was expensive for me when I was just starting out. I started with a $20/mo VPS before finally stopping when I had a $40/mo dedicated server. I spent probably 5 hours per month “maintaining” that server in a world before configuration management software existed (at least to me; I learned of Chef and Puppet circa 2013). More than half of the server’s RAM was dedicated to content scanning, mostly antispam and antivirus. I had three active inboxes plus aliases for another 5-10 people over that time. I spent a lot of time trying to avoid catastrophes and spending nights fixing things when suddenly someone calls me and says that they’ve not gotten an email from any Gmail addresses for two weeks.

                                      What ultimately killed my motive to maintain it was when a series of failures of backup systems and hardware took down my server at a very inopportune time. I learned the hard lesson in testing backup restorations and fortunately only lost about 2 years’ emails for myself – I’d configured both of my other inbox users email clients to download and cache everything locally. My email client was configured similarly, but I was using a proprietary email client (AirMail) with an undocumented mail storage system (SQLite backed) and a developer unwilling to fix a bug/implement a feature that allowed access to an inbox when the client couldn’t connect to the server anymore. I started on a recovery tool but decided that it wasn’t worth my time… I almost never look at my archive anyway and I’ve never once in my life needed an email that was more than a year old.

                                      I happily pay small companies to host things that work best at scale. I’d love to take the rugged individualistic approach to email but I’ve got other concerns, so I’ll go collectivist on it and pay small businesses participating in and improving the email ecosystem. Fastmail gets my dollars these days.

                                      1. 7

                                        Honestly, I don’t see the appeal of hosting your own email.

                                        The process itself isn’t very rewarding (IMO) and there are a ton of good (and cheap) hosted options available.

                                        1. 3

                                          I think setting up YunoHost on a VPS provider, and hosting all of your family member’s emails and file/picture/link sharing needs (and may be ‘family networking needs’)– is the ‘right thing to do’. It should be like part of a ‘right of passage’ for a family promotion to the ‘IT geek’ ranks :-).

                                          It is not difficult, it works, it is rewarding, and it betters family relations.

                                          With regards to spam filtering, using email aliases for your shopping/one time engagements (like manyme.com) is a better practice, than constantly ‘beefing up’ a centralized email infrastructure with spam filters that read your emails….

                                          Certain things that you invest a significant your time in, should benefit more than one person. From that perspective, hand building an email server from a bunch of micro-components for just one person’s use – might not be very rewarding…

                                          1. 2

                                            For me, the appeal is knowing I can and having the ability to do so if I needed. Maybe sort of like trying to grow your own food. Yes, I can’t compete with the current food supply chain, but knowing how to do some things yourself is valuable even if sometimes unnecessary or overkill. In a changelog interview Richard Hipp of SQLite said something along the lines of “taking on responsibility is freedom”. He said it better but that was the idea and it really stuck with me.

                                          2. 3

                                            I know this is a bit “out there” but for some people “running your own” means it’s not on a cloud VPS, it’s running “on-premises” on a server you own. There are tons of easy ways to set that up, things like mailcow, mailu, and yunohost to name a few. But even then, as far as I know, those tools won’t make the set up and maintenance process easy enough that “anyone can do it”; you still have to manually set up the DNS records, configure port forwarding, most likely even configure some sort of VPN or socks proxy because your home ISP will block port 25.

                                            I think the barrier to entry is this high because there’s no money to be made on self-hosting on a small scale, the amount of work required is just astronomical compared to any potential for revenue. It’s all built on volunteer effort, and too often the volunteers are only willing to work pro-bono on things for their own community; the “artisan” technical communities who already know how the stuff works and what the constraints are. So no wonder it’s “artisanal”.

                                            IMO if I don’t like this direction the tech world is going, where I can’t have “nice things” unless they are allowed to form huge scary power structures, then I have to start volunteering to build things which are exceptionally easy to use, easy to manage, and that work for everyone. It’s extremely difficult work! It’s not something I can do in my spare time. So I quit my job to work on it. Maybe in a few years I’ll have my own email server that’s as mundane and mass-produced as possible. Something that you can copy and paste on to your own “raspberry pi running in a cupboard” without all the blood, sweat, and tears.

                                            1. 3

                                              I dislike how that article puts a huge amount of claims on the table, but completely fails at providing any example, technical reason or even an explanation why that would be other than “they are bigger”, which in my opinion isn’t a strong one. You can say that be about anything.

                                              You can claim that it’s artisanal to make video games, cook for yourself, make music, create an app, any kind of software, run a medium like a blog, because for all of these there’s companies which have more money, time, etc. to Invest into it and make all these things more secure and better quality.

                                              Some counter arguments. Smaller, less complex systems that don’t just add features for a longer feature list tend to be more secure. An example is the OpenBSD project and pretty much all software it does. Google, Amazon, Apple, IBM, Microsoft, RedHat, you name then have more resources, experts, time to do these more securely, convinient, etc. and actually most of them tried in some way.

                                              It’s just ridiculous to act like Gmail didn’t have massive problems (as in downtime or things but being usable) in the last two decades that you could have avoided by simply having had a simple setup yourself.

                                              I don’t even claim that the major statement, that running your own email is artisanal or that you should always run your own email. But bold claims without any real discussion of them and handwavy arguments like “but Google” don’t really feel right for a technical community.

                                              So here some examples that I thought of to support the security claims, even though I then very much disagree with “it could never be done” which is always just a wild claim and a bit of a red flag.

                                              Two things that tend to be missing from small email setups in terms of security that benefit from AFAIK non standardized Implementations on mail clients which to me seem like the biggest benefitn of bigger providers are two convenient factor authentication and heuristically detecting bad logins (strange country IP for a user, etc.). I think these would be solvable by adequate open source software. Of course this would have to be done cross platform and in a way it makes sense (which is different on a web and mobile client, see Gmail).

                                              1. 3

                                                Not a new take on the subject - Seeing stuff with this framing of self-hosted email for years, while me and a couple of friends share a self-hosted postfix server for years that’s handled 100,000s (maybe millions?) of mails without any serious issues, and honestly not even much ongoing maintenance…

                                                1. 3

                                                  Having had to set up outgoing email for work, I concur.

                                                  (Incoming email is corporate GMail, which I wholeheartedly recommend.)

                                                  Incredibly easy part: setting up Postfix on Ubuntu, sending SMTP over SSL. Absolutely the easiest thing ever.

                                                  Near-impossible part: getting Google, Microsoft and Yahoo! to accept the emails, even with full SPF/DMARC/DKIM in place. This involved multiple supplicant emails sent to unresponsive inboxes at the providers in question.

                                                  Short answer: spammers mean we can’t have nice things.

                                                  (No, not sending email to the largest webmail providers was absolutely not a feasible option, and I’m appalled at how many geeks thought this was a sensible suggestion to make.)

                                                  Email is three huge webmail companies and a few stragglers now. And I still get 200+ spam a day in my personal GMail.

                                                  1. 2

                                                    As true today as it was 1, 2, 3, 5, 8, 13 and more years ago.

                                                    I’ve still be running my own, with a friend, since 1997. No plans on stopping.

                                                    1. 2

                                                      I’ve run a mail server on my home setup fine. Webmail is still a bit blech, but using actual apps gives a good enough/sometimes better experience. I think the biggest issue is making sure reverse DNS is configured, and in my small experience, ISPs actually know how to set up a reverse DNS entry nowadays, which helps a lot with deliverability issues.

                                                      1. 2

                                                        Things like your webmail, your spam filtering, and almost certainly your general security will not be as good as they have.

                                                        This is really the key. It takes a lot to have great security and spam detection. It is almost certain Big Tech Co is able to do this better than anyone else.

                                                        1. 2

                                                          Honestly, in my experience spam filtering with rspamd on my mail server is more effective than gmail’s spam filtering. I don’t know what it’s doing, but rspamd is genuinely magical.

                                                          The point about security is well made of course - it’s very hard to match Google here.