I’ve read the review copy (I sponsored the book writing) and it’s going to be good. I wanted to run my own email for quite some time, but the amount of learning required was scary. MWL thought me to FreeBSD (Absolute FreeBSD) and this book is the same quality.
The unformatted manuscript is almost 250 pages long.
Michael W. Lucas hasn’t updated his OpenBSD book in years. I wonder if he simply stopped being actively interested by it. (I would love to see an updated version of his OpenBSd book)
I don’t think it’s as easy as systems like mail-in-a-box but it gives you total control over your setup and gets you a nice roundcube webmail interface with rspamd protection etc.
Going to sponsor this book since yeah this knowledge should be out there.
I’m thorn because I absolutely love Michael W. Lucas books, I consider them perfectly on-topic here but I agree that Kickstarter links are really annoying.
I love all kinds of authors–even technical ones!–but wouldn’t shill them here.
The front page of Lobsters is slow moving and very valuable. Posting and tolerating ads (whether it’s content marketing, Kickstarter or Patreon pages, newsletter signups, or whatever else) paints a big red target on our community for growth hackers and self-promoters. The incentive for each of them, individually, is to get as many prospective eyeballs on their wares as possible, regardless of what damage it does to the community here.
Even with the folks that I like (some of whom are users here!) letting them do their thing normalizes it for other bad-faith and abusive actors. So, we need to just cut it off at the root and not allow anything of the sort.
(This has been going on for years here, and is one of the reasons the signal-to-noise ratio here is stronger than it would be otherwise.)
EDIT:
For the folks who originally flagged me as “unkind”: that’s your business, but consider whether it is kinder to harm the discourse here than to firmly and succinctly push back on advertising colonization.
Wouldn’t it make sense then to forbid a bunch of domains like kickstarter, patreon, etc.
After all, if a project should really be posted here on lobste.rs, it would be better to have a blog post talking about it instead of the marketing mumbo-jumbo usually seen on those pages between two videos.
Or you can read the book, because it is much more comprehensive than these scripts. These are fine to get started with, but neither gives you the knowledge and understanding to debug and fix your stack when things inevitably go wrong.
While I’m also somewhat excited about the book (coincidentally exactly at a time where I’ve been close to just giving up after twenty years) I wouldn’t discount the “understanding” part from any “turnkey” solution, be it those scripts (that I haven’t checked) or mailcow or docker-mailserver or mailu or sovereign or what-have-you. Configuring the stuff mostly correctly isn’t the hard part imho and debugging the stuff that doesn’t work… I still am not hopeful the book will help because so much is tied up not in proven standards but in reputation management, small details that the big email hosters have implemented correctly or incorrectly and overall it’s kinda dire.
Or maybe I am oversimplifying it because I looked at those configs generated by one of the complete packages I mentioned and they looked fine to me, and I didn’t have to change a lot, they were just like my self-written postfix configs from 10y earlier.
I still am not hopeful the book will help because so much is tied up not in proven standards but in reputation management, small details that the big email hosters have implemented correctly or incorrectly and overall it’s kinda dire.
I would be quite skeptical, except… It’s by Michael W. Lucas. He is very good at finding people who really know what they’re talking about and persuading them to do technical reviews, so tends not to miss things like ‘this is a good overview of what you think the problem is, but actually the hard bit is over there’. He’s also a very engaging writer.
I have recently set up my own Stalwart instance and it makes running your own email so much easier. If someone wants to try self-hosting email, it is a bliss. And it allows me to have “weird” email addresses as much as I want.
It’s not that hard, you just have to add new security features (DKIM, SPF, and so on) regularly. I think there’s probably some reputation system at work as well, as I’ve been running my own mail server for nigh-on twenty years, and only rarely have problems with emails being rejected. Truth be told, I don’t get all the people who say running your own mail server is impossible.
It’s not that hard if the other parties play by the same rules. I have to regularly email people who are behind allow-lists (government stuff, the kids’ kindergarten, soon their school, to name a few), and they don’t even know it. Thus, my outgoing mail is sent through a third party service, because I can’t afford to get on the allow lists myself.
But sending mail to the likes of Gmail and Outlook are problematic too: I have all the things set up. My IP has been stable for years, and there’s no spam coming out of my system. Yet, I have no reputation, because I don’t send enough mail to build one. Therefore, my mail ends up in Spam folders. Thus, I use a third party service to send mail, because then it actually gets delivered, because they have the reputation I do not, and the reputation I cannot get unless I start sending 100x more mail (which I will not).
It’s not just about mail getting rejected, its about mail getting flagged as spam, or silently swallowed, never to arrive at the destination, without you knowing about it, even though DKIM, SPF, and all the rest are properly set up.
I’m happy you have a much better experience hosting your mail. I am not that fortunate.
I agree and have run into quite the issue with my emails (now hosted on the same server for almost 10 years with a clean reputation!) getting blackholed into the spam bin – or worse, simply getting denied (and dropping a sad mailer daemon message in my desktop!). Since it’s just my personal server (and I send about ~1-2 emails a month), there’s no way I can amass the volume of good emails required to get a reputation.
Additionally, some of these services (like Microsoft/Outlook) have online form that look helpful – you can submit a petition in order to get on the whitelist (with some real human processing). In my experience those forms have never worked.
Truth be told, I don’t get all the people who say running your own mail server is impossible.
At a guess, it’s because you don’t try to email the same people that they do. DKIM, SPF, etc. are not enough to reach some corporate inboxes.
I have a domain that I registered in 1998 and I started running a mailserver for it soon after. It was never on any blacklist (at least, none of the public ones). DNS was always setup correctly (including rDNS). I set up SPF, DKIM, and DMARC when they came out. I never had any trouble sending mail to Gmail or most personal email accounts. But a couple of years ago, my mail stopped reaching some corporate inboxes: it would be accepted by their server but then just disappeared. I eventually managed to talk to an admin at one of the companies and found out that they have explicit whitelists and silently delete email from everyone else.
If you don’t ever need to send email to people like that, you’ll be fine. When they are your customers, you’re out of luck.
When I was a university postmaster between 10 and 20 years ago, I reorganized the IP addressing for our mail relay cluster so that the service addresses visible to other mail servers were basically constant regardless of the coming and going of the hardware that provided the services. There were several reasons, partly problems with clients that would do one DNS lookup and keep using the result until restart, and partly anti-spam IP reputation, among other less pressing issues.
As far as I know (having happily moved on from that job ages ago) it’s still the case that the most important thing for mail deliverability is to have an IP address with a consistent reputation for sending messages that the recipient is happy to receive.
Huh, this came in right at the correct time since just recently I had some complaints about Proton which I’m mainly using and started thinking of switching mail providers to something else. His other books are definitely great!
Stephen, I disagree. It’s a complex topic. I don’t think your comment adds anything to the discussion. Nine days ago, you posted this in another thread:
Wow, this is a very rude an un-called-for move
I have a hard time reading your comment except in the same framing.
I’ve read the review copy (I sponsored the book writing) and it’s going to be good. I wanted to run my own email for quite some time, but the amount of learning required was scary. MWL thought me to FreeBSD (Absolute FreeBSD) and this book is the same quality.
The unformatted manuscript is almost 250 pages long.
I’m wondering why he doesn’t use OpenBSD and OpenSMTPD; still, it’s good to have some more resources on this topic out there.
Michael W. Lucas hasn’t updated his OpenBSD book in years. I wonder if he simply stopped being actively interested by it. (I would love to see an updated version of his OpenBSd book)
He stated that 3rd ed is planned.
Awesome news!
Interesting point, I also saw his book on OpenBSD, which made me wonder even more why he didn’t propose it as a mail server OS.
I run my own email server based on the ispmail guide:
https://workaround.org/ispmail-bookworm/
I don’t think it’s as easy as systems like mail-in-a-box but it gives you total control over your setup and gets you a nice roundcube webmail interface with rspamd protection etc.
Going to sponsor this book since yeah this knowledge should be out there.
Flagged. Kickstarter can-rattling has been a long-standing thing we’ve tried to discourage.
Please don’t use Lobsters to shill for products and signup pages.
I’m thorn because I absolutely love Michael W. Lucas books, I consider them perfectly on-topic here but I agree that Kickstarter links are really annoying.
I love all kinds of authors–even technical ones!–but wouldn’t shill them here.
The front page of Lobsters is slow moving and very valuable. Posting and tolerating ads (whether it’s content marketing, Kickstarter or Patreon pages, newsletter signups, or whatever else) paints a big red target on our community for growth hackers and self-promoters. The incentive for each of them, individually, is to get as many prospective eyeballs on their wares as possible, regardless of what damage it does to the community here.
Even with the folks that I like (some of whom are users here!) letting them do their thing normalizes it for other bad-faith and abusive actors. So, we need to just cut it off at the root and not allow anything of the sort.
(This has been going on for years here, and is one of the reasons the signal-to-noise ratio here is stronger than it would be otherwise.)
EDIT:
For the folks who originally flagged me as “unkind”: that’s your business, but consider whether it is kinder to harm the discourse here than to firmly and succinctly push back on advertising colonization.
Wouldn’t it make sense then to forbid a bunch of domains like kickstarter, patreon, etc.
After all, if a project should really be posted here on lobste.rs, it would be better to have a blog post talking about it instead of the marketing mumbo-jumbo usually seen on those pages between two videos.
Or here is a guide/script to do so:
Or you can read the book, because it is much more comprehensive than these scripts. These are fine to get started with, but neither gives you the knowledge and understanding to debug and fix your stack when things inevitably go wrong.
The book will.
While I’m also somewhat excited about the book (coincidentally exactly at a time where I’ve been close to just giving up after twenty years) I wouldn’t discount the “understanding” part from any “turnkey” solution, be it those scripts (that I haven’t checked) or mailcow or docker-mailserver or mailu or sovereign or what-have-you. Configuring the stuff mostly correctly isn’t the hard part imho and debugging the stuff that doesn’t work… I still am not hopeful the book will help because so much is tied up not in proven standards but in reputation management, small details that the big email hosters have implemented correctly or incorrectly and overall it’s kinda dire.
Or maybe I am oversimplifying it because I looked at those configs generated by one of the complete packages I mentioned and they looked fine to me, and I didn’t have to change a lot, they were just like my self-written postfix configs from 10y earlier.
I would be quite skeptical, except… It’s by Michael W. Lucas. He is very good at finding people who really know what they’re talking about and persuading them to do technical reviews, so tends not to miss things like ‘this is a good overview of what you think the problem is, but actually the hard bit is over there’. He’s also a very engaging writer.
I’m sure it will be, but you can’t read it today, can you?
I have recently set up my own Stalwart instance and it makes running your own email so much easier. If someone wants to try self-hosting email, it is a bliss. And it allows me to have “weird” email addresses as much as I want.
Does it require foundationdb? Seems like a bit too much maintenance burden for a single user instance
It does not require it. For single user instance you can use SQLite or RocksDB (default).
Nice! Will give it a go
Receiving mail is easy, but will this book also explain how my outgoing mail won’t get marked by spam by big tech companies?
It’s not that hard, you just have to add new security features (DKIM, SPF, and so on) regularly. I think there’s probably some reputation system at work as well, as I’ve been running my own mail server for nigh-on twenty years, and only rarely have problems with emails being rejected. Truth be told, I don’t get all the people who say running your own mail server is impossible.
It’s not that hard if the other parties play by the same rules. I have to regularly email people who are behind allow-lists (government stuff, the kids’ kindergarten, soon their school, to name a few), and they don’t even know it. Thus, my outgoing mail is sent through a third party service, because I can’t afford to get on the allow lists myself.
But sending mail to the likes of Gmail and Outlook are problematic too: I have all the things set up. My IP has been stable for years, and there’s no spam coming out of my system. Yet, I have no reputation, because I don’t send enough mail to build one. Therefore, my mail ends up in Spam folders. Thus, I use a third party service to send mail, because then it actually gets delivered, because they have the reputation I do not, and the reputation I cannot get unless I start sending 100x more mail (which I will not).
It’s not just about mail getting rejected, its about mail getting flagged as spam, or silently swallowed, never to arrive at the destination, without you knowing about it, even though DKIM, SPF, and all the rest are properly set up.
I’m happy you have a much better experience hosting your mail. I am not that fortunate.
I agree and have run into quite the issue with my emails (now hosted on the same server for almost 10 years with a clean reputation!) getting blackholed into the spam bin – or worse, simply getting denied (and dropping a sad mailer daemon message in my desktop!). Since it’s just my personal server (and I send about ~1-2 emails a month), there’s no way I can amass the volume of good emails required to get a reputation.
Additionally, some of these services (like Microsoft/Outlook) have online form that look helpful – you can submit a petition in order to get on the whitelist (with some real human processing). In my experience those forms have never worked.
[Comment removed by author]
At a guess, it’s because you don’t try to email the same people that they do. DKIM, SPF, etc. are not enough to reach some corporate inboxes.
I have a domain that I registered in 1998 and I started running a mailserver for it soon after. It was never on any blacklist (at least, none of the public ones). DNS was always setup correctly (including rDNS). I set up SPF, DKIM, and DMARC when they came out. I never had any trouble sending mail to Gmail or most personal email accounts. But a couple of years ago, my mail stopped reaching some corporate inboxes: it would be accepted by their server but then just disappeared. I eventually managed to talk to an admin at one of the companies and found out that they have explicit whitelists and silently delete email from everyone else.
If you don’t ever need to send email to people like that, you’ll be fine. When they are your customers, you’re out of luck.
True, thanks for the reasoned reply. I’ve “only” been using the server for my personal e-mail.
When I was a university postmaster between 10 and 20 years ago, I reorganized the IP addressing for our mail relay cluster so that the service addresses visible to other mail servers were basically constant regardless of the coming and going of the hardware that provided the services. There were several reasons, partly problems with clients that would do one DNS lookup and keep using the result until restart, and partly anti-spam IP reputation, among other less pressing issues.
As far as I know (having happily moved on from that job ages ago) it’s still the case that the most important thing for mail deliverability is to have an IP address with a consistent reputation for sending messages that the recipient is happy to receive.
I have his other books, they’re a lot of fun. I’ve enjoyed every one of his talks is seen at Penguicon, it’s cool to see his book pop up here.
Pledged 15$. Wanted to get the physical copy, but it doesn’t ship to my country.
All the best!
It won’t cover notqmail but I can hardly hold that against MWL. Absolutely the sort of thing I support.
Huh, this came in right at the correct time since just recently I had some complaints about Proton which I’m mainly using and started thinking of switching mail providers to something else. His other books are definitely great!
[Comment removed by author]
Glendower: I can install postfix and send emails into the vasty deep.
Hotspur: Why, so can I, or so can any man;
But will the recipients receive them?
Stephen, I disagree. It’s a complex topic. I don’t think your comment adds anything to the discussion. Nine days ago, you posted this in another thread:
I have a hard time reading your comment except in the same framing.
This reply is exactly why YOU should read the book.