1. 19

    Self-hosted on OpenBSD with OpenSMTPD and dovecot. Self-hosting my emails for over a decade so I’ve been through all ups and downs. I like to run my own stuff, have a maximum level of privacy and always learn new stuff. On the downside, I nearly lost my complete inbox twice (restored from backups, so take backups!), learned very fast that having a primary and a backup MX is different from having two primaries.

    1. 5

      I am also self-hosting using OpenBSD, OpenSMTPD and dovecot for a number of years. I’ve got a primary and a secondary server with SPF and DKIM. My netblock was blacklisted by outlook.com but was easy enough to fix by filling into an online form.

      I also recommend to get yourself onto whitelists like https://www.dnswl.org/.

      1. 4

        I think it’s really cool that you are self-hosted but I have to ask; how are your delivery rates? Do you have DKIM and SPF records? I know it’s quite the challenge to develop a good sending reputation so I am always curious to see how others fare.

        1. 3

          I have SPF records (mainly to make google happy) but no DKIM. However, DKIM is not a hassle to set up. There are plenty of good howtos out there.

          I cannot complaint about reputation, it seems all my email reach the recipient (and yes, also the ones at gmail). I once had some trouble with outlook.com and German Telekom when I had a system at Hetzner because their IP addresses have a very bad reputation. Once I moved away, everything works fine.

        2. 2

          Did the same 4/5 years ago. Never looked back and would not go back to a third-party provider for a million bucks.

        1. 3

          It jumps out to me that you had to include an estimate of your connection speed in the configuration. What’s the behavior if Comcast either gives you a free performance boost, or runs in a degraded state with lower performance?

          1. 3

            There’s not much you can do about that. You could always monitor it with a cron job and update the params/rules with a script I suppose.

            1. 3

              Right, but what happens? How does the system behave in those circumstances?

              1. 7

                If the actual bandwidth is more than what’s specified, it won’t hurt but you will be artificially limiting yourself. If it is less than what’s specified, then it probably won’t work very well because even though the buffers on your router will remain empty (LAN<->WAN both at 1Gbps), the next device in sequence will start buffering and that’s outside of your control at this point. In other words, you want the router that is doing QoS to be the bottleneck.

                On OpenBSD, if you don’t specify the bandwidth param, then it will default to whatever rate the NICs are running at (10/100/1000Mbps for example).

                1. 3

                  Thanks!

          1. 11

            Note that SMT doesn’t necessarily have a posive effect on performance; it highly depends on the workload. In all likelyhood it will actually slow down most workloads if you have a CPU with more than two cores.

            In case you’re wondering, this refers to OpenBSD’s giant-locked kernel. Some parts of this kernel are now unlocked (e.g. network stack) but for some workloads 2 CPUs can be faster than 3 or more due to lock contention.

            1. 1

              Per my understanding, every “physical” CPU can have many cores, and each core can have multiple hardware thread if SMT is supported. So every “hardware thread” is a “logical” CPU. For OpenBSD kernel, does it do special operations according to physical CPU, core and hardware thread? Or just consider “logic” CPU? Thanks!

              1. 2

                As far as I know the SMT threads were simply exposed as additional CPUs to the scheduler.

                1. 1

                  @stsp Thanks for your response!

                  If I understand correctly, disable SMT means cut half the “logical” CPU, right? For example, if the server has one CPU, 2 cores, and every core has 2 hardware threads, in theory, the server has 4 “logical” CPUs. Assume my workload has 4 thread, and every thread is independent and computing-intensive (mostly user-space computation, not involved kernel part, such as syscall, or accessing network, etc.). Currently the workload can occupy the whole 4 “logical” CPUs. But now, if the count of “logical” CPU is halved, and my workload’s 4 thread need to contend for 2 “logical” CPUs. So in this scenario, the workload’s performance should be downgraded.

                  Is it correct? Thanks in advance!

                  1. 3

                    At least when HT was new, it also meant the caches would be halved unless you disabled HT in bios. So if your threads are doing different things they might suffer from it.

                    1. 1

                      As far as I understand, it doesn’t mean that all 4 threads can progress in parallel, it will depend on which unit in the CPU each thread is utilizing.

              1. 2

                I have a TODO file in my home directory for generic tasks and separate TODO files in the target project directories.

                1. 8

                  I have tried surf and ran for for quite some days, but there were a number of problems I had with it.

                  • It is unstable. It crashed way too often for me to be used as my day-to-day browser.
                  • I know no reliable way to do adblocking with it. Just fiddling with /etc/hosts is not really enough; many pages look weird if you do that. Adblockers in Firefox have never shown this problem for me. And keeping /etc/hosts up-to-date is a pain.
                  • Enabling JavaScript for a page only on demand does not work. I want it off by default.
                  • surf rejects a number of SSL websites Firefox accepts for no obvious reason (especially bad with lets-encrypt sites). In contrast to what the article says, surf does support SSL, though. Just not in the stable version I have found.
                  • I have no idea how to create the facility of a plugin I very much like, Flagfox. It displays a little country flag in Firefox' URL bar depending on where it thinks the IP is from.
                  • Bookmarks. I know I can manage them with scripts, but until now I have been too lazy for that, since a proper script would allow me to search the bookmark list and then directly follow the link. I often find myself remembering the title of an article I read and bookmarked, but not the page. So it is required that the bookmark link is stored together with the title and can be selected by either title or URL. As said, possible, but I’m just too lazy for that.
                  • How to forbid 3rd party cookies? How to delete all cookies after quitting the last surf instance?

                  There were probably more points which I don’t remember anymore.

                  Now I’m back on Firefox and have turned on the start-search-by-typing option. This gives me the required level of keyboard navigation I need – I can just type in the text of a link and Firefox will select it. There is a surprising amount of useful keyboard shortcuts in Firefox that is a little bit hidden (for example, by typing ‘ [single quote] with the start-search-by-typing option enabled you search only the links of a page, very useful).

                  1. 2

                    The SSL woes are more than likely because Firefox caches sub-CAs it sees in the wild to handle all the badly configured webservers that do not serve the whole certificate chain when connecting.

                    I really wish browsers did not do this as it masks a problem that to the sysadmin running the site looks just like a temporary glitch in the matrix that they can ignore.

                    I hate the web.

                    1. 1

                      For adblocking, you could use http://git.codemadness.nl/surf-adblock/ with surf-webkit2.

                      1. 1

                        surf rejects a number of SSL websites Firefox accepts for no obvious reason (especially bad with lets-encrypt sites). In contrast to what the article says, surf does support SSL, though. Just not in the stable version I have found.

                        There are some issues with TLS, at least on my Fedora system (visiting the badssl.com dashboard). For example, no host matching, no check for expired certificate, etc.

                      1. 7

                        I really like these kind of writeups, both tedu’s but also the post mentioned from poolp.org. I do think it’s an unfortunate trend that all these lovely things are buried away from openbsd.org or undeadly. Maybe the world needs a ‘Planet OpenBSD’ where all the developer’s blogs are syndicated?

                        planet.openbsd.org doesn’t appear to currently be a thing.

                          1. 3

                            Argh, no full text RSS feed. Why do people persist in doing that (and making me jump through [minor] hoops to work around it)?

                            1. 2

                              In my case, because it would push tons of unnecessary traffic.

                              1. 1

                                I’d rather your feed had a single but fulltext entry than 10 but abbreviated ones. (At least as long as you don’t post twice within half a day or so… which I don’t remember seeing.)

                                1. 1

                                  Do you happen to know which readers replace content when it changes? That was my other concern, that i update something, but readers cache a frozen version.

                                  1. 2

                                    Don’t all of them? I can’t remember seeing one that doesn’t. No doubt they do exist, but I doubt they are at all common. I can remember ones all the way out at the opposite extreme, where they version they content and offer diffs in the UI. NewsBlur has that in some capacity, and there was a desktop one on the Mac that did this – probably old NetNewsWire.

                                    Frozen caches really happen when items get updates after falling off the bottom of the feed. Obviously aggregators won’t see content you didn’t put in the feed… so item inclusion for the feed must be based on update date rather than creation date, if that’s a concern.

                                    (Btw, while we’re here… could you use proper <category>s in the item, instead of putting a line with <p>tagged: at the bottom of your description and then me having to sed your feed to fix that?)

                                    1. 1

                                      Oh? category is a thing? that seems doable. the perils of writing everything from scratch.

                                      1. 2

                                        Yup. I recommend http://www.rssboard.org/rss-profile for reference, which is lamentably difficult to stumble upon serendipitously. It includes recommendations based on surveys of publishers and aggregators in the wild… well, from 10 years ago, but still.

                                        Hm, if that peril is also the reason you don’t have a <guid>… that would be nice, because in absence of it, aggregators must guess how to identify an item as being the same one throughout edits. For flak you can just switch the <link> to <guid> I think (you never change those URLs, right?)… or have both if you worry about edge-case aggregators. For inks, I’ve noticed you number the blocks in the HTML, so you already have an identifier to reuse – keep the <link> and add a <guid isPermaLink="false">, probably with a tag: URL, maybe tag:www.tedunangst.com,2016:inks:37 (where only the trailing number varies; the date is just any point in time you controlled the domain, it can be constant). That would go a long way to ensuring that your updates to items do come through as updates, rather than showing up as dupes. (That’s part of the reason I sed your feed – I’d get dupes all the time when you edited your inks tags, which you do quite a bit, whereas metadata doesn’t figure into the deduping in Liferea, so now I only get dupes anymore when you actually update the item description.)

                                        1. 1

                                          Ah, cool. My understanding of RSS readers is heavily influenced by the one I wrote, which is also odd in its own way.

                                          1. 1

                                            Hey, thanks for all the fixes! Much appreciated.

                        1. 1

                          I’m still disappointed that VAX support is no longer present, but pulling it was the right decision. I guess we’ll always have 4.3 Quasijarus!

                          I just hope SPARC isn’t next to go…

                          1. 2

                            sparc was just removed on OpenBSD -current. sparc64 is still there.

                            1. 5

                              I missed that announcement :( I had heard rumours that it was nearing the door, but didn’t realise it was going to be so soon. Guess it’s the passing of an era as it was Theo’s massive patchset for NetBSD/sparc that was key during the lead up to the fork (for those who’ve never read it, coremail is a fascinating read - lobsters story).

                              I still have a few 32-bit SPARC systems (not used for anything productive - I’m a huge fan of the SPARCstation 20) - I guess NetBSD is the only viable option now.

                              1. 2

                                Keep them. My best recommendation for dealing with potential NSA subversion was putting root of trust on old, esp ancient, hardware that likely predated subversion. One can put a trusted interface in front of them to force simple, mediated communication to the app. Yet, gotta make sure hardware itself isn’t bacdoored. Odds strongly against that on a SPARCstation 20 or a VAX. ;)

                                Got a list of them here: https://www.schneier.com/blog/archives/2013/09/surreptitiously.html#c1762647

                                Note: Another benefit is in chasing the holy grail of automated generation of correct, secure, and portable software. Need lots of ISA’s and machines to test such tooling on. A tool with 10 implementations running full coverage testing on 50 machines from mutually-suspicious countries with same, correct output for every input inspires much confidence. For me at least.

                                Note 2: Intel’s i960 should be on that list. It’s still available in watered-down form. The original was one of their best designs. They’re the assholes that locked up Alpha’s, too. Briefly licensed by them and Samsung. They need to FOSS the last Alpha implementation if they still have it given OpenPOWER and OpenSPARC. I wan’t PALcode damnit! :)

                          1. 1

                            is this an old text? This fact should be mentioned in the title, shouldn’t it ?

                            1. 1

                              I’ve amended the title to include the date.

                            1. 8

                              Looks too complicated to be a useful starting point for anyone not comfortable writing their own Makefile. I think make(1) should be studied in the same way one studies sh(1), yacc(1) etc. Once the main points are understood it is fairly trivial to write a minimal Makefile that gets the job done.

                              1. 1

                                Yep. The whole mess was started at the time I was not quite enjoying makefile. easymake has gave me some sweet time. It’s not quit generic or extensible, though.

                              1. 1

                                “It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration.” - Edsger W. Dijkstra

                                1. 14

                                  QuickBASIC, of course, has almost nothing other than a common set of keywords and sigils to do with the language that Dijkstra was railing against. It’s structured in exactly the sense he supported. It still supports GOTO, but you’ll find it rarely if at all in idiomatic QuickBASIC code.

                                  1. 13

                                    One of the hazards of quoting a bon mot without pausing to understand it.

                                1. 8

                                  https://arcetera.moe/git/pg/

                                  no more, no less, just a pager

                                  1. 2

                                    Cool. What do you use to host your git projects?

                                    1. 4

                                      That’s stagit: http://git.2f30.org/stagit/

                                  1. 1

                                    On OpenBSD I use ksh. On my workstation at work, I use mksh.

                                    1. 1

                                      Just blindly installed it yesterday on a old PC (after I failed trying to reinstall 9front.. 3 times), I was not expecting to find 5.9 but I haven’t been keeping up with the release cycles so I thought I’d just have forgotten about it.

                                      Sadly it’s old enough to miss both VT-x, amd64 and UEFI, so I don’t get to try any of the new goodness.. thought I guess “pledge” works?

                                      1. 2

                                        vmm(4) is not enabled in 5.9 so you are not missing out on that.

                                      1. 1

                                        Skipping signify verification I see. What’s the point in that?

                                        1. 3

                                          Follow the recommended security practices. This guide is only useful to show the details of performing the installation on that particular machine without attempting to replicate information found in the OpenBSD manpages or FAQ.

                                        1. 3

                                          The author should consider hosting his own server and using whatever markdown implementation is comfortable with instead of relying on github.

                                          1. 1

                                            In case you are interested in the Xen support: http://www.openbsd.org/papers/asiabsdcon2016-xen-paper.pdf

                                            1. 1

                                              From a quick look, it appears that the authors use C++ instead of C so the tag is misleading.

                                              1. 2

                                                The c tag is described as “C, C++, Objective C programming”.

                                              1. 23

                                                “It’s the world’s tiniest open source violin”

                                                https://xkcd.com/743/

                                                1. 4

                                                  So what’s the alternative to GitHub that we should be using?

                                                  1. 14

                                                    Phabricator. It’s used successfully by Wikimedia, LLVM, FreeBSD, Blender, and many more communities. A bot to help bridge would be great (e.g. submit a pull request on Github, the bot creates a Phabricator review and directs the submitter there).

                                                    1. 1

                                                      Side note: anyone using Phabricator know of a good Not Rocket Science testing system? I’m a little new to it still and am not sure how to make Revisions work how I want.

                                                    2. 13

                                                      Gitlab. Open-source, with a hosted option if that’s the service you need, but open-source so you can run it yourself, or pay someone else to, and contribute changes if you need them.

                                                      1. 1

                                                        I’ve run a small/mid-sized project on here for the past few months, and I’ve been quite happy with it. Does everything I need, except the primary gitlab.com instantiation does not allow commenting over email, though this can be enabled for private installs.

                                                      2. 9

                                                        IMO, BitBucket is superior to GitHub in every way except for CI/CD integration. Which I believe they are working on. It’s still possible to at least kick off jenkins jobs and what not but it’s a bit janky and there is no feedback yet. Otherwise, I find BitBucket to be very well done.

                                                        EDIT: I’m responding to the above from a feature/quality perspective. Not based on the xkcd cartoon.

                                                        1. 8

                                                          Bitbucket recently got CI status integration. As an Atlassian employee I’ve seen some really cool Bitbucket and CI integration being used internally. I’m sure some of this slickness will be shown using public projects soon.

                                                          1. 2

                                                            you can’t even search in repositories in bitbucket online.

                                                            why do you prefer it?

                                                            i use both, and find bitbucket mostly worse in most web user experience: no searching, can’t see sources vs forks easily, dashboard shows repos and not activity of people you follow as primary thing (i use this on github a lot).

                                                            1. 2

                                                              The two things you mention are two things I basically never use. Most of the repositories I interact with are ones I’m using locally and have in my various tooling already and most of the programming I do is in organizations where forks aren’t really useful at all. BitBucket has robust branch permissions which I make more use of.

                                                              The Pull Request system, which is my main use for any tool like this, is significantly superior to GitHub’s for my usecases. It has Reviewers, real Approve buttons, and Tasks, all of which I use a lot. I don’t really care about the social/activity aspect that GitHub is aiming for, I mostly care abotu a tool around development, which I find BitBucket does a lot better. I also have to use GHE at work which I find very aggravating to use.

                                                          2. 7

                                                            Set up your own server. Use a mailing list for reviews.

                                                            1. 7

                                                              I used self hosted gogs for a bit, but ended up returning to github because I missed the social/community features. Sure, they technically exist on gogs too, but who’s going to sign up for my gogs instance just to say post an issue, or star/what/whatever it?

                                                              1. 2

                                                                One can use cgit and use email for reviews. No need to create an account. Although the barrier of entry may be a little bit higher as not many people use git format-patch/apply-patch, this is more an issue familiarity than something inherent to the process. I like it more than github’s pull requests as it is easier to go back and forth.

                                                                1. 1

                                                                  For open-source projects with outside contributions/contributors, dead right. For my purposes though gogs is ideal. I’ve been using it for personal projects for a few months. Works well enough that I moved all my private repos from Github onto it and saved myself cash money. Fast, simple and regularly updated, often with nice new features that so far have all seemed pretty well-tested and working. For my v low-complexity requirements, natch. YMMV.

                                                                  1. 2

                                                                    If it’s for private repos, why not just have bare git repositories on an ssh server?

                                                                    1. 1

                                                                      Well, sure, in terms of raw git operations, no reason - but private repos can still have multiple contributors, and even single-contributor projects can benefit from organisational tools like the issue tracker, milestones, wiki for notes, etc. Mostly though I just like the UI, the graphical, easily-click-through-able display of a range of projects at a glance, and the visual diffs are simple and easy to get at. Sure, none of this is anything Github/Bitbucket/etc doesn’t do, but it does all the bits that I need and like, well enough for me, for free, on my server.

                                                                2. 3

                                                                  I agree that there’s no shortage of OSS GitHub alternatives out there, and most of them work really well.

                                                                  What kills me is the lack of a hosted free-software alternative to Google Groups. I have a couple projects on librelist.com, but it’s been down for almost a month now, and I haven’t gotten a response about what’s up. Hosting your own mailing list is really easy to screw up.

                                                                  1. -4

                                                                    Well you did not host your own mailing list.

                                                                3. 1

                                                                  Kallithea, although it desperately needs a larger community of contributors to add features like pull requests and CI integration.

                                                                  1. 1

                                                                    I see no one has mentioned Launchpad yet. Launchpad supports git repositories now, and they’re improving it steadily. The Launchpad blog has info on their progress.

                                                                    Keep in mind that I work for Canonical, who started Launchpad and who employ everyone I know of who works on Launchpad development (I’m not really up on who’s doing what, though). There are other organizations who use LP, e.g. Openstack.

                                                                    My own opinions of LP are mixed. I like it, and I used it heavily for a couple of years, but eventually moved to git, and moved off to mostly use GitHub, back before LP added git support.

                                                                    LP’s bug tracking is more featureful than github’s issues. There are lots of other features that may or may not be useful, such as PPAs, translation support, blueprints, etc etc.

                                                                1. 3

                                                                  inetd is pretty unused these days, isn’t it? I can’t even recall the last time I used it.
                                                                  Maybe it should be removed from a few base installs, and put into ports or something?

                                                                  Aside: That is one thing I really like about the OpenBSD project. They actually remove crufty old things sometimes!

                                                                  1. 2

                                                                    OpenBSD has inetd(8) in base. It is just disabled by default.

                                                                    1. 2

                                                                      One handy trick I’ve used it for recently in production is wiring-up port redirections using nc as the server spawned by inetd.

                                                                      1. 1

                                                                        indeed

                                                                        1. 1

                                                                          OpenBSD still includes a CGI daemon too.

                                                                          Like CGI, you just wish that fork, exec, and reap were faster. :-)

                                                                        1. 1

                                                                          I don’t want to be Debbie Downer, but is this page new? heavily updated? or was it just now discovered by someone and deemed interesting?

                                                                          Regardless of which it is, I’m not complaining, just curious if I missed something.

                                                                          1. 2

                                                                            It looks like a tutorial that appeared on BSDNow a few years ago. According to the 2 days ago commit message, it was written by the same person.

                                                                            1. 1

                                                                              Are these pages in a CVS repo somewhere? Where did you find a commit message?