1. 3

    Note that U2F keys are a new OpenSSH key type, so the server must support it too.

    Anyone know why this would be the case? I was hoping the keypair could just be presented to the server as a regular ECDSA key.

    1. 4

      It was not possible to enable U2F keys using the existing SSH ECDSA key format because, despite U2F devices using ECDSA for signatures, the signature format differs from the plain signatures used in SSH by including some additional fields.

      1. 1

        I noticed that, but I didn’t understand why those additional fields are there, or why it would cause a problem.

        My best guess now, after seeing ClashTheBunny’s reply, is that 1) the additional fields allow for at least the signature counter, and 2) the existing signature validation mechanism doesn’t tolerate unknown, additional fields (which would be a reasonable choice, really.)

        1. 2

          In most crypto schemes the thing that is signed is actually a digest of some more complex byte structure. That they don’t match here is not surprising. U2F for example requires adding origin (domain name) and a challenge. Even if the required data were the same the binary encoding could be different and that alone would make the signatures incompatible.

      2. 3

        I get the question now.

        I wonder if it is to “promise” that the secret key material is on a hardware store and not a software store?

        1. 4

          It does seem that there is attestation PROTOCOL.u2f:

          U2F also offers a number of features that are attractive in the context of SSH authentication. They can be configured to require indication of “user presence” for each signature operation (typically achieved by requiring the user touch the key). They also offer an attestation mechanism at key enrollment time that can be used to prove that a given key is backed by hardware. Finally the signature format includes a monotonic signature counter that can be used (at scale) to detect concurrent use of a private key, should it be extracted from hardware.

      1. 3

        This one from 2.5 Distraction 2: minimizing privilege really made me think, especially in relation to the benefits of sandboxing techniques like pledge(2):

        I have become convinced that this “principle of least privilege” is fundamentally wrong. Minimizing privilege might reduce the damage done by some security holes but almost never fixes the holes. Minimizing privilege is not the same as minimizing the amount of trusted code, does not have the same benefits as minimizing the amount of trusted code, and does not move us any closer to a secure computer system.

        If I understand correctly then his argument is that the principle of least privilege is no more than a security mitigation, limiting collateral damage at best. Whereas proper separation and having a bug-free TCB are the only security solutions. Or in other words an unprivileged process (from the operating system point of view) that handles all incoming mail for all users on the system still breaks the users security requirements once exploited since the process has access to all incoming mail of all users. If the same program with the same security vulnerability was configured to only handle one incoming message and then exit (and thus be forked for every incoming message), it could never break the security requirement of any user, even when exploited.

        1. 2

          That’s a very interesting observation, and I feel sort of reveals a difference in the underlying assumptions: do you expect that There’s Always One More Bug, and so the mitigations are useful, or do you expect you can make a truly secure core kernel of your system and design accordingly?

          1. 2

            Whereas proper separation and having a bug-free TCB are the only security solutions

            That is how the strongest solutions implement least privilege. The separation architecture with whitelisted communications starts with no privilege before giving just a few for IPC. It’s doing the same thing. From there, bug-free TCB’s are hard enough that high-assurance security reduced bug-free to tiniest components (eg separation kernel) that they could. Then, they tried to expand what those capabilities could handle with medium-assurance techniques, like memory safety and covert-channel analysis, handling other components with less impact on security policy.

            In fact, the Least-Privilege, Separation Kernels achieved higher assurance than Bernstein’s designs given their proofs covered all modeled states for both correctness and leak freedom assuming trustworthy hardware. Then, someone found a problem in his before others found thousands of bugs in kernel it ran on. Then, there’s no covert-channel analysis of either qmail or Linux that I know of. Least privilege designs with EAL7 process still wins over his highly-respectable work.

            It lost in long run due to relying on COTS hardware with vulnerabilities and leaks. The assumption was never correct on high-performance, production CPU’s. That was predicted back in 1995, though, after an analysis of Intel CPU’s for leaks called for leak-proof components in processors. Verified processors such as VAMP DLX and Rockwell-Collins’ AAMP7G were created with latter in real-world use. They’re not desktop or server grade, though.

            1. 1

              I was hoping you would chime in, thanks!

              In 5.2 Bernstein does mention that his assumption that the CPU does not leak secret information from one process to another, is debatable.

              1. 2

                “I am also assuming that the CPU does not leak secret information from one process to another. This assumption is debatable.”

                It good that you brought it up. I either forgot or sort of glanced over that statement. I wonder what he was thinking when he wrote it. Quite a few people worried about things like prior state of the registers leaking information or CPU just glitching. Was it as little as that or something leaning toward bigger or deeper leaks? We won’t know. Anyway, just add that observation to his security cred. :)

                Far as non-leaking CPU’s, this pdf is a good one among those that aren’t paywalled. Always remember, too, that much of the “proof” benefit can be gained by precise specifications, detailed reviews, static analysis, and lots of testing. The proof parts are impressive and capable but not necessary if aiming for qmail-like assurance (Medium - stop many attackers). 80/20 rule applies in INFOSEC, too, if not aiming for perfection.

          1. 2

            Actually, the linked article: https://blog.cloudflare.com/warp-technical-challenges/ offers more in-depth description of their hurdles and how they overcame them.

            1. 24

              Fortunately, WireGuard has a three byte block in its header which is not currently used by other clients. We decided to put our identifier in this region

              I believe the technical term for referring to this is “a dick move”. First CF decides to just go off and make their one wireguard impl rather than contributing to the existing opensource ones now they’re not even following the protocol? That’s not very neighbourly. I would have thought that CF would have had to deal with the problems of protocol ossification enough to know that they shouldn’t do this.

              1. 4

                I want to share my thought process here. At first, I sneered “If their 1.1.1.1 app (client) never connects to any server other than cloudflare, what’s it matter?” Please note that I prefer to use clients that use protocols to connect to servers of my choosing. I like to participate on the Internet rather than merely consume…

                But then I thought about how the FTP protocol changed over time as network position of FTP clients changed over time: first on mostly single-activity-at-a-time machines, to multi-user multi-process machines, to machines on networks that were not professionally managed, to machines behind consumer NAT devices. Some of you just know what I’m talking about–the history of ftp clients, ftp daemons, router manufacturers, and ISPs, and what a mess it was. If you don’t, I guess this might be a legit starting poing? https://duckduckgo.com/?q=FTP+PASV+NAT

                But azdle’s point is spot on. The CF engineers surely do know that whole sordid history–and the histories of http, of browsers, of transparent http proxies, etc. Sticking to an open protocol and contributing to it as needed is absolutely the right answer and CF surely knows better–even if their shiny new thing is merely an “app”.

              2. 3

                I’m a bit sad that, from a technical point of view, there seems to be an influx of lower quality articles posted on Lobste.rs for the last couple of months. I’m seeing more and more web stuff and commercial blog postings that are only light on technical details. This is a good example of what I think should have been posted instead of the OP.

              1. 24

                I see too many people rolling PHP-FPM only to show an IP address to the client. So, I wanted to share a simpler method which, I hope, can save you some time.

                1. 3

                  This seems very elegant, but just to be thorough are there any drawbacks/tradeoffs?

                  1. 4

                    The same T&Cs apply as when using nginx for standard stuff. This will/might be wrong if this nginx is behind another nginx, then you should look at X_FORWARDED_FOR (or whatever it’s called exactly).

                    1. 2

                      Beware if you use another public facing server in front of nginx. For example, if you have a reverse proxy (HAproxy for example), then the variable $remote_addr can represent the IP address of the proxy, not the initial HTTP client.

                      1. 4

                        Have a look at the realip module that allows Nginx to set the remote address based on a header set by the frontend proxy provided it’s one you have decided to trust that it’s setting correct headers.

                        Doing this over a custom solution in the application has the advantage that all remote address based features continue to work unaltered. Like geoip detection or logging addresses to web log files using built-in standard formats

                    2. 2

                      Yes this method also has my preference and we use that for years now. Years ago we used to use php (without php-fpm) for this, more or less like this:

                      <?php
                      echo $_SERVER['REMOTE_ADDR'] . PHP_EOL;
                      

                      But I was wondering: do you have suggestions for making it output both IPv4 and IPv6 addresses (like https://ip6.nl and others do) without adding additional complexity/dependencies like php (preferably with stock nginx or apache).

                      1. 4

                        To show both IPv4 and IPv6, the client needs to make two separate requests, to two separate domains that are configured differently, one with only an A record and one with only an AAAA record. Any given HTTP request is only going to be coming in on one or the other IP version.

                        ip6.nl makes XHR requests to https://4only.ip6.nl/myip.plp and https://6only.ip6.nl/myip.plp and displays the results on the page, again with Javascript. While those servers could very well be running the nginx config in the linked article, the ability to show both on the same page is much more complicated, tech-wise.

                        1. 2

                          You might be able to do it with redirects. Have the IPv4 server redirect to the IPv6 server with ?v4=a.b.c.d, and vice versa. Both servers would display both addresses once available.

                          It falls apart if you only have one type of address, since the redirect would be broken, but there’s probably a way around that. Maybe include the single address in the body of the 303, so if the redirect fails to connect you still have the initial IP address you used?

                          1. 3

                            The case where the caller can only connect on one protocol is probably very, very common still.

                        2. 3

                          But I was wondering: do you have suggestions for making it output both IPv4 and IPv6 addresses (like https://ip6.nl and others do) without adding additional complexity/dependencies like php (preferably with stock nginx or apache).

                          The tcp/ip stack of the client decides whether to try to connect using v4 or v6 first. I’ve added two extra dns entries, one with only a v4 address, and one with only a v6 address atop of one that has both: http://ip.netsend.nl http://ip4.netsend.nl http://ip6.netsend.nl

                        3. 2

                          Nice trick! Thanks!

                          However, you could add links to the relevant nginx-pages to your blog post as well.

                        1. 4

                          There is also vis, comparison of vis and kakoune here.

                          1. 6

                            One major difference is that kakoune is being actively developed and has an active community around it, while vis seems to have gone a bit quiet.

                            I tried vis in the past and ran into quite some rough edges; it didn’t seem to be in a state where it’s “done”.

                            1. 1

                              What kind of “rough edges”?

                              1. 1

                                I particularly remember having to force quit it after it became unresponsive, e.g. related to search prompt editing and resizing the terminal.

                            2. 3

                              I’ve looked into vis yesterday (as well as kakoune) as I’m looking for an editor that supports auto-complete and can jump to definitions like ctags in vi(1). My main reason is to make exploring new and possibly big code bases easier.

                              It wasn’t clear to me if these features are already supported by vis (I saw mentions of ctags in the issues on github, but nothing concrete). I’ve also seen vis-complete, but it wasn’t quite clear to me how to set this up to span all source code in a repository. Are these things you have working in your vis setup?

                              1. 1

                                There is apparently a plugin for ctags in vis, though I’ve never used them, nor have I used autocomplete, sorry.

                                1. 1

                                  kak-lsp will give you both features (and more, including floating docs, param documentation, etc)

                              1. 1

                                Oh no, not again.

                                It’s clearly time exim was sent to the great mailer-daemon in the sky. How many RCE CVE’s in the last two years? Too many.

                                Sadly there don’t seem to be any open source SMTP servers written in memory safe langauges around. Unless I’ve missed one?

                                1. 1

                                  Fortunately there are SMTP servers with a proper design that greatly reduce the severity of the effects of memory corruption.

                                  1. 2

                                    I disagree that those 2 examples are good solutions: Postfix’s configuration is even less readable than Exim’s, and OpenSMTPd is really under-documented and looks much more trouble to run on Linux than it’s worth. Exim is still the least bad of the bunch.

                                    1. 1

                                      All of them should be run in a container (or jail/chroot) if not a VM (QubesOS). Furthermore, we need to get rid of root.

                                      **edit, oh jebus. It looks like ASN.1 parsing strikes again. The most profitable back door in the history of computers.

                                  1. 1

                                    One thing that I don’t think this cookie thing solves is CSRF when it comes to POSTs done by unauthenticated users.

                                    If you have a comment form on your site but no anti-CSRF token in the submit form, then people could just trigger POST requests for your website from theirs.

                                    the anti-CSRF token lets you verify that the form was generated by you, so to speak.

                                    I would be pretty interested in not needing to deal with anti-CSRF tokens on validated forms in general though…

                                    1. 3

                                      That’s what this brings you. Doesn’t it? You can get it NOW by explicitly setting your cookies as SameSite. All major browsers already support the SameSite attribute, but not all set it by default.

                                      1. 2

                                        No? The question being asked here is whether you can get CSRF protection on forms with no required cookies. A mitigation that limits cookies won’t fix that.

                                        1. 4

                                          If the form is anonymous, is CSRF even a thing? If there’s no user to attack, then there’s no harm. At best it seems like nuisance comparable to hotlinking.

                                          1. 1

                                            Kind of, though it’s a bit more work than just what CSRF normally takes.

                                            A typical site that accepts anonymous submissions will use IP address rate limits. A site operator that wants to screw over another site could embed a script that submits a hidden form in an iframe over and over, essentially using the visitors as DoS reflectors. (Obviously, you could make them deny service with just image hotlinking, but having everybody that visits your site automatically submit a question to Stack Overflow in the background would be much more damaging with much fewer visitors).

                                            1. 1

                                              Not even a nuisance: a feature! Lots of useful things can rely on forms from one site POSTing to another – in fact, even CSRF would be useful if there were some way a browser could warn a user to prevent the attack-mode version of it.

                                            2. 2

                                              Oh, thank you. I misread that. You can soon get this with fetch-metada though.

                                          2. 2

                                            From a 2017 article by the same author, just set SameSite=lax on the cookies you send to your visitors:

                                            Set-Cookie: sess=abc123; path=/; SameSite=lax
                                            

                                            The OP is that Chrome is about to make this the default for all received cookies in an upcoming release.

                                            1. 2

                                              Anti-CSRF works by generating a secret when you visit a page. This allows you to present that secret the next time you take an action. This provides trust that you are making the request, not some malicious javascript from a different site (eg, cross site).

                                              This cookie option allows the cookie to be sent with a request only when the browser is viewing the page, and is not sent for cross site requests. In other words, if a server is getting the cookie, the server must have sent it, the browser supports the option, and the option was set to “Lax,” then there’s no reason for the anti-csrf token.

                                              1. 1

                                                You could issue one SameSite cookie to all unauthenticated users and check for that cookie (or one for a logged-in session) when validating the form submission.

                                              1. 7

                                                This is great news! Now I can finally recommend OpenBSD to my friends that need to run Apache / PHP / NGINX / Samba :)

                                                1. 6

                                                  One statistic I’m interested in is the number of submitted stories authored by ppl from the community. I think this is a strong differentiator of Lobste.rs compared to other link aggregation sites.

                                                  1. 3

                                                    There appears to be an increase in them recently, too.

                                                    1. 3

                                                      I had a spot of insomnia and got curious, so I wrote the query. (Reminder I’m happy to run queries, see /about.) The user_is_author boolean was added to stories on 2019-07-30.

                                                      > select extract(year from created_at) as y, extract(month from created_at) as m, sum(user_is_author) as authored, count(*) as "all", round(sum(user_is_author)/count(*)*100) as p
                                                       from stories where created_at >= '2015-07-01' group by 1, 2;
                                                      +------+------+----------+------+------+
                                                      | y    | m    | authored | all  | p    |
                                                      +------+------+----------+------+------+
                                                      | 2015 |    7 |        2 |  608 |    0 |
                                                      | 2015 |    8 |       88 |  541 |   16 |
                                                      | 2015 |    9 |      106 |  549 |   19 |
                                                      | 2015 |   10 |      117 |  737 |   16 |
                                                      | 2015 |   11 |      109 |  791 |   14 |
                                                      | 2015 |   12 |      124 |  761 |   16 |
                                                      | 2016 |    1 |      116 |  972 |   12 |
                                                      | 2016 |    2 |      117 |  849 |   14 |
                                                      | 2016 |    3 |      131 |  736 |   18 |
                                                      | 2016 |    4 |      131 |  739 |   18 |
                                                      | 2016 |    5 |      132 |  786 |   17 |         
                                                      | 2016 |    6 |      157 |  798 |   20 |         
                                                      | 2016 |    7 |      129 |  812 |   16 |                                                          
                                                      | 2016 |    8 |      124 |  797 |   16 |
                                                      | 2016 |    9 |      110 |  731 |   15 |                                                          
                                                      | 2016 |   10 |      115 |  779 |   15 |         
                                                      | 2016 |   11 |      116 |  835 |   14 |  
                                                      | 2016 |   12 |      119 |  852 |   14 |                                                          
                                                      | 2017 |    1 |      152 | 1037 |   15 |       
                                                      | 2017 |    2 |      139 | 1068 |   13 |
                                                      | 2017 |    3 |      182 | 1194 |   15 |  
                                                      | 2017 |    4 |      147 |  947 |   16 |         
                                                      | 2017 |    5 |      160 |  979 |   16 |
                                                      | 2017 |    6 |      175 |  941 |   19 |                                                          
                                                      | 2017 |    7 |      206 | 1109 |   19 |
                                                      | 2017 |    8 |      170 | 1111 |   15 |
                                                      | 2017 |    9 |      202 |  974 |   21 |                                                                                                                                                            
                                                      | 2017 |   10 |      223 |  985 |   23 |                                                          
                                                      | 2017 |   11 |      180 |  924 |   19 |                                                          
                                                      | 2017 |   12 |      165 |  922 |   18 |         
                                                      | 2018 |    1 |      227 |  961 |   24 |                                                          
                                                      | 2018 |    2 |      181 |  846 |   21 |
                                                      | 2018 |    3 |      212 | 1058 |   20 |                                                          
                                                      | 2018 |    4 |      213 |  983 |   22 |         
                                                      | 2018 |    5 |      200 |  982 |   20 |                                                          
                                                      | 2018 |    6 |      188 |  886 |   21 |         
                                                      | 2018 |    7 |      198 | 1017 |   19 |                                                          
                                                      | 2018 |    8 |      207 |  985 |   21 |         
                                                      | 2018 |    9 |      212 |  869 |   24 |                                                                                                                                                            
                                                      | 2018 |   10 |      207 |  918 |   23 |                                                          
                                                      | 2018 |   11 |      221 |  924 |   24 |
                                                      | 2018 |   12 |      225 |  905 |   25 |
                                                      | 2019 |    1 |      218 | 1058 |   21 |
                                                      | 2019 |    2 |      207 |  924 |   22 |
                                                      | 2019 |    3 |      246 |  978 |   25 |
                                                      | 2019 |    4 |      271 |  967 |   28 |
                                                      | 2019 |    5 |      229 |  954 |   24 |
                                                      | 2019 |    6 |      260 |  925 |   28 |
                                                      | 2019 |    7 |       62 |  187 |   33 |
                                                      +------+------+----------+------+------+
                                                      49 rows in set (0.97 sec)
                                                      

                                                      We’ve had rumblings in #lobsters that there’s been too much self-promotion/content marketing recently, and there’s certainly more author-submitted posts the last four months (as @nickpsecurity observed). Maybe someone would like to write queries to get at this?

                                                      1. 3

                                                        Appreciate the data. It confirms that authored by’s went up with it being steadily over 200 since 08/2018. Our curators are pretty good at flagging away, countering or just dropping self promotion with no content. I’m not too worried about it. I probably need to get back to messaging these people about what kinds of content Lobsters prefer. I used to do it sometimes for people whose submissions got no reaction or a negative one.

                                                    1. 3

                                                      This had the useful result that you could no longer accidentally cat a directory and get all sorts of gibberish spewed on your screen, without requiring cat (and everything else that reads files) to explicitly refuse to touch directories. This feature does not seem to have spread to Solaris or the *BSDs, at least as far as I can see.

                                                      Failing a read(2) with EISDIR was introduced in OpenBSD 6.1 (released in 2017). See this comment and the rest of the thread for some additional insights into history.

                                                      1. 2

                                                        As I’m a bit ignorant about these things, just wondering if anyone can explain the motivation for the Xorg setuid change?

                                                        1. 10

                                                          A local root hole in X.org discovered around the release of 6.4: https://marc.info/?l=openbsd-tech&m=154050351216908&w=2

                                                          1. 2

                                                            https://marc.info/?l=openbsd-tech&m=154050351216908&w=2

                                                            Wow, that’s… Very interesting.

                                                            OTOH, I think it’s pretty cool that Theo would reveal such info even though it may seem to undermine credence about revealing security vulnerabilities to his own project. I mean, if his own people don’t tell him about the upcoming embargoed bugs due to the known stance of The OpenBSD Project against security embargoes, what can be expected of other projects and bigger vendors?

                                                            I’m happy to see matthieu is still a committer, though, and has been committing throughout Oct and Nov last year around this controversy, too; i.e., at least from the public eye, there’s no evidence to suggest that his account was ever disabled; but this is some harsh reality come OpenBSD way…

                                                          2. 6

                                                            Heh, I’ve used startx(1) for as long as I’ve used Unix systems, guess I have to finally start using a login manager!

                                                          1. 11

                                                            Some personal favorites:

                                                            • Work has started on a ISC-licensed rsync-compatible program called OpenRSYNC. In this release it has basic functionality such as -a, –delete, but lacks –exclude. Work will continue.
                                                            • unveil(2) has been improved to understand and find covering unveil matches above the working directory of the running process for relative path accesses. As a result many programs now can use unveil in broad ways such as unveil(”/”, “r”).
                                                            • Now using unveil(2) in ospfd(8), ospf6d(8), rebound(8), getconf(1), kvm_mkdb(8), bdftopcf(1), Xserver(1), passwd(1), spamlogd(8), spamd(8), sensorsd(8), snmpd(8), htpasswd(1), ifstated(8). Some pledge(2) changes were required to accommodate unveil.
                                                            • RETGUARD replaces the stack protector on amd64 and arm64, since RETGUARD instruments every function that returns and provides better security properties than the traditional stack protector.
                                                            • tcpdump(8) already used privsep, pledge(2) and unveil(2) containment. It now also drops root privileges completely (switching to a reserved uid).
                                                            • malloc(3) now uses sysctl(2) to get its settings, making it respect the system-wide settings in chroots as well.
                                                            • LibreSSL 2.9.1: Added support for XChaCha20 and XChaCha20-Poly1305.
                                                            • Removed some ASN.1 related code from libcrypto that had not been used since around 2000.
                                                            • ssh(1): When prompting whether to record a new host key, accept the key fingerprint as a synonym for “yes”. This allows the user to paste a fingerprint obtained out of band at the prompt and have the client do the comparison for you.
                                                            • Xorg(1), the X window server, is no longer installed setuid. xenodm(1) should be used to start X.
                                                            • Mandoc 1.14.5: Much better HTML output, in particular with respect to paragraphs, line breaks, and vertical spacing in tagged lists. Tooltips are now implemented in pure CSS, the title attribute is no longer abused.

                                                            I guess the new filtering engine of OpenSMTPD isn’t ready yet.

                                                            1. 14

                                                              “O Deep Thought computer,” he said, “the task we have designed you to perform is this. We want you to tell us….” he paused, “The Answer.”
                                                              “The Answer?” said Deep Thought. “The Answer to what?”
                                                              “Life!” urged Fook.
                                                              “The Universe!” said Lunkwill.
                                                              “Everything!” they said in chorus.
                                                              Deep Thought paused for a moment’s reflection.
                                                              “Tricky,” he said finally.
                                                              “But can you do it?”
                                                              Again, a significant pause.
                                                              “Yes,” said Deep Thought, “I can do it.”
                                                              “There is an answer?” said Fook with breathless excitement.
                                                              “Yes,” said Deep Thought. “Life, the Universe, and Everything. There is an answer. But, I’ll have to think about it.”

                                                              Fook glanced impatiently at his watch.
                                                              “How long?” he said.
                                                              “Seven and a half million years,” said Deep Thought.
                                                              Lunkwill and Fook blinked at each other.
                                                              “Seven and a half million years…!” they cried in chorus.
                                                              “Yes,” declaimed Deep Thought, “I said I’d have to think about it, didn’t I?”

                                                              [Seven and a half million years later…. Fook and Lunkwill are long gone, but their descendents continue what they started]

                                                              “We are the ones who will hear,” said Phouchg, “the answer to the great question of Life….!”
                                                              “The Universe…!” said Loonquawl.
                                                              “And Everything…!”
                                                              “Shhh,” said Loonquawl with a slight gesture. “I think Deep Thought is preparing to speak!”
                                                              There was a moment’s expectant pause while panels slowly came to life on the front of the console. Lights flashed on and off experimentally and settled down into a businesslike pattern. A soft low hum came from the communication channel.

                                                              “Good Morning,” said Deep Thought at last.
                                                              “Er..good morning, O Deep Thought” said Loonquawl nervously, “do you have…er, that is…”
                                                              “An Answer for you?” interrupted Deep Thought majestically. “Yes, I have.”
                                                              The two men shivered with expectancy. Their waiting had not been in vain.
                                                              “There really is one?” breathed Phouchg.
                                                              “There really is one,” confirmed Deep Thought.
                                                              “To Everything? To the great Question of Life, the Universe and everything?”
                                                              “Yes.”
                                                              Both of the men had been trained for this moment, their lives had been a preparation for it, they had been selected at birth as those who would witness the answer, but even so they found themselves gasping and squirming like excited children.
                                                              “And you’re ready to give it to us?” urged Loonsuawl.
                                                              “I am.”
                                                              “Now?”
                                                              “Now,” said Deep Thought.
                                                              They both licked their dry lips.
                                                              “Though I don’t think,” added Deep Thought. “that you’re going to like it.”
                                                              “Doesn’t matter!” said Phouchg. “We must know it! Now!”
                                                              “Now?” inquired Deep Thought.
                                                              “Yes! Now…”
                                                              “All right,” said the computer, and settled into silence again. The two men fidgeted. The tension was unbearable.
                                                              “You’re really not going to like it,” observed Deep Thought.
                                                              “Tell us!”
                                                              “All right,” said Deep Thought. “The Answer to the Great Question…”
                                                              “Yes..!”
                                                              “Of Life, the Universe and Everything…” said Deep Thought.
                                                              “Yes…!”
                                                              “Is…” said Deep Thought, and paused.
                                                              “Yes…!”
                                                              “Is…”
                                                              “Yes…!!!…?”
                                                              “the new filtering engine of OpenSMTPD isn’t ready yet,” said Deep Thought, with infinite majesty and calm.

                                                              1. 2

                                                                Is this the new “Is KERNSEAL ready yet?”

                                                            1. 18

                                                              I have long had trouble understanding how the blockchain gets the very many magical properties ascribed to it. Every time I’ve asked, I have gotten a lecture in hashes and proof of work. I know the fundamentals of blockchains, what I don’t understand is how the fundamentals lead to these amazing emergent properties.

                                                              This article sorta kinda makes me think I might not be missing anything at all – the people talking about it may have been full of shit.

                                                              1. 8

                                                                I have been wrong before but to me it seems like a mass psychological phenomenon. That many people and that much money cannot be wrong! So they add more people and money.

                                                                In the best case, some companies use the label “block chain” to market some established cryptographic techniques that are not block chain at all…

                                                                1. 7

                                                                  yep. You might think “blockchain” meant something like “append-only ledger with a consensus mechanism”, but it turns out in practice to literally just mean “whatever I’m trying to sell you today”.

                                                                  I was talking about this a few months ago with a well-meaning non-techie, who suggested that Uber - the money-burning minicab firm with an app - was an example of a “decentralised system.” More than that - it was a model for how blockchain could succeed.

                                                                  I think they’d never thought about the concept of distributed systems of any sort ever before in their lives.

                                                                  “It’s like blockchain, because anyone can sign up to be an Uber driver!”
                                                                  “Uh … anyone can sign up to be a minicab driver.”

                                                                  or the very concept of “open source” only being possible with “blockchain”.

                                                                  1. 4

                                                                    The weird thing is that I know intelligent, technical people that advocate for this. If asked for specifics, some variant of “we still have to figure out the specifics” is used.

                                                                    Well, chances are that you never will…

                                                                  2. 4

                                                                    The hype cycle became self fulfilling. I got a look at the internal roadmap for one of the pieces of legacy software at the big enterprise I work at - crusty old barely touched 90s technology that’s critical for parts management and ordering.

                                                                    2020 plans? Traceability of parts on the blockchain.

                                                                    1. 1

                                                                      Traceability, correct me if I’m wrong, was one of the actual things a distributed append-only ledger was good at. The way I see it, it’s a good decision with regards to what tech to use, at least until someone puts the wrong data in.

                                                                  3. 4

                                                                    As well as that non-tech explanation talk, I have the longer and more techy version to an infosec group. (My mission to get across to them: “please don’t get into blockchains”)

                                                                    1. 4

                                                                      You’re sending the wrong message. Instead, tell them to come up with something useful, pitch a blockchain version, build the useful thing first with the money, open source it, and then build the “better” version with blockchain. We’ll steadily get more useful stuff out of blockchain investments.

                                                                    2. 4

                                                                      I’d like to refer to this article about Bitcoin from 2011, before all the mass hysteria began: https://paulbohm.com/articles/bitcoins-value-is-decentralization/

                                                                      To elaborate: Bitcoin isn’t just a currency but an elegant universal solution to the Byzantine Generals’ Problem, one of the core problems of reaching consensus in Distributed Systems. Until recently it was thought to not be practically solvable at all, much less on a global scale. Irrespective of its currency aspects, many experts believe Bitcoin is brilliant in that it technically made possible what was previously thought impossible.

                                                                      /edit quote

                                                                      1. 4

                                                                        Herd behavior. It’s usually irrational except for the scheming people fueling and benefiting from it.

                                                                        Blockchain looks like herd behavior. Similarly, most of it has a tiny group of people that will get rich if enough buy in. That’s similar to how companies like Goldman create, pop, and profit from bubbles in the market.

                                                                        1. 3

                                                                          irrational exuberance meets unjustifi-ed/able faith in technology.

                                                                          if you bought into a blockchain, you want to hype it up because that’s how you get paid. If you didn’t buy into it, well you got bored of trying to reason with people a long time ago.

                                                                          It’s probably the most interesting social phenomenon of recent years.

                                                                          1. 2

                                                                            It sounds basically like a pyramid scheme when you put it like that…

                                                                          2. 2

                                                                            The flip side of this is that some companies are actually trying to look for the blockchain-based “killer app”, if such an app ever exists. I did develop a few blockchain based proof of concepts, which didn’t go anywhere, but there wasn’t any attempt to trick anyone. It’s just about experimenting with a new technology and see what can be done with it.

                                                                          1. 1

                                                                            I left a Git branch stall for a few weeks and, when I got back to it, it was behind by almost 60,000 commits. I think one can argue that a single person cannot possibly read all the code that is added to Windows every day, let alone read what was written during the past thirty years!

                                                                            What about peer review and quality assurance?!

                                                                            1. 28

                                                                              It’s entirely possible that Microsoft has more than one person doing peer review and QA on Windows.

                                                                              1. 3

                                                                                Given that it is a monorepo, the vast majority of those commits are probably not touching the area of the codebase that you’re personally working on.

                                                                              1. 33

                                                                                Am I missing something, or is that a whole article? Looks like an excerpt or preface though :)

                                                                                But from what I see, your problem isn’t a vim itself, but rather plugins and overloading yourself with them. A common mistake for Vim newcomers is that they come in and just skip the vi grammar part (calling it “the boring theory stuff”) and trying things too hard. Then they discover plugins and – more importantly – plugin managers, which lower the bar dramatically. Within such conditions, they start to feel “safe” and install as many plugins as they can handle to mimic their previous IDE, their colleagues’ IDE or any IDE they pretend to be better from, just to show off maybe.

                                                                                And after all, you end up with a pretty much beefied up editor which you don’t really know. Okay, well, you know how to open it, type the text, save and call it a day. You know “how”, but now “why”.

                                                                                In every editor which has been thought reasonably from the ground up (not just vi - it could be emacs, joe, sam, acme, kakoune, vis, CygnusED, EDLIN…) you need to understand the rules. It’s like speaking in new dialect of your language or swimming - won’t come overnight which sounds unpleasant in age of instant grafitication, but its long-term benefits are hard to overlook.

                                                                                So, in other words (or as people say these days - tl;dr), your problem is that you don’t grok vi (or any other editor, as I said).

                                                                                Of course, if you want to just click ok and go on with things, you might just spawn up your web browser^W^Wtext editor called vscode, throw a ton of plugins on yourself and cheer. Would be that faster? Maybe. But only in short-term goal or if your project isn’t going to last more than year or two (which is a standard nowadays). Or if you just don’t want to “spend time on these neckbeardy things”.

                                                                                1. 3

                                                                                  So, in other words (or as people say these days - tl;dr), your problem is that you don’t grok vi (or any other editor, as I said).

                                                                                  I’m not sure how these words hold up to someone who calls himself a “VIM master from using others’ plugins to rolling out my own”.

                                                                                  After seeing some colleagues working at lightning speed with PhpStorm I gave up thinking vi(m) is always faster, especially on the cases the OP mentions like renaming/refactoring classes, functions and files in a big project. The reason I still use vi for everything, is because today I mainly develop in C, which is natively supported in vi and because I can use vi for everything, not only development but also sysadmin, mail composing etc.

                                                                                  1. 3

                                                                                    You said what I was thinking in more words.

                                                                                    1. 2

                                                                                      Am I missing something, or is that a whole article? Looks like an excerpt or preface though :)

                                                                                      This is ‘part 1’, so I presume they split it up to increase their exposure, which means we will probably see part 2, 3, 4,…,N of this poor article here over time.

                                                                                    1. 5

                                                                                      However, benchmarks quickly showed that wireguard-go falls very short of the performance offered by the kernel module. This is because while the Go language is very good for writing servers, it is not so good for raw packet processing, which a VPN essentially does.

                                                                                      Is the difference between Go versus Rust really that big? I would suspect the difference between kernel and userland is way bigger.

                                                                                      Rust is a modern, safe language that is both as fast as C++ and is arguably safer than Go (it is memory safe and also imposes rules that allow for safer concurrency), […].

                                                                                      Interesting point about Rust being arguably safer than Go.

                                                                                      1. 12

                                                                                        Go is pretty fast and more than enough for lots of tasks, especially in its niche of higher-level network-bound servers.

                                                                                        However, in this case there’s a bunch of cryptography, packet parsing and shuffling of data from one place to another, where LLVM optimized code, zero-cost abstractions and no-GC memory slices shine.

                                                                                      1. 3

                                                                                        Though there are already objections to do ECDH and key derivation in the kernel

                                                                                        I’ve read some objections by @zx2c4 as well about putting timers and handshake code outside of the kernel when this was proposed by someone developing a kernel version for OpenBSD: https://lists.zx2c4.com/pipermail/wireguard/2018-December/003656.html

                                                                                        In my own userland implementation I’ve separated the long term secrets into a small privileged process apart from the rest.

                                                                                        1. 2

                                                                                          In my own userland implementation I’ve separated the long term secrets into a small privileged process apart from the rest.

                                                                                          That’s the right way to do it. :)

                                                                                          EDIT: Finally reading mailing list (shorter than I thought), his counter is pretty good. We keep stuff out of the kernel by default due to higher damage upon code injection. Another principle for trusted code is keeping it simple enough to know with confidence that won’t happen. He says he did that. The last factor is whether it’s easier to analyze his code for immunity vs yours plus its interactions with kernel code. The latter might be considerably more complex. This is good example of where exception might be made. So, either sounds good in this case. One might still apply some verification or hard-hitting analyzers to it if wanting extra assurance.

                                                                                        1. 4

                                                                                          So I wasn’t sure whether or not to say I’m the author here. I’m not the author, but I am running the instance of writefreely on OpenBSD. I’m hoping to build this into a federated space for others interested in reading and writing medium-high quality content, particularly around the BSD community.

                                                                                          I’ll have an open beta from April, but if anyone’s interested in writing content here, drop me a line for an invite. It’s totally open, all your data is yours, you can read content without javascript and there’s no visitor tracking beyond post view counts accessible only by a post’s individual author.

                                                                                          It’d be really nice to get some people interested in writing about lesser known operating system experiences, Open/NetBSD use, less common FreeBSD use and so on.

                                                                                          1. 2

                                                                                            “So I wasn’t sure whether or not to say I’m the author here. I’m not the author”

                                                                                            You’re the author if you wrote the content, built the tools, etc. That’s probably the original intent. It’s also the usage in almost all situations I see “authored by.”

                                                                                            1. 3

                                                                                              Yeah it was h3artbl33d that wrote the piece, hit publish, I just gave him an account. It’s the submit tick box I’m unsure about. I don’t want to claim authorship, but certainly there’s a connection and I’m not sure the best way to make that clear.

                                                                                              1. 1

                                                                                                Maybe you/chargen.one can be seen as a publisher, comparable to the publisher of a journal.

                                                                                          1. 39

                                                                                            I’m really burning out on “simplicity” posts. I get it, simplicity is good. But that doesn’t actually inform me as a developer. Why do things become complex? What kinds of simplicity are there? How do we detect simplicity? How do we know when we shouldn’t simplify? None of these posts ever answer that.

                                                                                            It’s like if I stood on stage and said “Be good! Don’t be evil! Being evil is bad!” Sure, everybody agrees with that, but does it actually help people make moral choices?

                                                                                            (Also the analogy is dumb. Yes, we should totally base our engineering practice on a movie! A movie where the engineers are wrong because of magic.)

                                                                                            1. 11

                                                                                              Why do things become complex? What kinds of simplicity are there? How do we detect simplicity? How do we know when we shouldn’t simplify? None of these posts ever answer that.

                                                                                              Because your questions are difficult and answers are dependent on a lot of factors.

                                                                                              I’ll tell you what I do to detect simplicity, maybe you’ll find it useful. Let’s start with a real-life example.

                                                                                              I needed tokens for authorization, I reviewed existing formats, JWTs look conservative and Macaroons look powerful.

                                                                                              What do I do? I dissect the formats. For JWTs I read the RFCs and implemented software to create them and verify them (each in 2 languages) for various options (key algorithms).

                                                                                              For Macaroons I read the whitepaper, then implemented verifier based on the whitepaper, reviewed existing implementations, found out differences between the whitepaper and de-facto code with explanations. While comparing my implementation I found out some security issues with existing code. Additionally I implemented the rest of the stack (de/serialization, UI for manipulation of Macaroons). After two months I knew precisely where does complexity lie in Macaroons and of course there are the only spots all blogposts don’t mention (spoilers: cycles in third party caveats, no standards for encoded caveats…)!

                                                                                              Then I looked at my JWT proof-of-concept code - it uses base64(url) and JSON, primitives that basically all programming environments have built-in. After limiting the algorithms used the entire verifier takes just a couple of lines of code! It’s vastly simpler than the Macaroon one.

                                                                                              What’s the moral here? That you need a lot of time to see for yourself what is simple and what is complex. Now every time I see a post recommending Macaroons I can already see the author didn’t use them in practice (compare that with the Tess Rinearson post linked at the end of that article).

                                                                                              That’s only the example, I routinely implement various protocols and re-implement software (ActivityPub, Mailing Lists, roughtime client) and each time I discover what’s simple or what’s complex in each one of them.

                                                                                              (By the way your book is excellent!)

                                                                                              1. 9

                                                                                                I get it, simplicity is good.

                                                                                                Alas, not everybody gets it. The best that these kinds of exhortations can do (all that they aim to do, as far as I can tell) is to persuade people to modify their own set of values. This doesn’t immediately result in better code… but I think it’s a necessary precondition. The only developers who will even ask the good questions you suggest (let alone look for good answers) are the developers who hold simplicity as a value.

                                                                                                (The analogy is pretty dumb though, and not especially motivating.)

                                                                                                1. 10

                                                                                                  I’ve never met a developer who does not claim to hold simplicity as a value. But as a concept it is so subjective that this is meaningless. It’s extremely common for two developers arguing for opposing approaches each to claim that their approach is the simpler one.

                                                                                                  1. 7

                                                                                                    I get the value of exhortations. I think more examples would be better. Pairs of solutions where the simple one meets requirements with a number of better attributes. Developers often prefer to see the difference and benefits instead of being told.

                                                                                                  2. 6

                                                                                                    Exactly. This is one of those things you can’t explain in a book. When to compose, when to decompose. When to extract methods, when to inline methods. When to add a layer of abstraction, when to remove one. When is it too flexible, when is it too simplistic?

                                                                                                    No amount of rules of thumb is going to answer those question. I only know of one way to learn it: practice. Which takes effort and most importantly, time. Rendering this kind of posts mostly useless.

                                                                                                    1. 3

                                                                                                      P.S. They do feel good to write though, so people will keep writing them, and there’s nothing wrong with it either.

                                                                                                    2. 5

                                                                                                      I agree that anecdotes like this can get old, but I’ve been meaning to actually write a similar post to this… on something I’ve been calling the “too many buttons” syndrome. This issue pops up a ton in large pieces of software (Though I’m specifically thinking of projects like JRA and Confluence) where there’s an option for everything.

                                                                                                      Not everyone gets that simplicity is good because it can be harder to sell. “If a user wants it, we should do it” is something I’ve heard just a few too many times without bothering to look at the use case or if it could be done better. Sometimes it’s worth stepping back and looking at the complexity something will add to the project (in both code and testing… especially when it comes to options and how they interact with each other) rather than just adding all the little features.

                                                                                                      1. 5

                                                                                                        In my experience a lot of commercial companies that develop under tight deadlines produce a lot of suboptimal and dreadful code. Often it takes more time, to produce less code simply because the more time you spend on a difficult problem, the better you understand it. I think the reason that most a lot of software is bloated and complex is because it’s “good enough” which is optimal from an economic point of view.

                                                                                                        The other day there was a discussion here on Lobsters about all the required pieces needed to run a Mastodon instance and the popular solution of abstracting all that away in a Docker container. There are alternative implementations that depend on a smaller number of components alleviating the need for dumping everything in a container (of course the question is, do these alternatives offer the same functionality).

                                                                                                        How do we detect simplicity?

                                                                                                        For me personally simplicity has to do with readability, maintainability and elegance of code or infrastructure. If someones solution involves three steps, and someone else can do it in two steps (with comparable cognitive load per step), I would say it’s more simple.

                                                                                                        How do we know when we shouldn’t simplify?

                                                                                                        If that would cut some features you cannot miss.

                                                                                                        1. 5

                                                                                                          You are so right. After years of experience, I only start to clarify my idea of “simplicity”. There are different kind of simplicity most of them are not totally compatible. And in my opinion some need to be preferred to other, but there is no clear rule. To make a choice between different complexity I still use a lot of intuition and I debate a lot, and I am still unsure my choice are the best.

                                                                                                          • only using basic feature of a language (do not use advanced programming language feature) is certainly the most important aspect in simplicity. It will make your code easy to read by more people.
                                                                                                          • don’t use too much intermediate functions, and if possible don’t disperse those function in many different files before really feel you are copy/pasting too much. My rule of thumb is, 2 or 3 times duplications is totally fine and superior to centralisation of code. It start to be really clear that code factorisation is good when you start repeating yourself more than 6 to 10 times
                                                                                                          • only really use advanced feature of the language after having tried not to use it for some time and really lack the ability of that advanced feature. Some examples of what I call advanced feature of a language are; class heritage, protocols in Clojure, writing your own typeclasses in Haskell, meta programming (macros in LISP), etc…
                                                                                                          • prefer stateless functions to objects/service with internal states
                                                                                                          • prefer pure functions (side effect free) other procedures (functions with side effects)
                                                                                                          • give a lot of preference to composable solutions ; composable in the algebraic meaning. For example, I do my best not to use LISP macros, because most of the time macros break composability. The same could be said when you start to deal with type-level programming in Haskell, or when you are doing meta-programming in ruby/python.

                                                                                                          For now, all those rules are still quite artisanal. I don’t have any really hard metrics or strong rules. Everything I just said is “preferable” but I’m pretty sure we can find exception to most of those rules.

                                                                                                          1. 5

                                                                                                            Amen, +1, etc. “Simplicity” often just means that a concept fits cleanly in the maker’s head at a particular point in time. How many times have I returned to a project I thought was simple only to find I had burdened it with spooky magic because I didn’t benefit from critical distance at the time? When was the last time I deemed another person’s work “too complex” because I couldn’t understand it in one sitting and wasn’t aware of the constraints they were operating under? Answers: too often and too recently.

                                                                                                            1. 3

                                                                                                              What kinds of simplicity are there?

                                                                                                              This is a good question (as are the others). Borrowing from Holmes, I’d say there’s a continuum from naive simplicity, to complexity, to simplicity on the other side of complexity (which is what is truly interesting)

                                                                                                              For example, “naively simple” code would only cover a small subset (say, the happy path) of a business problem. Complex code would handle all, or most, of the business complexity but in a messy, complicated way. “Other side” simplicity refines that complex code into something that can handle the business complexity without itself becoming overly complicated.

                                                                                                              1. 2

                                                                                                                What happens to simplicity? We trade it for a other things of course. For example, you can have simple regular expressions, but most people prefer less simple and more powerful implementation like Perls.

                                                                                                                Simplicity is often a tradeoff versus easyness, performance, flexibility, reusability, useability, etc. So simplicity is good, but those other things are also good.

                                                                                                                1. 1

                                                                                                                  Most people seem to agree that simplicity is best. However, when it comes down to simplicity for the user versus the developer, I have seen disagreement. Each trade off is going to be situation and implementation dependent, but at my job I’ve been pushing for a simpler developer environment.

                                                                                                                  In my office, there is a tendency to create exceptions to rules because it makes things simpler for the user. Since the environment has more exceptional circumstances, it tends to have more errors when people forget the undocumented exception case. In my opinion, this causes an uneven experience for the user despite being “simpler.”

                                                                                                                  My experience is coming from a medium sized, non-tech company. I work in the IT department so we are a cost center. There is an emphasis on white glove treatment of the revenue producing portions of the company. YMMV

                                                                                                                1. 65

                                                                                                                  In the Mastodon universe, technically-minded users are encouraged to run their own node. Sounds good. To install a Mastodon node, I am instructed to install recent versions of

                                                                                                                  • Ruby
                                                                                                                  • Node.JS
                                                                                                                  • Redis
                                                                                                                  • PostgreSQL
                                                                                                                  • nginx

                                                                                                                  This does not seem like a reasonable set of dependencies to me. In particular, using two interpreted languages, two databases, and a separate web server presumably acting as a frontend, all seems like overkill. I look forward to when the Mastodon devs are able to tame this complexity, and reduce the codebase to a something like single (ideally non-interpreted) language and a single database. Or, even better, a single binary that manages its own data on disk, using e.g. embedded SQLite. Until then, I’ll pass.

                                                                                                                  1. 22

                                                                                                                    Totally agree. I heard Pleroma has less dependencies though it looks like it depends a bit on which OS you’re running.

                                                                                                                    1. 11

                                                                                                                      Compared to Mastodon, Pleroma is a piece of cake to install; I followed their tutorial and had an instance set up and running in about twenty minutes on a fresh server.

                                                                                                                      From memory all I needed install was Nginx, Elixir and Postgres, two of which were already set up and configured for other projects.

                                                                                                                      My server is a quad core ARMv7 with 2GB RAM and averages maybe 0.5 load when I hit heavy usage… it does transit a lot of traffic though, since the 1st January my server has pushed out 530GB of traffic.

                                                                                                                      1. 2

                                                                                                                        doesnt Elixir require Erlang to run?

                                                                                                                        1. 2

                                                                                                                          It does. Some linux distributions will require adding the Erlang repo before installing elixir but most seem to have it already included: https://elixir-lang.org/install.html#unix-and-unix-like meaning its a simple one line command to install e.g pkg install elixir

                                                                                                                      2. 7

                                                                                                                        I’m not a huge social person, but I had only heard of Pleroma without investigating it. After looking a bit more, I don’t really understand why someone would choose Mastodon over Pleroma. They do basically the same thing, but Pleroma takes less resources. Anyone who chose Mastodon over Pleroma have a reason why?

                                                                                                                        1. 6

                                                                                                                          Mastodon has more features right now. That’s about it.

                                                                                                                          1. 4

                                                                                                                            Pleroma didn’t have releases for a looong time. They finally started down that route. They also don’t have official Docker containers and config changes require recompiling (just due to the way they have Elixir and builds setup). It was a pain to write my Docker container for it.

                                                                                                                            Pleroma also lacks moderation tools (you need to add blocked domains to the config), it doesn’t allow remote follow/interactions (if you see a status elsewhere on Mastodon, you can click remote-reply, it will ask your server name, redirect you to your server and then you can reply to someone you don’t follow) and a couple of other features.

                                                                                                                            Misskey is another alternative that looks promising.

                                                                                                                            1. 2

                                                                                                                              it doesn’t allow remote follow/interactions (if you see a status elsewhere on Mastodon, you can click remote-reply, it will ask your server name, redirect you to your server and then you can reply to someone you don’t follow)

                                                                                                                              I think that might just be the Pleroma FA - if I’m using the Mastodon FE, I get the same interaction on my Pleroma instance replying to someone on a different instance as when I’m using octodon.social (unless I’m radically misunderstanding your sentence)

                                                                                                                              1. 1

                                                                                                                                Thanks, this is a really great response. I actually took a quick look at their docs and saw they didn’t have any FreeBSD guide set up, so I stopped looking. I use Vultr’s $2.50 FreeBSD vps and I didn’t feel like fiddling with anything that particular night. I wish they did have an official docker container for it.

                                                                                                                              2. 3

                                                                                                                                Pleroma has a bunch of fiddly issues - it doesn’t do streaming properly (bitlbee-mastodon won’t work), the UI doesn’t have any “compose DM” functionality that I can find, I had huge problems with a long password, etc. But they’re mostly minor annoyances than show stoppers for now.

                                                                                                                              3. 7

                                                                                                                                It doesn’t depend - they’ve just gone further to define what to do for each OS!

                                                                                                                                1. 4

                                                                                                                                  I guess it’s mainly the ImageMagick dependency for OpenBSD that got me thinking otherwise.

                                                                                                                                  OpenBSD

                                                                                                                                  • elixir
                                                                                                                                  • gmake
                                                                                                                                  • ImageMagick
                                                                                                                                  • git
                                                                                                                                  • postgresql-server
                                                                                                                                  • postgresql-contrib

                                                                                                                                  Debian Based Distributions

                                                                                                                                  • postgresql
                                                                                                                                  • postgresql-contrib
                                                                                                                                  • elixir
                                                                                                                                  • erlang-dev
                                                                                                                                  • erlang-tools
                                                                                                                                  • erlang-parsetools
                                                                                                                                  • erlang-xmerl
                                                                                                                                  • git
                                                                                                                                  • build-essential
                                                                                                                                  1. 3

                                                                                                                                    imagemagick is purely optional. The only hard dependencies are postgresql and elixir (and some reverse proxy like nginx)

                                                                                                                                    1. 4

                                                                                                                                      imagemagick is strongly recommended though so you can enable the Mogrify filter on uploads and actually strip exif data

                                                                                                                                2. 3

                                                                                                                                  Specifically, quoting from their readme:

                                                                                                                                  Pleroma is written in Elixir, high-performance and can run on small devices like a Raspberry Pi.

                                                                                                                                  As to the DB, they seem to use Postgres.

                                                                                                                                  The author of the app posted his list of differences, but I’m not sure if it’s complete and what it really means. I haven’t found a better comparison yet, however.

                                                                                                                                3. 16

                                                                                                                                  Unfortunately I have to agree. I self-host 99% of my online services, and sysadmin for a living. I tried mastodon for a few months, but its installation and management process was far more complicated than anything I’m used to. (I run everything on OpenBSD, so the docker image isn’t an option for me.)

                                                                                                                                  In addition to getting NodeJS, Ruby, and all the other dependencies installed, I had to write 3 separate rc files to run 3 separate daemons to keep the thing running. Compared to something like Gitea, which just requires running a single Go executable and a Postgres DB, it was a massive amount of toil.

                                                                                                                                  The mastodon culture really wasn’t a fit for me either. Even in technical spaces, there was a huge amount of politics/soapboxing. I realized I hadn’t even logged in for a few weeks so I just canned my instance.

                                                                                                                                  Over the past year I’ve given up on the whole social network thing and stick to Matrix/IRC/XMPP/email. I’ve been much happier as a result and there’s a plethora of quality native clients (many are text-based). I’m especially happy on Matrix now that I’ve discovered weechat-matrix.

                                                                                                                                  I don’t mean to discourage federated projects like Mastodon though - I’m always a fan of anything involving well-known URLs or SRV records!

                                                                                                                                  1. 11

                                                                                                                                    Fortunately the “fediverse” is glued by a standard protocol (ActivityPub) that is quite simple so if one implementation (e.g. Mastodon) doesn’t suit someone’s needs it’s not a big problem - just searching for a better one and it still interconnects with the rest of the world.

                                                                                                                                    (I’ve written a small proof-of-concept ActivityPub clients and servers, it works and federates, see also this).

                                                                                                                                    For me the more important problems are not implementation issues with one server but rather design issues within the protocol. For example established standards such as e-mail or XMPP have a way to delegate responsibility of running a server of a particular protocol but still use bare domain for user identifies. In e-mail that is MX records in XMPP it’s DNS SRV records. ActivityPub doesn’t demand anything like it and even though Mastodon tries to provide something that would fix that issue - WebFinger, other implementations are not interested in that (e.g. Pleroma). And then one is left with instances such as “social.company.com”.

                                                                                                                                    For example - Pleroma’s developer’s id is lain@pleroma.soykaf.com.

                                                                                                                                    1. 16

                                                                                                                                      This is a completely reasonable and uncontroversial set of dependencies for a web app. Some of the largest web apps on the Internet run this stack. That is a good thing, because when Fediverse nodes need to scale there are well-understood ways of doing it.

                                                                                                                                      Success in social networking is entirely about network effects and that means low barrier to entry is table stakes. Yeah, it’d be cool if someone built the type of node you’re talking about, but it would be a curiosity pursued only by the most technical users. If that were the barrier to entry for the network, there would be no network.

                                                                                                                                      1. 39

                                                                                                                                        This is a completely reasonable and uncontroversial set of dependencies for a web app. Some of the largest web apps on the Internet run this stack.

                                                                                                                                        Yes, but not for a web app I’m expected to run on my own time, for fun.

                                                                                                                                        1. 6

                                                                                                                                          I’m not sure that’s the exact expectation, that we all should run our single-user Mastodon instances. I feel like the expectation is that sysadmin with enough knowledge will maintain an instance for many users. This seems to be the norm.

                                                                                                                                          That, or you go to Mastohost and pay someone else for your own single-user instance.

                                                                                                                                          1. 2

                                                                                                                                            You’re not expected to do that is my point.

                                                                                                                                          2. 16

                                                                                                                                            completely reasonable and uncontroversial

                                                                                                                                            Not true. Many people are complaining about the unmanaged proliferation of dependencies and tools. Most projects of this size and complexity don’t need more than one language, bulky javascript frameworks, caching and database services.

                                                                                                                                            This is making difficult to package Mastodon and Pleroma in Debian and Ubuntu and making it more difficult for people to make the service really decentralized.

                                                                                                                                            1. 1

                                                                                                                                              I’m not going to defend the reality of what NPM packaging looks like right now because it sucks but that’s the ecosystem we’re stuck with for the time being until something better comes along. As with social networks, packaging systems are also about network effects.

                                                                                                                                              But you can’t deny that this is the norm today. Well, you can, but you would be wrong.

                                                                                                                                              This is making difficult to package Mastodon and Pleroma in Debian and Ubuntu

                                                                                                                                              I’m sure it is, because dpkg is a wholly unsuitable tool for this use-case. You shouldn’t even try. Anyone who doesn’t know how to set these things up themselves should use the Docker container.

                                                                                                                                              1. 1

                                                                                                                                                I think the most difficult part of the Debian packaging would be the js deps, correct?

                                                                                                                                                1. 3

                                                                                                                                                  Yes and no. Unvendorizing dependencies is done mostly for security and requires a lot of work depending on the amount of dependencies. Sometimes js libraries don’t create serious security concerns because they are only run client-side and can be left in vendorized form.

                                                                                                                                                  The Ruby libraries can be also difficult to unvendorize because many upstream developers introduce breaking changes often. They care little about backward compatibility, packaging and security.

                                                                                                                                                  Yet server-side code is more security-critical and that becomes a problem. And it’s getting even worse with new languages that strongly encourage static linking and vendorization.

                                                                                                                                                  1. 1

                                                                                                                                                    I can’t believe even Debian adopted the Googlism of “vendor” instead of “bundle”.

                                                                                                                                                    That aside, Rust? In Mastodon? I guess the Ruby gems it requires would be the bigger problem?

                                                                                                                                                    1. 2

                                                                                                                                                      The use of the word is mine: I just heard people using “vendor” often. It’s not “adopted by Debian”.

                                                                                                                                                      I don’t understand the second part: maybe you misread Ruby for Rust in my text?

                                                                                                                                                      1. 1

                                                                                                                                                        No, I really just don’t know what Rust has to do with Mastodon. There’s Rust in there somewhere? I just didn’t notice.

                                                                                                                                                        1. 2

                                                                                                                                                          AFAICT there is no Rust in the repo (at least at the moment).

                                                                                                                                                          1. 1

                                                                                                                                                            Wow, I’m so dumb, I keep seeing Rust where there is none and misunderstanding you, so sorry!

                                                                                                                                              2. 7

                                                                                                                                                Great. Then have two implementations, one for users with large footprints, and another for casual users with five friends.

                                                                                                                                                It is a reasonable stack if you will devote 1+ servers to the task. Not for something you might want to run on your RPI next to your irc server (a single piece of software in those stacks too)

                                                                                                                                                1. 4

                                                                                                                                                  Having more than one implementation is healthy.

                                                                                                                                                  1. 2

                                                                                                                                                    Of course it is. Which is why it’s a reasonable solution to the large stack required by the current primary implementation.

                                                                                                                                              3. 6

                                                                                                                                                There’s really one database and one cache there. I mean, I guess technically Redis is a database, but it’s almost always used for caching and not as a DB layer like PSQL.

                                                                                                                                                You can always write your own server if you want in whatever language you choose if you feel like Ruby/Node is too much. Or, like that other guy said, you can just use Docker.

                                                                                                                                                1. 4

                                                                                                                                                  There’s really one database and one cache there. I mean, I guess technically Redis is a database, but it’s almost always used for caching . . .

                                                                                                                                                  A project that can run on a single instance of the application binary absolutely does not need a cache. Nor does it need a pub/sub or messaging system outside of its process space.

                                                                                                                                                  1. 2

                                                                                                                                                    It’s more likely that Redis is being used for pub/sub messaging and job queuing.

                                                                                                                                                  2. 11

                                                                                                                                                    This does not seem like a reasonable set of dependencies to me

                                                                                                                                                    Huh. I must be just used to this, then. At work I need to use or at least somewhat understand,

                                                                                                                                                    • Postgres
                                                                                                                                                    • Python 2
                                                                                                                                                    • Python 3
                                                                                                                                                    • Django
                                                                                                                                                    • Ansible
                                                                                                                                                    • AWS
                                                                                                                                                    • Git (actually, Mercurial, but this is my choice to avoid using git)
                                                                                                                                                    • Redis
                                                                                                                                                    • Concourse
                                                                                                                                                    • Docker
                                                                                                                                                    • Emacs (My choice, but I could pick anything else)
                                                                                                                                                    • Node
                                                                                                                                                    • nginx
                                                                                                                                                    • Flask
                                                                                                                                                    • cron
                                                                                                                                                    • Linux
                                                                                                                                                    • RabbitMQ
                                                                                                                                                    • Celery
                                                                                                                                                    • Vagrant (well, optional, I actually do a little extra work to have everything native and avoid a VM)
                                                                                                                                                    • The occasional bit of C code

                                                                                                                                                    and so on and so forth.

                                                                                                                                                    Do I just work at a terrible place or is this a reasonable amount of things to have to deal with in this business? I honestly don’t know.

                                                                                                                                                    To me Mastodon’s requirements seem like a pretty standard Rails application. I’m not even sure why Redis is considered another db – it seems like an in-memory cache with optional disk persistence is a different thing than a persistent-only RDBMS. Nor do I even see much of a problem with two interpreted languages – the alternative would be to have js everywhere, since you can’t have Python or Ruby in a web browser, and js just isn’t a pleasant language for certain tasks.

                                                                                                                                                    1. 38

                                                                                                                                                      I can work with all that and more if you pay me. For stuff I’m running at home on my own time, fuck no. When I shut my laptop to leave the office, it stays shut until I’m back again in the morning, or I get paged.

                                                                                                                                                      1. 2

                                                                                                                                                        So is Mastodon unusual for a Rails program? I wonder if it’s simply unreasonable to ask people to run their own Rails installation. I honestly don’t know.

                                                                                                                                                        Given the amount of Mastodon instances out there, though, it seems that most people manage. How?

                                                                                                                                                        1. 4

                                                                                                                                                          That looks like a bog-standard, very minimal rails stack with a JS frontend. I’m honestly not sure how one could simplify it below that without dropping the JS on the web frontend and any caching, both of which seem like a bad idea.

                                                                                                                                                          1. 7

                                                                                                                                                            There’s no need to require node. The compilation should happen at release time, and the release download tarball should contain all the JS you need.

                                                                                                                                                            1. -3

                                                                                                                                                              lol “download tarball”, you’re old, dude.

                                                                                                                                                              1. 7

                                                                                                                                                                Just you wait another twenty years, and you too will be screaming at the kids to get off your lawn.

                                                                                                                                                            2. 2

                                                                                                                                                              You could remove Rails and use something Node-based for the backend. I’m not claiming that’s a good idea (in fact it’s probably not very reasonable), but it’d remove that dependency?

                                                                                                                                                              1. 1

                                                                                                                                                                it could just have been a go or rust binary or something along those lines, with an embedded db like bolt or sqlite

                                                                                                                                                                edit: though the reason i ignore mastodon is the same as cullum, culture doesn’t seem interesting, at least on mastodon.social

                                                                                                                                                              2. 4

                                                                                                                                                                If security or privacy focused, I’d try a combo like this:

                                                                                                                                                                1. Safe language with minimal runtime that compiles to native code and Javascript. Web framework in that language for dynamic stuff.

                                                                                                                                                                2. Lwan web server for static content.

                                                                                                                                                                3. SQLite for database.

                                                                                                                                                                4. Whatever is needed to combine them.

                                                                                                                                                                Combo will be smaller, faster, more reliable, and more secure.

                                                                                                                                                                1. 2

                                                                                                                                                                  I don’t think this is unusual for a Rails app. I just don’t want to set up or manage a Rails app in my free time. Other people may want to, but I don’t.

                                                                                                                                                              3. 7

                                                                                                                                                                I don’t think it’s reasonable to compare professional requirements and personal requirements.

                                                                                                                                                                1. 4

                                                                                                                                                                  The thing is, Mastodon is meant to be used on-premise. If you’re building a service you host, knock yourself out! Use 40 programming languages and 40 DBs at the same time. But if you want me to install it, keep it simple :)

                                                                                                                                                                  1. 4

                                                                                                                                                                    Personally, setting up all that seems like too much work for a home server, but maybe I’m just lazy. I had a similar issue when setting up Matrix and ran into an error message that I just didn’t have the heart to debug, given the amount of moving parts which I had to install.

                                                                                                                                                                    1. 3

                                                                                                                                                                      If you can use debian, try installing synapse via their repository, it works really nice for me so far: https://matrix.org/packages/debian/

                                                                                                                                                                      1. 1

                                                                                                                                                                        Reading other comments about the horror that is Docker, it is a wonder that you dare propose to install an entire OS only to run a Matrix server. ;)

                                                                                                                                                                        1. 3

                                                                                                                                                                          i’m not completely sure which parts of you comment are sarcasm :)

                                                                                                                                                                    2. 0

                                                                                                                                                                      Your list there has lots of tools with overlapping functionality, seems like pointless redundancy. Just pick flask OR django. Just pick python3 or node, just pick docker or vagrant, make a choice, remove useless and redundant things.

                                                                                                                                                                      1. 3

                                                                                                                                                                        We have some Django applications and we have some Flask applications. They have different lineages. One we forked and one we made ourselves.

                                                                                                                                                                    3. 6

                                                                                                                                                                      Alternatively you install it using the Docker as described here.

                                                                                                                                                                      1. 32

                                                                                                                                                                        I think it’s kinda sad that the solution to “control your own toots” is “give up control of your computer and install this giant blob of software”.

                                                                                                                                                                        1. 9

                                                                                                                                                                          Piling another forty years of hexadecimal Unix sludge on top of forty years of slightly different hexadecimal Unix sludge to improve our ability to ship software artifacts … it’s an aesthetic nightmare. But I don’t fully understand what our alternatives are.

                                                                                                                                                                          I’ve never been happier to be out of the business of having to think about this in anything but the most cursory detail.

                                                                                                                                                                          1. 11

                                                                                                                                                                            I mean how is that different from running any binary at the end of the day. Unless you’re compiling everything from scratch on the machine starting from the kernel. Running Mastodon from Docker is really no different. And it’s not like anybody is stopping you from either making your own Dockerfile, or just setting up directly on your machine by hand. The original complaint was that it’s too much work, and if that’s a case you have a simple packaged solution. If you don’t like it then roll up the sleeves and do it by hand. I really don’t see the problem here I’m afraid.

                                                                                                                                                                            1. 11

                                                                                                                                                                              “It’s too much work” is a problem.

                                                                                                                                                                              1. 5

                                                                                                                                                                                Unless you’re compiling everything from scratch on the machine starting from the kernel

                                                                                                                                                                                I use NixOS. I have a set of keys that I set as trusted for signature verification of binaries. The binaries are a cache of the build derivation, so I could theoretically build the software from scratch, if I wanted to, or to verify that the binaries are the same as the cached versions.

                                                                                                                                                                                1. 2

                                                                                                                                                                                  Right, but if you feel strongly about that then you can make your own Dockerfile from source. The discussion is regarding whether there’s a simple way to get an instance up and running, and there is.

                                                                                                                                                                                  1. 3

                                                                                                                                                                                    Docker containers raise a lot of questions though, even if you use a Dockerfile:

                                                                                                                                                                                    • What am I running?
                                                                                                                                                                                    • Which versions am I running?
                                                                                                                                                                                    • Do the versions have security vulnerabilities?
                                                                                                                                                                                    • Will I be able to build the exact same version in 24 months?

                                                                                                                                                                                    Nix answers these pretty will and fairly accurately.

                                                                                                                                                                                2. 2

                                                                                                                                                                                  Unless you’re compiling everything from scratch on the machine starting from the kernel.

                                                                                                                                                                                  You mean starting with writing a bootstrapping compiler in assembly, then writing your own full featured compiler and compiling it in the bootstrapping compiler. Then moving on to compiling the kernel.

                                                                                                                                                                                  1. 1

                                                                                                                                                                                    No no, your assembler could be compromised ;)

                                                                                                                                                                                    Better write raw machine code directly onto the disk. Using, perhaps, a magnetized needle and a steady hand, or maybe a butterfly.

                                                                                                                                                                                    1. 2

                                                                                                                                                                                      My bootstrapping concept was having the device boot a program from ROM that takes in the user-supplied, initial program via I/O into RAM. Then passes execution to it. You enter the binary through one of those Morse code things with four buttons: 0, 1, backspace, and enter. Begins executing on enter.

                                                                                                                                                                                      Gotta input the keyboard driver next in binary to use a keyboard. Then the display driver blind using the keyboard. Then storage driver to save things. Then, the OS and other components. ;)

                                                                                                                                                                                    2. 1

                                                                                                                                                                                      If I deploy three Go apps on top of a bare OS (picked Go since it has static binaries), and the Nginx server in front of all 3 of them uses OpenSSL, then I have one OpenSSL to patch whenever the inevitable CVE rolls around. If I deploy three Docker container apps on top of a bare OS, now I have four OpenSSLs to patch - three in the containers and one in my base OS. This complexity balloons very quickly which is terrible for user control. Hell, I have so little control over my one operating system that I had to carefully write a custom tool just to make sure I didn’t miss logfile lines in batch summaries created by cron. How am I supposed to manage four? And three with radically different tooling and methodology to boot.

                                                                                                                                                                                      And Docker upstream, AFAIK, has provided nothing to help with the security problem which is probably why known security vulnerabilities in Docker images are rampant. If they have I would like to know because if it’s decent I would switch to it immediately. See this blog post for more about this problem (especially including links) and how we “solved” it in pump.io (spoiler: it’s a giant hack).

                                                                                                                                                                                      1. 3

                                                                                                                                                                                        That’s not how any of this works. You package the bare minimum needed to run the app in the Docker container, then you front all your containers with a single Nginx server that handles SSL. Meanwhile, there are plenty of great tools, like Dokku for managing Docker based infrastructure. Here’s how you provision a server using Let’s Encrypt with Dokku:

                                                                                                                                                                                        sudo dokku plugin:install https://github.com/dokku/dokku-letsencrypt.git
                                                                                                                                                                                        okku letsencrypt:auto-renew
                                                                                                                                                                                        

                                                                                                                                                                                        viewing logs isn’t rocker science either:

                                                                                                                                                                                        dokku logs myapp
                                                                                                                                                                                        
                                                                                                                                                                                        1. 1

                                                                                                                                                                                          OK, so OpenSSL was a bad example. Fair enough. But I think my point still stands - you’ll tend to have at least some duplicate libraries across Docker containers. There’s tooling around managing security vulnerabilities in language-level dependencies; see for example Snyk. But Docker imports the entire native package manager into the “static binary” and I don’t know of any tooling that can track problems in Docker images like that. I guess I could use Clair through Quay but… I don’t know. This doesn’t feel like as nice of a solution or as polished somehow. As an image maintainer I’ve added a big manual burden keeping up with native security updates in addition to those my application actually directly needs, when normally I could rely on admins to do that, probably with lots of automation.

                                                                                                                                                                                          1. 3

                                                                                                                                                                                            you’ll tend to have at least some duplicate libraries across Docker containers

                                                                                                                                                                                            That is literally the entire point. Application dependencies must be separate from one another, because even on a tight-knit team keeping n applications in perfect lockstep is impossible.

                                                                                                                                                                                            1. 1

                                                                                                                                                                                              OS dependencies are different than application dependencies. I can apply a libc patch on my Debian server with no worry because I know Debian works hard to create a stable base server environment. That’s different than application dependencies, where two applications are much more likely to require conflicting versions of libraries.

                                                                                                                                                                                              Now, I run most of my stuff on a single server so I’m very used to a heterogeneous environment. Maybe that’s biasing me against Docker. But isn’t that the usecase we’re discussing here anyway? How someone with just a hobbyist server can run Mastodon?

                                                                                                                                                                                              Thinking about this more I feel like a big part of what bothers me about Docker, and therefore about Clair, is that there’s no package manifest. Dockerfile does not count, because that’s not actually a package manifest, it’s just a list of commands. I can’t e.g. build a lockfile format on top of that, which is what tools like Snyk analyze. Clair is the equivalent of having to run npm install and then go trawling through node_modules looking for known vulnerable code instead of just looking at the lockfile. More broadly, because Docker lacks any notion of a package manifest, it seems to me that while Docker images are immutable once built, the build process that leads you there cannot be made deterministic. This is what makes it hard to keep track of the stuff inside them. I will have to think about this more - as I write this comment I’m wondering if my complaints about duplicated libraries and tracking security there is an instance of the XY problem or if they really are separate things in my mind.

                                                                                                                                                                                              Maybe I am looking for something like Nix or Guix inside a Docker container. Guix at least can export Docker containers; I suppose I should look into that.

                                                                                                                                                                                              1. 2

                                                                                                                                                                                                OS dependencies are different than application dependencies.

                                                                                                                                                                                                Yes, agreed.

                                                                                                                                                                                                Thinking about this more I feel like a big part of what bothers me about Docker, and therefore about Clair, is that there’s no package manifest. Dockerfile does not count, because that’s not actually a package manifest, it’s just a list of commands. I can’t e.g. build a lockfile format on top of that, which is what tools like Snyk analyze.

                                                                                                                                                                                                You don’t need a container to tell you these things. Application dependencies can be checked for exploits straight from the code repo, i.e. brakeman. Both the Gemfile.lock and yarn.lock are available from the root of the repo.

                                                                                                                                                                                                The container artifacts are most like built automatically for every merge to master, and that entails doing a full system update from the apt repository. So in reality, while not as deterministic as the lockfiles, the system deps in a container are likely to be significantly fresher than a regular server environment.

                                                                                                                                                                                            2. 1

                                                                                                                                                                                              You’d want to track security vulnerabilities outside your images though. You’d do it at dev time, and update your Dockerfile with updated dependencies when you publish the application. Think of Docker as just a packaging mechanism. It’s same as making an uberjar on the JVM. You package all your code into a container, and run the container. When you want to make updates, you blow the old one away and run a new one.

                                                                                                                                                                                      2. 4

                                                                                                                                                                                        I have only rarely used Docker, and am certainly no booster, so keep that in mind as I ask this.

                                                                                                                                                                                        From the perspective of “install this giant blob of software”, do you see a docker deployment being that different from a single large binary? Particularly the notion of the control that you “give up”, how does that differ between Docker and $ALTERNATIVE?

                                                                                                                                                                                        1. 14

                                                                                                                                                                                          Ideally one would choose door number three, something not so large and inauditable. The complaint is not literally about Docker, but the circumstances which have resulted in docker being the most viable deployment option.

                                                                                                                                                                                        2. 2

                                                                                                                                                                                          You have the dockerfile and can reconstruct. You haven’t given up control.

                                                                                                                                                                                          1. 5

                                                                                                                                                                                            Is there a youtube video I can watch of somebody building a mastodon docker image from scratch?

                                                                                                                                                                                            1. 1

                                                                                                                                                                                              I do not know of one.

                                                                                                                                                                                      3. 3

                                                                                                                                                                                        I totally agree as well, and I wish authors would s/Mastodon/Fediverse/ in their articles. As others have noted, Pieroma is another good choice and others are getting into the game - NextCloud added fediverse node support in their most recent release as a for-instance.

                                                                                                                                                                                        I tried running my own instance for several months, and it eventually blew up. In addition to the large set of dependencies, the system is overall quite complex. I had several devs from the project look at my instance, and the only thing they could say is it was a “back-end problem” (My instance had stopped getting new posts).

                                                                                                                                                                                        I gave up and am now using somebody else’s :) I love the fediverse though, it’s a fascinating place.

                                                                                                                                                                                        1. 4

                                                                                                                                                                                          I just use the official Docker containers. The tootsuite/mastodon container can be used to launch web, streaming, sidekiq and even database migrations. Then you just need an nginx container, a redis container, a postgres container and an optional elastic search container. I run it all on a 2GB/1vCPU Vultr node (with the NJ data center block store because you will need a lot of space) and it works fairly well (I only have ~10 users; small private server).

                                                                                                                                                                                          In the past I would agree with out (and it’s the reason I didn’t try out Diaspora years ago when it came out), but containers have made it easier. I do realize they both solve and cause problems and by no means think they’re the end all of tech, but they do make running stuff like this a lot easier.

                                                                                                                                                                                          If anyone wants to find me, I’m @djsumdog@hitchhiker.social

                                                                                                                                                                                          1. 2

                                                                                                                                                                                            Given that there’s a space for your Twitter handle, i wish Lobste.rs had a Mastodon slot as well :)

                                                                                                                                                                                          2. 2

                                                                                                                                                                                            Wait, you’re also forgetting systemd to keep all those process humming… :)

                                                                                                                                                                                            You’re right that this is clearly too much: I have run such systems for work (Rails’ pretty common), but would probably not do that for fun. I am amazed, and thankful, for the people who volunteer the effort to run all this on their week-ends.

                                                                                                                                                                                            Pleroma does look simpler… If I really wanted to run my own instance, I’d look in that direction. ¯_(ツ)_/¯

                                                                                                                                                                                            1. 0

                                                                                                                                                                                              I’m waiting for urbit.org to reach useability. Which I expect for my arbitrary feeling of useability to come about late this year. Then the issue is coming up to speed on a new language and integrated network, OS, build system.

                                                                                                                                                                                              1. 2

                                                                                                                                                                                                Urbit is apparently creating a feudal society. (Should note that I haven’t really dug into that thread for several years and am mostly taking @pushcx at his word.)

                                                                                                                                                                                                1. 1

                                                                                                                                                                                                  The feudal society meme is just not true, and, BTW, Yarvin is no longer associated with Urbit. https://urbit.org/primer/

                                                                                                                                                                                              2. 1

                                                                                                                                                                                                I would love to have(make) a solution that could be used locally with sqlite and in aws with lambda, api gateway and dynamodb. That would allow scaling cost and privacy/controll.

                                                                                                                                                                                                1. 3

                                                                                                                                                                                                  https://github.com/deoxxa/don is sort of in that direction (single binary, single file sqlite database).