1. 14

    Thanks, I feel like a dick, now.

    On a serious note, that hit me right in the feels. I wouldn’t go as far as saying that I am overconfident, but I’m definitely a loudmouth, and tend to argue to exhaustion when I think I’m right.

    I’ve been working for a while in recognizing that I might be wrong and try to clearly communicate that I am open (and in fact, often hoping) to be proven wrong, but it’s a process, and I can’t help but wonder how often have I just been a massive dick and prevented other people from even throwing an idea =(

    1. 8

      I’m almost exactly the opposite. I realize that usually my preferences boil down to: “I like this way because I’m used to it”, or “I don’t think it matters”. I used to be the most experienced C# programmer on my team. My teammates used to ask me questions like “Should I assign a type explicitly or just use ‘var’?” any I would say that that’s a personal preference and it doesn’t really matter. Another programmer with a much more pronounced opinion joined the team, and I noticed that people liked his answers a lot more. Sometimes people just like to be told what to do, especially when the consequences of the choices aren’t so clear.

      1. 5

        Sometimes people just like to be told what to do, especially when the consequences of the choices aren’t so clear.

        This, and also cases where the consequences on either side of a tradeoff are so minor, that the energy spent deliberating them out is costlier than just making a call and moving on. I spent far longer learning this, and fixing my behavior, than I wish I had.

        On so many of these low-impact-low-cost details, where I did not strongly care, where I could expect another senior to reasonably disagree (even just for reasons of “this is what I am used to instead”), I would give teammates a full-depth, longwinded explanation, only to end in an equivocation and let them decide. Maybe one out of a dozen times they learned something, but often I was wasting people’s time when they just wanted a boolean result from the ask_swifthand function.

        The first step in fixing this was realizing why I always led with longwinded explanations. It turned out to be misapplied empathy (which I assume-by-default that I lack, hence it took a while to realize).

        I personally value knowing the detailed ins-and-outs wherever possible, and so I saw extending a full explanation to another person as a sign of respect: share all the learning, and help them make their own decision. But when that comes by wasting their time (on a low-impact-low-cost choice), was not being perceived that way.

        Change was slow. I first changed my explanations to lead with the decision before launching into the explanation, giving a TL;DR to frame their expectation. After some time doing this, I began leading with the decision and then asking “There are reasons I could dive into, but they are subtle and inconsequential. So I’ll leave it there, unless you are curious?” followed by a long, awkward pause while I wait for an answer.

        Due to some personality quirks that pause is the hardest part for me, but it gets the job done. It balances respecting their time by giving them the option to walk away with my desire to extend a courtesy that I would (personally) value of learning more.

        Ironically, I do not always extend this sort of preemptive empathy to the rest of my life’s conversations. The last paragraph in the article really hit home, as I know friends and family often feel my SOLH is overconfidence layered with stubbornness.

        1. 3

          Sometimes people just like to be told what to do, especially when the consequences of the choices aren’t so clear.

          Newbies need to be fed opinions because they don’t have enough experience to have any of their own, and the ones they might have at the moment are more likely to be wrong than right. Like, yes, you can write thousand-line subroutines in Python, that’s certainly something the language allows, but more mature opinion will state that writing smaller subroutines is better for reasons of ease of modification a neophyte hasn’t run into yet.

          Gnomonic pronouncements make things go faster, but once the learner has some experience of their own, pushback is to be expected and should be encouraged. Except with things like styleguides. Those are completely arbitrary and set in stone for precisely that reason.

          So teachers need to have strong opinions because they can’t go over every single thing when they answer a single question, but those opinions must be held loosely enough that students can successfully challenge them as they gain enough knowledge to exercise their own agency.

          1. 1

            This is one thing that’s true: people like to be told what to do. People should be encouraged to think for themselves. The best of both worlds is perhaps by explaining the difference and explaining why you lean the way you do. That way you teach thought and give a straight answer.

          2. 5

            tend to argue to exhaustion

            that doesn’t sound “loosely held” :)

            1.  

              I recently had an argument that went on for an hour until I conceded from frustration. Once we switched from the what to the how they immediately took to my original argument. Loosely held seems like it can be in the eye of the beholder.

              1. 1

                Well, for some definition of loosely =P

              2. 3

                I am open (and in fact, often hoping) to be proven wrong

                Say that clearly and directly. Tell the other person that you are not feeling personally attacked and hope to be proven wrong.

              1. 65

                In the Mastodon universe, technically-minded users are encouraged to run their own node. Sounds good. To install a Mastodon node, I am instructed to install recent versions of

                • Ruby
                • Node.JS
                • Redis
                • PostgreSQL
                • nginx

                This does not seem like a reasonable set of dependencies to me. In particular, using two interpreted languages, two databases, and a separate web server presumably acting as a frontend, all seems like overkill. I look forward to when the Mastodon devs are able to tame this complexity, and reduce the codebase to a something like single (ideally non-interpreted) language and a single database. Or, even better, a single binary that manages its own data on disk, using e.g. embedded SQLite. Until then, I’ll pass.

                1. 22

                  Totally agree. I heard Pleroma has less dependencies though it looks like it depends a bit on which OS you’re running.

                  1. 11

                    Compared to Mastodon, Pleroma is a piece of cake to install; I followed their tutorial and had an instance set up and running in about twenty minutes on a fresh server.

                    From memory all I needed install was Nginx, Elixir and Postgres, two of which were already set up and configured for other projects.

                    My server is a quad core ARMv7 with 2GB RAM and averages maybe 0.5 load when I hit heavy usage… it does transit a lot of traffic though, since the 1st January my server has pushed out 530GB of traffic.

                    1. 2

                      doesnt Elixir require Erlang to run?

                      1. 2

                        It does. Some linux distributions will require adding the Erlang repo before installing elixir but most seem to have it already included: https://elixir-lang.org/install.html#unix-and-unix-like meaning its a simple one line command to install e.g pkg install elixir

                    2. 7

                      I’m not a huge social person, but I had only heard of Pleroma without investigating it. After looking a bit more, I don’t really understand why someone would choose Mastodon over Pleroma. They do basically the same thing, but Pleroma takes less resources. Anyone who chose Mastodon over Pleroma have a reason why?

                      1. 6

                        Mastodon has more features right now. That’s about it.

                        1. 4

                          Pleroma didn’t have releases for a looong time. They finally started down that route. They also don’t have official Docker containers and config changes require recompiling (just due to the way they have Elixir and builds setup). It was a pain to write my Docker container for it.

                          Pleroma also lacks moderation tools (you need to add blocked domains to the config), it doesn’t allow remote follow/interactions (if you see a status elsewhere on Mastodon, you can click remote-reply, it will ask your server name, redirect you to your server and then you can reply to someone you don’t follow) and a couple of other features.

                          Misskey is another alternative that looks promising.

                          1. 2

                            it doesn’t allow remote follow/interactions (if you see a status elsewhere on Mastodon, you can click remote-reply, it will ask your server name, redirect you to your server and then you can reply to someone you don’t follow)

                            I think that might just be the Pleroma FA - if I’m using the Mastodon FE, I get the same interaction on my Pleroma instance replying to someone on a different instance as when I’m using octodon.social (unless I’m radically misunderstanding your sentence)

                            1. 1

                              Thanks, this is a really great response. I actually took a quick look at their docs and saw they didn’t have any FreeBSD guide set up, so I stopped looking. I use Vultr’s $2.50 FreeBSD vps and I didn’t feel like fiddling with anything that particular night. I wish they did have an official docker container for it.

                            2. 3

                              Pleroma has a bunch of fiddly issues - it doesn’t do streaming properly (bitlbee-mastodon won’t work), the UI doesn’t have any “compose DM” functionality that I can find, I had huge problems with a long password, etc. But they’re mostly minor annoyances than show stoppers for now.

                            3. 7

                              It doesn’t depend - they’ve just gone further to define what to do for each OS!

                              1. 4

                                I guess it’s mainly the ImageMagick dependency for OpenBSD that got me thinking otherwise.

                                OpenBSD

                                • elixir
                                • gmake
                                • ImageMagick
                                • git
                                • postgresql-server
                                • postgresql-contrib

                                Debian Based Distributions

                                • postgresql
                                • postgresql-contrib
                                • elixir
                                • erlang-dev
                                • erlang-tools
                                • erlang-parsetools
                                • erlang-xmerl
                                • git
                                • build-essential
                                1. 3

                                  imagemagick is purely optional. The only hard dependencies are postgresql and elixir (and some reverse proxy like nginx)

                                  1. 4

                                    imagemagick is strongly recommended though so you can enable the Mogrify filter on uploads and actually strip exif data

                              2. 3

                                Specifically, quoting from their readme:

                                Pleroma is written in Elixir, high-performance and can run on small devices like a Raspberry Pi.

                                As to the DB, they seem to use Postgres.

                                The author of the app posted his list of differences, but I’m not sure if it’s complete and what it really means. I haven’t found a better comparison yet, however.

                              3. 16

                                Unfortunately I have to agree. I self-host 99% of my online services, and sysadmin for a living. I tried mastodon for a few months, but its installation and management process was far more complicated than anything I’m used to. (I run everything on OpenBSD, so the docker image isn’t an option for me.)

                                In addition to getting NodeJS, Ruby, and all the other dependencies installed, I had to write 3 separate rc files to run 3 separate daemons to keep the thing running. Compared to something like Gitea, which just requires running a single Go executable and a Postgres DB, it was a massive amount of toil.

                                The mastodon culture really wasn’t a fit for me either. Even in technical spaces, there was a huge amount of politics/soapboxing. I realized I hadn’t even logged in for a few weeks so I just canned my instance.

                                Over the past year I’ve given up on the whole social network thing and stick to Matrix/IRC/XMPP/email. I’ve been much happier as a result and there’s a plethora of quality native clients (many are text-based). I’m especially happy on Matrix now that I’ve discovered weechat-matrix.

                                I don’t mean to discourage federated projects like Mastodon though - I’m always a fan of anything involving well-known URLs or SRV records!

                                1. 11

                                  Fortunately the “fediverse” is glued by a standard protocol (ActivityPub) that is quite simple so if one implementation (e.g. Mastodon) doesn’t suit someone’s needs it’s not a big problem - just searching for a better one and it still interconnects with the rest of the world.

                                  (I’ve written a small proof-of-concept ActivityPub clients and servers, it works and federates, see also this).

                                  For me the more important problems are not implementation issues with one server but rather design issues within the protocol. For example established standards such as e-mail or XMPP have a way to delegate responsibility of running a server of a particular protocol but still use bare domain for user identifies. In e-mail that is MX records in XMPP it’s DNS SRV records. ActivityPub doesn’t demand anything like it and even though Mastodon tries to provide something that would fix that issue - WebFinger, other implementations are not interested in that (e.g. Pleroma). And then one is left with instances such as “social.company.com”.

                                  For example - Pleroma’s developer’s id is lain@pleroma.soykaf.com.

                                  1. 16

                                    This is a completely reasonable and uncontroversial set of dependencies for a web app. Some of the largest web apps on the Internet run this stack. That is a good thing, because when Fediverse nodes need to scale there are well-understood ways of doing it.

                                    Success in social networking is entirely about network effects and that means low barrier to entry is table stakes. Yeah, it’d be cool if someone built the type of node you’re talking about, but it would be a curiosity pursued only by the most technical users. If that were the barrier to entry for the network, there would be no network.

                                    1. 39

                                      This is a completely reasonable and uncontroversial set of dependencies for a web app. Some of the largest web apps on the Internet run this stack.

                                      Yes, but not for a web app I’m expected to run on my own time, for fun.

                                      1. 6

                                        I’m not sure that’s the exact expectation, that we all should run our single-user Mastodon instances. I feel like the expectation is that sysadmin with enough knowledge will maintain an instance for many users. This seems to be the norm.

                                        That, or you go to Mastohost and pay someone else for your own single-user instance.

                                        1. 2

                                          You’re not expected to do that is my point.

                                        2. 16

                                          completely reasonable and uncontroversial

                                          Not true. Many people are complaining about the unmanaged proliferation of dependencies and tools. Most projects of this size and complexity don’t need more than one language, bulky javascript frameworks, caching and database services.

                                          This is making difficult to package Mastodon and Pleroma in Debian and Ubuntu and making it more difficult for people to make the service really decentralized.

                                          1. 1

                                            I’m not going to defend the reality of what NPM packaging looks like right now because it sucks but that’s the ecosystem we’re stuck with for the time being until something better comes along. As with social networks, packaging systems are also about network effects.

                                            But you can’t deny that this is the norm today. Well, you can, but you would be wrong.

                                            This is making difficult to package Mastodon and Pleroma in Debian and Ubuntu

                                            I’m sure it is, because dpkg is a wholly unsuitable tool for this use-case. You shouldn’t even try. Anyone who doesn’t know how to set these things up themselves should use the Docker container.

                                            1. 1

                                              I think the most difficult part of the Debian packaging would be the js deps, correct?

                                              1. 3

                                                Yes and no. Unvendorizing dependencies is done mostly for security and requires a lot of work depending on the amount of dependencies. Sometimes js libraries don’t create serious security concerns because they are only run client-side and can be left in vendorized form.

                                                The Ruby libraries can be also difficult to unvendorize because many upstream developers introduce breaking changes often. They care little about backward compatibility, packaging and security.

                                                Yet server-side code is more security-critical and that becomes a problem. And it’s getting even worse with new languages that strongly encourage static linking and vendorization.

                                                1. 1

                                                  I can’t believe even Debian adopted the Googlism of “vendor” instead of “bundle”.

                                                  That aside, Rust? In Mastodon? I guess the Ruby gems it requires would be the bigger problem?

                                                  1. 2

                                                    The use of the word is mine: I just heard people using “vendor” often. It’s not “adopted by Debian”.

                                                    I don’t understand the second part: maybe you misread Ruby for Rust in my text?

                                                    1. 1

                                                      No, I really just don’t know what Rust has to do with Mastodon. There’s Rust in there somewhere? I just didn’t notice.

                                                      1. 2

                                                        AFAICT there is no Rust in the repo (at least at the moment).

                                                        1. 1

                                                          Wow, I’m so dumb, I keep seeing Rust where there is none and misunderstanding you, so sorry!

                                            2. 7

                                              Great. Then have two implementations, one for users with large footprints, and another for casual users with five friends.

                                              It is a reasonable stack if you will devote 1+ servers to the task. Not for something you might want to run on your RPI next to your irc server (a single piece of software in those stacks too)

                                              1. 4

                                                Having more than one implementation is healthy.

                                                1. 2

                                                  Of course it is. Which is why it’s a reasonable solution to the large stack required by the current primary implementation.

                                            3. 6

                                              There’s really one database and one cache there. I mean, I guess technically Redis is a database, but it’s almost always used for caching and not as a DB layer like PSQL.

                                              You can always write your own server if you want in whatever language you choose if you feel like Ruby/Node is too much. Or, like that other guy said, you can just use Docker.

                                              1. 4

                                                There’s really one database and one cache there. I mean, I guess technically Redis is a database, but it’s almost always used for caching . . .

                                                A project that can run on a single instance of the application binary absolutely does not need a cache. Nor does it need a pub/sub or messaging system outside of its process space.

                                                1. 2

                                                  It’s more likely that Redis is being used for pub/sub messaging and job queuing.

                                                2. 11

                                                  This does not seem like a reasonable set of dependencies to me

                                                  Huh. I must be just used to this, then. At work I need to use or at least somewhat understand,

                                                  • Postgres
                                                  • Python 2
                                                  • Python 3
                                                  • Django
                                                  • Ansible
                                                  • AWS
                                                  • Git (actually, Mercurial, but this is my choice to avoid using git)
                                                  • Redis
                                                  • Concourse
                                                  • Docker
                                                  • Emacs (My choice, but I could pick anything else)
                                                  • Node
                                                  • nginx
                                                  • Flask
                                                  • cron
                                                  • Linux
                                                  • RabbitMQ
                                                  • Celery
                                                  • Vagrant (well, optional, I actually do a little extra work to have everything native and avoid a VM)
                                                  • The occasional bit of C code

                                                  and so on and so forth.

                                                  Do I just work at a terrible place or is this a reasonable amount of things to have to deal with in this business? I honestly don’t know.

                                                  To me Mastodon’s requirements seem like a pretty standard Rails application. I’m not even sure why Redis is considered another db – it seems like an in-memory cache with optional disk persistence is a different thing than a persistent-only RDBMS. Nor do I even see much of a problem with two interpreted languages – the alternative would be to have js everywhere, since you can’t have Python or Ruby in a web browser, and js just isn’t a pleasant language for certain tasks.

                                                  1. 38

                                                    I can work with all that and more if you pay me. For stuff I’m running at home on my own time, fuck no. When I shut my laptop to leave the office, it stays shut until I’m back again in the morning, or I get paged.

                                                    1. 2

                                                      So is Mastodon unusual for a Rails program? I wonder if it’s simply unreasonable to ask people to run their own Rails installation. I honestly don’t know.

                                                      Given the amount of Mastodon instances out there, though, it seems that most people manage. How?

                                                      1. 4

                                                        That looks like a bog-standard, very minimal rails stack with a JS frontend. I’m honestly not sure how one could simplify it below that without dropping the JS on the web frontend and any caching, both of which seem like a bad idea.

                                                        1. 7

                                                          There’s no need to require node. The compilation should happen at release time, and the release download tarball should contain all the JS you need.

                                                          1. -3

                                                            lol “download tarball”, you’re old, dude.

                                                            1. 7

                                                              Just you wait another twenty years, and you too will be screaming at the kids to get off your lawn.

                                                          2. 2

                                                            You could remove Rails and use something Node-based for the backend. I’m not claiming that’s a good idea (in fact it’s probably not very reasonable), but it’d remove that dependency?

                                                            1. 1

                                                              it could just have been a go or rust binary or something along those lines, with an embedded db like bolt or sqlite

                                                              edit: though the reason i ignore mastodon is the same as cullum, culture doesn’t seem interesting, at least on mastodon.social

                                                            2. 4

                                                              If security or privacy focused, I’d try a combo like this:

                                                              1. Safe language with minimal runtime that compiles to native code and Javascript. Web framework in that language for dynamic stuff.

                                                              2. Lwan web server for static content.

                                                              3. SQLite for database.

                                                              4. Whatever is needed to combine them.

                                                              Combo will be smaller, faster, more reliable, and more secure.

                                                              1. 2

                                                                I don’t think this is unusual for a Rails app. I just don’t want to set up or manage a Rails app in my free time. Other people may want to, but I don’t.

                                                            3. 7

                                                              I don’t think it’s reasonable to compare professional requirements and personal requirements.

                                                              1. 4

                                                                The thing is, Mastodon is meant to be used on-premise. If you’re building a service you host, knock yourself out! Use 40 programming languages and 40 DBs at the same time. But if you want me to install it, keep it simple :)

                                                                1. 4

                                                                  Personally, setting up all that seems like too much work for a home server, but maybe I’m just lazy. I had a similar issue when setting up Matrix and ran into an error message that I just didn’t have the heart to debug, given the amount of moving parts which I had to install.

                                                                  1. 3

                                                                    If you can use debian, try installing synapse via their repository, it works really nice for me so far: https://matrix.org/packages/debian/

                                                                    1. 1

                                                                      Reading other comments about the horror that is Docker, it is a wonder that you dare propose to install an entire OS only to run a Matrix server. ;)

                                                                      1. 3

                                                                        i’m not completely sure which parts of you comment are sarcasm :)

                                                                  2. 0

                                                                    Your list there has lots of tools with overlapping functionality, seems like pointless redundancy. Just pick flask OR django. Just pick python3 or node, just pick docker or vagrant, make a choice, remove useless and redundant things.

                                                                    1. 3

                                                                      We have some Django applications and we have some Flask applications. They have different lineages. One we forked and one we made ourselves.

                                                                  3. 6

                                                                    Alternatively you install it using the Docker as described here.

                                                                    1. 31

                                                                      I think it’s kinda sad that the solution to “control your own toots” is “give up control of your computer and install this giant blob of software”.

                                                                      1. 9

                                                                        Piling another forty years of hexadecimal Unix sludge on top of forty years of slightly different hexadecimal Unix sludge to improve our ability to ship software artifacts … it’s an aesthetic nightmare. But I don’t fully understand what our alternatives are.

                                                                        I’ve never been happier to be out of the business of having to think about this in anything but the most cursory detail.

                                                                        1. 11

                                                                          I mean how is that different from running any binary at the end of the day. Unless you’re compiling everything from scratch on the machine starting from the kernel. Running Mastodon from Docker is really no different. And it’s not like anybody is stopping you from either making your own Dockerfile, or just setting up directly on your machine by hand. The original complaint was that it’s too much work, and if that’s a case you have a simple packaged solution. If you don’t like it then roll up the sleeves and do it by hand. I really don’t see the problem here I’m afraid.

                                                                          1. 11

                                                                            “It’s too much work” is a problem.

                                                                            1. 5

                                                                              Unless you’re compiling everything from scratch on the machine starting from the kernel

                                                                              I use NixOS. I have a set of keys that I set as trusted for signature verification of binaries. The binaries are a cache of the build derivation, so I could theoretically build the software from scratch, if I wanted to, or to verify that the binaries are the same as the cached versions.

                                                                              1. 2

                                                                                Right, but if you feel strongly about that then you can make your own Dockerfile from source. The discussion is regarding whether there’s a simple way to get an instance up and running, and there is.

                                                                                1. 3

                                                                                  Docker containers raise a lot of questions though, even if you use a Dockerfile:

                                                                                  • What am I running?
                                                                                  • Which versions am I running?
                                                                                  • Do the versions have security vulnerabilities?
                                                                                  • Will I be able to build the exact same version in 24 months?

                                                                                  Nix answers these pretty will and fairly accurately.

                                                                              2. 2

                                                                                Unless you’re compiling everything from scratch on the machine starting from the kernel.

                                                                                You mean starting with writing a bootstrapping compiler in assembly, then writing your own full featured compiler and compiling it in the bootstrapping compiler. Then moving on to compiling the kernel.

                                                                                1. 1

                                                                                  No no, your assembler could be compromised ;)

                                                                                  Better write raw machine code directly onto the disk. Using, perhaps, a magnetized needle and a steady hand, or maybe a butterfly.

                                                                                  1. 2

                                                                                    My bootstrapping concept was having the device boot a program from ROM that takes in the user-supplied, initial program via I/O into RAM. Then passes execution to it. You enter the binary through one of those Morse code things with four buttons: 0, 1, backspace, and enter. Begins executing on enter.

                                                                                    Gotta input the keyboard driver next in binary to use a keyboard. Then the display driver blind using the keyboard. Then storage driver to save things. Then, the OS and other components. ;)

                                                                                  2. 1

                                                                                    If I deploy three Go apps on top of a bare OS (picked Go since it has static binaries), and the Nginx server in front of all 3 of them uses OpenSSL, then I have one OpenSSL to patch whenever the inevitable CVE rolls around. If I deploy three Docker container apps on top of a bare OS, now I have four OpenSSLs to patch - three in the containers and one in my base OS. This complexity balloons very quickly which is terrible for user control. Hell, I have so little control over my one operating system that I had to carefully write a custom tool just to make sure I didn’t miss logfile lines in batch summaries created by cron. How am I supposed to manage four? And three with radically different tooling and methodology to boot.

                                                                                    And Docker upstream, AFAIK, has provided nothing to help with the security problem which is probably why known security vulnerabilities in Docker images are rampant. If they have I would like to know because if it’s decent I would switch to it immediately. See this blog post for more about this problem (especially including links) and how we “solved” it in pump.io (spoiler: it’s a giant hack).

                                                                                    1. 3

                                                                                      That’s not how any of this works. You package the bare minimum needed to run the app in the Docker container, then you front all your containers with a single Nginx server that handles SSL. Meanwhile, there are plenty of great tools, like Dokku for managing Docker based infrastructure. Here’s how you provision a server using Let’s Encrypt with Dokku:

                                                                                      sudo dokku plugin:install https://github.com/dokku/dokku-letsencrypt.git
                                                                                      okku letsencrypt:auto-renew
                                                                                      

                                                                                      viewing logs isn’t rocker science either:

                                                                                      dokku logs myapp
                                                                                      
                                                                                      1. 1

                                                                                        OK, so OpenSSL was a bad example. Fair enough. But I think my point still stands - you’ll tend to have at least some duplicate libraries across Docker containers. There’s tooling around managing security vulnerabilities in language-level dependencies; see for example Snyk. But Docker imports the entire native package manager into the “static binary” and I don’t know of any tooling that can track problems in Docker images like that. I guess I could use Clair through Quay but… I don’t know. This doesn’t feel like as nice of a solution or as polished somehow. As an image maintainer I’ve added a big manual burden keeping up with native security updates in addition to those my application actually directly needs, when normally I could rely on admins to do that, probably with lots of automation.

                                                                                        1. 2

                                                                                          you’ll tend to have at least some duplicate libraries across Docker containers

                                                                                          That is literally the entire point. Application dependencies must be separate from one another, because even on a tight-knit team keeping n applications in perfect lockstep is impossible.

                                                                                          1. 1

                                                                                            OS dependencies are different than application dependencies. I can apply a libc patch on my Debian server with no worry because I know Debian works hard to create a stable base server environment. That’s different than application dependencies, where two applications are much more likely to require conflicting versions of libraries.

                                                                                            Now, I run most of my stuff on a single server so I’m very used to a heterogeneous environment. Maybe that’s biasing me against Docker. But isn’t that the usecase we’re discussing here anyway? How someone with just a hobbyist server can run Mastodon?

                                                                                            Thinking about this more I feel like a big part of what bothers me about Docker, and therefore about Clair, is that there’s no package manifest. Dockerfile does not count, because that’s not actually a package manifest, it’s just a list of commands. I can’t e.g. build a lockfile format on top of that, which is what tools like Snyk analyze. Clair is the equivalent of having to run npm install and then go trawling through node_modules looking for known vulnerable code instead of just looking at the lockfile. More broadly, because Docker lacks any notion of a package manifest, it seems to me that while Docker images are immutable once built, the build process that leads you there cannot be made deterministic. This is what makes it hard to keep track of the stuff inside them. I will have to think about this more - as I write this comment I’m wondering if my complaints about duplicated libraries and tracking security there is an instance of the XY problem or if they really are separate things in my mind.

                                                                                            Maybe I am looking for something like Nix or Guix inside a Docker container. Guix at least can export Docker containers; I suppose I should look into that.

                                                                                            1. 2

                                                                                              OS dependencies are different than application dependencies.

                                                                                              Yes, agreed.

                                                                                              Thinking about this more I feel like a big part of what bothers me about Docker, and therefore about Clair, is that there’s no package manifest. Dockerfile does not count, because that’s not actually a package manifest, it’s just a list of commands. I can’t e.g. build a lockfile format on top of that, which is what tools like Snyk analyze.

                                                                                              You don’t need a container to tell you these things. Application dependencies can be checked for exploits straight from the code repo, i.e. brakeman. Both the Gemfile.lock and yarn.lock are available from the root of the repo.

                                                                                              The container artifacts are most like built automatically for every merge to master, and that entails doing a full system update from the apt repository. So in reality, while not as deterministic as the lockfiles, the system deps in a container are likely to be significantly fresher than a regular server environment.

                                                                                          2. 1

                                                                                            You’d want to track security vulnerabilities outside your images though. You’d do it at dev time, and update your Dockerfile with updated dependencies when you publish the application. Think of Docker as just a packaging mechanism. It’s same as making an uberjar on the JVM. You package all your code into a container, and run the container. When you want to make updates, you blow the old one away and run a new one.

                                                                                    2. 4

                                                                                      I have only rarely used Docker, and am certainly no booster, so keep that in mind as I ask this.

                                                                                      From the perspective of “install this giant blob of software”, do you see a docker deployment being that different from a single large binary? Particularly the notion of the control that you “give up”, how does that differ between Docker and $ALTERNATIVE?

                                                                                      1. 14

                                                                                        Ideally one would choose door number three, something not so large and inauditable. The complaint is not literally about Docker, but the circumstances which have resulted in docker being the most viable deployment option.

                                                                                      2. 2

                                                                                        You have the dockerfile and can reconstruct. You haven’t given up control.

                                                                                        1. 5

                                                                                          Is there a youtube video I can watch of somebody building a mastodon docker image from scratch?

                                                                                          1. 1

                                                                                            I do not know of one.

                                                                                    3. 3

                                                                                      I totally agree as well, and I wish authors would s/Mastodon/Fediverse/ in their articles. As others have noted, Pieroma is another good choice and others are getting into the game - NextCloud added fediverse node support in their most recent release as a for-instance.

                                                                                      I tried running my own instance for several months, and it eventually blew up. In addition to the large set of dependencies, the system is overall quite complex. I had several devs from the project look at my instance, and the only thing they could say is it was a “back-end problem” (My instance had stopped getting new posts).

                                                                                      I gave up and am now using somebody else’s :) I love the fediverse though, it’s a fascinating place.

                                                                                      1. 4

                                                                                        I just use the official Docker containers. The tootsuite/mastodon container can be used to launch web, streaming, sidekiq and even database migrations. Then you just need an nginx container, a redis container, a postgres container and an optional elastic search container. I run it all on a 2GB/1vCPU Vultr node (with the NJ data center block store because you will need a lot of space) and it works fairly well (I only have ~10 users; small private server).

                                                                                        In the past I would agree with out (and it’s the reason I didn’t try out Diaspora years ago when it came out), but containers have made it easier. I do realize they both solve and cause problems and by no means think they’re the end all of tech, but they do make running stuff like this a lot easier.

                                                                                        If anyone wants to find me, I’m @djsumdog@hitchhiker.social

                                                                                        1. 2

                                                                                          Given that there’s a space for your Twitter handle, i wish Lobste.rs had a Mastodon slot as well :)

                                                                                        2. 2

                                                                                          Wait, you’re also forgetting systemd to keep all those process humming… :)

                                                                                          You’re right that this is clearly too much: I have run such systems for work (Rails’ pretty common), but would probably not do that for fun. I am amazed, and thankful, for the people who volunteer the effort to run all this on their week-ends.

                                                                                          Pleroma does look simpler… If I really wanted to run my own instance, I’d look in that direction. ¯_(ツ)_/¯

                                                                                          1. 0

                                                                                            I’m waiting for urbit.org to reach useability. Which I expect for my arbitrary feeling of useability to come about late this year. Then the issue is coming up to speed on a new language and integrated network, OS, build system.

                                                                                            1. 2

                                                                                              Urbit is apparently creating a feudal society. (Should note that I haven’t really dug into that thread for several years and am mostly taking @pushcx at his word.)

                                                                                              1. 1

                                                                                                The feudal society meme is just not true, and, BTW, Yarvin is no longer associated with Urbit. https://urbit.org/primer/

                                                                                            2. 1

                                                                                              I would love to have(make) a solution that could be used locally with sqlite and in aws with lambda, api gateway and dynamodb. That would allow scaling cost and privacy/controll.

                                                                                              1. 3

                                                                                                https://github.com/deoxxa/don is sort of in that direction (single binary, single file sqlite database).

                                                                                            1. 2

                                                                                              Throwing a few pounds of copper onto a decade-old motherboard.

                                                                                              My desktop is still an old system from 2010 with a Westmere Xeon in it. I am finally going to overclock it to help catch up with a few years’ worth of increased bloat performance needs in various applications.

                                                                                              The northbridge (yes, it’s so old that it still has a northbridge) runs dangerously hot. Not a huge deal normally because I’m not trying to overclock the memory, but the motherboard connects the (rather weak) heatsinks on the power delivery to the heatsink on the northbridge with a heatpipe. So overclocking it is going to add a lot of heat to the NB via that heatpipe, and as they are physically connected, I cannot upgrade the cooling of just one of them.

                                                                                              Hence removing the cooling and throwing a few pounds of fresh copper onto it.

                                                                                              Inviting a younger friend over who has only ever owned laptops, and going to narrate my teardown and rebuild, as he wanted to learn about building a desktop.

                                                                                              1. 2

                                                                                                This is brilliant.

                                                                                                Having developed a sixth-sense or intuition for a lot of this, whether from curiosity or battle scars, I often struggle to succinctly impart that knowledge to more junior teammates. Going to be linking to this a lot and emphasizing details.

                                                                                                Thank you!

                                                                                                1. 6

                                                                                                  I’ve seen and even tried many XP themes and such over the years, especially during the mid-00’s era of the smarmy Linux ideologues tempting people to switch from Windows (which I will shamefully admit to being part of, occasionally).

                                                                                                  I am in awe at how much further this goes beyond that, and at the attention put into faithfully recreating some parts of these.

                                                                                                  That screenshot of “Windows 95/98/2000/ME” just brought on a type of comfy nostalgia that I didn’t realize I was capable of feeling. But there is at least one person for whom Windows Vista offers that type of comfy nostalgia, and this project has them covered too.

                                                                                                  Edit: It looks like the creator has a whole suite of well-made historical desktop OS themes, from older MacOS, to Solaris and even BeOS!

                                                                                                  1. 5

                                                                                                    I’d be interested to see a side-by-side comparison of kitty to alacritty. In particular, I’ve been using alacritty at work for a while and while it’s barebones at the moment, it’s exceptionally fast (which is probably my core feature for terminal emulators). That said, kitty looks like a fine emulator.

                                                                                                    1. 6

                                                                                                      Honest question: what need do you have for a fast terminal emulator?

                                                                                                      1. 7

                                                                                                        I have a minor obsession with input latency and scroll jank. It seems to creep up everywhere and is hard to stamp out (Sublime Text is a shining counterexample). I noticed a bit of weird input latency issues when using Terminal.app (purely anecdotal), and haven’t seen the same thing since using alacritty. So that’s the need I have for a fast emulator, it enables a smooth input and output experience.

                                                                                                        1. 3

                                                                                                          I am sensitive to the same.

                                                                                                          This is what kept me on Sublime Text for years, despite open source alternatives (Atom, VS Code and friends). I gave them all at least a week, but in the end the minor latency hiccups were a major distraction. A friend with similar sensitivity has told me that VS Code has gotten better lately, I would give it another go if I weren’t transitioning to Emacs instead.

                                                                                                          I sometimes use the Gmail web client and, for some period of time, I would experience an odd buffering of my keystrokes and it would sometimes completely derail my train of thought. It’s the digital equivalent of a painful muscle spasm. Sometimes you ignore it and move on, but sometimes you stop and think “Did I do something wrong here? Is there something more generally broken, and I should fear or investigate it?”

                                                                                                          1. 1

                                                                                                            Web-based applications are particularly bad, because often they don’t just buffer, but completely reorder my keystrokes. So I can’t just keep typing and wait for the page to catch up; I have to stop, otherwise I’m going to have to do an edit anyway.

                                                                                                        2. 3

                                                                                                          I have to admit, I thought for certain this was going to be Yet Another JavaScript Terminal but it turns out it’s written in Python. Interesting.

                                                                                                          Anyway I would have a hard time believing it’s faster than xfce4-terminal, xterm, or rxvt. It’s been a long time since I last benchmarked terminal emulators, maybe I smell a weekend project coming on.

                                                                                                          1. 6

                                                                                                            kitty is written is about half C, half Python, Alacritty is written in Rust.

                                                                                                            There were some benchmarks done for the recent Alacritty release that added scrollback, which include kitty, urxvt, termite, and st. https://jwilm.io/blog/alacritty-lands-scrollback/#benchmarks

                                                                                                            1. 2

                                                                                                              I just did a few rough-and-ready benchmarks on my system. Compared to my daily driver (xfce4-terminal), kitty is a little under twice as fast, alacritty and rxvt are about three times as fast. If raw speed was my only concern, I would probably reach for rxvt-unicode since it’s a more mature project.

                                                                                                              Alacritty is too bare-bones for me but I could be sold on kitty if I took the time to make it work/behave like xfce4-terminal.

                                                                                                              1. 1

                                                                                                                I like xfce4-terminal, but it renders fonts completely wrong for me. It’s most noticeable when I run tmux and the solid lines are drawn with dashes. If I pick a font where the lines are solid, then certain letters look off. It’s a shame, because other vte-based terminals (e.g. gnome-terminal) tend to be much slower.

                                                                                                          2. 2

                                                                                                            For me it’s the simple stuff that gets annoying when it’s slow. Tailing high-volume logs. less-ing/cat-ing large files. Long scrollbacks. Makes a difference to my day by just not being slow.

                                                                                                            1. 2

                                                                                                              I don’t care that much about the speed it takes to cat a big file, but low latency is very nice and kitty is quite good at that. I cannot use libvte terminals anymore, they just seem so sluggish.

                                                                                                              1. 2

                                                                                                                For one thing, my workflow involves cutting and pasting large blocks of text. If the terminal emulator can’t keep up, blocks of text can come through out of order etc, which can be a bad time for everyone involved.

                                                                                                              2. 3

                                                                                                                I’m on macOS.

                                                                                                                I used alacritty for a while, then switched to kitty as I’d get these long page redraws when switching tmux windows—so kitty is at least better for me in that regard. Both have similar ease of configuration. I use tmux within both, so I don’t use kitty’s scrolling or tabs. The way I was using them, they were more or less the same.

                                                                                                                I’m going to try alacritty again to see if it’s improved. I’d honestly use the default Terminal app if I could easily provide custom shortcuts (I bind keys to switching tmux panes, etc).

                                                                                                                1. 4

                                                                                                                  I came back to Alacritty on MacOS just the other day after trying it last maybe 6 months ago and finding it “not ready” in my head. It’s been significantly updated, there’s a DMG installer (and it’s in brew), a lot more polished overall and it works really well and really fast. No redraws in tmux switches. Weirded redraw artifiact while resizing main window, but snaps to fixed immediately you stop, and doesn’t bother me much. Using it as a full-time Terminal replacement right now, liking it so far, will see how it goes!

                                                                                                                  1. 1

                                                                                                                    Good to know! I’ve installed it via brew now and double-checked my old config. My font (as in, not the default Menlo. I’m using a patched Roboto Mono) looks a bit too bold, so just gotta figure out what’s wrong there.

                                                                                                                    1. 2

                                                                                                                      They’ve updated config files with additional info about aliasing and rendering fonts on Mac. So take a look at that if you are using your old config. It’s not a bad idea to start from scratch.

                                                                                                                      1. 1

                                                                                                                        Thanks for the tip! I did start from scratch and moved over changes bit by bit, but I’ll have to check the new macOS specific lines.

                                                                                                                  2. 3

                                                                                                                    Cool, thanks for your input! I also use tmux, and I haven’t seen anything like what you described (I also don’t really use tmux panes, only tabs). I know there has been a longstanding vim + tmux + osx bug as well, but I haven’t used vim proper in a while.

                                                                                                                    1. 2

                                                                                                                      I think that’s my exact problem (turns out I’m even subscribed to the issue haha). I use neovim so I think it is/was applicable to both

                                                                                                                  3. 1

                                                                                                                    do any of those really measure up when benchmarked.

                                                                                                                    I remember doing some writing to stdout and it alacritty turned out to be slower than say gnome-terminal or whatever.

                                                                                                                    Might’ve been that there was a bug with my intel graphics card though, don’t remember to well.

                                                                                                                  1. 17

                                                                                                                    Somebody needs to solve the mismatch between the value generated by free software and the inability to get paid. Programmers shouldn’t have to take a huge pay cut to work on libre software.

                                                                                                                    Having to ask for ‘donation’ is an insult to the dignity of a competent programmer who can otherwise get a very lucrative offer for his/her skills.

                                                                                                                    1. 10

                                                                                                                      I honestly think to a large degree this has been solved if we follow the example of SQLite. Rather than trying to reach out all all possibly users of SQLite, trying to get a monthly donation of like $1/$2/$5 from each user, they focus on the corporate users of the software and ask for significant investment for things that corporate users specifically care about and aren’t “donations”:

                                                                                                                      • Email Support: $1500 a year.
                                                                                                                      • Phone Support: $8000 a year base, for SLA response time goes up to $50,000.
                                                                                                                      • Consortium Membership: $85,000+
                                                                                                                      • Title/License: $6000
                                                                                                                      • AES Custom Version: $2000
                                                                                                                      • CEROD Custom Version: $2000
                                                                                                                      • ZIPVFS Custom Version: $4000
                                                                                                                      1. 2

                                                                                                                        Note that this doesn’t work if the software isn’t going to be used by companies. For instance I have a hard time picturing a company pay for sway or aerc.

                                                                                                                        1. 1

                                                                                                                          Absolutely, stuff that is of no use to a corporation is harder to deal with this way. I would argue that at certain levels of corporate dependency, even niche products like text editors and diff tools can get widespread financial backing. I have seen both (text editors and diff tools) get major contributions in terms of cash from corporations.

                                                                                                                      2. 9

                                                                                                                        Donations are difficult to justify by companies both legally and in terms of business. They also cannot guarantee any continuity to the recipient. Moreover, donations are inherently unfair to donors VS non-donors.

                                                                                                                        Public funding has been invented exactly for this.

                                                                                                                        1. 2

                                                                                                                          Moreover, donations are inherently unfair to donors VS non-donors.

                                                                                                                          Could you elaborate “fair” a little?

                                                                                                                          I cannot settle on a definition of fairness around donations (esp. to develop open source software) that I, myself, would use in this situation, and so I would surely fail at assuming the definition intended in your comment.

                                                                                                                          1. 4

                                                                                                                            Forgive me for the platitude: If a company donates a lot and a competitor does not (while still benefiting from the same shared public good), the latter has an advantage. This little prisoner dilemma around donations encourage greed over cooperation. That’s why taxes are mandatory.

                                                                                                                            1. 2

                                                                                                                              If a company donates a lot and a competitor does not (while still benefiting from the same shared public good), the latter has an advantage.

                                                                                                                              That sounds right but might not be. IBM backed Linux when Microsoft and SCO were going to patent sue everyone trying to use it. IBM both was donating labor to Linux and would counter-sue to keep it open. The result over time was IBM got massive benefit from Linux while proprietary offerings folded or went into legacy mode over time.

                                                                                                                              I mean, IBM and Linux are both kind of unique. That success might not extrapolate. It might for companies who can stand to gain a lot from a community around their product. The community might be bigger if the company significantly invests in open source and community activities.

                                                                                                                            2. 3

                                                                                                                              I assume the rational is that open source code is a public good in the same way that clean water or science is. If you spend a lot of money making sure that your local river has clean water, or a lot of money to study physics then the benefits are shared by everybody but the costs were incurred by just you.

                                                                                                                              “Fairness” in the context of shared resources generally means that the costs of providing the resource are shared by the users of the resource in proportion to the benefit those users each receive.

                                                                                                                            3. 2

                                                                                                                              I agree that public funding was meant to solve problems much like this, but that doesn’t make it an easy solution.

                                                                                                                              There are thousands of new libraries created every day, which ones will you fund? How much money should you give Pixie Lang?

                                                                                                                              The NSF gives grants to researchers who are motivated to publish papers, which journals will only accept if the papers reveal something new and important. If you give money to open source developers do they have any external motivation to produce useful things? What is preventing them from adding a million new features to OpenSSL rather than carefully auditing the code and fixing tricky bugs?

                                                                                                                              If ruby is given enough public funding to hire 10 developers, won’t that make the developers who weren’t chosen feel like they’re not as important? Would they continue contributing as much as they have when they know somebody else is getting paid?

                                                                                                                              Many open source projects have contributors from many different nations. Is the agency doing public funding okay with giving money to non-nationals?

                                                                                                                              1. 2

                                                                                                                                public funding was meant to solve problems much like this, but that doesn’t make it an easy solution

                                                                                                                                It worked better than other alternatives during the last 100 years to develop phones, early computers, semiconductors, satellites, a lot of medicine, aeronautics, chemistry… Anything that does not have a short or medium-term return.

                                                                                                                                Is the agency doing public funding okay with giving money to non-nationals?

                                                                                                                                A lot of long-term scientific research is funded through global nongovernmental organizations.

                                                                                                                            4. 6

                                                                                                                              Not a great comfort to a libertarian, I’m sure - but for those who believe in government intervention, taxpayer-funded work on core infrastructure is an obvious way to share the load (since broadly speaking, society at large benefits from access to improved technology).

                                                                                                                              IIRC at least one of the OpenSSL core team was funded - for years - off the German social security pension. RMS’s work at MIT was largely funded through the US government. DARPA paid for a ton of computing infrastructure.

                                                                                                                              1. 4

                                                                                                                                Who is this somebody who needs that?

                                                                                                                                Describing your own desires as someone else’s needs is a cop-out.

                                                                                                                                1. 1

                                                                                                                                  I discuss this a lot. It usually breaks the moment I bring in the notion that this somebody should probably be paid, at around, say, 10-20% if what the developers get.

                                                                                                                                2. 1

                                                                                                                                  If the software is valuable, you can license it such that you can sell it for money.

                                                                                                                                  1. 6

                                                                                                                                    This is a pretty often mentioned, but not every FOSS software has a straight forward business model attached. For example, programming languages are far too remote from an actual product for people to actually invest in them on large scale. Yet, they certainly have huge value! If you see the struggle to get a widely used project as MRI funded…

                                                                                                                                    Sure, I could get my money by consulting in that programming language and being an expert in it, but there, the incentive is actually again to have other people developing it and just run around using their stuff.

                                                                                                                                    Also, not every programmer wants to become a salesperson or build end-user products.

                                                                                                                                    1. 3

                                                                                                                                      You can also license it freely and sell it for money. There’s no inherent contradiction in “commercial free software”. Indeed, sr.ht seems like it fits this category.

                                                                                                                                      1. 1

                                                                                                                                        Great example (and congrats again) :)

                                                                                                                                        In my experience, most such software is very hard to deploy for yourself (since the primary maintainer has no real reason to work on that aspect and nobody else tends to step up).

                                                                                                                                        This is in no way a jab at your fantastic work - merely an observation of how this, like every funding structure, exerts a pull on the project funded.

                                                                                                                                        1. 1

                                                                                                                                          Congrats? For what? I’m not Drew.

                                                                                                                                          1. 1

                                                                                                                                            Huh, somehow I got my wires crossed, sorry.

                                                                                                                                      2. 1

                                                                                                                                        I wonder if that’s true, and if not, why.

                                                                                                                                        You’ve done it. And perhaps I have too (although one might tell my own story in different ways). But the people who manage to create functioning open source software from scratch have failed to earn real money from it with such frequency that I wonder whether there’s some blocker. That some personality trait that’s necessary to create that software also blocks generating income from the software.

                                                                                                                                        1. 1

                                                                                                                                          I absolutely believe this is the case, personality traits that draw people to open source software tend to push them away from the obvious avenues of income. I think they also fear angering their communities if they start to value corporate users over regular users. I think this fear is misguided, if regular users get much a much better product because of that corporate support, I believe they will be very understanding / support (ala sqlite).

                                                                                                                                          1. 1

                                                                                                                                            That some personality trait that’s necessary to create that software also blocks generating income from the software.

                                                                                                                                            I don’t believe this is the case. FOSS comes out of a culture where many people could make their ends meet. Either by being employed by MIT or by having a paid day job.

                                                                                                                                            It’s something our community - as a whole - could easily ignore and not build structure around. That’s falling on our feet now. Those structures will take years to build.

                                                                                                                                      1. 3

                                                                                                                                        Super excited for you!

                                                                                                                                        (and excited for your projects, albeit a little selfishly ;] )

                                                                                                                                        1. 1

                                                                                                                                          Thanks! I’m excited for my projects, too ;)

                                                                                                                                        1. 8

                                                                                                                                          This is the first time I have heard “ethics theatre” as a term.
                                                                                                                                          I am going to be lifting that for future use.

                                                                                                                                          1. 2

                                                                                                                                            What a marvelous term, and it explains so much.

                                                                                                                                            1. 1

                                                                                                                                              Isn’t that synonymous with “virtue signalling”?

                                                                                                                                            1. 14

                                                                                                                                              Not that it’s hard to find, but for anyone whose curiosity was piqued by the opening paragraph of the README, here’s the commit where they switched from parody to serious.

                                                                                                                                              1. 16
                                                                                                                                                - ## Fork me, daddy ¯`·.¸><(((º>  
                                                                                                                                                + ## Contribtuions
                                                                                                                                                

                                                                                                                                                There aren’t many commits like these.

                                                                                                                                                1. 3

                                                                                                                                                  The original README screenshot was actually quite good.
                                                                                                                                                  Also smol and comfy.

                                                                                                                                                  1. 3

                                                                                                                                                    OwO

                                                                                                                                                1. 7

                                                                                                                                                  The day I discovered wmctrl and xdotool was life-changing.
                                                                                                                                                  Low-tier life-changing, but definitely life-changing.

                                                                                                                                                  I even have some old, never-finished Ruby wrappers built around them that was purpose-build to aid in a single project, very much in line with what the article suggests: Set up my window placement for each project I work on across various monitor layouts and WMs, via a hotkey, or running a command from dmenu (or at login if I wanted, but I never want that, personally).

                                                                                                                                                  One relic from that still gets used on my desktop, and I have since used the two for other things.

                                                                                                                                                  In one case I was able to brute-force my way through a form on a mis-configured government website that kept kicking me out to page 1 of a multi-step form, depending on which of their servers was serving my request. And of course none of the data persisted, or would re-reappear when I jumped between the different servers, but I simply xdotool’d a bit and only had to hit a hotkey to fill out the dozen-or-so fields on each page, and submit, until it worked.

                                                                                                                                                  1. 9

                                                                                                                                                    About three years ago I did a “deep enough” dive (i.e. not a true deep dive) into ZFS vs btrfs for my home and home office use (in Linux). I tend to have relatively small amounts of data, with separate backups, so I have happily run outright RAID1 for years for the sheer simplicity. I intended to use either in a pure mirror setup.

                                                                                                                                                    The things that pushed me toward btrfs at that time were:

                                                                                                                                                    1. The lack of native encryption in ZFS, and my uncertainty with combining ecryptfs or LUKS with ZFS. Running btrfs on separate LUKS volumes, and combining those into a RAID setup surely isn’t ideal or efficient, but at least it is simple and I know how each piece works.
                                                                                                                                                    2. Mirror expansion in btrfs works how I (personally) think it should: Take a 2-disk mirror and add a 3rd disk? Just rebalance and going forward each piece of data is on two devices out of the three. Not efficient, but simple. ZFS requires 2 new drives as a separate mirror from the first two. That feels like an unnecessary constraint (and expense) to have to always buy drives in pairs.
                                                                                                                                                    3. A general sense of unease that ZoL wasn’t really a first-class citizen in the ZFS world. That Illumos and the BSDs would always get the advanced, and most stable features, and Linux users might never get this-or-that feature, and that “Just use FreeBSD instead” seemed like an accepted response to questions about ZoL stability (I do not say that in reference to anyone involved in the projects, of course. More from various end-user communities and conversations).

                                                                                                                                                    Despite going with LUKS and btrfs (again I emphasize: mirrors), and that setup being great so far, tools like Sanoid have long left me with ZFS envy, although I will always have a soft spot for Bacula (now Bareos). So I was already watching the ZoL releases and the 0.8 milestone, with hopes to try out the encryption.

                                                                                                                                                    Now hearing that RAIDZ Expansion is actually on the way and this FreeBSD-and-ZoL news, so close in time, seals the deal. As soon as 0.8 is marked stable, I will be switching to ZFS.

                                                                                                                                                    1. 5

                                                                                                                                                      You might also be interested in searching other areas in the Midwest or the South, since the cost of living is waaaaaaaaaaaaaaay lower and they’re hungry for devs.

                                                                                                                                                      Down in Houston, off the top of my head you might want to try EOG Resources–they’re hiring frontend devs, React/Redux right now. Really good pay, adult company (none of that startup nonsense), etc. Various medical center academic groups might also be hiring.

                                                                                                                                                      Oklahoma also offers a stipend for tech folks to move there right now iirc.

                                                                                                                                                      1. 5

                                                                                                                                                        Hey friendlysock,

                                                                                                                                                        The secret sauce to Seattle is that I have friends and family I can crash with while we get back on our feet.

                                                                                                                                                        It’s interesting that you mentioned the midwest—we were actually looking at St. Louis for a while but I couldn’t find software positions that weren’t defense contracting jobs for Boeing or otherwise. Do you know of anything in the area?

                                                                                                                                                        1. 4

                                                                                                                                                          I don’t know St. Louis, but I’ll throw in another Houston shoutout (because cost of living is so low) that matches your recent experience (Scala and JavaScript): iOFFICE.

                                                                                                                                                          I know some of their engineers personally and their team culture seems great, and some of them are involved in the local Functional Programmers meetup.

                                                                                                                                                          I also know some of their engineers don’t have a college degree, and others have a degree in something nontechnical and are self-taught in software. So they understand alternative paths toward practical skills.

                                                                                                                                                          And a quick search shows the are hiring.

                                                                                                                                                          EDIT: After reading my link a little closer, they explicitly say:

                                                                                                                                                          Our only requirements are real world experience and passion. Don’t worry about your resume (we don’t care) tell me why you love coding and send over your github, twitter and whatever else you can talk about and lets talk!

                                                                                                                                                          1. 2

                                                                                                                                                            Ah, gotcha gotcha.

                                                                                                                                                            I don’t know of anything out in St. Louis, unfortunately. :(

                                                                                                                                                            1. 2

                                                                                                                                                              I have some software engineer acquaintances in the STL area: there’s not much. The more ambitious feel trapped unless they are on a good track at one of the few premier employers.

                                                                                                                                                          1. 2

                                                                                                                                                            Well executed. The only thing this is missing is a truncated y-axis on the graph.

                                                                                                                                                            1. 4

                                                                                                                                                              This is really neat. After explaining a few dozen times to people who are new to Ruby how namespaces differ from their prior language experience (often Node or Java), I had begun to idly wonder if this sort of thing could be bolted on to Ruby, ideally in an optional way.

                                                                                                                                                              Really excited to dig into the source of this and just see how it’s built.

                                                                                                                                                              Edit: Digging done. The way Modulation::Builder::make uses Modulation::ModuleMixin to make imports happen is really slick.

                                                                                                                                                              1. 5

                                                                                                                                                                People who’re participating, what languages are you using? I’m starting to learn Typescript, so I’m thinking of using that for practical reasons, though I’d probably rather do OCaml or something a bit more esoteric.

                                                                                                                                                                1. 8

                                                                                                                                                                  Last year I used 6 languages, and this year my goal is to use 12! Carp, Fennel, and Reason are some new-to-me languages I hope to touch on. I will likely do most of the problems in clojure because I enjoy manipulating data-structures in it so much, but rarely feel like it’s the practical choice for my projects.

                                                                                                                                                                  1. 2

                                                                                                                                                                    So, the 12 languages of Christmas?

                                                                                                                                                                  2. 6

                                                                                                                                                                    If it is fine with you to not be able to complete a task in 24h than AoC is a great way to learn new language.

                                                                                                                                                                    1. 6

                                                                                                                                                                      I’m learning Clojure and liking it so far, so my AoC has a perfect timing this year to be full of parenthesis

                                                                                                                                                                      1. 2

                                                                                                                                                                        Let’s compare solutions!

                                                                                                                                                                      2. 3

                                                                                                                                                                        Planning on giving ReasonML and Elm a shot.

                                                                                                                                                                        1. 3

                                                                                                                                                                          I did 2015 and 2017 in Racket, and had a pretty good time. I tried 2016 in Erlang, and really struggled.

                                                                                                                                                                          This year, I’m going to try it with Pony.

                                                                                                                                                                          1. 2

                                                                                                                                                                            As much as I am tempted to try ReasonML, I wrote a modest amount of Elixir back in 1.2 (three years ago, and stable is now 1.7), and have an upcoming need for it again. Will probably do that to brush up on any changes.

                                                                                                                                                                            In an attempt to make it challenging is I am going to try and pick up Emacs while doing so. I have heard that the lineage inherited from Erlang provides a decent ecosystem in Emacs for other BEAM languages. Does anyone know if that perception is correct?

                                                                                                                                                                            1. 1

                                                                                                                                                                              Cannot edit my post anymore, but it looks like not only is that the case, there are multiple Elixir-specific packages as well (elixir-mode and alchemist).

                                                                                                                                                                            2. 2

                                                                                                                                                                              I’ll probably take a shot with Haskell again, or possibly F#.

                                                                                                                                                                              1. 2

                                                                                                                                                                                Crystal. Figured I should brush up on making actual programs instead of one-off scripts and thus style and “architecture” will be my focus.

                                                                                                                                                                                1. 2

                                                                                                                                                                                  Pretty ballsy that a project will try to reclaim that name from the horror that is Crystal Reports. Kudos!

                                                                                                                                                                                2. 2

                                                                                                                                                                                  I use Perl 5. Life + day job doesn’t give much time for actual programming so I’m really happy to just try to solve problems in a language I know well.

                                                                                                                                                                                  Plus Perl makes parsing the problem input a breeze.

                                                                                                                                                                                  1. 1

                                                                                                                                                                                    I’ve been looking into D a bit recently so going to give that a try.

                                                                                                                                                                                    1. 1

                                                                                                                                                                                      I think this year I’ll forego going for leaderboard, and use the opportunity to brush up on my Elixir skills. I haven’t had a good chance to touch it for quite a while, so it should be a good refresher. I really enjoy thinking in and solving problems with the toolset available in Elixir.

                                                                                                                                                                                      1. 2

                                                                                                                                                                                        Jose Valim is going to stream his own solutions to each challenge one day afterward (in Elixir of course):

                                                                                                                                                                                        http://blog.plataformatec.com.br/2018/11/lets-learn-elixir-together-with-advent-of-code/

                                                                                                                                                                                        1. 1

                                                                                                                                                                                          Fantastic! Thanks for the heads up!

                                                                                                                                                                                    1. 1

                                                                                                                                                                                      This is fantastic! Sent this to two junior engineers for whom I know it will be very timely.

                                                                                                                                                                                      1. 2

                                                                                                                                                                                        There is a public_send method that respects method visibility - to be completely honest I’m not sure how it deals with protected status, though, since protected is not typically seen in idiomatic Ruby afaict.

                                                                                                                                                                                        I do like the perspective of looking at method dispatch in terms of sending messages, though. Decent write up!

                                                                                                                                                                                        1. 2

                                                                                                                                                                                          public_send does not call protected methods. And, yeah, protected rarely happens in Ruby. I consider it a mild code smell as it implies the use of inheritance.

                                                                                                                                                                                          1. 1

                                                                                                                                                                                            And, yeah, protected rarely happens in Ruby. I consider it a mild code smell as it implies the use of inheritance.

                                                                                                                                                                                            I’ve always considered private to be the wrong choice, in any language, because it signals that I think I know every way anyone might ever want to extend my class. Inheritance may not always (or even often) be the right solution, but making certain extension options impossible is just going to drive future coders crazy.

                                                                                                                                                                                            1. 2

                                                                                                                                                                                              I use private to lower the cost for future internal refactors, and upgrades, by providing a signal that “Hey, this method is not part of the public API, I do not condone its external use, and reserve the right to change it wildly.”

                                                                                                                                                                                              It communicates that there consciously was no consideration for what might happen if external objects began to depend on said private method. Later changes by anyone (not just the original author) can and will be free to change the private method at will: refactor, upgrade, remove entirely, and there is less to hold in your head and worry about when upgrading them later.

                                                                                                                                                                                              However, behavior of a private method that is observable through the a public method is fair game, and test cases should exist for those. The flip side of this is that, except in some rare cases, I favor not testing private methods. Any behavior that is possible via a private method, but which cannot be exercised or observed through use of public methods, should not be considered as supported behavior on the object. That does not preclude a new public method from exposing more of the private method’s behavior later, but at that point it is part of the public interface, and should be presented and documented as such.

                                                                                                                                                                                              As for making certain extension options impossible, at least in the context of Ruby, private does not do this.

                                                                                                                                                                                              class PrivateGreeter
                                                                                                                                                                                                private
                                                                                                                                                                                                def greeting
                                                                                                                                                                                                  "Hello."
                                                                                                                                                                                                end
                                                                                                                                                                                              end
                                                                                                                                                                                              
                                                                                                                                                                                              class PublicGreeter < PrivateGreeter
                                                                                                                                                                                                def speak
                                                                                                                                                                                                  greeting
                                                                                                                                                                                                end
                                                                                                                                                                                              end
                                                                                                                                                                                              
                                                                                                                                                                                              irb> PrivateGreeter.new.greeting
                                                                                                                                                                                              # => NoMethodError: private method `greeting' called for #<PrivateGreeter>
                                                                                                                                                                                              irb> PublicGreeter.new.speak
                                                                                                                                                                                              # => "Hello."
                                                                                                                                                                                              

                                                                                                                                                                                              You can even give it the same name as the parent class’ private method if you really, truly need to let external objects utilize the method as part of the subclass’ public API with the exact same name:

                                                                                                                                                                                              class PublicGreeter < PrivateGreeter
                                                                                                                                                                                                def greeting
                                                                                                                                                                                                  super
                                                                                                                                                                                                end
                                                                                                                                                                                              end
                                                                                                                                                                                              
                                                                                                                                                                                              irb> PrivateGreeter.new.greeting
                                                                                                                                                                                              # => NoMethodError: private method `greeting' called for #<PrivateGreeter>
                                                                                                                                                                                              irb> PublicGreeter.new.greeting
                                                                                                                                                                                              # => "Hello."
                                                                                                                                                                                              

                                                                                                                                                                                              Since an inherited class can make use of its parent class’ private method, the question then becomes: should it? If we have a stipulation that the parent class reserves the right to refactor its private methods or change them wildly, should we rely on a parent class’ private method?

                                                                                                                                                                                              I think the answer differs between application code and library code. If the parent class comes from a library, or some internal, but still cross-domain, utility code, I would not want to rely on the existence, the arity, or the behavior of a private method. If the behavior is so critical, I would rather capture the behavior permanently as part of the subclass by copying the code itself over to the subclass (or not use inheritance at all, but I would still definitely copy the code). Everything I have seen makes me agree that duplication is far cheaper than the wrong abstraction.

                                                                                                                                                                                              In the case of application code, it is more acceptable to rely on a parent class’ internals. I still would fight the urge, but it can be expected that the author modifying the subclass and parent class has access and authority to work on both in order to get their job done, that the tests for both classes live in the same suite, and other niceties that make maintaining that dependency across time situation more tenable.

                                                                                                                                                                                          2. 1

                                                                                                                                                                                            Thanks for your great feedback ! That’s motivates me to do even more :-)

                                                                                                                                                                                          1. 2

                                                                                                                                                                                            Hate to be that guy but I can’t believe it is 2018 and the Rails world still holds so tightly to the active record pattern despite it hurting so much.

                                                                                                                                                                                            1. 2

                                                                                                                                                                                              OP here.

                                                                                                                                                                                              It’s a legacy Rails app from 2011. Moving the whole chunk to something other than the AR pattern is not feasible, nor will it ever be.

                                                                                                                                                                                              The best we can do is to rewrite pieces of it at a time with better patterns, and that’s exactly what we are doing now.

                                                                                                                                                                                              1. 1

                                                                                                                                                                                                Much of the perceived productivity wins that out-of-the-box Rails provides are derived from the tight coupling between ActiveRecord (or at least ActiveModel) and the rest of the provided libraries. I agree it is a hindrance, but choosing not to use it in the initial phases of a project causes other sorts of friction unless you really know the ecosystem and know what you want to do instead. After all, isn’t ActiveRecord almost half of the lines of code in Rails in the first place?

                                                                                                                                                                                                With respect to tests, the community’s preference for factory libraries (instead of the built-in fixtures) exacerbates the performance issue quite a bit.

                                                                                                                                                                                              1. 10

                                                                                                                                                                                                Pattern matching is one of the only features I regularly feel is lacking in Ruby.
                                                                                                                                                                                                There are other things I dislike or wish weren’t present, but the only missing feature is this.
                                                                                                                                                                                                Combining conditionals and assignment just feels 10x better than having them separate.

                                                                                                                                                                                                I even built a pattern matching library with some gross fun hacks on top of a specific namespace, thread-local singleton instance variables and caller_locations to allow for kinda-mostly pattern matching. I’m going to see if I can dig up the code, because aside from a first/rest pairing, I managed to get nested matching across Arrays and Hashes, and an Any object to match anything.
                                                                                                                                                                                                Then I bumped into a Ruby Tapas video on the topic and stole the hubcaps to implement Erlang-like guard clauses.

                                                                                                                                                                                                1. 6

                                                                                                                                                                                                  Have you looked at Qo at all? If so, any strong opinions about where it’s lacking? (Admittedly, it’s a giant hack, but it’s a pretty good giant hack.)

                                                                                                                                                                                                  1. 12

                                                                                                                                                                                                    Hello! Author of Qo here. Yeah, you’re right, it is a bit of a giant hack XD

                                                                                                                                                                                                    Anyways, I’d love to hear any ideas or suggestions on it. Typically I use the issue tracker to keep tabs on what I want to do with it, so feel free to jump in there as well.

                                                                                                                                                                                                    Admittedly I’m also the one who wrote a small-scale novel on pattern matching in the description to try and cover bases as I really really really want this feature, as evidenced by writing Qo in the first place.

                                                                                                                                                                                                    1. 3

                                                                                                                                                                                                      What do you think about the %p() or %m() syntaxes? I think they’re really ugly personally. Is it possible to make a spin on case which parses the when clauses as patterns without any special extra delimiters? You somewhat hit on that when talking about Scala style I think. Something like this maybe?

                                                                                                                                                                                                      match val 
                                                                                                                                                                                                      when [:constant, "constant", [1], variable_binding]
                                                                                                                                                                                                        variable_binding + 1
                                                                                                                                                                                                      else
                                                                                                                                                                                                        0
                                                                                                                                                                                                      end
                                                                                                                                                                                                      

                                                                                                                                                                                                      If you already covered that, apologies, I read your comments quickly last night and might have missed it.

                                                                                                                                                                                                      1. 4

                                                                                                                                                                                                        EDIT Codefied my idea here - https://bugs.ruby-lang.org/issues/14709#note-6

                                                                                                                                                                                                        match(value) do |m|
                                                                                                                                                                                                          m.when(/name/, 42) { |name, age| Person.new(name, age) }
                                                                                                                                                                                                          m.else { |object| raise "Can't convert!" }
                                                                                                                                                                                                        end
                                                                                                                                                                                                        

                                                                                                                                                                                                        An example practical usage:

                                                                                                                                                                                                        def get_url(url)
                                                                                                                                                                                                          Net::HTTP.get_response(URI(url)).then(&match do |m|
                                                                                                                                                                                                            m.when(Net::HTTPSuccess) { |response| response.body.size }
                                                                                                                                                                                                            m.else { |response| raise response.message }
                                                                                                                                                                                                          ))
                                                                                                                                                                                                        end
                                                                                                                                                                                                        

                                                                                                                                                                                                        Original:

                                                                                                                                                                                                        Not a problem, there are a lot of them (may Matz and Koichi forgive me)

                                                                                                                                                                                                        I hadn’t quite covered it yet. That’s always been the trick about this: what should it look like?

                                                                                                                                                                                                        Truthfully I don’t know quite yet, but I’m working on ideas. In regards to the %p and %m I would agree with some Reddit comments that they can tend slightly Perl-ish. I’d like a more fluent syntax if possible that reads intuitively, and I don’t think that quite does it.

                                                                                                                                                                                                        I had initially proposed this:

                                                                                                                                                                                                        new_value = match value
                                                                                                                                                                                                          when %m(:_, 20..99) { |_, age| age + 1 }
                                                                                                                                                                                                          else { |object| ... }
                                                                                                                                                                                                        end
                                                                                                                                                                                                        

                                                                                                                                                                                                        …which is quite similar to your suggestion. I’d almost consider switching the syntax a bit to something more like this:

                                                                                                                                                                                                        new_value = match value
                                                                                                                                                                                                          when matches(:_, 20..99) { |_, age| age + 1 }
                                                                                                                                                                                                          else matches(:_) { |object| ... }
                                                                                                                                                                                                        end
                                                                                                                                                                                                        

                                                                                                                                                                                                        When possible I would prefer very clear words as syntax over the percent shorthands. With this you could almost amend case to look for matches

                                                                                                                                                                                                        1. 1

                                                                                                                                                                                                          Ahhh, I hadn’t seen how binding the match results as block arguments would be valuable, but your example of using === overloads like Class.=== and Regexp.=== have convinced me. I learned pattern matching in Erlang so I was thinking about the Erlang style mostly, and I didn’t think of how pattern matching would be most useful in Ruby. Blocks are a good way to reuse the case style matching that’s already well understood.

                                                                                                                                                                                                        2. 1

                                                                                                                                                                                                          OOH I like this! You should counter-propose this syntax.

                                                                                                                                                                                                      2. 2

                                                                                                                                                                                                        I only just saw it via this proposal. Hope to find time to play with it today after work.