1. 21

    I have watched the videos behind this text and I’m a bit frustrated. The most problems they have are either hardware problems or problems because they expect thinks work like on Windows (or believe they work on Windows).

    For the hardware the somehow acknowledge that this is more the problem of the vendors then of Linux. It still sounds the most time more like Linux is bad because this super fancy hardware don’t work. Yes I know the problems behind this are complex and as a normal user this is frustrating.

    And of course they expect a Windows like behavior, they have used Windows for years. What bugs me is that they claim that the Windows way is the better way without understanding what the problem is. There are two examples for this:

    First the Linus broke his Pop!_OS installation while he tried to install steam. This was because the steam package had a dependency problem which could only resolved by removing essential packages. The GUI tells him there was an error with some suggestions what might case the problems and output from apt hidden behind a details button. He reads out loud: “Warning: you trying to remove the following essential packages”. So he googled and found the command line to install steam. So the command prompted him a lot of text and at the end following two lines:

    You are about to do something potentially harmful

    To continue type in the phrase ‘Yes, do as I say!’

    So he typed in “Yes, do as I say!” and his installation was broken. He claimed later: “the thinks that I did are not entirely ridiculous or unreasonable”. He ignored all warnings and “dictated the computer” “Yes, do as I say!”, how is this not a clear user error[0]?

    So lets look what would had happen with a similar issue under Windows. First so similar we don’t get the issue, because under Windows there is no package manager accessible for somehow third party software. So lets assume there is an windows update which removes the wrong file and breaks your system. On the install the update would remove the wrong file and breaks your system. Other example the steam installer manage to have a bug with removes some necessary files from your Windows installation. Is there anything Windows protect you from this bug[1]?

    It’s late and the other stuff about the file exertion issue I might write tomorrow.

    [0] Of course this has also something to do with the spirit of some developers to create popups/warnings/info messages. Which leads users to ignore these messages.

    [1] I don’t know, but a few years ago windows installers where just executable which required to run as administrator.

    1. 27

      And of course they expect a Windows like behavior, they have used Windows for years

      I think the “Windows-like behaviour” in this case is that on Windows Steam works perfectly, you don’t have to think about installing it, there’s no chance it’s going to break your OS, nor will you have to you choose between installing an application you want and having a working OS.

      We could imagine a hypothetical Steam bug that somehow wrecks Windows installations, but in reality those don’t exist.

      1.  

        I think those kinds of comparisons don’t work very well, because of the range of options. For the Steam installation issue, on windows you basically have two options: you install it and it works or it doesn’t. In Linux you have the same two options + playing around with various tweaks and installation methods.

        If we were going with a typical user windows-like approach, he’d declare it a failure after Steam failed to install from the default source. Going further with other solutions is both a good thing because it’s possible and a bad thing, because newbies get into a situation like a broken desktop. So once you start going past the basics it’s really on the user to understand what they’re doing. Otherwise it’s comparable to “I messed with some windows DLLs / registry trying to get Steam to work, despite warnings and now it doesn’t boot” - but that’s just not something average users do.

        1. 12

          on windows you basically have two options: you install it and it works or it doesn’t

          On Windows you install Steam and it works. Installing Steam and it not working isn’t really an experience people have with Steam on Windows.

          In Linux you have the same two options + playing around with various tweaks and installation methods.

          I guess? But the Linux (Pop!_OS?) equivalent of “I messed with some windows DLLs / registry trying to get Steam to work, despite warnings and now it doesn’t boot” is [0] kind of the only experience that was available? It seems like there was no way to install it and have it work, or even install it and have it just not work. The only way to install it broke the OS?

          [0] Disclaimer: I didn’t watch the videos, so I’m going off my understanding of the comment I originally replied to

          1. 9

            Installing Steam and it not working isn’t really an experience people have with Steam on Windows.

            Not just that but you actually do have a lot of tweaks to play around with. They’re not common knowledge because it’s incredibly rare to need it in order to get something like Steam working. You don’t really need them unless you’re developing software for Windows.

            I had this “it’s a black box” impression for a long time but 10+ years ago I worked in a Windows-only shopped that did a lot of malware analysis and the like. It’s quite foreign, since it comes from a different heritage, but the array of “power tools” you have on Windows is comparable to that of Linux. The fact that typical users don’t need them as frequently is a good thing, not an evil conspiracy of closed-source vendors to make sure you don’t have control over your hardware.

            1.  

              Installing Steam and it not working isn’t really an experience people have with Steam on Windows.

              That’s a bit hard to quantify, but sure they do. Just search for “Steam won’t start” or “steam installer fails” on Reddit or their forums. It’s also common enough for many SEO-spam sites to have listicles for that phrase that are actually steam-specific.

              And my point was that this wasn’t there only experience available. The alternative was not to type “yes I’m sure I know what I’m doing” (or whatever the phrase was) when he did not. He went out of his way to break the system after the GUI installer refused to do it. I think you really should watch they fragment for the discussion context.

          2.  

            there’s no chance it’s going to break your OS

            Presumably because it’s the primary platform they test for.

            1.  

              there’s no chance it’s going to break your OS

              Of course with a simple installer (copy all files to a directory and add an entry to the windows registry) it’s quite hard to have a bug with breaks your OS. But a simple installer don’t have the features a package management system, i.e. central update mechanism. I don’t want to say package manager are better then the installer way used on Windows[0]. The problem I have with this case it’s not he has clicked some random button and then everything was broken. He has read the error, ignored all warnings and typed the prompt char by char and then wounder why it’s goes wrong.

              I don’t say the UI[1] is perfect. The problem I have is this “I ignore all warnings and complain if it goes wrong” mentality[2]. apt is not a program witch bugs you with unnecessary questions or warnings. Install a package only ask for conformation if it does more then only install the requested package. The annoying conformation question is there only if you try to remove essential packages and is designed to give enough hassle to bring the user to question about this commands.

              [0] I think systems with package manager are better, but this is not the point of the comment

              [1] The error message in the GUI and the handling in the command line

              [2] Yes some (or most) users don’t want to understand error messages, but shouldn’t they not stop at the error and look for (professional) help? And no copy paste a command from a random blog post is not help, if you don’t understand the error or the blog post.

            2. 19

              At the risk of being that condescending Linux user (which would be pretty awful since I’m not really using Linux anymore) my main takeaway from these videos is “don’t use hipster distros”.

              Or, okay, hipster distros is where innovation happens. I get it, Gentoo was a hipster distro when I started using it, too. Okay, maybe don’t recommend hipster distros to beginners?

              I saw Manjaro mentioned here. I tried Manjaro. It’s not a beginners’ distro. It’s great if you’re a burned out Arch user and you like Arch but you already know the instructions for setting up a display manager by heart and if you have to do it manually again you’re going to go insane. There’s a (small!) group of people who want that, I get it. But why anyone would recommend what is effectively Arch and a rat’s nest of bash scripts held together with duct tape to people who wouldn’t know where to begin debugging a broken Arch installation is beyond me. I mean the installer is so buggy that half the time what it leaves you with is basically a broken Arch installation for heaven’s sake! Its main value proposition is in a bunch of pre-installed software, all of which can be trivially installed on Ubuntu.

              I haven’t used Pop!_OS but IMHO a distribution that can’t get Steam right, Steam being one of the most popular Linux packages, is just not a good distribution. It’s particularly unsettling when it’s a distro that’s supposed to have some level of commercial backing, and Steam is one of the most popular packages, so presumably one of the packages that ought to get the most testing. Hell even Debian has instructions that you can just copy-paste off their wiki without breaking anything. And the only reason why they’re “instructions”, not just apt install steam, is that – given their audience – the installation isn’t multilib by default.

              There’s certainly a possibility that the problem here was in the proverbial space between the computer and the chair, sure. But if that’s the case again, maybe it’s just time we acknowledged that the way to get “better UX” (whatever that is this year) for Linux is not to ship Gnome with the umpteenth theme that looks like all other theme save for the colors and a few additional extensions. It’s safe to say that every combination of Gnome extensions has already been tried and that’s not where the magic usability dust is at. Until we figure it out, can we just go back to recommending Ubuntu, so that people get the same bad (I suppose?) UX, just on a distribution with more exposure (and, thus, testing) and support channels?

              Also, it’s a little unsettling that the Linux community’s approach to usability hasn’t changed since the days of Mandrake, and is still stuck in the mentality of ESR’s ridiculous Aunt Tilly essay. Everyone raves about consistency and looking professional. Meanwhile, the most popular computer OS on the planet ships two control panels and looks like anime, and dragging things to the thrash bin in the second most popular OS on the planet (which has also been looking like anime for a few years now) either deletes them or ejects them, which doesn’t seem to deter anyone from using them. Over here in FOSS land, the UI has been sanitized for consistency and distraction-free visuals to the point where it looks like a frickin’ anime hospital, yet installing Steam (whether through the terminal or the interface it makes no difference – you can click “Yes” just as easily as you can type “Yes”) breaks the system. Well, yeah, this is what you get if you treat usability in terms of “how it looks” and preconceived notions about “how it’s used”, rather than real-life data on how it’s used. It’s not an irredeemable state of affairs, but it will stay unredeemed as long as all the debate is going to be strictly in terms of professional-looking/consistent/beautiful/minimal/distraction-free interfaces and the Unix philosophy.

              1. 12

                The issue about Linux distro here is that they didn’t know the differences between them, why that matters, and that Linux isn’t one thing. Without a knowledgeable person to ask what to use, this is how they ended up with these different flavours. They also didn’t know about desktop environments, or how much influence they have over their Linux experience.

                It’s unfortunately a hard lens for many technical people to wrap their head around. Heck, we are starting to see people that don’t need to interact with hierarchical file systems anymore. Something natural to everyone here, but becoming a foreign concept to others.

                1.  

                  Certainly. My response was mostly in the context of an underlying stream of “Ubuntu hate” that’s pretty prevalent in the circles of the Linux community that also have a lot of advice to give about what the best results for “best Linux distro for gaming” should be. I know I’m going to be obtuse again but if the l33t h4x0rz in the Linux community could just get over themselves and default to Ubuntu whenever someone says “I’ve never touched Linux before, how can I try it?” a lot of these problems, and several distributions that are basically just Ubuntu with a few preinstalled programs and a custom theme, would be gone.

                  There’s obviously a huge group of people who don’t know and are not interested in knowing what a distribution is, what their desktop environment is, and so on. As the Cheshire Cat would put it, then it doesn’t really matter which one they use, either, so they might as well use the one most people use, since (presumably) their bugs will be the shallowest.

                  I know this releases all sorts of krakens (BUT MINT WORKS BETTER OUT OF THE BOX AND HAS A VERY CONSISTENT INTERFACE!!!1!!) but the competition is a system whose out-of-the-box experience includes Candy Crush, forced updates, a highly comprehensive range of pre-installed productivity apps of like ten titles, featuring such amazing tools like Paint 3D and a Calculator that made the Win32 calculator one of the most downloaded programs in history, two control panels and a dark theme that features white titlebars. I’m pretty sure any distribution that doesn’t throw you to a command prompt on first boot can top that.

                  1.  

                    Oh, I totally agree, I was just clarifying that they did some googling to try and find something to use, and it’s how they ended up with this mess of difficulties.

                2.  

                  Until we figure it out, can we just go back to recommending Ubuntu, so that people get the same bad (I suppose?) UX, just on a distribution with more exposure (and, thus, testing) and support channels?

                  I wish Ubuntu offered an easier flow for getting a distribution with the right drivers out of the gate. This is what Pop_OS! does (source):

                  Pop!_OS comes in two versions: Intel/AMD and NVIDIA. This allows us to include different settings and the proprietary NVIDIA driver for NVIDIA systems, ensuring the best performance and use of CUDA tools one command away. On Oryx Pro systems, you can even switch between Intel and Nvidia graphics using a toggle in the top right corner of your screen.

                  IMO this is superior to in Ubuntu where you need to follow complex instructions to get NVIDIA proprietary drivers: https://help.ubuntu.com/community/BinaryDriverHowto/Nvidia

                  And you need to follow different instructions for AMD graphics

                  Also if you buy a System76 laptop all the drivers for your computer come set up, no driver manager needed. With Ubuntu you can buy from Dell but not with the same variety of hardware as System76.

                  I agree that Ubuntu is a good option but I would like to see it improve in these aspects before I would recommend it to a random non-power user who wants to play video games.

                  1.  

                    I haven’t used Ubuntu in a while, and that page doesn’t help because the instructions look like they haven’t been updated since Ubuntu 12.04, but the way I remember it all you needed to do was go to “Additional Drivers” or whatever it was called, choose the proprietary driver, hit OK and reboot. Has that changed in the meantime? Last time I used a machine with an NVidia card I was running Arch and it was literally just pacman -S nvidia, please tell me Ubuntu didn’t make it more complicated than that!

                    Also… is the overlap between “people who write CUDA code” and “people who can’t install the proprietary NVidia drivers” really that big? Or is this aimed at people using third-party CUDA applications, who know statistics but suck at computers in general (in which case I get the problem, I’ve taught a lot of engineers of the non-computer kind about Linux and… yeah).

                    Also if you buy a System76 laptop all the drivers for your computer come set up, no driver manager needed.

                    If you choose the “Ubuntu LTS” option when ordering, doesn’t it come with the right drivers preloaded? I mean… I get that Pop!_OS is their thing, but shipping a pre-installed but unconfigured OS is not exactly the kind of service I’d expect in that price range.

                3. 17

                  The entire point of Linus’ challenge is that desktop Linux is full of barriers and traps for new users who don’t (yet) know what they’re doing.

                  Explaining “well, it’s like that because you told it to florb the waggis instead of confeling rolizins, so it’s all your fault” may very well be technically correct, but it doesn’t change the fact that the OS hasn’t worked well for the user. “I want to install Steam in 5 minutes without learning about package sudoku solvers, or bricking my computer” is an entirely reasonable use-case.

                  Web dev community had a reckoning with this, and thinking has changed from “users are too stupid to understand my precious site” to “all my new users know only other sites, so I must meet their expectations”. If Linux wants to get new users it needs to be prepared for users who know only Windows, macOS, or even just Android/iOS.

                  1.  

                    Explaining “well, it’s like that because you told it to florb the waggis instead of confeling rolizins, so it’s all your fault” may very well be technically correct, but it doesn’t change the fact that the OS hasn’t worked well for the user. “I want to install Steam in 5 minutes without learning about package sudoku solvers, or bricking my computer” is an entirely reasonable use-case.

                    That’s well and good, but there is a perfectly good fast path for this; install Pop!_OS or Ubuntu on a day where there’s not a bug in the packaging system, which is the vast majority of all days. Yep, it sucks that there was a bug, but that’s simply not going to affect anyone going forward - so why are LTT giving advice based on it?

                    1. 8

                      For every distro D there exists a problem P that is solved in a distro E.

                      That endless cycle of “then abandon your whole OS and install a completely new one” thing is another annoying problem “Linux desktop” has. It’s not any single distro’s fault, but it’s a pain that users need to deal with.

                      In my case: I want to use Elementary, but I hosed it trying to update Nvidia drivers. So I was told to switch to Pop!_OS — they do it right. But this one gets stuck seemingly forever when trying to partition my disk, presumably because of the combination of NVMe and SATA that I have. Manjaro worked with my disks, but I’ve run into bugs in its window manager, which wouldn’t be an issue in Elementary. I still use macOS.

                      1.  

                        For every distro D there exists a problem P that is solved in a distro E.

                        Right, I agree that in general this is a problem; we need better ways to integrate the best ideas from multiple projects.

                        But for the problem stated, which was “I want to install Steam in 5 minutes without learning about package sudoku solvers, or bricking my computer”, Pop!_OS or Ubuntu are the way to go. Your problem is not that; it’s “I want Pantheon and a fast Nvidia card,” and Nvidia have intentionally made that harder than it needs to be.

                        To be totally clear, I’m under no illusions that every user can simply pick up a free desktop and be on their way, but I think it’s pretty unhelpful to cultivate a discourse which simultaneously says “Users should have a fast path for these common use cases” and “Users should be able to get whichever window manager, packaging system, and customizations they want.” Those are both valuable goals, but the former inherently precludes the latter, especially in a world where some hardware companies, like Nvidia, are actively hostile to free desktop projects.

                    2.  

                      I switched from windows to Mint a couple of years back for gaming, in a similar experiment to this one (only not so public). I had no issues at all, steam was in the official applications, it installed with one click. Every game that steam claimed worked on linux did work. There were issues with my non-standard multi-monitor set up (there were issues with this in windows too, but they were worse in linux*) but nothing that prevented playing the games. It was only once I enabled the steam beta program which sets steam to attempt to open all games in wine that I had to get down in the weeds with configuring stuff and some things didn’t work. Steam has pretty clear warnings about this when you turn it on though.

                      I feel like for a tech tips site those guys are pretty non-technical. I never really watched their stuff anyway but now it seems like they should be calling me for help (and I am pretty noob when it comes to linux). This is the biggest criticism for me of this whole experiment. If these guys are an authority on computer tech informing users, they should simply be better at what they do. It is almost like they are running an investment advice channel and going ‘oh no I lost all my money investing in a random startup, guys don’t do the stockmarket it’s broken’. They should be informing people interested in linux what to do and what not to do, and if they are not qualified to do that they should state that and recommend alternatives sources of advice.

                      *I have a suspicion most of these issues were on the application level not the OS level. Games were probably getting the monitors list from the wrong place. Ironically once I set my monitors up in the way that the developers on both windows and linux were expecting me too, the problems on linux disappeared, but a few small issues persisted on windows.

                      1.  

                        The entire point of Linus’ challenge is that desktop Linux is full of barriers and traps for new users who don’t (yet) know what they’re doing.

                        I have understand this. The problem I have with some of the complains is they proclaim the one or other way is clear better during the challenge. This is a bit more obvious in the file extension example[0]. I completely understand the experience is frustrating. But the “it is frustrating for me because the system don’t behave like I expect from the knowledge of an other system” don’t mean this system is bad.

                        Yes systems try to adopt behavior from other systems to make it for users better to adopt. But this has it down side, because you can’t change a bad design after the user get used to it. In this example users get used to ignore errors and warnings and just “click ok”.

                        Explaining “well, it’s like that because you told it to florb the waggis instead of confeling rolizins, so it’s all your fault” may very well be technically correct, but it doesn’t change the fact that the OS hasn’t worked well for the user

                        I don’t want to imply they are dump or just don’t want to learn the system. It is frustrating, if a system don’t work the way you expect. I would like to see a discussion after the challenges with an expert explaining why the UI behaves different.

                        [0] Which I don’t write today, it’s late again

                        1.  

                          When doing usability evaluations, it’s normal to discount specific solutions offered by frustrated users, but never the problems they face.

                          There were a few problems here:

                          • Lack of hardware support. Sadly, that’s a broad problem, and difficult to fix.

                          • User needed to download and run a script from GitHub. I think distros could improve here. For a random developer who wrote a script, it’s difficult to distribute the code in a better way. There’s a very high bar for getting something to be an official package, and hardly any other viable alternative. There are several different packaging formats (some tedious to build, some are controversial), a few unofficial package repositories, a few “app stores” for some distros. All this fragmentation is a lot of work and maintenance headache. It makes just dumping a script on GitHub very easy and attractive in comparison. It may not be a failing of any single person or distro, but it is a failing of “Linux desktop” in aggregate.

                          • Browser and file manager did a crappy job by treating HTML with .sh extension as if it was a totally normal thing a user may want to do. The fight about file extensions has been lost in the ‘90s. I’ve been there, tweaking detection by magic bytes in my Directory Opus on AmigaOS, and fiddling with creator codes when copying stuff from classic MacOS. The reality is that file extensions exist, and are meaningful. No normal person stores web pages as “.sh” files.

                      2. 8

                        I mean, I have Big Opinions® on the subject, but my tl;dr is that Linux isn’t Windows, we shouldn’t give false expectations, have our own identity, etc. etc. But….

                        So he typed in “Yes, do as I say!” and his installation was broken. He claimed later: “the thinks that I did are not entirely ridiculous or unreasonable”. He ignored all warnings and “dictated the computer” “Yes, do as I say!”, how is this not a clear user error[0]?

                        I mean, the system should refuse to do that. Alpine’s and others refuse to allow the system to enter a boned state. One of the Alpine developers was rightly criticizing Debian for this issue in apt, citing it as one of the reasons why they stopped using Debian. The attention to the problem Linus gave in an embarrassing light was the push finally needed to fix it.

                        1.  

                          I like luke’s perspective that some distros should do different things. I think it’s reasonable for arch to be a ‘power user distro’ that is willing to bork itself. But PopOS is ‘an operating system for STEM and creative professionals’, so it probably should have some safeguards.

                          That being said I don’t think arch should ever be recommended to a brand new user. Linus shouldn’t even be on arch because 1) there should be better resources for picking a good distro for absolute beginners and 2) PopOS never should have that broken of a steam package in the first place.

                          1.  

                            That being said I don’t think arch should ever be recommended to a brand new user.

                            I would qualify this; there are many users for whom arch was there first distro and it went great, but the key thing is these are not your typical computer user; they are people who are technically minded (not necessarily with deep deep knowledge of anything in particular, but they’re probably at least the person their friends ask for help), are up to and interested in learning about the system, and generally have been given some idea of what they’re getting into. That is to say, arch is definitely for “power users,” but that set includes some users who have not actually used Linux before.

                            For my part, Arch was the first distro that was actually reliable enough for me to do more than play with; I spent a year or so fussing with other stuff while dual booting windows, and Arch is the first one that actually worked well enough for me wipe the windows partition and stay. This was 15 years ago and I haven’t left, though I keep eyeing NixOS these days.

                            I think at the time folks still had fresh memories of before Linux desktop environments were even a thing, and there was this mentality that the barrier to entry was mostly around user interfaces. People hadn’t really internalized the fact that Linux had caught up decently well even by then (this was KDE 3.x era), but the problem was stuff needed to work better out of the box, and it needed to not break whenever you upgraded to the next release of the distro.

                          2.  

                            Knowing how Internet guides work, now all guides will say “apt --allow-solver-remove-essential <do dangerous stuff> instead of “Type Yes, do as I say at the prompt”.

                          3. 6

                            To answer some of your questions:

                            First so similar we don’t get the issue, because under Windows there is no package manager accessible for somehow third party software.

                            Technically not true; there is a Windows package manager and has been for a long time, and that’s the Windows Installer (MSI files). There’s also APIs and supported methods for 3rd-party installers to register an install with the system. What’s historically been missing are official package repositories for installing and updating applications (ala. APT, RPM, etc… repos). That’s slowly changing with the Microsoft Store, winget, and others, but this is an area Linux has long been very far ahead.

                            So lets assume there is an windows update which removes the wrong file and breaks your system. On the install the update would remove the wrong file and breaks your system.

                            This is incredibly rare. I won’t claim it hasn’t happened, but more common (while still very rare) is an update which causes a bluescreen on boot or exposes a bug in a critical system process. In either case, we’re talking very rare, but I’d suggest that’s true of Linux too.

                            Other example the steam installer manage to have a bug with removes some necessary files from your Windows installation. Is there anything Windows protect you from this bug[1]?

                            Yes, several things, and this is a genuine major contrast to Linux. Off the top of my head:

                            1. Window system files cannot be modified by default even with administrator privileges. You can’t simply run an elevated Command Prompt and run the equivalent of rm -rf C:\Windows. That’s because most operating system files are both owned and only writeable by a special account (TrustedInstaller). You can still modify or delete these files, but you have to jump through several hoops. At a minimum, you need administrator privileges (ala. root), and would have to take ownership of the file(s) of interest and subsequently grant yourself the relevant privileges. There are other ways you could gain the relevant access, but the point is it’s not a thing you could do by accident. That’s similarly true for installers, which also would need to take the same approach.

                            2. Windows has long had numerous recovery options for when things go pear shaped. Notable ones include Safe Mode and its various permutations (since forever), the ability to uninstall operating system updates (also forever), System Restore (since XP?), System Reset (Windows 10?), and a dedicated recovery partition with a minimal Windows 10 installation to serve as a recovery environment wholly independent of the main operating system. Obviously, none of these are a guarantee for recovery of an appropriately damaged system, but it’s long been the case that Microsoft has implemented numerous recovery/rollback mechanisms.

                            On Linux, it’s usually limited to one or more previous kernel versions, and that’s about it? Yes, there’s single-user mode, but that just drops you into a root shell, which is wholly unsuitable for non-experts to use.

                            1.  

                              On Linux, it’s usually limited to one or more previous kernel versions, and that’s about it?

                              https://documentation.suse.com/sles/11-SP4/html/SLES-all/cha-snapper.html

                              By default Snapper and Btrfs on SUSE Linux Enterprise Server are set up to serve as an “undo tool” for system changes made with YaST and zypper. Before and after running a YaST module or zypper, a snapshot is created.

                          1. 25

                            No; Docker doesn’t require as much new learning. Solutions which let you lift-and-shift from previous infra really often win.

                            1. 8

                              This.

                              Also, in addition to being a containerization strategy, (Get ready to cringe hard line bloat hating ascetics!) Docker has also become a super convenient container for server side software distribution, and for that use case I think it would take a long time for nix to overtake the convenience of DockerHub and private docker registries.

                              1. 5

                                Definitely agree. I just installed NixOS in a vm to start attempting to learn it and seems like a pretty steep hill to climb, Docker was challenging when I first encountered it years ago, and while I’m far from being comfortable with it, I don’t think it had near the learning curve that Nix/NixOS does.

                                I want to continue to look into it, but it’s not going to replace any of my Docker containers any time soon.

                                1. 4

                                  Wait actually on second thought, maybe? It’s entirely plausible that Docker will implode horribly due to terrible decisions induced by share price chasing. Leaving something else to take the field from it with no competition.

                                  Then Nix would have some users and Docker would have roughly zero users (even though the former Docker users moved to something else really similar under a different banner) and Nix would have overtaken Docker.

                                  1. 3

                                    Perhaps a silly question - can’t we fall back to Podman? Red Hat has several Docker alternatives.

                                    1. 2

                                      Sure. Anything that can run containers.

                                1. 12

                                  I really dislike the tone of this article. It is overly condescending to the person who made this mistake. Especially this sentence:

                                  However, this was clearly too robust of a solution to the problem of determining where to install the pkg-config files, and so innovation™️ happened

                                  The tone detracts from the content of the post.

                                  1. 3

                                    I tried the recommended pkg-config command on several systems, and on some, it worked, and on others, it didn’t.

                                    1. 3

                                      In such circumstances, it would have been better to fail than to use a “solution” as in the article.

                                      1. 1

                                        Where did it fail?

                                        1. 1

                                          On an older Linux system I have sitting around. About 10, maybe 12 years old at this point.

                                          1. 1

                                            It worked on a fairly recent RHEL, openSUSE, FreeBSD, OpenBSD, macOS and MSYS2 though (yes, I’ve checked all of those)

                                      2. 1

                                        I think the tone is appropriate to the garbage produced here. Like, sometimes you have to produce garbage to make something work, but that doesn’t make it not garbage.

                                        I think software developers, myself absolutely included, need to avoid getting attached to the trash we produce. I think there needs to be a shared understanding of - no one person is to blame for the perennially bad state of the ecosystem, but neither can we coddle and defend bad decisions. Bad code impacts everyone; users, maintainers, and not least the original developer.

                                        Everyone produces trash sometimes. Lord knows I produce trash. At work, at home, in $HOME, on Github. You don’t need to take a stand and say “don’t say trash, that’s insulting.” To me, it’s insulting to think that I can’t stand being called out on bad ideas. I am not my code; if anything, I am the intent behind the code, and the intent is not furthered by obfuscating my mistakes from me.

                                        At least there should be a comment with a rationale.

                                        1. 3

                                          It’s not OK to hurl abuse at your coworkers or their work when they send out a suboptimal solution. I only see this childish behavior in open source projects, but I guess it could happen anywhere. It’s a good way to get people to leave your workplace and for others to avoid working there.

                                          Everyone produces trash sometimes. Lord knows I produce trash. […]

                                          Your argument boils down to you not caring about abusive behavior. Ok, but why would anyone else feel the same as you? There is a HUGE disconnect between your words and how myself and my coworkers interact. I would explain what to do instead, but it’s pretty fucking obvious.

                                          no one person is to blame

                                          When the poster writes “and so innovation™️ happened,” that’s blaming.

                                          At least there should be a comment with a rationale.

                                          No thanks! We screen for kindness in my company’s hiring process. It’s that easy for me to avoid these interactions. Most people are capable of learning kindness by the age of 12.

                                          1. 1

                                            Kind of annoyed that you didn’t respond to what I consider the core argument in my post, starting with “I am not my code.”

                                            Your argument boils down to you not caring about abusive behavior.

                                            I fundamentally disagree with this take on my argument. I suspect that you could construct a scenario where I would think otherwise, but to a first approximation, I don’t think abuse of a person through criticism of their code should be something that people consider a thing. I am not my code; you are not your code; nobody is their code. Calling out bad code is not an accusation of any sort, nor is it a character slight. We learn through feedback.

                                            There are trash fires in the codebase at the company where I work that people, even people still working here, at various points contributed to. Everyone agrees these projects are trash, because everyone can see that; nobody takes that personally. In fact, I suspect more people would take it personally if you told them that you couldn’t call it trash, because that would destroy all hope that it would some day get better.

                                            When the poster writes “and so innovation™️ happened,” that’s blaming.

                                            I don’t even think that it is; at most, it’s griping. Griping is healthy. I don’t know how people can get through the day without griping, considering the state of the field.

                                            At least there should be a comment with a rationale.

                                            No thanks! We screen for kindness in my company’s hiring process.

                                            I fail to see what that has to do with anything? I’m saying if there is a point to this, it should be documented in the file.

                                            edit: There’s several files that I’ve personally written where I could imagine a new hire coming to me and being like, “feep, isn’t this code kind of … shit?” And I’d be like “careful!………. it’s shit for a reason.” But it is, definitely, shit. I don’t see what the benefit is supposed to be in pretending otherwise.

                                            1. 2

                                              Kind of annoyed that you didn’t respond to what I consider the core argument in my post, starting with “I am not my code.”

                                              I mostly agree with the sentiment in your argument applied in general, but I am pre-empting it with my claim that this instance crosses the line into abuse.

                                              I don’t think abuse of a person through criticism of their code should be something that people consider a thing.

                                              I think there is a spectrum of reasonable criticism, and this linked post is beyond that point. To compare to cooking, there can be harsh criticism (of a cooked meal), and there is Hell’s Kitchen criticism. I believe the average person would not consider Hell’s Kitchen to be a healthy work environment. Ramsay’s criticism was probably often correct, but that is beside the point. He tore people down. I do not want our work environment to be Hell’s Kitchen.

                                              I don’t even think that it is; at most, it’s griping. Griping is healthy. I don’t know how people can get through the day without griping, considering the state of the field.

                                              This is a poor excuse for poor behavior. The author of the post did the digital equivalent of shaking hands for the first time with someone they never met and saying “Hi, nice to meet you, your work is really shit.” Griping privately is fine. Griping publicly and shitting on the VERY FIRST MISTAKE other random people make is not going to accomplish anything.

                                              No thanks! We screen for kindness in my company’s hiring process.

                                              I fail to see what that has to do with anything? I’m saying if there is a point to this, it should be documented in the file

                                              My bad, I misunderstood your comment. Please disregard.

                                              edit: There’s several files that I’ve personally written where I could imagine a new hire coming to me and being like, “feep, isn’t this code kind of … shit?” And I’d be like “careful!………. it’s shit for a reason.” But it is, definitely, shit. I don’t see what the benefit is supposed to be in pretending otherwise.

                                              Yep, I’ve been in this situation before too. So let’s run with this example. What if they didn’t come up and tell you it’s shit privately? They were just hired, run git blame on the code and write a long rant about how shit it is and send it to the author (you) and CC all your coworkers? That is what happened here. How many people would that convince? I think zero. If it convinces you, remove yourself from the example and re-run it with a random reasonable person. Ranting is not a useful rhetorical device. I’ll now expand on this idea a bit more.

                                              Let’s go back to my original post:

                                              I really dislike the tone of this article. It is overly condescending to the person who made this mistake. Especially this sentence:

                                              However, this was clearly too robust of a solution to the problem of determining where to install the pkg-config files, and so innovation™️ happened

                                              The tone detracts from the content of the post.

                                              I made the narrow claim that “The tone detracts from the content of the post.” It did. It was less convincing. I was reading the article thinking “this person is right, but they are being such an asshole that I don’t care.” And look what the maintainer of ncurses said: https://lists.gnu.org/archive/html/bug-ncurses/2021-10/msg00051.html Basically the same thing. It doesn’t matter how right you are. If you are an asshole, no one will care.

                                              When I find something shit I might rant about it privately, with the understanding that it will accomplish nothing. If I actually want to fix the problem, I write a proposal for the solution and send it out. If that proposal is ignored or treated like it’s stupid, I will escalate. Here is an example of a justified rant: https://lwn.net/Articles/629259/

                                              1. 1

                                                Yep, I’ve been in this situation before too. So let’s run with this example. What if they didn’t come up and tell you it’s shit privately? They were just hired, run git blame on the code and write a long rant about how shit it is and send it to the author (you) and CC all your coworkers? That is what happened here. How many people would that convince? I think zero. If it convinces you, remove yourself from the example and re-run it with a random reasonable person. Ranting is not a useful rhetorical device. I’ll now expand on this idea a bit more.

                                                Yeah, I mostly agree with that. On the other hand, every time I see a post on r/programminghorror I nervously check if I recognize the code…

                                                I think to a large extent this is a disagreement about perception of hostility in social media. To me, the innovation™ line is sufficiently dissociated from the maintainer that I wouldn’t feel targeted if it was me. I might go “now come on here” but I wouldn’t feel attacked¹ by it. As such, to me this seems somewhat beneficial with no downside, whereas I suspect to you it seems marginally beneficial with a clear downside? I see the point more as an attempt to consensus build code quality than to make anyone look bad.

                                                They were just hired, run git blame on the code and write a long rant about how shit it is and send it to the author (you) and CC all your coworkers?

                                                Honestly? Thinking about it, I kind of want this as a newsletter in the company I work now. As long as I’m not singled out as a target, this sounds fun.

                                                ¹ “Attacked” to me are posts where the poster is explicitly trying to attack my standing or relations, or make people think worse of me as a person. Those raise my hackles like nothing else. I don’t feel that here, I don’t think the author and the maintainer have a feud. I think they could grab a beer together. (And I don’t even drink beer, but it’s the spirit of the thing.) So my brain classes it as “harmless community interaction” and moves on.

                                                edit: A fun metric I came up with on the spot to gauge the quality of hostility is, “in the Reddit/Hnews/lobsters comments to this article, would the maintainer be upvoted or downvoted?” I think this is still on the upvote side.

                                                edit edit: I think you’re coming at this from a reasonable perception, but I can’t help but note that a community that is more sensitive to hostility is also a community that’s going to have fewer interactions by some people, because it presents them with higher effort costs. Maybe the quality of interactions or the gain from the people who have lower effort with less hostility makes up for this, but I could easily even see it be a net loss. I don’t think you’re wrong here, but I do think it’s a matter of - like, I think a programming community with this level of hostility as normal is valid. I think it should ought to be able to exist, even in a perfect world. Don’t you?

                                                1. 2

                                                  Thanks for the thoughtful responses. I did read your entire reply. I don’t mean to give TMI, but I will cut straight to the point.

                                                  I think a programming community with this level of hostility as normal is valid. I think it should ought to be able to exist, even in a perfect world. Don’t you?

                                                  It’s impossible for me to separate out my own neuroses enough to pass such judgement. When I see hostility of this level in a programming discussion, it is very, very, difficult for me to process it in a constructive manner. I can barely do it, after tens of minutes calming myself down and much stress. What you are getting at, basically “constructive hostility,” I am blind to. I think I have experienced too much true hostility quite early in my life. So I see no value in it. If it bleeds into my community, I fight against it. I know that other people see things another way, but that does not change my mind.

                                                  I am not saying that anyone owes me anything. I just have zero tolerance for hostility. And I will keep arguing against it.

                                                  1. 1

                                                    Fair enough, makes sense.

                                      1. 3

                                        The RFC on incorporating OSH into Nix is super exciting

                                        1. 1

                                          Work: Doing the finishing steps of transitioning my team’s build system from Makefiles to Bazel, and upstreaming the open source fixes along the way.

                                          Life: Closing on my first home purchase this Thursday or maybe Friday since Thursday is a bank holiday. And repairing my bike.

                                          1. 3

                                            This is an insane amount of effort. I’d like to know more about how they test their patches.

                                            1. 2

                                              I did not know about these practical limitations of path lengths on Linux. I was curious to find more information about ENAMETOOLONG. I found this:

                                              [ENAMETOOLONG] is coming from the kernel. It’s also filesystem dependent hence why you can’t find SYMLINK_MAX defined anywhere

                                              From https://stackoverflow.com/questions/36757577/in-linux-can-the-value-of-a-symlink-be-longer-than-path-max

                                              1. 4
                                                1. 3

                                                  And its Lobsters submission: https://lobste.rs/s/xhct5o/things_unlearned

                                                1. 8

                                                  The vendor has been notified us that file:// has been silently patched the vulnerability in Big Sur and has not assigned it a CVE. We have notified Apple that FiLe:// (just mangling the value) doesn’t appear to be blocked, but have not received any response from them since the report has been made. As far as we know, at the moment, the vulnerability has not been patched.

                                                  Ok, but how long did they wait before publicly disclosing the vulnerability? It’s not clear from this description whether the security researcher attempted responsible disclosure. Normally the researcher only posts a public notice of the vulnerability after a multiple month window following initial contact with the vendor. Am I missing something?

                                                  EDIT: wording

                                                  1. 1

                                                    Like you said, this all leads back to the problem that move constructor implementations can vary in how expensive they are to call. Google protobuf performing a full copy if other arena id != this arena id is surprising to me, although it does comply with the C++ standard. It would be better if they reassigned ownership in their Arena allocator. Does anyone know of an easy way to instrument C++ move constructors to detect how expensive they are? If there is a solution with static analysis that would be even better. It would also be helpful if we could manually decorate a move constructor that is expensive in such a way that std::optional warns us if it calls the expensive move.

                                                    A crude way to do this would be to define an interface that has a method bool moveIsExpensive() and log when std::optional is used with an expensive move constructor. And by making a wrapper myproj::optional:

                                                    namespace myproj {
                                                    // interface to describe whether move constructor is expensive
                                                    class MoveCost {
                                                    public:
                                                    bool moveIsExpensive()=0;
                                                    };
                                                    class Object : public MoveCost {
                                                    public:
                                                    ....
                                                    Object(Object&& other) { ExpensiveCopy(other) }
                                                    bool moveIsExpensive() { return true; }
                                                    };
                                                    template <class T>
                                                    class optional {
                                                    private:
                                                    std::optional<T> m_optional;
                                                    public:
                                                    optional(T o) : m_optional(o) {
                                                      if (DEBUG) {
                                                        auto expense = dynamic_cast<MoveCost>(&o);
                                                        if (expense) {
                                                          std::cerr << "warning at " << __file__ << ':': << __line__
                                                            << ": myproj::optional calling an expensive move constructor." << std::endl;
                                                        }
                                                      }
                                                    }
                                                    // delegate the rest to m_optional
                                                    ...
                                                    };
                                                    }
                                                    

                                                    Then you could make a myproj::optional<Object> and get a warning at runtime.

                                                    EDIT: As an aside - would it be better to prefer std::unique_ptr over std::optional for code that returns Google protobuf messages? That would only ever call a single std::move. Or are the ownerships semantics too restrictive for this use case?

                                                    1. 2

                                                      About your EDIT paragraph:

                                                      I think that’s better for binary size, but I still think output parameters is the way to go because you should leave the decision of where to allocate the protobuf object to the caller.

                                                      If the caller wants to use an Arena (like Google backends do), you make it impossible to use your function by allocating the object in the unique_ptr. Impossible without a copy which is undesirable and avoidable.

                                                      What if the caller has that object from a previous iteration in a loop? Reusing protobuf objects reduces the number of memory allocations as google:: protobuf::ParseFromString reuses the allocated buffers when populating the message again.

                                                      1. 2

                                                        You could allow the allocator and/or deleter to be overloaded via template parameters of the function that returns the unique_ptr. But at that point the function signature would be pretty noisy. And the caller would have to provide said allocators and deleters. I agree that the output parameters is the way to go.

                                                        1. 2

                                                          Noisy and you would lose the ability to compile them separately — templates have to be declared and defined inline.

                                                          http://foldoc.org/Separate+compilation

                                                    1. 6

                                                      Quite nice! I love little tools like this; I’ve had trash-cli aliased to rm for a long time and never regretted it.

                                                      1. 9

                                                        A big motivation for this project was that I wanted something like trash-cli but that was a single static binary instead of a larger Python application (nothing wrong with Python, I just like static binaries).

                                                        Also I wanted to actually mirror the semantics of rm as much as possible, since trash-put does not and it is not recommended that you alias it.

                                                        1. 3

                                                          I package trash-cli for Debian. Someone in the debian community once showed me some esoteric invocation of some existing xdg tools (that we almost certainly have installed) that did almost everything trash-cli does, but missed something crucial (probably restore, haha). I wish I could remember what it was.

                                                          1. 5

                                                            My original plan for this project was more or less that; patching together various standard commands using Bash. But this got clunky very quickly so I gave up on writing it in Bash and switched to D.

                                                          2. 1

                                                            A big motivation for this project was that I wanted something like trash-cli but that was a single static binary instead of a larger Python application (nothing wrong with Python, I just like static binaries).

                                                            You can probably compile trash-cli into a static binary using Nuitka and its --onefile option assuming all the dependencies play nicely. If you run Nuitka within a conda environment you can get all the C dependencies statically compiled in, too. There are downsides to doing this, like the binary can get pretty large and compile times can be multiple minutes. I am still happy you made this project, though.

                                                        1. 2

                                                          Going canoeing along the Allegheny river in Western Pennsylvania. Will be camping along the riverside. Really psyched!!

                                                          1. 17

                                                            I tried this and it actually is pretty darn fast. Coming from completion-nvim, it’s a massive difference in speed. If only the author licensed it properly

                                                            1. 19

                                                              Everything about this ‘edgy’, meme marketing reeks of immaturity – down to naming it Coq right after news of Coq realizing it probably needs a new name. While there is room on the scale for more fun/serious (no, putting emoji in the docs or CLI is not ‘fun’), I think this well overshot into gawdy and something I can’t take seriously.

                                                              1. 8

                                                                I’m not a huge fan of the copy, but it is pretty good software so I’ll judge it by that metric.

                                                                1. 5

                                                                  I wouldn’t want to raise an issue or collaborate with the project though

                                                                2. 3

                                                                  Very edgy. I respect the author’s choice to represent their project however they like, but it all comes across very unprofessional to me. Profanity aside, the commit log alone makes me wonder about the quality of the project.

                                                                  1. 6

                                                                    I don’t get why professionalism matters here? This is a personal project they made in their spare time and released for other people to use if they want. There’s nothing professional about it.

                                                                    1. 6

                                                                      Profanity aside, the commit log alone makes me wonder about the quality of the project.

                                                                      Ouch… I just took a look at it, and yes, I understand your reluctance… I never look at the commit log of projects to assess their quality, now I’m thinking that I should start doing that.

                                                                      Thanks for saying this!

                                                                      1. 4

                                                                        I think icy’s point is a good one. If its good software, then who cares. The commit log being organized says nothing about the quality of the software. If the author is working on a thing primarily by themselves, then it doesn’t matter too much if the commit log is clean as they are the only ones that are hurt by it.

                                                                        If the software solves a problem, then that’s a worthy reason for its existence imho

                                                                        1. 2

                                                                          You’re welcome! The log certainly isn’t the only indicator of project quality, but when the readme concerns me I like to check the logs.

                                                                          1. 1

                                                                            The r/vim sub didn’t take kindly either https://redd.it/p5931v considering it’s Neovim only and the react doesn’t inspire confidence

                                                                            1. 1

                                                                              If it’s good software, isn’t that evidence you should care less about the commit log?

                                                                              1. 5

                                                                                Reading commit logs is a great first step towards contributing to the project. Whenever I’m learning how a project works, often times I’ll look at the log messages, especially when I want to run git blame or git annotate.

                                                                                Proper log messages not only help others, but yourself, too. I’ve forgotten a lot of the code I’ve written over the period of my hobbyist/career. I’ve needed to go back and make changes to code I’ve written. So that I can provide the best fix possible, it’s helpful to understand the “why” of a commit. The changes a commit produces the “what” and log messages (should) provide the “why”.

                                                                                1. 2

                                                                                  None of that is an argument for why a chaotic commit log is evidence that a project is not good or that the software is bad

                                                                                  1. 2

                                                                                    That’s not the point I was making.

                                                                          2. 2

                                                                            Moreover… I don’t know if you understand French but “gawdy” is probably a good adjective to describe the linked video at the beginning of the readme.

                                                                          3. 8

                                                                            I wrote a github comment on that license issue: https://github.com/ms-jpq/coq_nvim/issues/15#issuecomment-900956033

                                                                            Usually I don’t care too much, stuff like the WTFPL is a bit stupid but ultimately mostly harmless. But this “license” is really dangerous and could end up getting people hurt, if any part of it turns out to be enforceable.

                                                                            1. 4

                                                                              Yeah this neovim completion engine has me shaking in my boots.

                                                                              I find it all refreshing that this guy doesn’t care about using his github as a linkedin or about people who think his license is dangerous.

                                                                              1. 2

                                                                                Are you making fun of the idea that giving up your rights to sue anyone ever can be dangerous? I don’t think I’m understanding you.

                                                                                1. 3

                                                                                  Interpreting this as “can’t sue for any reason ever” should definitely and obviously be unenforceable right? If that could ever work, that’s not an issue with the license, rather that’s a huge issue with the legal system in question.

                                                                                  1. 3

                                                                                    I mean, I agree. It’s probably not enforceable. But I don’t know, I’m not a lawyer and neither is the author - and I’d not be willing to test it.

                                                                                    I have a lot of confidence that the “you can’t sue me” part of the license is unenforceable, so users of software under this license are probably safe. I assume. Again, not a lawyer. But the part where the license author promises not to ever sue the licensee? I have no idea how that works in court. Could a lawyer argue that the author of the license didn’t know what they were doing so that the license he wrote doesn’t apply to himself? Are there other protections in place to protect license authors from themselves? I really, really wouldn’t want to find out if I was in his shoes.

                                                                                    If there are any lawyers out there who could bring some clarity to this, I’d love to hear it. But the obvious practical solution is to pick a real license.

                                                                                  2. 2

                                                                                    Yes

                                                                              2. 6

                                                                                It has now been relicensed as GPL v3

                                                                                1. 2
                                                                                  1. 1

                                                                                    Have you compared it to coc-nvim?

                                                                                    1. 2

                                                                                      No. I’m using Neovim’s built in LSP.

                                                                                  1. 3

                                                                                    It’s even worse than this. Backgrounds are also per output device. So if you use one or more external monitors, you need to change the background for each desktop on every single display you use. It’s nuts.

                                                                                    1. 4

                                                                                      Oh! Thanks for mentioning this. I just realized from testing multiple monitors that the AppleScript iterator tell every desktop actually iterates across monitors. I thought it was supposed to be used for the desktops listed in Mission Control. I just updated my tool to support multiple monitors now, using that iterator. I could only test for 2 monitors since I’m on M1.

                                                                                      1. 1

                                                                                        That’s awesome!

                                                                                    1. 7

                                                                                      Queuing theory. Retry with exponential back-off and jitter. Is this stuff people pick up from blog posts or are they actually taught in a typical undergraduate CS program?

                                                                                      1. 4

                                                                                        Pick up from asking friends while struggling with an issue, or from reading papers and blog posts. Or at university. I think the statistics professor gave me a nudge. I’d read the TCP papers at the university library and asked about something after, I think, a lecture about the Poisson distribution, and got a really helpful monologue about skewed distribution of input. (Translated to programmerese: use backoff and make sure you don’t accidentally synchronise.)

                                                                                        There are many good papers. It’s a bit difficult to explain how to recognise the good ones. Reading the morning paper archives and the papers it links to might help.

                                                                                        1. 8

                                                                                          use backoff and make sure you don’t accidentally synchronise

                                                                                          I once took down pretty much the whole Google App Engine platform, in the middle of a Republican convention that Google/YouTube was sponsoring/covering (yes, that happened), because I didn’t think about that. Whoops.

                                                                                          (Also just terrible project management, ridiculous demands from Google/YouTube to me, the solo developer at the agency they’d outsourced to, and other reasons. But definitely the thing that technically happened is I DDOSed App Engine, and Google paid me to do it)

                                                                                        2. 1

                                                                                          Yes, I was specifically taught these concepts at my undergraduate CS program in my networking programming course. Graduated in 2019 from Edinboro University of Pennsylvania, which is not an especially well known program. But of high quality!

                                                                                        1. 1

                                                                                          This was a very difficult article to parse. I think it should have been edited for clarity. For example:

                                                                                          We develop our code on Windows and have a Windows version of our servers that can run under Windows.

                                                                                          This would not have passed proofreading, the sentence is redundant. The rest of the article has similar problems.

                                                                                          It would have been immensely more useful and clear if they provided code examples of their scripts in this pipeline.

                                                                                          The script downloads the Docker container that was built with that version, extracts the server binary from it, along with certain shared runtime libraries for gdb’s use. (This avoids gdb compatibility problems you may encounter if your WSL version does not exactly match the deployed Linux version.) The script writes to ~/.gdbinit to set up the shared libraries as system libraries for the debug session.

                                                                                          Couldn’t they just show the script? Omit any proprietary paths and just show the setup needed to get the right libstdc++.so extracted! I would have sent this article to my office, but it is too far from helpful.

                                                                                          1. 2

                                                                                            The script downloads the Docker container that was built with that version, extracts the server binary from it, along with certain shared runtime libraries for gdb’s use. (This avoids gdb compatibility problems you may encounter if your WSL version does not exactly match the deployed Linux version.) The script writes to ~/.gdbinit to set up the shared libraries as system libraries for the debug session.

                                                                                            Couldn’t they just show the script? Omit any proprietary paths and just show the setup needed to get the right libstdc++.so extracted! I would have sent this article to my office, but it is too far from helpful.

                                                                                            Recreating the script shouldn’t be to difficult, what I would do is something like this:

                                                                                            • Use ldd to get a list of the libraries loaded by the program and copy all of these files
                                                                                            • put these gdb commands a .gdbinit file (if solib-search-path doesn’t work right try something on this page instead https://sourceware.org/gdb/onlinedocs/gdb/Files.html)

                                                                                            set solib-search-path /tmp/dbg/lib/

                                                                                            file /tmp/dbg/executable

                                                                                            core-file /tmp/dbg/core

                                                                                            Also as an aise that setup they describe sounds a bit over-complicated, it would be much simpler if the “Debug on WSL” option worked with docker containers using the WSL backend (it might I haven’t tried yet).

                                                                                          1. 7

                                                                                            For a couple years around 2012 or so my laptop broke and I couldn’t afford a new one. I used mpv with the youtube-dl backend to watch videos on my XP-era compaq laptop running Linux. The combination of Linux and mpv/youtube-dl really breathes life back into old hardware which would just be electronic junk otherwise. I am very thankful for this tool!

                                                                                            1. 1

                                                                                              This was a good read. But the variable names (a, b, etc) obfuscated the code examples and made them hard to follow.

                                                                                              1. 18

                                                                                                What has been the problem with Python/Flask/SQLA?

                                                                                                1. 28

                                                                                                  Python: the size of the codebase and number of moving parts has reached a point where the lack of static typing has become the main source of programmer errors in the code. There are type annotations now, but they don’t work very well IMO, are not used by most of our dependencies, and would be almost as much to retrofit onto our codebase as switching to a type-safe language would be. The performance of the Python VM is also noticably bad. We could try PyPy, but again… we’re investing a lot of effort just to stick to a language which has repeatedly proven itself poorly suited to our problem. The asyncio ecosystem helps but it’s still in its infancy and we’d have to rewrite almost everything to take advantage of it. And again, if we’re going to rewrite it… might as well re-evaluate our other choices while we’re at it.

                                                                                                  Flask: it’s pretty decent, and not the main source of our grief (though it is somewhat annoying). My main feedback for Flask would be that it tries to do just a little bit too much. I wish it was a little bit more toolkit-oriented in its design and a more faithful expression of HTTP as a library.

                                                                                                  SQLAlchemy: this is now my least favorite dependency in our entire stack. It’s… so bad. I just want to write SQL queries now. The database is the primary bottleneck in our application, and hand-optimizing our SQL queries is always the best route to performance improvements. Some basic stuff is possible with SQLAlchemy, simple shit like being smart about your joins and indicies, but taking advantage of PostgreSQL features is a pain. It’s a bad ORM - I’m constantly fighting with it to just do the shit I want it to and stop dicking around - and it’s a bad database abstraction layer - it’s too far removed from Postgres to get anything more than the basics done without a significant amount of grief and misery. Alembic is also constantly annoying. Many of the important improvements I want to do for performance and reliability are blocked by ditching these two dependencies.

                                                                                                  Another problem child that I want to move away from is Celery. It just isn’t flexible enough to handle most of the things I want to do, and we have to use it for anything which needs to be done asyncronously from the main request handling flow. In Go it’s a lot easier to deal with such things. Go also allows me to get a bit closer to the underlying system, with direct access to syscalls and such*, which is something that I’ve desired on a few occasions.

                                                                                                  For the record, the new system is not without its flaws and trade-offs. Go is not a perfect tool, nor GraphQL. But, they fit better into the design I want. This was almost a year of research in the making. The Python codebase has served us well, and will continue to be useful for some time to come, in that it (1) helped us understand the scope necessary to accomplish our goals, and (2) provided a usable platform quickly. Nothing quite beats Python for quickly and easily building a working prototype, and it generally does what you tell it to, in very few lines of code. But, its weaknesses have become more and more apparent over time.

                                                                                                  * Almost. The runtime still gets on my nerves all the time and is still frustratingly limiting in this respect.

                                                                                                  1. 9

                                                                                                    Thanks for responding. I think static typing in Python works really well once configured so I’m surprised to hear you say that. I think it’s better than the static typing in most other languages because generics are decent and the inference is pretty reasonable. For example it seems better thought out than Java, C and (in my limited experience) Go. My rough feeling is that 75% of the Python ecosystem either has type annotations or has type stubs in typeshed. Where something particularly important is untyped, I tend to just wrap it and give it an explicit annotation (this is fairly rare). I’ve written some tips on getting mypy working well on bigger projects.

                                                                                                    I don’t think you have the right intuition that asyncio would help you if your problem is speed. I pretty convinced that asyncio is in fact slower than normal Python in most cases (and am currently writing another blogpost about that - UWSGI is for sure the fastest and most robust way to run a python webservice). Asyncio stuff tends to fail in weird ways under load. I also think asyncio is a big problem for correctness - it actually seems quite hard to get asyncio programs right and there are a lot of footguns around.

                                                                                                    Re: SQLAlchemy - I’m also very surprised. I think SQLAlchemy is a good ORM and I’ve used postgres specific features (arrays, json, user defined functions, etc) from it a great deal. If you want to write SQL-level code there is nothing stopping you from using the “core” layer rather than the “ORM” layer. There’s also nothing stopping you using SQL strings with the parameterisation, ie "select col_a from table where col_b = :something - I do that sometimes too. I have to say I have never had trouble with hand optimising a SQL query in SQLA - ever - because it gives you direct control over the query (this is even true at the ORM level). One problem I have run into is where people decide to use SQLA orm objects as their domain objects and…that doesn’t end happily.

                                                                                                    Celery however is something that I do think is quite limited. It’s really just a task queue. I am not sure that firing off background tasks as goroutines is a full replacement though as you typically need to handle errors, retry, record what happened, etc. I think even if you were using go every serious system ends up with a messaging subsystem inside it - at least for background tasks. People do not usually send emails from their webserving processes. Perhaps the libraries for this in go land are better but in Python I don’t think there is a library that gets this kind of thing wholly right. I am working on my own thing but it’s too early to recommend it to anyone (missive). I want to work on it more but childcare responsibilities are getting in the way! :)

                                                                                                    Best of luck in your rewrite/rework. I have not been impressed with GraphQL so far but I haven’t used the library you’re planning to use. My problems with GraphQL so far are that a) it isn’t amenable to many of the optimisations I want to do with it b) neither schema first nor code first really work that well and c) it’s query language is much more limited than it looks - much less expressive than I would like. You may not find that the grass is greener!

                                                                                                    1. 5

                                                                                                      I don’t think you have the right intuition that asyncio would help you if your problem is speed.

                                                                                                      I don’t want asyncio for speed, I want it for a better organizational model of handling the various needs of the application concurrently. With Flask, it’s request in, request out, and that’s all you get. I would hope that asyncio would improve the ability to handle long-running requests while still servicing fast requests, and also somewhat mitigate the need for Celery. But still, I’ve more or less resigned from Python at this point, so it’s a moot point.

                                                                                                      I am not sure that firing off background tasks as goroutines is a full replacement though as you typically need to handle errors, retry, record what happened, etc.

                                                                                                      Agreed. This is not completely thought-out yet, and I don’t expect the solution to be as straightforward as fire-and-forget.

                                                                                                      My problems with GraphQL so far are that a) it isn’t amenable to many of the optimisations I want to do with it b) neither schema first nor code first really work that well and c) it’s query language is much more limited than it looks - much less expressive than I would like.

                                                                                                      I have encountered and evaluated all of the same problems, and still decided to use GraphQL. I am satisfied with the solutions to (a) and (b) presented by the library I chose, and I feel comfortable building a good API within the constraints of (c). Cheers!

                                                                                                    2. 3

                                                                                                      So do you plan to keep the web UI in Python using Flask, and have it talk to a Go-based GraphQL API server? Or do you plan to eventually rewrite the web UI in Go as well? If the latter, is there a particular Go web framework or set of libraries that you like, or just the standard library?

                                                                                                      1. 4

                                                                                                        To be determined. The problems of Python and Flask become much less severe if it’s a frontend for GraphQL, and it will be less work to adapt them as such. I intend to conduct more research to see if this path is wise, and also probably do an experiment with a new Golang-based implementation. I am not sure how that would look, yet, either.

                                                                                                        It’s also possible that both may happen, that we do a quick overhaul of the Python code to talk to GraphQL instead of SQL, and then over time do another incremental rewrite into another language.

                                                                                                      2. 3

                                                                                                        I’m curious about why you consider that Flask does “a little bit too much”. It’s a very lightweight framework, and the only “batteries included” thing I can think of is the usage of Jinja for template rendering. But if I’m not wrong, sourcehut uses it a lot so I don’t thing this is what annoys you.

                                                                                                        Regarding SQLAlchemy, I totally agree with you. It’s a bad database abstraction layer. When you try to make simple queries it becomes cumbersome because of SQLAlchemy’s supposed low level abstractions. But when you want to make a fine-grained query it’s also a real pain and you end up writing raw SQL because it’s easier. In some cases you can embed some raw SQL fragment inside the ORM query, but it is often not the case (for example, here is a crappy piece of code I’m partially responsible of). Not having a decent framework-agnostic ORM is the only thing that makes me miss Django :(

                                                                                                        1. 8

                                                                                                          Regarding Flask, I recently saw Daniel Stone give a talk wherein he reflected on the success of wlroots compared to the relative failure of libweston, and chalked it up to the difference between a toolkit and a midlayer, where wlroots is the former. Flask is a midlayer. It does its thing, and provides you a little place to nestle your application into. But, if you want to change any of its behavior - routing, session storage, and so on - you’re plugging into the rails its laid down for you. A toolkit approach would instead have the programmer always be in control, and reach for the tools it needs - routing, templating, session management, and so on - as they need them.

                                                                                                          1. 1

                                                                                                            I’ve personally found falcon a bit nicer to work with than flask, as an api/component.
                                                                                                            That said, as a daily user for some mid-sized codebases (some 56k odd lines of code), I very much agree with what you said about python and sqlalchemy.

                                                                                                          2. 4

                                                                                                            I find that linked piece of code perplexing because converting that from string-concat-based dynamic SQL into SQLA core looks straightforward: pull out the subqueries, turn them into python level variables and then join it all up in a single big query at the end. That would also save you from having a switch for sqlite in the middle of it - SQLA core would handle that.

                                                                                                          3. 1

                                                                                                            SQLAlchemy: this is now my least favorite dependency in our entire stack. It’s… so bad

                                                                                                            That’s also the only thing I remember about it from when I used it years ago. Maybe it’s something everyone has to go through once to figure out the extra layer might look tasty, but in the end it only gives you stomach ages.

                                                                                                          4. 13

                                                                                                            Yeah, I’d be very interested to hear more about that too. Not that I disagree, but I think his article was light on details. What were the things that “soured” his view of Python for larger projects, and why was he “unsatisfied with the results” of REST?

                                                                                                            1. 11

                                                                                                              I found REST difficult to build a consistent representation of our services with, and it does a poor job of representing the relationship between resources. After all, GraphQL describes a graph, but REST describes a tree. GraphQL also benefits a lot from static typing and an explicit schema defined in advance.

                                                                                                              Also, our new codebase for GraphQL utilizes the database more efficiently, which is the main bottleneck in the previous implementation. We could apply similar techniques, but it would require lots of refactoring and SQLAlchemy only ever gets in the way.

                                                                                                            2. 1

                                                                                                              Ive been using Flask and Gunicorn. I basically do native dev before porting it to web app. My native apps are heavily decomposed into functions. One thing that’s weird is they break when I use them in web setup. The functions will be defined before “@app” or whatever it is like in a native app. Then, Gunicorn or Flask tells me the function is undefined or doesn’t exist.

                                                                                                              I don’t know why that happens. It made me un-decompose those apps to just dump all the code in the main function. Also, I try to do everything I can outside the web app with it just using a database or something. My Flask apps have stayed tiny and working but probably nearing the limit on that.

                                                                                                            1. 4

                                                                                                              I have been doing a bit of JavaScript lately, and my thought so far is that the sync/async divide is a mistake, exposing details which shouldn’t matter in general. Seems like either code should always appear synchronous (and the runtime should do what’s necessary in the background) or it should always be asynchronous. But maybe I am missing something!

                                                                                                                1. 7

                                                                                                                  C# solved this by doing both. Named async functions that return a value need a Task<> type. The async keyword is there for anonymous functions (which have inferred types).

                                                                                                                  The await keyword avoids “callback hell” but because you’re dealing with a reified Task and not just a colored function you can also call other functions on it and define strategies and such.

                                                                                                                    1. 1

                                                                                                                      Great article! I think that’s part of why writing JavaScript has been so painful for me: I normally write in Go, which as Bob points out does all this right, and just works.

                                                                                                                    2. 6

                                                                                                                      the sync/async divide is a mistake

                                                                                                                      It is. Of course, the problem is, everyone puts up with this async/sync divide because of backwards compatibility.

                                                                                                                      An ideal distillation of this would involve an async-only runtime, with a programming model that is sync by default. If you wanted to do async-style things, you could, but it’d be opt-in, and only for the type of work that was traditionally async, such as receiving from N sockets at once. Think of something along the lines of Win32’s WaitForMultipleObjects() as a primitive.