1. 49
  1.  

  2. 61

    In a world where there is a serious shortage of volunteers to do all of this, it seems to me that a small army of “packagers” all doing duplicate work is perhaps not necessarily the best way to distribute the available manpower.

    Obviously there are also upsides to package mangers, but it’s not all roses and sunshine. I use the system’s package manger for some things, and don’t use it for others. It works better that way.

    Some developers may complain that distros ship their software too slowly, but you shouldn’t sweat it [..] You should use a distro that ships updates to you at the pace you wish, and let your users do the same.

    Make a vaguely popular Vim plugin and I will guarantee you that will get bug reports that it doesn’t work because people are using a Vim version that’s more than 2 years old, because that’s the version their Ubuntu ships with. I appreciate there are conflicting interest here and that there is no simple solution here, but handwaving away real problems isn’t helping.

    examine the software being packaged for anti-features like telemetry or advertisements, patching it out as necessary

    I would be rather annoyed if someone packaged my software “Example” as “Example” but with alleged “anti-features” removed, especially if those features allow me to actually pay my rent and food. You could also just choose to not package/use/install software if you don’t like it, instead of shipping it to users in a modified state (user who don’t really have a choice in the matter, by the way; who says all users share your values?)

    Patching software in general has its own risk; see the Debian OpenSSL incident for an extreme example, but generally “Example 1.0” is tested by those who know the most about it as released, not “as released with a bunch of modifications from a random Debian packager”.

    The “This article presumes that proprietary/nonfree software is irrelevant, and so should you” line is unbelievable pompous; who the hell are you to tell me what I should do? I think I will start replacing this sort of free software absolutism with similar lines about veganism: “This article presumes that a non-vegan diet is irrelevant, and so should you”. Sounds pretty silly, ‘ey? (not taking the piss at vegans; I am vegan; can also use religion or something else). It’s just arrogantly trying to impose your own preferences and set of morals on other as some sort of absolute truth, nothing more.

    Footnote: In general I am getting rather exasperated by Drew’s “this is the one true way to do it, or else you’re a bloody stupid[1]”-attitude that I’ve seen on several occasions. I already knew it was going to be an article like this based purely on the title and author.

    [1]: “bloody stupid” taken from HN comment, which seems to have inspired at least parts of this article.

    1. 15

      You should use a distro that ships updates to you at the pace you wish, and let your users do the same.

      That line is one of those that sound great and makes democracy seem perfect but actually glosses over the fundamental problem that people will solve their pains in order from greatest to least. I can vote for the politician that promises improvements to (for example) net neutrality, but if that same candidate wants to cut a government service I rely on, I’m going to vote for someone else regardless of their stance on net neutrality. Likewise, if my choice is between a distro that is fundamentally broken but provides quick updates to software packages versus a distro that’s well supported, stable, and functions for my job, I’m going to pick support and stability over broken with fast updates. I’m not going to switch my daily driver from (for example) Ubuntu to Kali because of their software update policy because Kali is fundamentally broken as a daily driver OS. Update policy has nothing to do with it.

      The #1 biggest problem in the Linux world is “if you don’t like it there’s a million choices to switch to” and the #2 biggest problem is “if you don’t like it, build your own”. That’s fine if your only target is the developer community, but end users won’t put up with that.

      1. 2

        Likewise, if my choice is between a distro that is fundamentally broken but provides quick updates to software packages versus a distro that’s well supported, stable, and functions for my job, I’m going to pick support and stability over broken with fast updates.

        I mean… having quick updates and being stable are basically opposing forces. So if you want stable normally you don’t want updates anyway, which is why you pick a stable OS…

        1. 1

          There are strategies though, as usually it’s not that black and white. For example, I like my base system to be stable, but the user-space tools (that mostly run without elevated permissions) to be reasonably up to date. I can deal with my editor or somesuch crashing every now and then (although, in reality that doesn’t really happen much anyway), but I want my ssh daemon to be well vetted, and new features are not important there (security updates are, but those are usually pushed out by the Distribution maintainers fairly quickly). So I’m running the nix package manager on top of Debian stable. There are other options, but I guess my point is that it does not have to be an ‘either/or’-situation…

      2. 5

        I would be rather annoyed if someone packaged my software “Example” as “Example” but with alleged “anti-features” removed, especially if those features allow me to actually pay my rent and food.

        Note that free software licenses must allow derived works per the FSF four freedoms, and the OSI’s open source definition. The OSI does allow use of licenses that require the modified software to carry a different name or version, although most open source licenses do not have this requirement. Given that the article is in the context of packaging open source software, isn’t this just saying that you’d rather not be writing open source software, thus meaning the article is talking about problem domain that you’re choosing to not be a part of?

        1. 4

          Sounds to me like they are saying “I should choose an open-source license which requires rebranding of modifications for my work”.

          1. 5

            I wasn’t really considering the legal aspects, more the moral ones. Let’s say I spend 6 months developing a Banana app, and then release it with ads to support myself and the first thing people do is patch out the ad support. This strikes me as … not great. I appreciate people can do this, but just because you can doesn’t mean you should.

            I think open source is the right way to make most software, but it does require some form of cooperation from the community as a whole to work well, especially with regards to the whole financial question. I’m not saying that ads are great, but an absolute “it’s an anti-feature we need to patch out” stance strikes me as rather unempathetic with the needs of the developers at best.

            1. 11

              I support this viewpoint. In my experience, some people just can’t grasp that one can or should draw a distinction between morals and laws. I don’t want to use laws to enforce certain things, but that doesn’t mean that following the law ensures I won’t think you’re a jerk. For example, almost all my software is released under a public-domain-like license. This technically permits folks to legally plagiarize my software. I support their legal right to do that. But I will think you’re an asshole and potentially use non-legal means to fix the situation.

              I see this as the same principle you’re eapousing with your ad example. Your example is just more inflammatory, as can be seen by some of the responses. But that doesn’t change the validity of your point.

              (I also agree with your perspective on the OP. He has always been this way, as far as I can remember. Whether intentional or not, it is extremely effective at driving traffic to his blog because virtually every post inspires controversy. And not the good kind, IMO.)

              1. 0

                If by “plagarize” you mean “not attribute to me” then I would point out that the right to be identified with a work is actually somewhat rarely conferred by law anyway. Copyright is (in many jurisdictions) concerned with copying (and copying-like things, such as performance) not with attribution.

                1. 0

                  Licenses like MIT require attribution by including the license: https://tldrlegal.com/license/mit-license

                  1. 0

                    Yes, but that is a special requirement the MIT license adds in order to be allowed to exercise rights under the license.

                    There is no such requirement under default copyright law, it’s just something that can be added (as can basically anything else that is tied to the use).

                    1. 2

                      I realize it’s a special requirement. But that’s immaterial to my point:

                      Me: Please don't plagiarize my software.
                      Person: But you used a license, like the public domain (or UNLICENSE) that said I could.
                      Person: If you don't want people plagiarizing your software, then use MIT.
                      Me: I don't believe in enforcing every moral value I hold with the weight of the legal system.
                      

                      Which is analogous to what’s happening in this conversation. Licenses define what is legal. Not necessarily what is moral. Some licenses combine these notions intentionally, but that is not a universal property of all software licenses.

                      Basically, you’re pointing out an interesting factoid, but it doesn’t negate my point.

              2. 19

                What’s the point oft free software if there are “moral obligations to not change it”? I use free software exactly because anti-features like ads can be patched out. I am not willing to be annoyed/manipulated by ads just so that the author of an application can get like 5 cents from it or so.

                1. 4

                  Defaults matter in practice – to the tune of potentially lots. (Just ask all the folks who used to turn a profit selling freeware CDs for which the ISOs were already available, including basically all linux vendors – or the whole economy of subscription services, most of which survive simply because people are slow to cancel in a way that, at scale, makes money. Or consider the hexchat model – windows users can build xchat from source for free, or find a binary from somebody else who has, or they can pay for the official binary.)

                  In other words, the problem is not that someone has stripped whatever features, but that a centralized node in the distribution network has stripped those features such that most users, by default, never knew they existed.

                  1. 4

                    I agree. I think some open source developers just aren’t aware of, or don’t accept, the market that they’re participating in. If a developer’s work is so valuable, people will pay for changes. But it’s often not, because there’ll be other people who will publish competing software for no payment.

                    The reality of the free software situation is that you have to be lucky enough to make it big with a project e.g. curl (small scale), or build a company around an open source product e.g. Redhat (large scale), and in both cases get paid by companies for maintenance and consulting. New models like GitHub sponsorships, crowdfunding, subscriptions, are still finding their footing.

                    1. 2

                      What’s the point oft free software if there are “moral obligations to not change it”?

                      That’s not really what I said; your paraphrasing is an exaggeration which is indeed silly. All that I was trying to point out is that things are a bit more nuanced than “these anti-features should be patched out by distros”. I think my comment was phrased sufficiently nuanced to make that clear.

                    2. 10

                      and then release it with ads to support myself and the first thing people do is patch out the ad support.

                      It’s because you shouldn’t have chosen ads to support yourself in the first place. Your product will be worse because of it and users or packagers are right to remove them. Charge for it instead.

                      1. 2

                        It’s because you shouldn’t have chosen ads to support yourself in the first place. Your product will be worse because of it and users or packagers are right to remove them. Charge for it instead.

                        I appreciate that’s your opinion, and I even share it to some degree, but also appreciate there are others who make different choices. I think we should respect that, too.

                        1. -5

                          I will never pay for an app.

                          1. 9

                            How do you think the creators of the apps should survive in the current scarcity-based society?

                            1. 0

                              Put in unintrusive ads, put in a donate button (I actually use those), put in a “pro mode” for extra functionality.

                              1. 9

                                put in a “pro mode” for extra functionality

                                Does this mean that you would pay for pro functionality? Or do you leave that to others as well?

                                In any case: you’re free to never pay for an application of course, but I for one am happy to support developers by actually paying for software I use and benefit from. Especially when they still release that work under an OS license.

                                And I’m happy that my users have been paying me for the last 10 years so I can work on open-source full time. I like to think it’s a win-win situation.

                                1. 2

                                  Does this mean that you would pay for pro functionality?

                                  Depends on the app and what pro features offer over regular features. If it’s a good app I’ll happily pay to remove even unintrusive ads.

                                  1. 3

                                    So you would pay for (good) software after all. Happy to hear it!

                                    1. 3

                                      Yeah, I just would never pay to install an app onto my phone

                        2. 4

                          It seems to me though that free software in particular was an attempt to give legal weight to a moral position, namely that users should be free to decide exactly what software they use. This is contrasted to the proprietary software world, which is built on the idea that producers deserve to be compensated for their work, and the means of doing so typically implies a vendor creating an unchangeable bundle where the user can decide whether to accept the entire bundle or none of it.

                          IMO, these are both valid moral positions. Morality is subjective. But I’m struggling to reconcile the idea that users shouldn’t patch out things they don’t like, but that they should have the ability to do so. There is a world that believes users shouldn’t be able to patch out things they don’t like, but it’s a world that Drew is explicitly ignoring, presumably because that world isn’t catering to his needs or wants.

                      2. 3

                        especially if those features allow me to actually pay my rent and food.

                        So, basically you’re implying that you would make your fine software a malware (let’s face it - ads, tracking and other unintended behavior in application is what makes it malware) just to exploit your target user and its machine to sustain your primary income, and then get angry at them for not complying to that terms and patching the code that runs on their own personal machines?

                        Crazy.

                        1. 10

                          Vim has an advert in its front page, and most people don’t consider it malware.

                          Because of course it’s not that simple. AdSense and its clones are malware, but advertising is not inherently so.

                        2. 1

                          In a world where there is a serious shortage of volunteers to do all of this, it seems to me that a small army of “packagers” all doing duplicate work is perhaps not necessarily the best way to distribute the available manpower.

                          I think this is actually the main problem. It shouldn’t take an army of packagers (in addition to arcane automation scripts no one understands, but everyone started to rely upon).

                          I’m really hoping that distributions can improve their approach to package management; distri is a breath of fresh air in that regard.

                          I’m hopeful that other issues than speed will also be tackled (like non-root/user-local package installs) by people who care about it, and that package managers can take the next step.

                          1. -1

                            Who took your candy away? Your reply seems awfully negative, let people have their opinions…

                          2. 23

                            I’m not a huge fan of that view. I love myself an awesome packager, but sadly, the maintainer/packager relationship is very broken - to the point where “I’m a student maintaining the xyz packages for distro abc” is a good indicator for a discussion going the wrong way.

                            The package maintainer community has to go through a phase of reflection about how their model is subtly broken.

                            But first things first: every distribution has the moral right to ship our software any way they want. All power to you! If you do ship a changed upstream, I do expect you to own up for users problems, though and not yell at the maintainers for not following your practices.

                            Here’s a number of practical indicators that I’ve seen:

                            • Distributions take no feedback from upstream maintainers and ship in absurd ways
                              • Example: Debian (and by extension Ubuntu) has shipped Ruby split in multiple packages, of which only very view combinations were practically useful. That was the a substantial driver behind development of things like rvm and ruby-install. For example, you could install rubygems without TLS support, which is useless, as rubygems was always using HTTPS by default.
                              • Icing on the cake: Ubuntu 14.04 wanted to fix all this, breaking the release 5 days before LTS and and then shipping it.
                              • Debian has single-handedly driven one of the top 10 programming languages into a culture to see package maintenance as enemies. This is sad, as MRI is an explicitly FOSS project, with a capital F, and would fit Debian very well.
                            • I have no path to ask a distribution to fix my packages in an active way except reading up on a distributions organisational policies. I can not proactively request a change of maintainer.
                            • Distribution policies trump practical cases. There’s a classic case where Riak used an ABI-compatible fork of leveldb, which is tuned towards server use, but package maintainers ripping it out and using the upstream leveldb, which performs vastly worse.

                            Package maintenance is broken on the level of package maintainers being primarily gatekeeper and developing hazardous behaviours towards upstream, by “knowing better”. Package maintenance should be much more a discussion of tailoring the practical decisions of upstream to the practical decisions of a distribution.

                            This might sound terribly negative, so, I’ll finish with this: I <3 myself a good package maintainer and I’d love to have more active conversations. I appreciate that all sides have very limited time, but mistakes can have year-long impact.

                            1. 20

                              Distros are anything but guaranteed to ever get to including your package into the repos, even if you volunteer to maintain that package yourself. In case of LTS distros, they may not include completely new packages in older versions even if there’s no chance it will break anything. Thus your “time to market” can be years.

                              It gets worse when libraries you are using are not in that distro either. Worse yet if you are using a language that distro maintainers don’t understand and their packaging is unidiomatic or ourgith broken.

                              Don’t get me wrong, I do think distro maintainers are doing a great and important work. I have most of my software installed from distro repos, and I try to contribute to packaging when I can. But sometimes as a developer you have no choice but to distribute your own software, and as a user you have to turn to out of distro packages.That’s just reality.

                              One annoying example offhand: Fedora 30 still ships MuseScore 2.x in its repos. The 3.x line has been stable for a long time, it’s perfectly compatible with old files, and it’s a huge improvement over 2.x By huge I mean huge. Still not there. Luckily, its maintainers make an AppImage, so I can just get it in one wget command. When distro maintainers update it to 3.x, I’ll switch immediately, but it should happen first.

                              1. 1

                                Distros are anything but guaranteed to ever get to including your package into the repos, even if you volunteer to maintain that package yourself

                                This is honestly my main tension these days. Of course one should never (or, advanced users only: rarely) install software that does not come from their OS. However… how does one become popular enough to be included in an OS without first getting installs via some other channel? It’s a bit chicken-and-egg.

                                Of course, that only matters to the upstream if the goal of writing your software is getting people to use it, which for FLOSS is often not the case at all (people being able to use it is a happy side effect of it being written unless you’re selling it).

                              2. 18

                                I’ve worked on an open source project. Not so tiny, it used to be preinstalled with several major distros, and is still quite popular.

                                Early 2018 we had a major CVE, with remote code execution. We had a patch ready within of 8 hours of discovery, had it tested and in our official releases within of a few days.

                                Debian took over a month to patch it (and continued using an old version with major bugs, only patching security issues themselves). And they were the fastest. Alpine 3.7 was the first to ship the fix, and that took an eternity. Previous alpine versions (at the time still officially supported) never got the patch.

                                Now, we’re moving towards snap/flatpak for desktop and docker for server, and building our own packages and bundles, because distro maintainers are basically useless, always ship ancient broken versions, users come to us to complain about stuff being broken (and distros refuse to ship bugfixes or versions from this decade), and the maintainers are never reachable, and even security updates are shipped at glacial speed.

                                Honestly, distro maintainers are a massive security risk, and after this experience, I’m kinda mind blown.

                                1. 9

                                  As an Arch packager, I can’t help but feel a little bit offended by what you said there. >:(

                                  1. 11

                                    Arch is actually one of the few distrso where this issue never existed - but that’s because arch, being rolling release, actually just uses our upstream sources, and updates frequently and reliably.

                                  2. 7

                                    because distro maintainers are basically useless

                                    That’s quite an offensive statement.

                                    1. 6

                                      If major software that’s preinstalled and in the default start menu of Kubuntu is so outdated that it has remotely exploitable bugs, months after developers have released patches for all version branches, including the one used by Debian/Ubuntu/etc, then how can you really trust the packages installed on your system?

                                      How many programs from the repos do you have installed which are not that common, or complicated to package. Are you sure they’re actually up to date? Are you sure there are no vulnerabilities in them?

                                      Ever after this, I can’t trust distros anymore.

                                      1. 3

                                        And that makes distro maintainers basically useless?

                                        1. 8

                                          Yes. If there’s no practical value add, that statement is true.

                                          It’s harsh to take, but yes, it’s okay to ask groups that insist on their status - especially in a role prone to gatekeeping - to stand for their value.

                                          1. 3

                                            If you can’t trust software distributed by your distro to be up-to-date and safe, what use does it have then? Stability is never more important than safety.

                                            The whole point people use distributions, and especially reputable ones, is because they want to ensure (a) stuff doesn’t break, and (b) stuff is secure.

                                            1. 2

                                              If you can’t trust software distributed by your distro to be up-to-date and safe, what use does it have then?

                                              Of course packagers try to keep stuff up to date and secure, but a) things move fast, and spare time and motivation can be at a premium; and b) there’s too much code to audit for security holes.

                                              distro maintainers are basically useless

                                              Come on now… I assure you, you’d be pretty upset if you had to build everything from source.

                                              1. 4

                                                Of course packagers try to keep stuff up to date and secure, but a) things move fast, and spare time and motivation can be at a premium; and b) there’s too much code to audit for security holes.

                                                And this is where @arp242’s sentiment comes from. “In a world where there is a serious shortage of volunteers to do all of this, it seems to me that a small army of ‘packagers’ all doing duplicate work is perhaps not necessarily the best way to distribute the available manpower.”

                                                1. 1

                                                  In a world where there is a serious shortage of volunteers

                                                  This is false. All too often it is difficult to find good software to package. A lot of software out there is either poorly maintained, or insecure, or painful to package due to bundled dependencies, or has hostile upstreams, or it’s just not very useful.

                                                  It’s also false to imply that all package maintainers are volunteers. There are many paid contributors.

                                                2. 2

                                                  Come on now… I assure you, you’d be pretty upset if you had to build everything from source.

                                                  I don’t necessarily have to — the distro can provide a clean base with clean APIs, and developers can package their own packages for the distro. As some operating systems already handle it.

                                        2. 3

                                          Various distributions, including Debian, backport security fixes to to stable versions even when upstream developers don’t do it. It’s not uncommon that the security fixes are released faster than upstream.

                                          Your case is an exception. Sometimes this can be due to applications difficult to package or difficult to patch or low on popularity.

                                          Besides, it’s incorrect to assume that the package mantainer is the only person doing security updates. Most well-known distributions have dedicated security teams that track CVEs and chase the bugs.

                                          1. 1

                                            We already provide backported security fixes, as .patch simply usable with git apply, and provide our own packages for old and recent branches. It’s quite simple to package too. Popularity, well, it was one of the preinstalles programs on Kubuntu, and is in Kubuntus start menu (not anymore recently, but on older versions it still is).

                                            The fact that many distro maintainers still take an eternity updating patches, and sometimes not even apply those, makes relying on distro packages quite an issue. I don’t trust distro maintainers anymore, not after this.

                                          2. 3

                                            Honestly, distro maintainers are a massive security risk, and after this experience, I’m kinda mind blown.

                                            I think this is mostly because you have a one-sided experience of this and it’s most likely a bit more nuanced and down to several factors.

                                            One of them being that the CVE system is broken and hard to follow. How did you disclose and announce the CVE and fix? Did the patches need backports for the given release and where those provided? I don’t know the CVE number, so this is hard to followup on. But the best approach is to announce on a place like oss-sec from open-wall and it should be picked up by all distribution security teams.

                                            The other side of this, which is what distribution maintainer see, but few upstreams realize, is patching dependencies is where most of the work is done. Distributing your app as a snap/flatpak works great if you also patch the dependencies and keep track of security issues with those dependencies. This is where most upstreams fails, and this is where distribution maintainers and the distro security teams improve the situation.

                                            1. 1

                                              The other side of this, which is what distribution maintainer see, but few upstreams realize, is patching dependencies is where most of the work is done. Distributing your app as a snap/flatpak works great if you also patch the dependencies and keep track of security issues with those dependencies

                                              That’s why, if you ever build such images yourself, you need to automate it, have it as CI, and update those dependencies at least daily, and generate a new image whenever new dependencies are available. Obviously, you need automated tests in your build procedure to ensure everything still works together, as sometimes some dependencies break important stuff even in patch releases.

                                              How did you disclose and announce the CVE and fix? Did the patches need backports for the given release and where those provided

                                              We provided patches for every version distros used, as nice patch files that could directly be applied with git apply, and in addition to the more common ways, we also directly contacted the package maintainers for our package for the important distros via email or instant messaging.

                                              In general, personally, I’m not a fan of the stable model anyway, though. We’ve done great work to ensure the software stays 100% binary compatible for all its protocols since 2009, we support every supported version of debian and ubuntu even with our absolutely newest builds, and yet, in the end, it’s the distro maintainers shipping not only outdated versions (apparently some users prefer buggy old versions), but also take time to apply security fixes.

                                              1. 2

                                                That’s why, if you ever build such images yourself, you need to automate it, have it as CI, and update those dependencies at least daily, and generate a new image whenever new dependencies are available. Obviously, you need automated tests in your build procedure to ensure everything still works together, as sometimes some dependencies break important stuff even in patch releases.

                                                Which again, few upstream do this, and they surely do not keep an eye on this at all. You sounds like a competent upstream and it’s nice when you encounter them :)

                                                We provided patches for every version distros used, as nice patch files that could directly be applied with git apply, and in addition to the more common ways, we also directly contacted the package maintainers for our package for the important distros via email or instant messaging.

                                                And this is how you should proceed. I would however contact the linux distro list if it’s a widely used piece of software multiple distributions package, and the CVE is critical enough. https://oss-security.openwall.org/wiki/mailing-lists/distros

                                                In general, personally, I’m not a fan of the stable model anyway, though. We’ve done great work to ensure the software stays 100% binary compatible for all its protocols since 2009, we support every supported version of debian and ubuntu even with our absolutely newest builds, and yet, in the end, it’s the distro maintainers shipping not only outdated versions (apparently some users prefer buggy old versions), but also take time to apply security fixes.

                                                The work is appreciated, but I’ll still urge you to not let one bad experience ruin the whole ordeal. Distribution security teams is probably one of the least resourceful teams and sometimes things do fall between two chairs.

                                                1. 3

                                                  The work is appreciated, but I’ll still urge you to not let one bad experience ruin the whole ordeal. Distribution security teams is probably one of the least resourceful teams and sometimes things do fall between two chairs.

                                                  But given that the main argument of distros is security, that statement flies directly in the face of their promises.

                                                  1. 2

                                                    But given that the main argument of distros is security, that statement flies directly in the face of their promises.

                                                    I don’t think it’s the main argument, but surely one them. If you want to be completely covered you need a well paid team able to respond. You wont get this with community based distribution, we are unpaid volunteers, just like most upstreams. You’ll have to use something backed by a paid team if you expect premium service and full coverage.

                                                    Anything else is only on a best effort basis. The CVE system is sadly hard to navigate, ingest and process. Some things are going to bubble up faster, and something is going to be missed.

                                                    1. 2

                                                      I have absolutely no issue with all your statements, but it is a cornerstone argument.

                                                      I’m fine with community distributions, if they own it, and agree that paid distros are a good way to go. RHEL licenses are actually worth their money.

                                                      I disagree with the reading of best-effort, though, because it goes both ways. If your work is impacting others, either through making them have more support requests or slowing down their iteration speed, you need to make sure you don’t add undue labor.

                                            2. 3

                                              With this attitude, which a lot of developers seem to have nowadays, it doesn’t make sense to have your software included in distributions. As a packager I’d call this a hostile upstream… Just distribute it as a flatpak and/or snap and be done with it.

                                              Relevant here may be a blog post from an upstream fully embracing the distribution instead of fighting it: https://www.enricozini.org/blog/2014/debian/debops/

                                              1. 3

                                                It allows me to rely on Debian for security updates, so I don’t have to track upstream activity for each one of the building blocks of the systems I deploy.

                                                That’s exactly what I used to believe in, too, but after this experience, the facade has cracked. I can deal with 90% of my packages being years out of date and full of bugs because the distro wants to be stable and refuses to apply bugfixes or update to newer versions, but if security updates aren’t reliably applied even if they have a CVE (and debian just ignores issues entirely if they have no CVE), then how can one still trust the distro for security updates? Having a remotely exploitable unauthenticated DoS if not even RCE in a publicly facing software for 30 days is absolutely not fine.

                                                As a packager I’d call this a hostile upstream… Just distribute it as a flatpak and/or snap and be done with it.

                                                We actively maintain all version branches, and provide even backported security patches as nice little .patch file even for all the major.minor.patch releases debian/ubuntu still use. You can build it nice and simple, you just have to apply one little patch. It’s not like this we’ve been actively hostile - what more should we have done, in your opinion?

                                                1. 2

                                                  how can one still trust the distro for security updates?

                                                  Fair enough. If they are not applied. I personally know at least one Debian package maintainer (not me, I don’t like Debian) that takes excellent care of their packages, including in the stable releases. So it may depend on the maintainer. But maybe that is your point, that there is no universal standard for maintainers…

                                                  what more should we have done, in your opinion?

                                                  I don’t know this specific case. There are a number of other ‘historical’ cases where packagers gave up on packaging ‘upstream’ software, e.g. https://www.happyassassin.net/2015/08/29/looking-for-new-maintainer-for-fedora-epel-owncloud-packages/. I also wrote a blog post about it in 2016: https://www.tuxed.net/fkooman/blog/owncloud_distributions.html I guess the best one can do is follow these discussions and if possible make it easier for distributions to package the software. Especially the ownCloud case back then bugged me a lot. But as you can see from some other people in those discussions, we just gave up on ownCloud and used something else instead…

                                              1. 7

                                                I’ve done quite an amount of packaging as a Debian developer, and inside Amazon (yes, the company uses OS packages), as a SuSE and HPE employee.

                                                It is sad to see how uncooperative upstream developers can be, mostly due to myths around software lifecycle and the needs of end users.

                                                Most software in the world is developed -> built -> distributed -> integrated -> deployed & mantained -> used by 4 to 6 different individuals/entities. Often different organizations.

                                                Software distributed with shorter “chains” is the minority and the majority of software and hardware deployed in the world lives for many years.

                                                This affect how software is packaged:

                                                • compile-time and run-time dependencies should be unbundled from sources to allow maintainers and system engineers to update components independently
                                                • software should be modular where reasonable: allow disabling ancillary features and remove dependencies
                                                • backward compatibility is important: hard dependencies on libraries released a week ago are painful; hard breaks on ABI/API/etc are more painful. Gentler deprecation and upgrade paths are preferred.
                                                • large set of dependencies on small / fast-moving libraries become stale very fast
                                                • runtime dependencies on remote assets e.g. loading js from a CDN need to be patched out. Same for spyware telemetries.
                                                • proper versioning is important. avoid “git pull from devel” or “curl | sudo bash”

                                                Unfortunately there’s some disconnect between these lifecycles and the ones known by the most vocal crowd of developers on the Internet.

                                                1. 2

                                                  It is sad to see how uncooperative upstream developers can be, mostly due to myths around software lifecycle and the needs of end users.

                                                  The exactly opposite view exists at the maintainer side: distributor packages can be real assholes about you not caring about details of their platform, relying often on nothing but power arguments. Everyone has different policies and it’s completely impossible to cover them all. But the communication breakdown here is so pervasive.

                                                  1. 6

                                                    But the communication breakdown here is so pervasive communication breakdown.

                                                    …that is why I bother to write suggestions and explanations instead of calling people names.

                                                    1. 2

                                                      You are right! Sorry for that, it’s a valid callout.

                                                      What I want to say: Everyone is frustrated, I see not a lot of action working against the frustration.

                                                2. 11

                                                  Packaging can be very, very hard. Alpine and Arch may have fairly simple systems that make providing your own into a community repository easy, but other distributions may not.

                                                  Debian, for example, not only has a very complex system, it also has byzantine policies: e.g. the wiki links to the new maintainer’s guide, which states in chapter 1 that it’s getting outdated, instead linking to the Guide for Debian Maintainers, which is supplemented by a policy manual and a developer’s reference, each having multiple pages of tables of contents alone.

                                                  How you’re going to get anything into Red Hat Enterprise Linux is beyond me, since you’ve got no control about it and they’re notoriously picky.

                                                  1. 2

                                                    I’m curious: is it just the fragmentation of the documentation that makes Debian packaging feel complex?

                                                    In general I find that sensible software can be packaged in minutes since it’s all automated.

                                                    1. 5

                                                      I’ll dig out the low-down from what I can tell and you be the judge if it feels complex (CC @federico3, who obviously may correct me if I’m wrong about any of this or have omitted something major):

                                                      1. To contribute a package, you must become a “maintainer”, who is a person that maintains a package—which is either a “Debian Maintainer”, who is a person that can upload specific packages directly, or someone who is a “sponsored maintaner”, who needs a sponsor to upload packages. When you start out, you’ll be the latter, so you’ll need a sponsor. What nearly nobody tells you: You’re allowed to become a maintainer for your own software as well. But usually the follow-up objection to that is if nobody asked for it on WNPP, chances are it will just get rejected for having no demand.
                                                      2. Try to generate most of the package automatically. If you’re lucky, you’re in one of the cases where it all works automatically and you “only” have the policy to worry about. If you’re unlucky, you first have to make it work.
                                                      3. Fill in the policy-mandated fields: A description (note: RFC 822, so you can’t use empty lines, you’ll need empty lines with a leading space and then a single dot), dependencies, build-time dependencies.

                                                      Now comply with the rest of the policy, such as:

                                                      1. If the source package contains hard links, those needs to be removed or replaced.
                                                      2. Split the package if necessary (always necessary for C/C++ libraries since you’ll need at least -dev and the main library).
                                                      3. If you’re packaging a library, also maintain a separate, semi-automated account of all symbols and the associated version.
                                                      4. If your package includes third-party code, remove that as well and instead patch it to use Debian’s copies of the third-party code if you have to.
                                                      5. Fill in the debian/copyright file. If you’re lucky, every file has a header that identifies the license. If you aren’t, you’re off to checking every single file’s license and try to infer it from other files (namely the COPYING or LICENSE file).
                                                      6. Write a package changelog, which is apparently a mix of changes made to packaging and upstream changes.
                                                      7. If the program is not covered by man pages, you should write man pages yourself; you can get bugs filed against this and if upstream is not cooperative, you’ll be writing them yourself. Upstream refusing is not an excuse.
                                                      8. A lot of mess regarding init systems, generally technically requiring you to still write init scripts in a systemd-centric distribution.
                                                      9. If your package falls in the scope of a team, join that first and check their policies as well.
                                                      10. Sign your package with GPG, so you also get all the extra fun of GPG key management.

                                                      etc. etc. There’s almost no way you’ll get all of this right on your own and that you won’t have a special case, so you should join the debian-mentors mailing list, but that means you get all the overhead of the list and the usual caveats apply (lurk for a while and see what the usual tone is before you ask a question, search the archives, etc.).

                                                      Now that you’re reasonably certain that you’ve handled all the corner cases in the documentation, which will usually get even the seemingly easiest of packages, you get to the fun part:

                                                      1. Create an account on mentors.debian.net.
                                                      2. Upload your signed package there.
                                                      3. Find a sponsor by filing a bug against the sponsorship-requests pseudo-package. Since the ratio of Debian Developers that can and want to be sponsors to prospective sponsored maintainers is rather imbalanced, chances are you’ll just get ignored entirely unless you can make a good case that your package is both low-risk for them and already well-made. If your first language isn’t English, this necessarily becomes comparatively much harder for you.
                                                      4. If you fail to find a sponsor, wait a few weeks, post a follow-up.
                                                      5. If that still fails, either give up or start bothering sponsors that work on related packages.

                                                      There is no equivalent to the Arch User Repository, so either your package eventually gets in, or it’s just in limbo forever if you can’t get anyone to bother sponsoring you.

                                                  2. 10

                                                    So, to developers: just don’t worry about distribution! Stick a tarball on your release page and leave the rest up to distros.

                                                    Because surely all the big disto’s will race to package anything I, as some anonymous developer, release.

                                                    1. 11

                                                      Note: This article presumes that proprietary/nonfree software is irrelevant, and so should you.

                                                      This seems to me to be a hell of a presumption.

                                                      1. 4

                                                        There are several ways to read that presumption, here are two I think are both reasonable:

                                                        1. Nonfree software cannot be included in a libre OS anyway, so it is out of the scope of this discussion
                                                        2. Nonfree software is harmful to the user, and so should not be installed – thus the fact that it cannot be installed from the libre OS directly is not a “big problem”

                                                        There are certainly other readings of the word “irrelevant” that would be much less reasonable. I cannot claim to know which combination of meanings Drew meant specifically.

                                                      2. 8

                                                        You should use a distro that ships updates to you at the pace you wish, and let your users do the same.

                                                        Many users want/need to use bleeding edge versions faster than distros can package them. By making it inconvenient to install bleeding edge versions, you are also not letting your users get updates at the cadence they wish. So maybe the title should be something like “Developers should rely on distributions in addition to distributing their own software”.

                                                        1. 7

                                                          There’s far more open-source software out there than distributions can or will ever maintain and distribute. If you’re writing niche software (open-source has a long tail of projects with a very small number of users), your project probably won’t get picked up and included in a distribution - so you’ll have to find some way to distribute it anyway.

                                                          Based on one of my own projects that has been included in multiple distributions, you can expect no communication if a distribution does pick up your project and package it. Users can pick the packages provided by their OS if they want, or grab it from an unmanaged software repository. Both of these are fine and part of a healthly ecosystem - users who want to trust everything they install can keep to their distributions packages, and users who want to use exactly what the developer provides can do that too. I think most users use a combination of packages from both managed and unmanaged repositories, and I doubt many of them could use only managed packages without giving up some of the projects they use.

                                                          1. 5

                                                            The true distributable form for Linux software, and rather for Unix software in general, is a .tar.gz file containing the source code.

                                                            I have tried several times to learn how to create .deb packages and I immediately unlearned it. I am glad to gnu guix is much simpler in this regard.

                                                            And, yes I package my own software, in my own channel and binaries substitutes.

                                                            I recognize there is chicken-egg problem here: no distribution <=> no users.

                                                            Again guix helps, because users can install in their profile whatever they want and it can not hurt other users.

                                                            1. 3

                                                              This sounds a lot like keenerd’s article about why maintainers matter http://kmkeen.com/maintainers-matter/

                                                              1. 5

                                                                Note: This article presumes that proprietary/nonfree software is irrelevant, and so should you.

                                                                Sometimes you’ve got to love Drew’s provocations – not that anything should be controversial about this statement.

                                                                1. 9

                                                                  It’s provocative and uncontroversial in the same way “the Earth is flat” is provocative and uncontroversial.

                                                                  1. 2

                                                                    I’m glad Drew can make a living from OSS development, but I guess that’s not everybody’s situation.

                                                                  2. 2

                                                                    The difference in trust between managed software repositories like Debian, Alpine Linux, Fedora, and so on; and unmanaged software repositories like PyPI, npm, Chrome extensions, the Google Play store, Flatpak, etc — is starkly obvious. Debian and its peers are full of quality software which integrates well into the host system and is free of malware. Unmanaged repositories, however, are constant sources for crapware and malware.

                                                                    Interesting. I’d never made the distinction between managed unmanaged repos before.

                                                                    Obviously a managed repo is preferred, but as he says, you need a dedicated team of (usually unpaid) porters on-hand to create and maintain binary packages. Not to mention the additional infrastructure. Whereas the unmanaged repos are effectively crowd-sourced…

                                                                    Food for thought.

                                                                    1. 1

                                                                      As a Developer, I consider the necessity of having a $PROJECT/platform tree one of the spices of life.

                                                                      I mean, this whole argument is moot if you treat all the end-target user platforms as a responsibility, not a liability.

                                                                      It is not unreasonable to be able to get most codebases, well-written and prepared for platform independence, ported to any of the things. The elephant in the room is that this is, of course, what F/OSS is all about: endless new platforms for the software to live on.

                                                                      But the premise that devs shouldn’t ‘distribute’ their own software is mind-bogglingly dumb I can only assume its a troll.

                                                                      Its the platform, duh!

                                                                      Putting Working Source on all the platforms is an ultimate developer goal .. it perpetuates the value of the software. (is the only way software can be valuable, i.e. its running on something in front of a user).

                                                                      Therefore, preparing for insertion into <build/packaging-system-de-jour> should be a no-brainer.

                                                                      And is, btw, why IDE-dependence is so sad to see. Choose the languages, not the syntax highlighting, kiddos!