1. 19
  1.  

  2. 4

    The big takeaway here for me is that “yanking” a package should not be as easy as it currently is in RubyGems and other package ecosystems. Most of the disruption here was caused by all existing versions of the dependency being yanked, immediately breaking the build of countless Rails applications, and Rails itself.

    Had the maintainer been forced to make a request to remove these versions which would be reviewed by the RubyGems team, they could have coordinated with Rails and others to greatly minimise disruption.

    Furthermore I’d argue that outside of very extreme cases, deleting a package version from a repository should not be permitted at all. A key promise of modern package managers is that a build plan will continue to work indefinitely into the future, and deleting packages breaks that promise. This doesn’t preclude marking a version as “bad” in some way such that new build plans will not choose it.

    1. 4

      What a mess!

      The tl;dr is that the mimemagic maintainer yanked all existing versions and new versions are licensed under GPL (I think because it was using GPL code by mistake). This is bad because Rails (specifically ActiveStorage) depends on it, and Rails is MIT licensed.

      The latest update from the rails core team is that they’re working on swapping to libmagic or a Ruby translation of libmagic.

      If you’re having trouble as a Rails dev, you can just update your dependency but if you’re in a…license sensitive…environment, check with your lawyers first.

      1. 1

        The dumbest thing is probably that he published a new GPLed version as 0.3.x, which now every tool is updating to automatically.

        1. 1

          So the older versions were incorrectly licensed under another license? Sounds like rolling back to a version that contains GPL code but incorrect license text is a bad solution. Even in the short term.

          1. 1

            Sounds like rolling back to a version that contains GPL code but incorrect license text is a bad solution.

            Disclaimer: I haven’t looked at the file in question, largely because this seems contentious enough to already have escalated against another project using it (see elsewhere in the comments on this post).

            But from reading the discussions, it appears the issue is not any “GPL code” – rather, it’s a data file that ships with a GPL’d package. That data file apparently contains at least some of the mappings of magic numbers to file types, or similar information. And that’s a different sort of question; as some of the threads have pointed out, US copyright law (which is what would matter for things like GitHub and a lot of open-source package hosts) does not recognize copyright on compilations of facts. So if that is indeed the content being claimed, there may be quite a strong case to be made that it’s not only not GPL’d, but not subject to copyright at all.

            1. 5

              I thought so too, but looking at the file:

               <magic priority="70">
                <match type="string" value="PK\003\004" offset="0">
                  <match type="string" value="mimetype" offset="30">
                    <match type="string" offset="38" value="application/vnd.sun.xml.calc"/>
                  </match>
                </match>
              </magic>
              

              Is this data or code? At what point does a list of matches with priorities become its own DSL? To me this definition looks closer to instructions than a data structure.

              1. 1

                I’m not entirely convinced that even this would clear the bar of creative content. And I’m less convinced of the authors’ understanding of the GPL every time I look at one of the threads. As someone else pointed out here, they (or someone who acts like they have the ability to act on behalf of the GPL’d project) keep suggesting workarounds like telling people to already have a copy of the file present. Which… is not how the GPL works. If the file really is GPL’d, then that’s not a workaround anymore than “require someone to have this .so already present to be loaded at runtime” would be a workaround.

                1. 1

                  They wouldn’t require someone to have that specific file present - just a file in that format. GPL specifically talks about linking and code, but would this really count? Or is it closer to ICC compiling GPL code which doesn’t make ICC itself GPL? (Compiling actually includes execution of constant expressions from the code)

        2. 4

          Incidents such as this are continual reminders to always cache your dependencies. Relying on package hosting always to be available is risky.

          1. 3

            To be fair that is a stop-gap solution to stop the build from breaking, but you still need to solve the original issue (unless you’re ok with GPL code in a bunch of other code).

          2. 3

            This same issue has affected the Golang community also. Sounds like tooling had removed some GPL notices, which at some point then got copied into these shared libraries. The tooling was fixed and recently some obvious IP/license violations have been identified. It’s a bummer, but seems to be getting resolved reasonably.

            1. 3

              That discussion got… heated. But interesting, because people are raising the (genuinely interesting and relevant) question of whether a database of mime-type information is even copyrightable – under US law – in the first place, as US copyright law generally does not extend to compilations of facts. There’s a certain amount of unfortunate misinformation and misunderstanding of the key concept there, too (generally, the “creative effort” required to make something copyrightable is not “it took creative effort to discover this fact”; otherwise you could copyright things like fundamental physical constants, since it takes “creative effort” to find a way to determine their values).

            2. 1

              Meanwhile 70 million downloads of a gem with transitive AGPL dependencies: https://rubygems.org/gems/mini_magick

              1. 1

                Did you report the issue?

                1. 3

                  What’s the issue? You can probably build imagemagick without ghostscript, you can buy a license or your can follow the AGPL and give your users rights to examine and modify the software they use.

                  This is something users will have to bring up if they use proprietary rails apps that process PDF or PostScript files. It’s a little disingenuous to license that gem as MIT but I have low expectations of license honesty these days :-(

                  1. 2

                    It uses those libraries by executing them as sub processes.0 That doesn’t trigger any *GPL licensing. That’s why I’m asking you to explain your comment.

                    1. 1

                      IANAL but one of the features of the AGPL is to close some of those holes with how users interact with the software and how developers link the software.

                      1. 2

                        The AGPL’s extra clauses kick in if you do both of:

                        1. Modify the original software, and
                        2. Make it accessible to users over a network

                        The (A)GPL family of licenses historically have claimed a lot of things as derivative that maybe wouldn’t quite hold up in a court, but as far as I know they’ve never tried to claim that they can reach across process boundaries/IPC.

              2. 1

                Two things remain unclear to me:

                1. Is your code a ‘derived work’ when it parses a GPL licensed file? If not, shouldn’t just the XML file itself remain licensed under the GPL, without necessarily ‘infecting’ the entire gem as everyone seems to be assuming?
                2. Given the solution (parse the file at runtime from a location where it’s already present) what has this copyright enforcement actually accomplished? Does anyone benefit from this episode?
                1. 2

                  In both cases the GPL applies, except if the parsing (1) or downloading at runtime (2) functionality would work with arbitrary inputs. Tailor-made code to “circumvent” the GPL doesn’t work.

                  1. 1

                    With respect to 1, I think you’re reiterating a certain position/interpretation, but I don’t think yours is obviously correct [1] and I’m wondering if that question was actually resolved somewhere in the discussions surrounding this issue. Preferably by someone with some measure of authority on the subject.

                    With respect to 2, I mean something entirely different: who benefits from the changes that have been made? In what way does the enforcement of this copyright have good consequences for the future? Would anything have been lost if these changes weren’t made?

                    [1] No functionality, except an identity function, can deal with arbitrary inputs. An implementation that can deal with this XML file can probably deal with the same XML file with other values in the structure. However, other values are pointless, because there is only a limited set of ‘correct’ values. It’s not obvious when something constitutes ‘a mere collection of facts’.

                    1. 2

                      This is exactly how Nvidia ships Linux GPU drivers without having to license them under the GPL:

                      They do not have a driver “for Linux”, they have a shared driver that works on Windows, macOS and Linux with the appropriate, permissively licensed shim in between that gives them plausible deniability, even if they call only into the kernel APIs that have been marked as non-GPL.

                2. 0

                  Tragedy of the commons

                  1. 2

                    It doesn’t read as such to me. Can you expand?

                    1. 3

                      My whole take on this is on the batteries included mind-set of rails. The number of transitive dependencies grows with each release (how many are they right now? 80 gems?). The long tail of those get unfortunately very little maintenance, usually from one person, who now has to acquiesce to the whims of the community. This situation brings him little recognition or even help, just more complaints with each rails breaking change that might break the integration.

                      And them something like this happens. Nevermind that a lot of apps handle file uploads via other alternatives like paperclip or shrine, which predate activestorage. Because of the Rais way of shipping everything by default, the build is now broken.

                      This would not have happened, for instance, with shrine, which supports many strategies for mime type detection, and leaves the choice to you.

                      This whole situation about to happen. The long tails of rails dependencies is filled of gems that receive very little maintenance or support, and are liable for such situations, like some copied file from a different license years ago (this case), or some maintainer rage quitting and taking the sink with him (leftpad, which didn’t happen yet in rails, but could)

                      1. 1

                        Thanks a lot for expanding!

                        1. 1

                          Because of the Rais way of shipping everything by default, the build is now broken.

                          Bundler has built in support for storing your dependencies in-repo so that you do not depend on a third-party service for your CI / deploy pipeline. You didn’t use it, and it bit you when the third-party service didn’t do what you expected. The same technique would also have stopped left-pad from being an issue.

                          1. 2

                            You’re preaching the preacher. I know how to set up CI caches. I also know how to use rails and not install activestorage. I prefer not to use rails though, safest option. But I’m not the default in the rails community. And saying “well you shoulda…” is hardly gonna make anyone feel better about it.