1. 108
  1. 14

    This is exciting stuff; valuable perspectives on the tension between developers giving their code and the companies that benefit from it. I’m looking forward to seeing where this goes.

    Since I don’t do any JS, I wonder how the ideas could be adapted to other languages and ecosystems.

    1. 5

      Answering my own question, the github repo description is “a package registry for anything, but mostly javascript” which looks promising: https://github.com/entropic-dev/entropic

      1. 6

        As package management seems to be a wheel oft-reinvented with the same lessons re-learned, it’s always worth looking at alternatives and prior art.

        I’ll note gx as well as Guix/Nix for non-Javascript-centric packaging, but I would like to see more too!

      2. 2

        Why don’t use CPAN? It’s proven to work and be sustainable.

      3. 9

        A great writeup and insight into the history of Node, and a good exploration of the issues of the ecosystem.

        A few things leap out at me from this writeup.

        It’s odd to me that they picked CouchDB, even as it seems that the folks I’ve asked have mentioned that it seems to be falling out of favor going back as far as 2013-2014 (Datastax employees, so take that with a grain of salt, but still). My guess is that the decision was a combination of hosting and desire to horizontally host, but I’d be interested in hearing why they went with what they did.

        The mention of ESR-style “open source” instead of RMS-style “free software” seems a little out of place to me. Most importantly the problems that npm faces/faced strike me as coming from that potlach culture and not from any specific licensing of the software. The problem seems to me not to be that the software was given away–since, after all, most of the software wasn’t npm’s!–but that the hosting was. Servers aren’t free, electricity isn’t free, bandwidth isn’t free. Software is free-as-in-beer to reproduce, hosting is emphatically not.

        There’s a brief rant in here about giving away work-product for nothing, and that I agree with heartily. We’re in one of the few fields where our artifacts, properly created, can truly obsolete us…that’s both the dream and the curse of software. Seeing how much profit has been given away by the JS ecosystem should be sobering to every developer out there. We ignore it at our peril.

        Edit: removed ranty stuff in favor of maybe a cooler blogpost some day.

        1. 9

          Here’s the talk video from JsConfEU 2019: https://www.youtube.com/watch?v=MO8hZlgK5zc

          1. 7

            I find it slightly disappointing that the author ended up with Apache 2 as the software license for his new project without any further justification. Even though the author clearly outlined problems with Eric Raymond style open source he doesn’t draw upon these reflections when deciding the license for his new decentralised project. He remains in the status quo and makes himself and his project vulnerable to all the issues he made the reader aware of.

            1. 6

              The tangent about copyleft is not the most important part. GPL would not have improved the npm situation in any way. The central problem discussed in the article is the VC-backed private control of the npm registry (which runs proprietary software, not permissively licensed software). Entropic is designed for people running their own registries, the registry software is designed for federation/mirroring.

              1. 2

                I trust you already know this but BSD-licensed distributed systems always devolve into SaaS businesses that give little back to the creators of the tech they are based on, e.g. most of AWS. With GPL, the Entropic team has a lot more leverage on the ecosystem.

                It’s an interesting idea: BSD, VC-backed vs GPL, Community-driven.

                1. 2

                  SaaS providers don’t need the original software all that much, they can reimplement APIs just fine.

                  If the whole idea from the beginning is a federated replication protocol (like here),

                  • SaaS can just implement it from scratch anyway; and
                  • community members just have to resist the SaaS temptation individually, fairly, without licensing tricks.

                  Even legally forcing hosting companies to release their modifications (which I’m not sure even AGPL does, isn’t that only achieved by the Commons Clause type crap?) doesn’t necessarily prevent SaaS expansion. It reduces the risk of embrace-extend-extinguish, but a lot of times people don’t use SaaS because of exclusive modifications and features, just the convenience of someone else taking care of the servers.

              2. 3

                Why is Apache bad for her project? Which license should she have chosen?

                1. 2

                  It looks to me that the comment pretty much answers your first question already. Although I appreciate the comment never stated that the Apache-2.0 license is bad for the project. Honestly I too was initially struck by the fact that despite “the author clearly outlined problems with Eric Raymond style open source he [sic] doesn’t draw upon these reflections”. On the other hand, the author of the article also comments that she thinks “it’s good for us as humans to give things to each other, and I think I’m at peace with the idea that some corporations will make money from it.” I guess it’s not a matter of justification but rather of acceptance, from what I can draw from this reading.

                  Regarding the second question, I would say that choosing the GPL would have felt more inclined towards the spirit of the comparison between open source and free software (what the author originally refers to as ESR- and RMS-style open source). On the other hand, given the remark I quoted from the article, choosing a license has also to do with your objectives. If you are “at peace with the idea that some corporations will make money from it” and you may want enterprise customers to be attracted by the product, the GPL can be quite a hurdle, as the General Public License is sometimes treated as poison, as the article itself recognizes.

                  Within the context of also introducing a possible replacement for NPM, as the article ultimately does, I believe choosing a permissive license is a smart choice, especially as the aim of the project, from what I read from the article, is to offer a way to put the control of the commons into the hands of the community, rather than one private entity.

                  1. 1

                    Yes, I’m not saying Apache-2.0 is bad for the project; I just felt that this new project remains susceptible to a lot of the issues the author raises with NPM. (And I’m sorry for using the wrong pronoun for the author in my original comment.) A license which forces sharing of derivative source code could help if for example the author of the article in the end goes the same route as the author of NPM and gives authority over the registry comes to a VC held firm.

                    Note that I never mentioned GPL in my original comment and never suggested it to be the solution. Though I do want to point out that the GPL and AGPL does “allow for corporations to make money from it”. But they do scare corporations; perhaps because it makes it a little harder for a private entity to do what the original author opposes in her article.

                    I don’t know which license she should have chosen. I guess I just wanted the argument to end up in a solution and not in the status quo. The solution could be anything. Perhaps I am missing the point and the implementation of her repository prevents corporate capture.

                2. 2

                  I think this paragraph is the reasoning?

                  The other confession I’ll make is that I believe in the potlatch economy despite everything. I think it’s good for us as humans to give things to each other, and I think I’m at peace with the idea that some corporations will make money from it.

                3. 4

                  What I don’t understand is why this pitches an alternative package manager with its own registry.

                  Wouldn’t it be much preferable to point to the repositories directly instead of registry-name+package name?


                  libfoo = http://git.examlpe/libfoo.git

                  Edit: asked the author on twitter

                  1. 8

                    Note this is addressed in the post a bit too:

                    Centralization has a lot of advantages for users. When you have one source for something, you can short-circuit a lot of work in finding the thing you’re looking for. I’ve been doing a lot of Go programming recently, and it’s very strange to try to find Go packages, because they’re everywhere, and the only way to find them is to Google for them, or look at old-fashioned text lists maintained by hand— you know, Yahoo’s original thing. And when you install Go packages, you install from Github repos that could just go away on you, and this is freaky to me. I have expectations that come from using npm for 8 years or so. The absence of a central registry for Go helps me appreciate what npm gave me.

                    1. 0

                      It’s interesting that they mention Go as Go already has index of their decentralized repositories: https://godoc.org/

                      It’s also worth mentioning that even though the web is decentralized the problem of discovery has been successfully solved with the introduction of search engines. I don’t see why it couldn’t be applied to packages.

                      Additionally now that package managers aggressively pin versions utilizing package digests can effectively avoid the problem of a package disappearing from any single location.

                      1. 2

                        Additionally now that package managers aggressively pin versions utilizing package digests can effectively avoid the problem of a package disappearing from any single location.

                        It’s not clear to me how that solves the problem at all

                        1. 2

                          It’s not clear to me how that solves the problem at all

                          Not by itself but having digests for everything makes it possible to securily run proxies and mirrors that replicate packages.

                  2. 3

                    I haven’t been following the NodeJS community, but there’s a few references in the transcript to NPM Inc. burning its community good-will in 2018; can anybody fill me in on the background to those comments? (my first guess was to check the Wikipedia article for a “controversy” section, but it doesn’t have one)

                    1. 10
                      1. 3

                        NPM took a new CEO on board who is well known to be brutish and rough to employees. They made a couple of firings that were questionable and currently in front of court because there’s the accusation of retaliation for attempting to form a union. The founders of NPM and other two C-levels have become surprisingly silent around the topic. They have ousted their CTO in a very cold manner.

                        Now, all this would be “another day in the industry” if it weren’t that NPM based their entire public image and advertisement around being a diverse, friendly and cool company with awesome employees for which they care for. This goes up to izs and seldo tweeting pro-union statements and generally playing the workers game. The inconsistency currently flies into their face (rightfully, so, IMHO).

                        Personally, I found those behaviours always a bit odd, being in the position of being a unionized Entrepreneur myself (I believe in collective bargaining being better for both sides). Still, there’s things an employee can consistently root for and things that is odd if you are in the power position. I try to emphasize with employees positions as much as possible, but performing as if you were in the position of an employee/worker is just inconsistent. Much like outlined in this talk, you have different goals and that shouldn’t be covered.

                      2. 3

                        Several of the people involved in early node figured out early on that package management for node would be very useful, and started writing package managers. Yes, more than one– there were several competitors.

                        dang, that makes me feel old.. I remember I used one with YAML files for package descriptions, I think it was called kiwi or something :D

                        Every package-lock file npm has ever seen is sitting in an s3 bucket somewhere, chock-full of interesting data nuggets about what you’ve been up to.

                        How did that data, out of all the datas, become valuable to the data capitalists? What profitable insights could possibly be extracted from package lock files?

                        Facebook’s social graph is mined because relationships between people and stuff can be used to figure out which stuff to advertise to which people to make them more likely to click. FOSS package dependency graph is mined to.. do what? What can you do with a dependency graph to make money? Figure out which packages to advertise commercial support or commercial alternatives for??

                        1. 2

                          Well, package-lock.json would contain references to internal libraries as well, potentially giving npm a way to track down new customers for enterprise offerings (to replace artifactory and similar.)

                          1. 1

                            Yeah, that’s the first thing that comes to mind. If I make a proprietary tool that replaces/augments an open source solution, or if I sell consulting services for a particular tech stack, being able to buy instant lists of potential customers sounds pretty useful.

                          2. 3

                            Is it a new problem that free software can be co-opted by corporations? The fact that folks didn’t get rich from their code or designs isn’t new and predates the licensing fad (heh) we’ve been in for the past 30 years. The sad story of person X getting rich from person Y’s work isn’t sad unless person Y feels slighted. This doesn’t ask, it just asserts that person Y wanted that big VC money (read debt/responsibility/less stable paycheck) for their work.

                            Making such assertions leaves me wondering if the author doesn’t value extra money (yacht money, not needs met money) as much as the VCs they rail against. Folks writing free software should know that anyone on the planet can monetize their work.

                            Even the AGPLv3 can’t stop 2 companies from competing with the same source code and one winning because they have a cooler name or better customer support or UX or color scheme or…

                            That said decentralized (aka mirrored aka old as dirt) package management is a good thing so I look forward to seeing where entropic goes long term.

                            1. 3

                              A rambling screed with a lofty title. “Economics” is not a useless synonym for “money”, but this “internet veteran” prefers the one with more syllables.

                              npm does not love you. npm cannot love you. npm Inc is a Delaware corporation founded as a financial instrument intended to turn money into more money for a handful of men.

                              Neither do volunteer organizations love you. Or rather their love is empty, because they’re subject to resource constraints like everyone else.

                              Federation spreads out costs.

                              Could have said that in fewer words. If the “mirror” model popular with linux distros can be made turnkey, then yes, you can spread out the costs and pretend that you’re distributed. But you still haven’t solved the incentive problem.

                              1. 2

                                The story is about how constraints and financial requirements shape incentives around NPM Inc, I think “economics” seems an apt term.. but I suppose that’s minor side track.

                                their love is empty, because they’re subject to resource constraints like everyone else.

                                Saying that because both organizational structures are subject to resource constraints means they are both incapable of “love” is equivalent to saying that because parents are subject to resource constraints they are incapable of love.

                                We are all subject to resource constraints. The love shows in what actions you take in response to them.

                                1. 1

                                  … is equivalent to saying that because parents are subject to resource constraints they are incapable of love.

                                  If the parents claim to love thousands of people they’re never met, then yes, the love will prove as generous as the resources.

                              2. 2

                                Package managers must be a hobby at Yahoo. They also had a modified FreeBSD ports tree that generated both FreeBSD packages and RHEL RPMs.

                                edit: yes, they installed packages on RHEL with the old FreeBSD pkg_add

                                1. 2

                                  Github is currently very much in a similar situation of power and conflicting interests.

                                  1. 1

                                    Yes. With the difference though that GitHub has been vastly better at keeping their public image. As Ceej says in the talk, it wouldn’t have been possible to give that talk half a year ago.

                                  2. 1

                                    I do wonder what prevents any of the big companies, for example, Microsoft through Github, to host an instance of Entropic, and adding too-good-to-be-true features on top. These could be automatic vulnerability alerts, detailed metrics, integration with code completion services, but only if you use their instance, and could serve to convince anyone to host mainly on Github’s instance. Then they roll their own CLI that supports Github-only features, and then they close API access to third party clients. The Apache 2 license would allow all of this, without problems.

                                    This has happened before with XMPP (Google Talk, Facebook Chat), IRC (Slack, Discord) and SMTP (Gmail). I think we’ll need more than federated protocols to solve the problem of VC-backed companies, but at least this is a start.

                                    1. 1

                                      I read this up until the advertisement for the new project, and mostly enjoyed it.

                                      I found it weird that the author skated right up next to the problem when talking about the difference between open source and free software, and then ignored that as a possible cause of the rest of the problems.

                                      Copyleft isn’t a catch-all magical potion that solves all problems everywhere, but I think it solves a surprising number of the problems described here. :(

                                      1. 8

                                        How exactly does copyleft solve centralized private control of a package repository??

                                        1. 2

                                          Copyleft isn’t a catch-all magical potion that solves all problems everywhere, but I think it solves a surprising number of the problems described here. :(

                                          I disagree. It solves almost none of the problems outlined here. Running an alternative NPM is a huge undertaking, just for the scale of it.