That’s quite unfair, since there are a lot of languages that were either created for educational, academic, esoteric or are just plain old, in the sense that they were created before “packages” were distributed in a anonymous manner as they are with most practical/production languages.
Also the article didn’t mention anything about situations like Chicken Scheme and Guile where one language can have many implementations with distinct repositories.
Guile doesn’t seem to have a package manager, but there’s a list of libraries. It doesn’t seem to have 100 packages so that would explain its absence from the table?
UPDATE: Sorry, miscounted. It has just over 100 libraries listed, so I think it should also be in the table.
Not to contradict you, because I don’t think its official in any context, but I’m pretty sure guix is the de facto package manager for guile now. It has a few libraries hosted in the main channel as well as a build system specific to guile.
The article doesn’t raise the question of whether having a central package repository is a good thing. I think an argument could be made that it’s not given security problems that seem to come along with them.
Is there a connection between a central package repository and more code reuse? If so, you could make the inverse argument that re-implementing code could also result in security problems.
But only if there’s a culture of not merely trusting the reused code, but still doing code reviews. Maybe the old “import manually” scheme (which requires some hand-holding for each dependency) facilitates that better than npm-style importers of dozens of leftpads?
But only if there’s a culture of not merely trusting the reused code, but still doing code reviews.
I don’t necessarily believe this. I’m having trouble reconciling your certainty here. You could be right, but I don’t have any idea on how to measure it an accurate way, but maybe you do? How do you know?
My point was less about saying “central package repos 4 lyfe” and more about “here’s the other side of that argument.”
Maybe the old “import manually” scheme (which requires some hand-holding for each dependency) facilitates that better than npm-style importers of dozens of leftpads?
I think this is one possible circular path back to mfeathers’ point. If you make depending on stuff harder, then you’re less likely to depend on stuff, and are thus less likely to suffer security vulnerabilities from your dependencies. But this doesn’t address the fact that since you depend on less stuff, you probably need to write more code yourself, and this just introduces a different vector for introducing security problems.
FWIW, I personally don’t see anyway to put centralized package repositories back in Pandora’s box. They—or something like them—are just too useful for speeding up development. With that said, I loathe how little back-pressure there now exists on adding a dependency. We’ve swung too far in that direction. So I think the way forward is something that mitigates the security risks imposed by centralized package repositories. That’s why I’m currently experimenting with crev.
Maybe the old “import manually” scheme (which requires some hand-holding for each dependency) facilitates that better than npm-style importers of dozens of leftpads?
I don’t really buy it. I’ve used various boost libraries many times. But I don’t know any teams that routinely and thoroughly audit each version bump before using them, and given the sheer complexity of some of their implementations (in terms of template metaprogramming and essentially unbounded potential for subtle sneakyness in C++) I’m not at all confident that if a boost maintainer’s credentials were to be compromised those teams are any more robustly prepared to catch them. In fact I half think an attack like that would go unnoticed for a lot longer due to the vastly larger surface area and potential for subtler attacks. I’m not a JS guy but I can at least audit a dozen leftpads in an hour or less.
Supply chain attack potential & severity doesn’t strike me as strongly coupled to the existence of a package library. Sure, I might depend on fewer suppliers, but they each deliver more complex and harder to audit libraries, AND I reinvent more wheels, which just means more bespoke security footguns instead of shared ones that get caught by someone else.
I rather like how Go does it: decentralised, but with some standards. It’d be cool were it even more pluggable (e.g. so one could add custom VCSes), but it’s good enough for the common case.
This surprised me. The median age of a language with a CR is 24 (1995)
Looking at the table, it is worth noting that in many cases the “appeared” date used is the date the language appeared, not the date at which the package manager appeared. For example, javascript is listed as appearing in 1995, but the npm package manager which the stats refer to didn’t appear until 2010 (according to wikipedia). It would be interesting to have a plot with the year the package manager appeared instead of the year the language appeared.
Also, I’d prefer if the y-axis scale was kept logarithmic, but labelled with numbers of packages, not log of number of packages.
Go has several competing package managers. It’s got no central server for package management, though - in practice, everything is fetched directly from the git repo where it’s developed.
How do Go people discover and integrate Open Source libraries?
Thankfully the languages parent company built a search tool which makes discovering things relatively straightforward.
Go modules is built in to the go command, and pretty much the “de-facto package manager” now. It took a while, but we got there.
A more central server is something that’s being worked on: https://proxy.golang.org, although it’s not quite the same as npm/gems/pypi, etc. (which is probably a good thing? YMMV)
One of the neat things about modules is GOPROXY: instead of making requests directly to example.com/proj, they’re made against $GOPROXY/example.com/proj, which can then either proxy to example.com/proj, or serve a cached version.
The proxy.golang.org is just a “default” GOPROXY, but it’s easy to run your own, which allows you to mirror external code to guarantee availability while also not adding a a lot of code to every project’s vendor directory. While there are still use cases for comitting the vendor directory (go mod vendor puts all dependencies in vendor/), I think this is a good middle ground for many projects.
Yeah, for projects where people don’t want vendoring for whatever reason, that sounds handy.
Personally, I vendor because
A) ‘dependency code is my responsibility’, so it should go through the same review process as everything else, which is easier if it’s in-tree, and
B) Build-time network dependencies (even if only to a proxy I control) are a source of flakiness. I have enough trouble getting builds to pass when the computers are all working properly.
There’s a lot more that goes into a package manager than go get (e.g. dependency tracking, duplication detection, integrity checks). If go get constitutes a package manager, then so does curl | bash, and I’d say that definition falls short quite a bit
Ah, gotcha. I can understand that package managers might be used to merge/integrate software, and that go get is used for that.
Is there an expectation that package managers are used to discover software? I don’t think I’ve ever used my package manager for that except to check if a package exists before I install it.
I also am a big fan of putting third party code in my repository and selectively merging changes from upstream. Usually it means that I understand the third party code more completely and I’m better able to modify it to fit my needs. And I don’t run into the issue where my app from three months ago no longer builds, because no code has changed unless I allowed it to.
This only works in situations with limited amounts of third party code and sufficient developer resources.
I never realy use the package manager in other languages to “discover” packages, either. And I rather like that I don’t have to deal with the hassle of “publishing” stuff to some package repository, but can just push to git and be done with it.
That’s quite unfair, since there are a lot of languages that were either created for educational, academic, esoteric or are just plain old, in the sense that they were created before “packages” were distributed in a anonymous manner as they are with most practical/production languages.
Also the article didn’t mention anything about situations like Chicken Scheme and Guile where one language can have many implementations with distinct repositories.
It looks quite incomplete. CHICKEN is in the list of languages and it certainly has more than 100 packages, so it should be in the table.
Guile doesn’t seem to have a package manager, but there’s a list of libraries. It doesn’t seem to have 100 packages so that would explain its absence from the table?
UPDATE: Sorry, miscounted. It has just over 100 libraries listed, so I think it should also be in the table.
Not to contradict you, because I don’t think its official in any context, but I’m pretty sure guix is the de facto package manager for guile now. It has a few libraries hosted in the main channel as well as a build system specific to guile.
The article doesn’t raise the question of whether having a central package repository is a good thing. I think an argument could be made that it’s not given security problems that seem to come along with them.
Is there a connection between a central package repository and more code reuse? If so, you could make the inverse argument that re-implementing code could also result in security problems.
But only if there’s a culture of not merely trusting the reused code, but still doing code reviews. Maybe the old “import manually” scheme (which requires some hand-holding for each dependency) facilitates that better than npm-style importers of dozens of leftpads?
I don’t necessarily believe this. I’m having trouble reconciling your certainty here. You could be right, but I don’t have any idea on how to measure it an accurate way, but maybe you do? How do you know?
My point was less about saying “central package repos 4 lyfe” and more about “here’s the other side of that argument.”
I think this is one possible circular path back to mfeathers’ point. If you make depending on stuff harder, then you’re less likely to depend on stuff, and are thus less likely to suffer security vulnerabilities from your dependencies. But this doesn’t address the fact that since you depend on less stuff, you probably need to write more code yourself, and this just introduces a different vector for introducing security problems.
FWIW, I personally don’t see anyway to put centralized package repositories back in Pandora’s box. They—or something like them—are just too useful for speeding up development. With that said, I loathe how little back-pressure there now exists on adding a dependency. We’ve swung too far in that direction. So I think the way forward is something that mitigates the security risks imposed by centralized package repositories. That’s why I’m currently experimenting with crev.
I don’t really buy it. I’ve used various boost libraries many times. But I don’t know any teams that routinely and thoroughly audit each version bump before using them, and given the sheer complexity of some of their implementations (in terms of template metaprogramming and essentially unbounded potential for subtle sneakyness in C++) I’m not at all confident that if a boost maintainer’s credentials were to be compromised those teams are any more robustly prepared to catch them. In fact I half think an attack like that would go unnoticed for a lot longer due to the vastly larger surface area and potential for subtler attacks. I’m not a JS guy but I can at least audit a dozen leftpads in an hour or less.
Supply chain attack potential & severity doesn’t strike me as strongly coupled to the existence of a package library. Sure, I might depend on fewer suppliers, but they each deliver more complex and harder to audit libraries, AND I reinvent more wheels, which just means more bespoke security footguns instead of shared ones that get caught by someone else.
It’s missing Quicklisp for Lisp …
I rather like how Go does it: decentralised, but with some standards. It’d be cool were it even more pluggable (e.g. so one could add custom VCSes), but it’s good enough for the common case.
Looking at the table, it is worth noting that in many cases the “appeared” date used is the date the language appeared, not the date at which the package manager appeared. For example, javascript is listed as appearing in 1995, but the npm package manager which the stats refer to didn’t appear until 2010 (according to wikipedia). It would be interesting to have a plot with the year the package manager appeared instead of the year the language appeared.
Also, I’d prefer if the y-axis scale was kept logarithmic, but labelled with numbers of packages, not log of number of packages.
Hm, I never thought about the fact that Go has no package manager. How do Go people discover and integrate Open Source libraries?
Go has several competing package managers. It’s got no central server for package management, though - in practice, everything is fetched directly from the git repo where it’s developed.
Thankfully the languages parent company built a search tool which makes discovering things relatively straightforward.
Go modules is built in to the
gocommand, and pretty much the “de-facto package manager” now. It took a while, but we got there.A more central server is something that’s being worked on: https://proxy.golang.org, although it’s not quite the same as npm/gems/pypi, etc. (which is probably a good thing? YMMV)
Ah, I’m a little out of date then.
For go code, I’m strongly in the “commit your dependencies” camp, so imo there’s less need for a central package repo with high availability.
One of the neat things about modules is GOPROXY: instead of making requests directly to
example.com/proj, they’re made against$GOPROXY/example.com/proj, which can then either proxy toexample.com/proj, or serve a cached version.The proxy.golang.org is just a “default” GOPROXY, but it’s easy to run your own, which allows you to mirror external code to guarantee availability while also not adding a a lot of code to every project’s vendor directory. While there are still use cases for comitting the vendor directory (
go mod vendorputs all dependencies invendor/), I think this is a good middle ground for many projects.Yeah, for projects where people don’t want vendoring for whatever reason, that sounds handy.
Personally, I vendor because
A) ‘dependency code is my responsibility’, so it should go through the same review process as everything else, which is easier if it’s in-tree, and B) Build-time network dependencies (even if only to a proxy I control) are a source of flakiness. I have enough trouble getting builds to pass when the computers are all working properly.
go get
There’s a lot more that goes into a package manager than
go get(e.g. dependency tracking, duplication detection, integrity checks). Ifgo getconstitutes a package manager, then so doescurl | bash, and I’d say that definition falls short quite a bitI wasn’t claiming that
go getis a pkg mgr; merely that it is how Go users integrate open src libs.There’s a new solution to the problems you mention, which does actually involve a pkg mgr: https://search.gocenter.io/stats
Ah, gotcha. I can understand that package managers might be used to merge/integrate software, and that
go getis used for that.Is there an expectation that package managers are used to discover software? I don’t think I’ve ever used my package manager for that except to check if a package exists before I install it.
go getused to be similarish tocurl | bash, but it now integrates withgo mod, which is Go’s package manager.I also am a big fan of putting third party code in my repository and selectively merging changes from upstream. Usually it means that I understand the third party code more completely and I’m better able to modify it to fit my needs. And I don’t run into the issue where my app from three months ago no longer builds, because no code has changed unless I allowed it to.
This only works in situations with limited amounts of third party code and sufficient developer resources.
I never realy use the package manager in other languages to “discover” packages, either. And I rather like that I don’t have to deal with the hassle of “publishing” stuff to some package repository, but can just push to git and be done with it.
That 9,889 total packages for Ruby seemed very wrong. Not sure where that came from but Rubygems itself reports 154,444.
C++ has vcpkg with over 1K packages.
vcpkg is not standard packager of the language, but still.. (and it runs on windows, macos, linux, and as a 2nd class citizen freebsd…)
https://github.com/microsoft/vcpkg/tree/master/ports