Did I forget to mention the one language environment that has a completely perfect, flawless packaging story?
Yes, the JVM. Even people who don’t accept maven as a build system use maven-compatible upstream dependencies and distribute their libraries in maven-compatible packages. There’s the right balance of a consensus on the “official” repository but ease of running private repositories and proxy repositories. Packages are namespaced and gpg-signed (at least in the official repository). It all just works, to the extent that you don’t see that many blog posts about how to make it work because there’s not a lot to say.
I think not being open-source for a while actually helped Java get it right - no-one was worrying about installing java libraries on linux systems for system utilities written in Java, so like Rust there was never really a concept of globally installed packages. Whereas for Python quite often the system package manager depends on Python, so you have a system install of Python and a concept of installing libraries for it to use and virtualenv is necessary to undo that mess. (The underlying problem is that linux package managers have no concept of locally-installed packages, only global ones, but I suspect they’ll be the last to migrate to a good approach).
Agree. SBT (after you have read the mandatory introduction page) is especially amazing.
I had a cross-platform project recently that consisted of a plugin to “provide” its functionality to other projects by just depending on it, a compiler plugin and multiple libraries that exist only on the JVM, libraries that exist only on JS and libraries that need to be compiled to both platforms.
Oh, and I wanted to support 3 major versions of Scala.
With SBT, it just worked. Most other build systems aren’t even close to supporting that.
I dislike SBT (introduction page is one thing, but I also need a reference page, and it doesn’t have one), but thankfully the ecosystem is mature/standard enough that I can depend on libraries built with SBT, and people using SBT can depend on my libraries, without needing to do anything special.
What’s the beef with Rust’s approach here? We don’t really have a virtualenv-like tool, as we don’t really have a concept of “globally installed packages” in the first place.
Rust doesn’t support multiple versions/compiler settings. Might be acceptable for early adopters, but completely unacceptable for professional development considering Rust’s release cadence.
No crate namespacing.
No separation between who provides crate and what’s in it.
Thanks. Number two is very clear, but I’m a little confused on the others. On point 1, at least for versions. Rust can and does include multiple versions of transitive deps. But then it seems like maybe you’re talking about the language at the end? And what would point three mean?
(1) Whenever you change Rust’s version or other compiler settings, every existing artifact is discarded and all dependencies are recompiled from scratch. Additionally, caching of artifacts only seems to be per-project. So even if you have the same dependency in many different projects with the same compiler version and options, it will compile multiple times instead of once.
(3) There needs to be a way for multiple vendors to declare “I am package X”, ideally by separating the artifact from its publisher.
Ahh, I see now. Thank you! I’m always trying to figure out how to make stuff better, so understanding criticism is crucial.
With 1, it’s tough, because any change in compiler settings can change everything. So this would mean that we’d have to encode exactly what flags were sent, and only re-use the ones with those specific sets of flags. That’s why it’s per-project now, because that’s where the settings for each project are set.
For 3, that makes sense in some way to me, but I don’t think I’ve seen that feature actually implemented before. Is there a language which does this?
(1) I think it’s a no-brainer. I don’t see the point in not doing it, it can only help but not hurt. As soon as that works, introduce repositories (local and global) from which people can request the artifacts. – Congrats, you reached the state of what Java did in 2005(?).
Scala ships bytecode, JS code and soon native code. Multiply that by at least three major version of Scala. No issues experienced or expected.
Given that Cargo builds are a lot less powerful, the website should actually be able to build artifacts for all configs and targets automatically after the library author uploaded the source.
This spawned an interesting discussion about conda and the split between the web and scientific python worlds on reddit and twitter:
https://www.reddit.com/r/Python/comments/4xnip4/python_packaging_is_good_now/d6gz7qs?st=is0q3ila&sh=504f7403
https://twitter.com/jakevdp/status/765308397050695680
Yes, the JVM. Even people who don’t accept maven as a build system use maven-compatible upstream dependencies and distribute their libraries in maven-compatible packages. There’s the right balance of a consensus on the “official” repository but ease of running private repositories and proxy repositories. Packages are namespaced and gpg-signed (at least in the official repository). It all just works, to the extent that you don’t see that many blog posts about how to make it work because there’s not a lot to say.
I think not being open-source for a while actually helped Java get it right - no-one was worrying about installing java libraries on linux systems for system utilities written in Java, so like Rust there was never really a concept of globally installed packages. Whereas for Python quite often the system package manager depends on Python, so you have a system install of Python and a concept of installing libraries for it to use and
virtualenvis necessary to undo that mess. (The underlying problem is that linux package managers have no concept of locally-installed packages, only global ones, but I suspect they’ll be the last to migrate to a good approach).Agree. SBT (after you have read the mandatory introduction page) is especially amazing.
I had a cross-platform project recently that consisted of a plugin to “provide” its functionality to other projects by just depending on it, a compiler plugin and multiple libraries that exist only on the JVM, libraries that exist only on JS and libraries that need to be compiled to both platforms. Oh, and I wanted to support 3 major versions of Scala.
With SBT, it just worked. Most other build systems aren’t even close to supporting that.
I dislike SBT (introduction page is one thing, but I also need a reference page, and it doesn’t have one), but thankfully the ecosystem is mature/standard enough that I can depend on libraries built with SBT, and people using SBT can depend on my libraries, without needing to do anything special.
The direction in the article to start every project with
virtualenvsuggests otherwise. (And unlike claimed in the article, Rust isn’t much better.)What’s the beef with Rust’s approach here? We don’t really have a
virtualenv-like tool, as we don’t really have a concept of “globally installed packages” in the first place.Thanks. Number two is very clear, but I’m a little confused on the others. On point 1, at least for versions. Rust can and does include multiple versions of transitive deps. But then it seems like maybe you’re talking about the language at the end? And what would point three mean?
Sorry for taking a bit to grok this!
(1) Whenever you change Rust’s version or other compiler settings, every existing artifact is discarded and all dependencies are recompiled from scratch. Additionally, caching of artifacts only seems to be per-project. So even if you have the same dependency in many different projects with the same compiler version and options, it will compile multiple times instead of once.
(3) There needs to be a way for multiple vendors to declare “I am package X”, ideally by separating the artifact from its publisher.
Ahh, I see now. Thank you! I’m always trying to figure out how to make stuff better, so understanding criticism is crucial.
With 1, it’s tough, because any change in compiler settings can change everything. So this would mean that we’d have to encode exactly what flags were sent, and only re-use the ones with those specific sets of flags. That’s why it’s per-project now, because that’s where the settings for each project are set.
For 3, that makes sense in some way to me, but I don’t think I’ve seen that feature actually implemented before. Is there a language which does this?
(1) Hashing could work.
(3) Probably every language that publishes things to Maven Central/Sonatype/Bintray/… Also, I think even npm supports it now.
Yeah, it could, the question is how much will it actually save you vs not.
Okay, I will try to look into Maven’s docs, thanks!
(1) I think it’s a no-brainer. I don’t see the point in not doing it, it can only help but not hurt. As soon as that works, introduce repositories (local and global) from which people can request the artifacts. – Congrats, you reached the state of what Java did in 2005(?).
Java provides bytecode, not native code, so there’s significantly less issues there.
Scala ships bytecode, JS code and soon native code. Multiply that by at least three major version of Scala. No issues experienced or expected.
Given that Cargo builds are a lot less powerful, the website should actually be able to build artifacts for all configs and targets automatically after the library author uploaded the source.