IMO this is a huge component missing from Go. Glad to see rust implementing this early on!
What does the go tool not do that you are still looking for? I haven’t used go get / go install extensively, so I don’t know its shortcomings.
Go has historically had this ‘vendor all the things!’ attitude. I hear godep is more similar to Cargo, though.
I really want to be able to easily pin a package at a given version, and a single-source search for all packages.
For package version pinning, you can just vendor the package code. It’s simple and effective.
For a single-source search, you can use godoc.org, which is now officially hosted by the Go team.
Godoc.org is almost there.. if I could do “go search sqlite3” and see a list of sqlite3 related packages I would be happy.
“If I could do a ‘go search sqlite3’”: you mean on the command line?
A simple wrapper around godoc.org API should do it.
Or an alias to curl:
$ curl -H 'Accept: text/plain' http://godoc.org/?q=sqlite3
github.com/mattn/go-sqlite3 sqlite3 driver for go that using database/sql
code.google.com/p/go-sqlite/go1/sqlite3 Package sqlite3 provides an interface to SQLite version 3 databases.
github.com/mxk/go-sqlite/sqlite3 Package sqlite3 provides an interface to SQLite version 3 databases.
github.com/turboroot/btcdb/sqlite3 Package sqlite3 implements a sqlite3 instance of btcdb.
github.com/kuroneko/gosqlite3 Package sqlite3 implements a Go interface to the SQLite database.
In Go I don’t miss such a component at all. I’d even say that not having to use a tool like Bundler in Ruby, pip in Python or npm in Node.js is refreshing.
Could you expand a bit on why that’s so?
Many reasons why I (and to my knowledge most Go developers) prefer vendoring over a tool like Bundler/pip/npm/cargo:
I have other points in mind, but I have to go! Don’t hesitate to reply if you want me to go on with this :-)
Thanks for taking the time to write a detailed reply. I feel completely differently, which is why I asked. I appreciate the perspective. :)
I understand where you come from :) When I started programming in Go, I feel disoriented by the lack of a package manager similar to pip/bundler/npm. It forced me into completely rethinking the way I used to manage dependencies. Now, I really prefer the “bare metal” approach of Go on this matter.
What about your experience? I’m curious about your perpective too.
I’ll start by responding with my thoughts on your thoughts, and at the end, see if there’s anything I didn’t cover.
I have to run bundle install so rarely that this doesn’t seem like a big deal to me. I guess I have a different tolerance. If I wanted it to be one, I could just make make run it, and then run whatever.
Adding one more command to a script that already has a bunch isn’t that big of a simplification to me.
I like that the manifest contains exact versions of my dependencies, because I can declare version ranges, but then assure my build is repeatable. Without ranges, I cannot have a tool verify for me that my transitive dependencies can be updated, at least not without hassle.
While it’s true that source "https://rubygems.org" is at the top of almost every Gemfile, just add another source line, and you’re good. It’s not a distributed authority, but it is a decentralized one. Good enough for me. I can run my own internal mirror if I want to be responsible for the uptime. Oh, and when Rubygems.org got hacked, these mirrors were supremely valuable for ensuring that the contents of gems hadn’t been tampered with. Just a note, centralized is bad, we agree. :)
Reproducible builds are awesome. However, you can manage this in other ways: as I mentioned, the lockfile helps with this, and you don’t even need Rubygems.org most of the time. And if you want it to be all of the time, you can run your own server.
While the process may be easier, understanding dependencies can be harder. Maybe it’s a discipline issue, but I’ve found that when vendoring, patches don’t move upstream as often, which can lead to a custom fork that’s private. And you can still psuedo-vendor in a way that is more transparent with regards to upstream.
Okay, so, some other things:
I don’t like that my Git repository has a whole bunch of junk in it that’s not actually my project.
I don’t like that I’m pushing up tons of code to my servers, when that code is already living on the server.
I don’t like that I will have common dependencies installed dozens of times.
I don’t like the extra work that’s required to re-try a dependency against upstream. So, for example, I have five Rails apps on my machine, and I have my own fork of Rails. I want to make sure I haven’t broken my apss, as an extra step before making the commit to Rails itself. With vendoring, I have to copy around my special rails in each project, then revert the changes. With Bundler, I can just point at my local Rails, and it’ll just work transparently. I like using tools instead of cp, even though that might be silly.
…. I think that’s all I got for now. This kind of thing is like vim vs. all those other irrelevant editors :wink:. It really boils down to what kind of tradeoffs you prefer.
all those other irrelevant editors
Thanks for sharing your thoughts too :)
The simplification does not come from not having to run another command, but from everything related to maintaining your own repo of Gems.
While the process may be easier, understanding dependencies can be harder.
I don’t get why dependencies would be harder to understand. They are all grouped in a dedicated directory like “third_party” or “vendor”. By having a look at this directory, you immediately know what your dependencies are. And you know where they are used by simply scanning the import statements. I want to emphasize this is a very scalable approach, used at Google internally, and also in open projects like Chromium or camlistore.
For a library, I agree. But for an application, I’d argue it’s an advantage to have everything in the same repo. I will repeat myself, but this is the single corporate-wide repo strategy adopted by Google or Facebook.
I don’t understand. Whatever you do, using code vendoring or using a package manager, you have to download dependencies at some point. Deploys following the initial download just need to download the new code. If you use git fetch or rsync, the dependencies source will not be downloaded again if there is no change.
I don’t get it too. I’m not sure about bundler, but if you use pip or npm in the recommended way, dependencies will be installed in environment local to each project. And anyway, this is irrelevant to Go which produces compiled binaries. Moreover, as people use containers like Docker more and more, your dependencies are installed in each container anyway.
This kind of thing is like vim vs. all those other irrelevant editors
Haha! I like this one :) I think you’re right.
A short comment would help understanding how is this incorrect…
Express.js’s main author is using Go since some time. Someone asked him in a tweet:
Also, wouldn’t you miss the Node/npm ecosystem?
not at all haha, I’m really enjoying not having a package manager, Go code is higher quality than stuff in npm
This is interesting considering that npm is probably one of the best package manager.
I think most go developers feel like they only rarely need to step out of the standard library to get things done. It’s an unsurprising result when you’re comparing code written by Ken Thompson, Rob Pike, Brad Fitzpatrick etc to packages that can be written by anybody in the world.
What do you do when a dependent package has an incompatible license? (say your app is BSD and the dep is GPLv3)
Interesting, coming from the land of OpenBSD (obviously not the norm for go-related stuff), re-licensing feels like a non-option. I haven’t hit this issue (needing to re-license something) specifically, but it is always in the back of my mind.
I wrote nothing about re-licensing in my comment. What do you mean by re-licensing (I think the term has several meanings)?
I was still thinking in terms of OpenBSD land (and in the context of my initial example of the BSD licensed app with a GPLv3 lib) - which put your: “you will either license the whole app under GPLv3 or use a library licensed under LGPLv3.” in the re-licensing context.. as in .. change it from BSD to GPLv3.
No need to re-license the app from BSD to GPLv3 (by the way, only the copyright holder can do that). You just have to license the derived work (the original version of the app + your modifications to the app + the GPLv3 dependency) under GPLv3, but each “original” part keep its original license.
your app is BSD licensed
Copyright holder has been assumed from the get go. Obviously the re-license would not have to be retroactive.. but that isn’t the point.
To do vendoring with a GPLv3 library you must jump through hoops that would not exist with a manifest style package config.
I agree that vendoring a GPLv3 library into a BSD application is impossible from a legal point of view. But what is the point of licensing the application under BSD if it requires a GPLv3 dependency downloaded independently? Because of the GPL dependency, you would not be authorized to distribute executable binaries of your app to your customers, for example. The problems with such a setup are not technical, code vendoring or not; they are legal. The obvious solution is to use a LGPL library (which is the case most of the time) or license the whole under GPL.
How can this be “incorrect”? I just expressed an opinion. Is this a way to have a discussion about something?