1. 25
  1.  

  2. 6

    IMO this is a huge component missing from Go. Glad to see rust implementing this early on!

    1. 2

      What does the go tool not do that you are still looking for? I haven’t used go get / go install extensively, so I don’t know its shortcomings.

      1. 3

        Go has historically had this ‘vendor all the things!’ attitude. I hear godep is more similar to Cargo, though.

        1. 1

          I really want to be able to easily pin a package at a given version, and a single-source search for all packages.

          1. 1

            For package version pinning, you can just vendor the package code. It’s simple and effective.

            For a single-source search, you can use godoc.org, which is now officially hosted by the Go team.

            1. 1

              Godoc.org is almost there.. if I could do “go search sqlite3” and see a list of sqlite3 related packages I would be happy.

              1. 2

                “If I could do a ‘go search sqlite3’”: you mean on the command line?

                1. 1

                  Yep

                  1. 4

                    A simple wrapper around godoc.org API should do it.

                    Or an alias to curl:

                    $ curl -H 'Accept: text/plain' http://godoc.org/?q=sqlite3
                    github.com/mattn/go-sqlite3 sqlite3 driver for go that using database/sql
                    code.google.com/p/go-sqlite/go1/sqlite3 Package sqlite3 provides an interface to SQLite version 3 databases.
                    github.com/mxk/go-sqlite/sqlite3 Package sqlite3 provides an interface to SQLite version 3 databases.
                    github.com/turboroot/btcdb/sqlite3 Package sqlite3 implements a sqlite3 instance of btcdb.
                    github.com/kuroneko/gosqlite3 Package sqlite3 implements a Go interface to the SQLite database.
                    ...
                    
        2. 2

          In Go I don’t miss such a component at all. I’d even say that not having to use a tool like Bundler in Ruby, pip in Python or npm in Node.js is refreshing.

          1. 4

            Could you expand a bit on why that’s so?

            1. 5

              Many reasons why I (and to my knowledge most Go developers) prefer vendoring over a tool like Bundler/pip/npm/cargo:

              • All dependencies are immediately available after a git checkout. No need to run another command.
              • As a consequence, deployment scripts and CI scripts are greatly simplified.
              • When I update a dependency, I don’t have to update a manifest file listing dependencies and their versions. I simply commit the change.
              • The whole ecosystem is fully decentralized and does not depend on a central authority. The official repositories of Bundler/pip/npm all suffered at some point from maintenance issues.
              • I get reproducible builds for free. Our business depends on our application, so we need to be able to build and deploy our application at any time, even when the central repository is unavailable or if the package author altered or deleted his code. This implies to keep a copy of all dependencies in a place we control. With code vendoring, the problem is already solved.
              • It’s easy to understand how code vendoring works. The mechanism is transparent and inspectable. With Bundler/pip/npm, you depend on a more complex tool, with a lot of edge cases. In particular, some of these tools try do too many things at the same time: dependency resolution, dependency download, dependency installation, dependency build. The last point is especially tricky to “standardize”. You end up either with something too simple and a lot users whose use case is not covered, or something too complex.

              I have other points in mind, but I have to go! Don’t hesitate to reply if you want me to go on with this :-)

              1. 5

                Thanks for taking the time to write a detailed reply. I feel completely differently, which is why I asked. I appreciate the perspective. :)

                1. 4

                  I understand where you come from :) When I started programming in Go, I feel disoriented by the lack of a package manager similar to pip/bundler/npm. It forced me into completely rethinking the way I used to manage dependencies. Now, I really prefer the “bare metal” approach of Go on this matter.

                  What about your experience? I’m curious about your perpective too.

                  1. 7

                    I’ll start by responding with my thoughts on your thoughts, and at the end, see if there’s anything I didn’t cover.

                    • I have to run bundle install so rarely that this doesn’t seem like a big deal to me. I guess I have a different tolerance. If I wanted it to be one, I could just make make run it, and then run whatever.

                    • Adding one more command to a script that already has a bunch isn’t that big of a simplification to me.

                    • I like that the manifest contains exact versions of my dependencies, because I can declare version ranges, but then assure my build is repeatable. Without ranges, I cannot have a tool verify for me that my transitive dependencies can be updated, at least not without hassle.

                    • While it’s true that source "https://rubygems.org" is at the top of almost every Gemfile, just add another source line, and you’re good. It’s not a distributed authority, but it is a decentralized one. Good enough for me. I can run my own internal mirror if I want to be responsible for the uptime. Oh, and when Rubygems.org got hacked, these mirrors were supremely valuable for ensuring that the contents of gems hadn’t been tampered with. Just a note, centralized is bad, we agree. :)

                    • Reproducible builds are awesome. However, you can manage this in other ways: as I mentioned, the lockfile helps with this, and you don’t even need Rubygems.org most of the time. And if you want it to be all of the time, you can run your own server.

                    • While the process may be easier, understanding dependencies can be harder. Maybe it’s a discipline issue, but I’ve found that when vendoring, patches don’t move upstream as often, which can lead to a custom fork that’s private. And you can still psuedo-vendor in a way that is more transparent with regards to upstream.

                    Okay, so, some other things:

                    • I don’t like that my Git repository has a whole bunch of junk in it that’s not actually my project.

                    • I don’t like that I’m pushing up tons of code to my servers, when that code is already living on the server.

                    • I don’t like that I will have common dependencies installed dozens of times.

                    • I don’t like the extra work that’s required to re-try a dependency against upstream. So, for example, I have five Rails apps on my machine, and I have my own fork of Rails. I want to make sure I haven’t broken my apss, as an extra step before making the commit to Rails itself. With vendoring, I have to copy around my special rails in each project, then revert the changes. With Bundler, I can just point at my local Rails, and it’ll just work transparently. I like using tools instead of cp, even though that might be silly.

                    …. I think that’s all I got for now. This kind of thing is like vim vs. all those other irrelevant editors :wink:. It really boils down to what kind of tradeoffs you prefer.

                    1. 1

                      Thanks for sharing your thoughts too :)

                      Adding one more command to a script that already has a bunch isn’t that big of a simplification to me.

                      The simplification does not come from not having to run another command, but from everything related to maintaining your own repo of Gems.

                      While the process may be easier, understanding dependencies can be harder.

                      I don’t get why dependencies would be harder to understand. They are all grouped in a dedicated directory like “third_party” or “vendor”. By having a look at this directory, you immediately know what your dependencies are. And you know where they are used by simply scanning the import statements. I want to emphasize this is a very scalable approach, used at Google internally, and also in open projects like Chromium or camlistore.

                      I don’t like that my Git repository has a whole bunch of junk in it that’s not actually my project.

                      For a library, I agree. But for an application, I’d argue it’s an advantage to have everything in the same repo. I will repeat myself, but this is the single corporate-wide repo strategy adopted by Google or Facebook.

                      I don’t like that I’m pushing up tons of code to my servers, when that code is already living on the server.

                      I don’t understand. Whatever you do, using code vendoring or using a package manager, you have to download dependencies at some point. Deploys following the initial download just need to download the new code. If you use git fetch or rsync, the dependencies source will not be downloaded again if there is no change.

                      I don’t like that I will have common dependencies installed dozens of times.

                      I don’t get it too. I’m not sure about bundler, but if you use pip or npm in the recommended way, dependencies will be installed in environment local to each project. And anyway, this is irrelevant to Go which produces compiled binaries. Moreover, as people use containers like Docker more and more, your dependencies are installed in each container anyway.

                      I don’t like the extra work that’s required to re-try a dependency against upstream. So, for example, I have five Rails apps on my machine, and I have my own fork of Rails. I want to make sure I haven’t broken my apss, as an extra step before making the commit to Rails itself. With vendoring, I have to copy around my special rails in each project, then revert the changes. With Bundler, I can just point at my local Rails, and it’ll just work transparently. I like using tools instead of cp, even though that might be silly.

                      Symbolic link?

                      This kind of thing is like vim vs. all those other irrelevant editors

                      Haha! I like this one :) I think you’re right.

                      1. 2

                        A short comment would help understanding how is this incorrect…

                      2. 1

                        Express.js’s main author is using Go since some time. Someone asked him in a tweet:

                        Also, wouldn’t you miss the Node/npm ecosystem?

                        He answered:

                        not at all haha, I’m really enjoying not having a package manager, Go code is higher quality than stuff in npm

                        This is interesting considering that npm is probably one of the best package manager.

                        Source: https://twitter.com/tjholowaychuk/status/483271990816436224

                        1. 1

                          I think most go developers feel like they only rarely need to step out of the standard library to get things done. It’s an unsurprising result when you’re comparing code written by Ken Thompson, Rob Pike, Brad Fitzpatrick etc to packages that can be written by anybody in the world.

                  2. 3

                    What do you do when a dependent package has an incompatible license? (say your app is BSD and the dep is GPLv3)

                    1. 1
                      • If the app is offered as an online service, then the app source and executable are kept internally and there is no problem.
                      • If the app is offered as an executable binary, then you must license it under GPLv3 anyway, because it uses a GPLv3 dependency.
                      • The only case where there is an actual issue is the one of an app distributed under source form, integrating a GPLv3 dependency. I agree that with a package manager, the app can be BSD licensed, and the dependency can still be GPLv3 licensed if it is downloaded separately by the package manager, which is impossible with code vendoring. But this is a very specific use case in my opinion. Most of the time, you will either license the whole app under GPLv3 or use a library licensed under LGPLv3.
                      1. 1

                        Interesting, coming from the land of OpenBSD (obviously not the norm for go-related stuff), re-licensing feels like a non-option. I haven’t hit this issue (needing to re-license something) specifically, but it is always in the back of my mind.

                        1. 1

                          I wrote nothing about re-licensing in my comment. What do you mean by re-licensing (I think the term has several meanings)?

                          1. 1

                            I was still thinking in terms of OpenBSD land (and in the context of my initial example of the BSD licensed app with a GPLv3 lib) - which put your: “you will either license the whole app under GPLv3 or use a library licensed under LGPLv3.” in the re-licensing context.. as in .. change it from BSD to GPLv3.

                            1. 1

                              No need to re-license the app from BSD to GPLv3 (by the way, only the copyright holder can do that). You just have to license the derived work (the original version of the app + your modifications to the app + the GPLv3 dependency) under GPLv3, but each “original” part keep its original license.

                              1. 1

                                your app is BSD licensed

                                Copyright holder has been assumed from the get go. Obviously the re-license would not have to be retroactive.. but that isn’t the point.

                                To do vendoring with a GPLv3 library you must jump through hoops that would not exist with a manifest style package config.

                                1. 1

                                  To do vendoring with a GPLv3 library you must jump through hoops that would not exist with a manifest style package config.

                                  I agree that vendoring a GPLv3 library into a BSD application is impossible from a legal point of view. But what is the point of licensing the application under BSD if it requires a GPLv3 dependency downloaded independently? Because of the GPL dependency, you would not be authorized to distribute executable binaries of your app to your customers, for example. The problems with such a setup are not technical, code vendoring or not; they are legal. The obvious solution is to use a LGPL library (which is the case most of the time) or license the whole under GPL.

                2. 3

                  How can this be “incorrect”? I just expressed an opinion. Is this a way to have a discussion about something?