This post talks about the downsides but does not acknowledge the underlying problem that is being corrected, namely that “go get” today does two completely different things: (1) add or update dependencies of a program, and (2) install binaries.
Nearly all the time, you only mean to do one of these things, not both. If you get in the habit of using “go get” to update your dependencies, you might run a command like “go get path/…” and be surprised when it installs “path/internal/ls” to your $HOME/bin directory. We have had people claim that’s a security issue, and whether that’s true or not, it’s certainly a usability issue. In this example, you only want the dependency change and are getting the install as an unwanted side-effect. After the transition, the rule will be simple: “go get only changes your dependency versions”.
On the other hand, we have commands in READMEs that say things like “go get -u rsc.io/2fa”. Today that means to fetch the latest version of rsc.io/2fa, then upgrade all its dependencies, recursively, producing a build configuration that may never have been tested, and then build the binary and install it. In this example, you only want the install and are getting the “update dependencies” as an unwanted side effect. If instead you use “go install rsc.io/2fa@latest” (which works today in Go 1.16), you get the latest tagged version of rsc.io/2fa, using exactly the dependency versions it declares and was tested with, which is what you really want when you are trying to install a binary. We didn’t just make a gratuitous change to the way the command is spelled: we fixed the semantics too.
The double meaning of “go get” was introduced in GOPATH mode many years ago. It is my fault, and all I can say is that it seemed like a good idea at the time, especially in the essentially unversioned context of GOPATH-based development. But it’s a mistake, and it is a big enough mistake to correct for the next million users of Go.
The post claims this is an abrupt deprecation schedule, but the change was announced in the Go 1.16 release notes and will take effect in Go 1.18. Once Go 1.17 is released, the two supported versions of Go will be Go 1.16 and Go 1.17, meaning all supported Go versions will know what “go install path@version” means, making it time to start nudging people over to using it with prints from the go command. This two-cycle deprecation schedule, where we make sure that we don’t encourage transition work until all supported versions of Go (the last two) support the new behavior, is our standard procedure for this. I suspect that Go 1.18 will keep the warning print, so you’d actually get two releases of warning prints as well.
We try very hard not to make breaking changes in tool behavior. This one is necessary, and we’ve done everything we can to make sure it is a smooth transition. The new behavior should be obvious - the tool tells you what is going on and what you need to do differently - and easy to learn.
I know it’s too late to change anything now, but in retrospect, would it have been better to leave go get in the old GOPATH world and introduce the module version of go get under a different name?
A quick note: “go get” does three things today, not two. The third is that it clones the version control repository for some Go package or module (into $GOPATH/src). In the process of doing this, it reaches through vanity import paths and works out the correct VCS system and clone path to use. As far as I know, in the non-GOPATH world there is currently no way to do this yourself, either for programs or for other packages. It would be nice at the very least to have a command that resolved through vanity import paths to tell you the underlying thing to clone, and with what.
(Cloning this way also locks you to the current location of the repository. One advantage of vanity import paths is that people can change them, and have, and you will automatically pick up that change.)
The other thing that is not possible in the non-GOPATH world is to install the very latest VCS version of a program (at least in any convenient way). As noted, the @latest syntax is the latest tagged version, not the latest VCS version. This means that if you want to have someone test your very latest development version, you need to either tag it or tell them to clone the repo and build it themselves; you can’t give them a simple, generic ‘go install …@…’ command to do it. (You can give them a VCS version, but there are various problems with this.)
Oh oops, that’s my mistake. I missed that in the Version queries section of the specification.
(If reaching for the specification strikes people as odd, I can only plead that I couldn’t readily find documentation of this in the “go” help and “go help modules” did point me to the specification.)
I agree. Changing go get to mean go get ...@latest makes more sense than bothering users with deprecation warnings and then changed behavior. Why break the user experience?
Personally I think it’s massively confusing to change the behaviour of a command based on which directory you’re in.
What’s the best path forward? I don’t know … but the sooner this “GOPATH/modules schism” goes away in the Go tooling the better IMHO. I’ve been confused about this as well, and I’ve been programming Go full-time for over five years – it’ll be even more confusing for people newer to Go who never worked with GOPATH.
it’ll be even more confusing for people newer to Go who never worked with GOPATH.
I’m one of these people! And I still have no idea what GOPATH is all about, other than it causing confusing issues for me when trying to build some go projects :(
Think of GOPATH as $HOME but only for Go, it’s stdlib, your go projects, and dependencies pulled in. The whole of the Go ecosystem on your computer in one directory.
I found being forced to use a specific location for projects, set in an env variable, to be confusing and annoying. After my comment above, I read a bit more about modules and that’s definitely how I would have expected things to work from the beginning. Better late than never I suppose!
I always disliked GOPATH, but it’s not very hard: all code goes in to GOPATH, and that’s pretty much it. Easy-peasy. It’s mostly the two different “modes” that the tooling can operate on that’s confusing and can take you by surprise, especially if you’re not familiar with GOPATH.
Indeed, the final section of that page you linked says that tools are exempt. I can’t think of any previous examples that broke the tooling as much as this will, though.
go get is invoked by a ton of other programs. For example, VSCode plugins love to run it to install missing developer tools. It’s effectively an API, but that’s going through the CLI instead of accessing a Go API directly.
There’s been loads of subtle breakage and changes in the Go tooling ever since modules were introduced. It rather sucks, but it’s better than the alternative of keeping it full of all sorts of legacy behaviour where commands do something different depending on the environment and/or which directory you’re in, which is super-confusing.
It’s easy to say it’s “short-sighted”, but all options suck – they just suck in different ways.
I used to use go get for grabbing even non-Go repos but had to stop because of similar workflow breaks over the past few years. I finally wrote a small tool to replace it I call grabhttps://github.com/jmhodges/grab to take over my “fetch the code and organize it into a tidy directory system” need
It actually uses the nice VCS library guts that go get does!
This is not meant to be a gotcha, but should you update your own install instructions? I.e., tell people who are using go 1.16 or up to run go install github.com/jmhodges/grab@latest rather than go get github.com/jmhodges/grab. I ask—and this is why it’s not meant to be a gotcha—because I’m new to go, and I want to make sure I understand the new recommendations.
This makes sense when seen inline with the death of GO111MODULE. The two having separate behaviors have caused a lot of problems with those who don’t understand the history of Go with no package management. I may finally get to stop having to help my developers understand how to get things into the $GOPATH versus their mod.
This post talks about the downsides but does not acknowledge the underlying problem that is being corrected, namely that “go get” today does two completely different things: (1) add or update dependencies of a program, and (2) install binaries.
Nearly all the time, you only mean to do one of these things, not both. If you get in the habit of using “go get” to update your dependencies, you might run a command like “go get path/…” and be surprised when it installs “path/internal/ls” to your $HOME/bin directory. We have had people claim that’s a security issue, and whether that’s true or not, it’s certainly a usability issue. In this example, you only want the dependency change and are getting the install as an unwanted side-effect. After the transition, the rule will be simple: “go get only changes your dependency versions”.
On the other hand, we have commands in READMEs that say things like “go get -u rsc.io/2fa”. Today that means to fetch the latest version of rsc.io/2fa, then upgrade all its dependencies, recursively, producing a build configuration that may never have been tested, and then build the binary and install it. In this example, you only want the install and are getting the “update dependencies” as an unwanted side effect. If instead you use “go install rsc.io/2fa@latest” (which works today in Go 1.16), you get the latest tagged version of rsc.io/2fa, using exactly the dependency versions it declares and was tested with, which is what you really want when you are trying to install a binary. We didn’t just make a gratuitous change to the way the command is spelled: we fixed the semantics too.
The double meaning of “go get” was introduced in GOPATH mode many years ago. It is my fault, and all I can say is that it seemed like a good idea at the time, especially in the essentially unversioned context of GOPATH-based development. But it’s a mistake, and it is a big enough mistake to correct for the next million users of Go.
The post claims this is an abrupt deprecation schedule, but the change was announced in the Go 1.16 release notes and will take effect in Go 1.18. Once Go 1.17 is released, the two supported versions of Go will be Go 1.16 and Go 1.17, meaning all supported Go versions will know what “go install path@version” means, making it time to start nudging people over to using it with prints from the go command. This two-cycle deprecation schedule, where we make sure that we don’t encourage transition work until all supported versions of Go (the last two) support the new behavior, is our standard procedure for this. I suspect that Go 1.18 will keep the warning print, so you’d actually get two releases of warning prints as well.
We try very hard not to make breaking changes in tool behavior. This one is necessary, and we’ve done everything we can to make sure it is a smooth transition. The new behavior should be obvious - the tool tells you what is going on and what you need to do differently - and easy to learn.
I know it’s too late to change anything now, but in retrospect, would it have been better to leave
go getin the oldGOPATHworld and introduce the module version ofgo getunder a different name?A quick note: “go get” does three things today, not two. The third is that it clones the version control repository for some Go package or module (into $GOPATH/src). In the process of doing this, it reaches through vanity import paths and works out the correct VCS system and clone path to use. As far as I know, in the non-GOPATH world there is currently no way to do this yourself, either for programs or for other packages. It would be nice at the very least to have a command that resolved through vanity import paths to tell you the underlying thing to clone, and with what.
(Cloning this way also locks you to the current location of the repository. One advantage of vanity import paths is that people can change them, and have, and you will automatically pick up that change.)
The other thing that is not possible in the non-GOPATH world is to install the very latest VCS version of a program (at least in any convenient way). As noted, the @latest syntax is the latest tagged version, not the latest VCS version. This means that if you want to have someone test your very latest development version, you need to either tag it or tell them to clone the repo and build it themselves; you can’t give them a simple, generic ‘go install …@…’ command to do it. (You can give them a VCS version, but there are various problems with this.)
You can use “go install …@branchname” to get the latest commit for whatever branch.
Oh oops, that’s my mistake. I missed that in the Version queries section of the specification.
(If reaching for the specification strikes people as odd, I can only plead that I couldn’t readily find documentation of this in the “go” help and “go help modules” did point me to the specification.)
I agree. Changing
go getto meango get ...@latestmakes more sense than bothering users with deprecation warnings and then changed behavior. Why break the user experience?Personally I think it’s massively confusing to change the behaviour of a command based on which directory you’re in.
What’s the best path forward? I don’t know … but the sooner this “GOPATH/modules schism” goes away in the Go tooling the better IMHO. I’ve been confused about this as well, and I’ve been programming Go full-time for over five years – it’ll be even more confusing for people newer to Go who never worked with GOPATH.
I’m one of these people! And I still have no idea what GOPATH is all about, other than it causing confusing issues for me when trying to build some go projects :(
Think of GOPATH as $HOME but only for Go, it’s stdlib, your go projects, and dependencies pulled in. The whole of the Go ecosystem on your computer in one directory.
I found being forced to use a specific location for projects, set in an env variable, to be confusing and annoying. After my comment above, I read a bit more about modules and that’s definitely how I would have expected things to work from the beginning. Better late than never I suppose!
I always disliked GOPATH, but it’s not very hard: all code goes in to GOPATH, and that’s pretty much it. Easy-peasy. It’s mostly the two different “modes” that the tooling can operate on that’s confusing and can take you by surprise, especially if you’re not familiar with GOPATH.
Does it mean that the promise to not break API doesn’t extend to CLI? https://golang.org/doc/go1compat#expectations
Keeping things stable is what makes the success of Go.
Indeed, the final section of that page you linked says that tools are exempt. I can’t think of any previous examples that broke the tooling as much as this will, though.
It seems short-sighted.
go getis invoked by a ton of other programs. For example, VSCode plugins love to run it to install missing developer tools. It’s effectively an API, but that’s going through the CLI instead of accessing a Go API directly.There’s been loads of subtle breakage and changes in the Go tooling ever since modules were introduced. It rather sucks, but it’s better than the alternative of keeping it full of all sorts of legacy behaviour where commands do something different depending on the environment and/or which directory you’re in, which is super-confusing.
It’s easy to say it’s “short-sighted”, but all options suck – they just suck in different ways.
I’ve been listening to the gotime podcast. Go get is a feature to people new to programming – not a bug.
I used to use
go getfor grabbing even non-Go repos but had to stop because of similar workflow breaks over the past few years. I finally wrote a small tool to replace it I callgrabhttps://github.com/jmhodges/grab to take over my “fetch the code and organize it into a tidy directory system” needIt actually uses the nice VCS library guts that
go getdoes!This is not meant to be a gotcha, but should you update your own install instructions? I.e., tell people who are using go 1.16 or up to run
go install github.com/jmhodges/grab@latestrather thango get github.com/jmhodges/grab. I ask—and this is why it’s not meant to be a gotcha—because I’m new to go, and I want to make sure I understand the new recommendations.Oh, this got fixed!
Good riddance!
This makes sense when seen inline with the death of GO111MODULE. The two having separate behaviors have caused a lot of problems with those who don’t understand the history of Go with no package management. I may finally get to stop having to help my developers understand how to get things into the $GOPATH versus their mod.