I’m really excited about this. The interface reminds me of Glide somewhat, which I’ve toyed with in the past and seems like the most mature of the unofficial dependency management options. This seems like a better option because (AFAICT) Glide has no support for pegging to a SemVer release (I could be wrong here).
With better support for dependencies on a per-project level, I’m curious if there will eventually be elimination of GOPATH. I can’t really think of a good reason for it still to exist, other than maybe for simple projects where you don’t want to have to bother with project-level dependencies.
Here’s hoping that GOPATH dies a fiery death. The requirement, by default, to keep my code 4 layers beneath my home directory always seemed completely obnoxious to me. Obviously there are ways around the various inconveniences this causes, but it’s still irritating, particularly because it’s so incredibly unnecessary.
My pet peeve with GOPATH is that I sometimes write libraries which support multiple languages. I want to use a single git repo with all the code in it but GOPATH forces me to either split up my archive or put GOPATH tendrils everywhere. It’s ugly. I just want to compile it in place like I can with any other language.
Specifically your home directory (surely you put it anywhere)
Because I normally keep my code in my home directory. It makes no difference where it is, the point is that it’s four layers of directories I don’t need or want.
Specifically four levels deep (I assume you refer to src/<domain>/<user>/<repo>)
Yep, you assumed correctly. For me, this means ~/Go/src/<domain>/<user>/<repo> instead of ~/repo. I find this endlessly annoying.
Unnecessary (there are trade-offs, to be sure, but it’s not like there are no nice features it enables)
Like what? What does it provide that would be infeasible with a different, less restrictive directory layout? I’m genuinely curious because I’ve literally never been in a situation where I said to myself “Gosh, I’m sure glad Go imposes this weird directory structure on me.”
The idea that something you dislike could not exist for a good reason…
I never said that. In fact, I only criticized the directory structure as the default. I feel that the current default is no longer the most reasonable default. Perhaps at one time it was the best default. That has no bearing on my opinion or argument.
I’m not saying we should stone Rob Pike because he chose the default stupidly, if that were the case then I would need to demonstrate that there were no good reasons for the current default (I would also need to make a strong case for stoning as the most appropriate reaction, but that’s another matter). But I’m saying the default should be changed, in the present.
For context, I use a GOPATH per project, so my app code is in eg src/server/main.go.
I also use an editor with a good fuzzy file search implementation, so I can type usrpkm instead of github.com/user/package/main.go to find a file. These both substantially improve the situation.
Like what? What does it provide that would be infeasible with a different, less restrictive directory layout?
Making the name of the package map 1:1 to its location on disk makes navigation (‘where is this code’) easy.
In eg ruby and node this is a runtime concern (and can be hooked by arbitrary code), which makes static analysis impossible.
An alternative directory layout would need to preserve the property of being easy for tools to navigate.
I’m saying the default should be changed, in the present.
One of the key reasons to use go is that they carefully avoid breaking changes.
totally agree that GOPATH needs to go away, although not just because of the deep directory structure. We now use the vendor directory for managing our projects' dependencies which works well, until a dependency is forgotten and is pulled out of GOPATH instead. this breaks builds (because we do not commit our deps, preferring to use glide to restore them to the correct versions) and makes deterministic builds more difficult. Another issue I’ve hit up against is that it becomes impossible to have two different working copies of a repo without setting up a second GOPATH, at which point, why not just use a project-based build tool to begin with?
Unsure what you mean by “pegging to a semver release” but with glide you can either specify the version as “1.2.3” which will use that exact release version and no others, or “^1.2.3” which means “semver compatible with 1.2.3” (it is shorthand for >=1.2.3, <2.0.0).
In short, SemVer provides clients (limited) warning of breaking changes. Alternatively, consider a scheme that provides guidance for upgrading without breakage. Concretely, this would mean replacing name@major.minor.rev with name@major-timestamp and integrating deprecation warnings in to go vet.
More verbosely, “breaking changes” should never happen in stable libraries without at least one major version’s worth of deprecation warnings for smooth migration. Adding new functionality should never cause breakage (Go has some unique problems here). SemVer formalizes bad library behavior as adds the unnecessary burden of fiddling with minor and revision numbers for every little change.
It’s very unlikely that we use something other than semver. I’ve spent plenty of time worshipping at the Altar of Hickey, but this is one case where I’d say his arguments don’t add a lot new to the discussion that we didn’t already know.
I’ve been hoping to find some time to write this up in greater detail, but it’s been a busy couple months. Simplest, shortest version I can conjure: Hickey, like most folks, seems to be expecting more out of fundamentally meaningless numbers than is really wise.
Hickey, like most folks, seems to be expecting more out of fundamentally meaningless numbers than is really wise
That’s a strange conclusion. He literally uses the words “not meaningful” when talking about version numbers. Half of his argument against SemVer is that it codifies non-useful information in to two thirds of its values.
The bulk of my proposal is that you simply replace minor and revision numbers with a timestamp. People who want to annoy their users with breaking changes can still bump the major version to do that. They just no longer have to fiddle with minor version numbers every time they make non-breaking changes.
That’s a strange conclusion. He literally uses the words “not meaningful” when talking about version numbers. Half of his argument against SemVer is that it codifies non-useful information in to two thirds of its values.
Indeed, but acknowledging that doesn’t preclude expecting unhelpful things from them. But like I said - shortest possible explanation. Not necessarily one that makes prima facie sense :) Maybe I’ll have time for a fuller writeup, sometime.
The bulk of my proposal is that you simply replace minor and revision numbers with a timestamp. People who want to annoy their users with breaking changes can still bump the major version to do that. They just no longer have to fiddle with minor version numbers every time they make non-breaking changes.
It might be better. Certainly, that’s encoding less subjective information. The crucial question is whether the benefit is large enough to merit the social disruption that would come from displacing a widely used standard. That’s where I’m dubious.
The crucial question is whether the benefit is large enough to merit the social disruption that would come from displacing a widely used standard.
To me, this is the only point worth discussing. I’m firmly in the camp that SemVer is actively harmful and trivially eliminated in any community that does not yet have a standardized package manager. What risks to you perceive?
I hope it works well, it doesnt really seem like many of the people i trust from the go team are working on it. I suppose google doesnt use a tool like this internally.
I’m not sure where you heard that, but I’ve worked with several of the people on the team and they are excellent. I’m really excited for the tool.
Google will either work from tip on every project (hence the support for aliasing, and the need to do large scale transformations across their multimillion line codebase), or use this tool for vendoring. They run Go betas and release candidates in production, there’s no reason they’d use a different or an adhoc tool.
My concern is it won’t be supported to the same degree as the rest of Go, it mostly seems to be developed by Edward Muller (someone not at google), who may well be doing it on his hobby time.
I looked at the contributor graph on the repo - https://github.com/tools/godep/graphs/contributors. Very few of the high ranking core team members on googles payroll have touched it much or at all (Russ Cox, Brad Fitz., Rob Pike, Robert Griesemer, Alan Donovan, Robert Griesemer, Andrew Gerrand …. etc), Even contributors I know who care about package management systems like Dave Cheney are not there (who wrote gb). I personally doubt google cares much because it has its own giant VCS repo where they vendor everything.
dep is part of a project that started last year, largely by Peter Bourgon. I was asked to join the team working on this project due to my involvement in a different tool: godep the OG dependency management tool for Go, which I inherited maintenance of from Keith Rarick & my work @ Heroku serving our customers who use Go. The other members of the team, aside from myself and Peter, are Jessie Frazelle, Andrew Gerrand and Sam Boyer. Andrew is part of the Go team @ Google. Jessie works at Google and is involved in large Go projects such as Docker and Kubernetes, among others. Sam wrote and maintains gps the SAT solver powering glide and dep.
This team has published a bunch of info about our progress over the course of our work:
To date, various other tool authors and concerned parties have also been involved in different ways.
In the end I’d say I’ve played a role in the development of dep, having contributed directly, indirectly by patching gps or via pairing sessions with other team members, but I would not say that the tool is being developed mostly by me in any way. My hope is that we can attract additional contributors to dep as soon as possible. As such I invite all Go users (especially the maintainers of other tools) to participate in the development by filing well researched issues, opening PRs and testing the tool on their own projects and repositories.
FWIW: The hope is that this tool sets the ground work for things to come as part of the go language distribution in the future, something I’ve also made minor contributions to in the past,
PS: I am not speaking for the Go team here, just myself.
Thanks for the reply - How usable is the tool currently?
edit: trying it now, the wording of the readme made me think it wasn’t even usable at all. “Typical usage on a new repo might be” made me think it wasn’t implemented yet.
edit2: Really happy it worked out of the box with one of my projects. Neat.
Edward Muller, specifically works on this as the language maintainer of Go at Heroku. He spends time on it for these reasons, among others I can’t speak to:
We have customers who would like to run Go apps on Heroku. Standard, accepted tooling reduces support burden and makes documentation, and support tooling much easier to maintain.
Heroku uses Go heavily internally. From the implementation of orchestration, to the custom metrics/monitoring stack I help build and maintain, to the git server that you push to to deploy…and everything in between.
I hope the prototype gets a better, more searchable name. So far, Go has done a great job of being terrible for Googling. Name of the most popular web toolkit? Gorilla. For that reason, any time I refer to Go, I instead make a point to call it Golang.
FWIW: this isn’t a troll, but a legitimate concern from someone uses Go a lot. go dep makes sense as a name, but I hope that the package itself on Github does not end up being named dep. As one of the early adopters of Golang in production, it was a real problem in the early day to help get the other programmers I worked with to find the right repo when talking about them.
We went back and forth over naming a fair bit internally, with the poor searchability of “dep” being a major driver. Ultimately, we couldn’t settle on an alternative name to the rather generic “dep,” so it’s sufficing for now.
On the plus side, assuming all goes well and the proposal is accepted, the planned trajectory for the tool is to integrate it seamlessly into the go toolchain - the name will go away. So, the name we use for now hopefully won’t matter too much.
Link to provisional docs.
I’m really excited about this. The interface reminds me of Glide somewhat, which I’ve toyed with in the past and seems like the most mature of the unofficial dependency management options. This seems like a better option because (AFAICT) Glide has no support for pegging to a SemVer release (I could be wrong here).
With better support for dependencies on a per-project level, I’m curious if there will eventually be elimination of GOPATH. I can’t really think of a good reason for it still to exist, other than maybe for simple projects where you don’t want to have to bother with project-level dependencies.
Here’s hoping that GOPATH dies a fiery death. The requirement, by default, to keep my code 4 layers beneath my home directory always seemed completely obnoxious to me. Obviously there are ways around the various inconveniences this causes, but it’s still irritating, particularly because it’s so incredibly unnecessary.
My pet peeve with GOPATH is that I sometimes write libraries which support multiple languages. I want to use a single git repo with all the code in it but GOPATH forces me to either split up my archive or put GOPATH tendrils everywhere. It’s ugly. I just want to compile it in place like I can with any other language.
Not sure why it’s:
src/<domain>/<user>/<repo>)The idea that something you dislike could not exist for a good reason is unhelpful in the extreme.
Because I normally keep my code in my home directory. It makes no difference where it is, the point is that it’s four layers of directories I don’t need or want.
Yep, you assumed correctly. For me, this means
~/Go/src/<domain>/<user>/<repo>instead of~/repo. I find this endlessly annoying.Like what? What does it provide that would be infeasible with a different, less restrictive directory layout? I’m genuinely curious because I’ve literally never been in a situation where I said to myself “Gosh, I’m sure glad Go imposes this weird directory structure on me.”
I never said that. In fact, I only criticized the directory structure as the default. I feel that the current default is no longer the most reasonable default. Perhaps at one time it was the best default. That has no bearing on my opinion or argument.
I’m not saying we should stone Rob Pike because he chose the default stupidly, if that were the case then I would need to demonstrate that there were no good reasons for the current default (I would also need to make a strong case for stoning as the most appropriate reaction, but that’s another matter). But I’m saying the default should be changed, in the present.
For context, I use a GOPATH per project, so my app code is in eg
src/server/main.go. I also use an editor with a good fuzzy file search implementation, so I can typeusrpkminstead ofgithub.com/user/package/main.goto find a file. These both substantially improve the situation.Making the name of the package map 1:1 to its location on disk makes navigation (‘where is this code’) easy. In eg ruby and node this is a runtime concern (and can be hooked by arbitrary code), which makes static analysis impossible.
An alternative directory layout would need to preserve the property of being easy for tools to navigate.
One of the key reasons to use go is that they carefully avoid breaking changes.
totally agree that GOPATH needs to go away, although not just because of the deep directory structure. We now use the vendor directory for managing our projects' dependencies which works well, until a dependency is forgotten and is pulled out of GOPATH instead. this breaks builds (because we do not commit our deps, preferring to use glide to restore them to the correct versions) and makes deterministic builds more difficult. Another issue I’ve hit up against is that it becomes impossible to have two different working copies of a repo without setting up a second GOPATH, at which point, why not just use a project-based build tool to begin with?
Unsure what you mean by “pegging to a semver release” but with glide you can either specify the version as “1.2.3” which will use that exact release version and no others, or “^1.2.3” which means “semver compatible with 1.2.3” (it is shorthand for >=1.2.3, <2.0.0).
I’m very glad to see Lockfiles in the design docs, but please consider alternatives to SemVer!
See Rich Hickey’s Spec-ulation talk for problems and my alternative scheme proposal for potential solutions.
In short, SemVer provides clients (limited) warning of breaking changes. Alternatively, consider a scheme that provides guidance for upgrading without breakage. Concretely, this would mean replacing
name@major.minor.revwithname@major-timestampand integrating deprecation warnings in togo vet.More verbosely, “breaking changes” should never happen in stable libraries without at least one major version’s worth of deprecation warnings for smooth migration. Adding new functionality should never cause breakage (Go has some unique problems here). SemVer formalizes bad library behavior as adds the unnecessary burden of fiddling with minor and revision numbers for every little change.
(one of the dep team members, here)
It’s very unlikely that we use something other than semver. I’ve spent plenty of time worshipping at the Altar of Hickey, but this is one case where I’d say his arguments don’t add a lot new to the discussion that we didn’t already know.
I’ve been hoping to find some time to write this up in greater detail, but it’s been a busy couple months. Simplest, shortest version I can conjure: Hickey, like most folks, seems to be expecting more out of fundamentally meaningless numbers than is really wise.
That’s a strange conclusion. He literally uses the words “not meaningful” when talking about version numbers. Half of his argument against SemVer is that it codifies non-useful information in to two thirds of its values.
The bulk of my proposal is that you simply replace minor and revision numbers with a timestamp. People who want to annoy their users with breaking changes can still bump the major version to do that. They just no longer have to fiddle with minor version numbers every time they make non-breaking changes.
Indeed, but acknowledging that doesn’t preclude expecting unhelpful things from them. But like I said - shortest possible explanation. Not necessarily one that makes prima facie sense :) Maybe I’ll have time for a fuller writeup, sometime.
It might be better. Certainly, that’s encoding less subjective information. The crucial question is whether the benefit is large enough to merit the social disruption that would come from displacing a widely used standard. That’s where I’m dubious.
To me, this is the only point worth discussing. I’m firmly in the camp that SemVer is actively harmful and trivially eliminated in any community that does not yet have a standardized package manager. What risks to you perceive?
I hope it works well, it doesnt really seem like many of the people i trust from the go team are working on it. I suppose google doesnt use a tool like this internally.
I’m not sure where you heard that, but I’ve worked with several of the people on the team and they are excellent. I’m really excited for the tool.
Google will either work from tip on every project (hence the support for aliasing, and the need to do large scale transformations across their multimillion line codebase), or use this tool for vendoring. They run Go betas and release candidates in production, there’s no reason they’d use a different or an adhoc tool.
My concern is it won’t be supported to the same degree as the rest of Go, it mostly seems to be developed by Edward Muller (someone not at google), who may well be doing it on his hobby time.
I looked at the contributor graph on the repo - https://github.com/tools/godep/graphs/contributors. Very few of the high ranking core team members on googles payroll have touched it much or at all (Russ Cox, Brad Fitz., Rob Pike, Robert Griesemer, Alan Donovan, Robert Griesemer, Andrew Gerrand …. etc), Even contributors I know who care about package management systems like Dave Cheney are not there (who wrote gb). I personally doubt google cares much because it has its own giant VCS repo where they vendor everything.
Edward Muller here….
dep is part of a project that started last year, largely by Peter Bourgon. I was asked to join the team working on this project due to my involvement in a different tool: godep the OG dependency management tool for Go, which I inherited maintenance of from Keith Rarick & my work @ Heroku serving our customers who use Go. The other members of the team, aside from myself and Peter, are Jessie Frazelle, Andrew Gerrand and Sam Boyer. Andrew is part of the Go team @ Google. Jessie works at Google and is involved in large Go projects such as Docker and Kubernetes, among others. Sam wrote and maintains gps the SAT solver powering glide and dep.
This team has published a bunch of info about our progress over the course of our work:
To date, various other tool authors and concerned parties have also been involved in different ways.
In the end I’d say I’ve played a role in the development of dep, having contributed directly, indirectly by patching gps or via pairing sessions with other team members, but I would not say that the tool is being developed mostly by me in any way. My hope is that we can attract additional contributors to
depas soon as possible. As such I invite all Go users (especially the maintainers of other tools) to participate in the development by filing well researched issues, opening PRs and testing the tool on their own projects and repositories.FWIW: The hope is that this tool sets the ground work for things to come as part of the go language distribution in the future, something I’ve also made minor contributions to in the past,
PS: I am not speaking for the Go team here, just myself.
Thanks for the reply - How usable is the tool currently?
edit: trying it now, the wording of the readme made me think it wasn’t even usable at all. “Typical usage on a new repo might be” made me think it wasn’t implemented yet.
edit2: Really happy it worked out of the box with one of my projects. Neat.
Great! It’s totally a work in progress. ;-)
Heroku hat time (though, I don’t have one).
Edward Muller, specifically works on this as the language maintainer of Go at Heroku. He spends time on it for these reasons, among others I can’t speak to:
If you look at the Go Packaging Proposal Process document, you’ll see that Andrew Gerrand, Dave Cheney, etc were all involved in the design.
I hope the prototype gets a better, more searchable name. So far, Go has done a great job of being terrible for Googling. Name of the most popular web toolkit? Gorilla. For that reason, any time I refer to Go, I instead make a point to call it Golang.
FWIW: this isn’t a troll, but a legitimate concern from someone uses Go a lot.
go depmakes sense as a name, but I hope that the package itself on Github does not end up being nameddep. As one of the early adopters of Golang in production, it was a real problem in the early day to help get the other programmers I worked with to find the right repo when talking about them.We went back and forth over naming a fair bit internally, with the poor searchability of “dep” being a major driver. Ultimately, we couldn’t settle on an alternative name to the rather generic “dep,” so it’s sufficing for now.
On the plus side, assuming all goes well and the proposal is accepted, the planned trajectory for the tool is to integrate it seamlessly into the go toolchain - the name will go away. So, the name we use for now hopefully won’t matter too much.