My policy: add an stdx package for extensions to the strand library, which doesn’t have dependencies beside std. Ideally, “no dependencies” is enforced (eg, “this space is intentionally left blank” comment in Cargo.toml).
It’s more useful to think in terms of dependencies, rather than naming:
Does it depend on your project? Then surely you can find appropriate place for it, because you control layout of your project.
It’s independent? To stdx it goes, because you can’t just add stuff to std (although it makes sense to upstream bits of stdx)
I don’t usually work on projects with a lot of dependencies, but I imagine the pattern extends to large foundational frameworks, so something like railsx or axumx might make sense.
Related patterns in this area:
Kitchen Sink — not every thing can be neatly organized. The overall cohesion increases if you have an explicit “unsorted” place somewhere.
Django’s shortcuts module — in layered architectures, it’s useful to create a separate module for stuff that spans several layers, such that it isn’t a part of any specific layer.
I’ve found that util hiearchies tend to fall into two categories:
Just a sign of poor organization; don’t do this.
Stuff that has nothing to do with the project, but it wasn’t in the stdlib or any of the other libraries the project is using, so it had to go somewhere.
The latter is I think just a fact of life; there’s always going to be random programming goop that you need that isn’t conceptually specific to your project. But thinking of it as stuff that “should” be in a library somewhere leads one to the sub-hierarchy approach the article discusses, which makes a ton of sense.
At a certain point I ended up maintaining my util out of the tree of any one project, since there were a few things I kept copying into new projects: https://github.com/zenhack/go-util
Much of what’s in there I hope will someday be obviated by the stdlib, but right now pretty much anything that requires generics to implement doesn’t exist, so…
I didn’t even allow a util directory on my teams. Put the code with the correct domain, if it’s shared, move it up a level and give it’s own domain relative to it’s use. Otherwise it can live in it’s feature package.
I do think this mailutil syntax is interesting but it honestly seems like you could then just strip util from the name and again as stated above apply it to the mail domain (portion of the code).
That’s what I made my team do, if someone needed an extra ticket pick up some tech debt clean-up. Before all of that we had a team session working through some of the obvious util code so everyone would know what to do.
“In the end, what defines a module is what pieces of the system it will never responsible for, rather what it is currently responsible for.”
—Tef
I am fond of all of that post of Tef’s, but that particular quote has stuck with me the most. As for this post by Brandur, very neat! A good way to prevent omnimodules. In the knapsack it goes, for it will come in handy someday soon.
Can’t find the exact quote, but this reminds me of someone saying that any large enough organisation system without a “miscellaneous” category is useless, because of a massive long tail of single-item categories.
I like the middle ground that accepts it and tries to create some order in “utils”, but not too strictly.
What I landed on, and have seen on many projects, is having util packages within other packages. You don’t get a top-level util, but util things will be landing places for helper functions that are useful in a specific domain without being “about” something.
I’ve never seen these grow too big, as generally there’s a willingness to pull out bigger chunks into their own files just out of pragmatism.
What I landed on, and have seen on many projects, is having util packages within other packages. You don’t get a top-level util, but util things will be landing places for helper functions that are useful in a specific domain without being “about” something.
This is why I’ve done too. I move common code within a package down into util, which might be exported and if usable documented, or internal, which are implementation details and should never be used.
A function for checking if a string exists in a slice of strings appears in almost all of my projects. It’s too small and simple to have a separate project and package and is clearer and cleaner as a function than inline. Ideally, Go should have an in keyword, like in Python, but for now it lives in util.go in several of my projects.
util/stringutil.go might be an improvement, but just stringutil.go or util/string.go would have less repetition of words in the selected path + filename.
That’s a good point. Go excels at enumeration, syntax wise. But there is also dictionaries:
for k, v in d:
...
vs
for k, v := range d {
...
}
An overall win for the in keyword, unless list enumeration is the main concern. Also range is a word that does not add anything, when shouting the line out loud, for the code heralds among us.
Go’s rule against circular dependencies at the package level makes splitting by dependency particularly important.
If one utility depends on any package in your app and another independently useful utility doesn’t, it’s possible someone will want the no-deps utility but be unable to use it because there’d be a circular dep with the other utility’s dependency. Besides the hard rule, avoiding circularity just tends to make things easier to think about.
I also sort of agree with another comment that I’d locate e.g. email utilities ‘near’ any other email code that lives in your world if possible, e.g. email/emailutil or email itself instead of util/emailutil. If someone writing future code knows they’re trying to do something with email they know the ‘email’ part for sure, and only might know that the particular thing they want is a ‘util’. However, if your utils are related to code outside your app (stdlib or otherwise third-party), “put the utils with the other code” may just not apply and the stdx sort of approach seems appealing.
This post boils down to “if you’re gonna util at least organize it by types of functions.” This makes me kind of sad because the “junk drawer” is still a junk drawer, even if it’s more organized.
I wish it were easier to reexport functions and types in Go. In that way, you could define your own standard library and colocate these “utils” in their proper place, along with the curated “best practices” for how to use the underlying stdlib.
At what point has a util package ever been a problem worth caring for? It’s no elegant but it’s like the least problematic thing ever. I always end up with a util package, and if I ever need to reorganize it because it gets too big I just do it, and if some parts become big enough that they’d be better extracted I just do it, what’s wrong with that?
My policy: add an
stdx
package for extensions to the strand library, which doesn’t have dependencies beside std. Ideally, “no dependencies” is enforced (eg, “this space is intentionally left blank” comment in Cargo.toml).It’s more useful to think in terms of dependencies, rather than naming:
stdx
it goes, because you can’t just add stuff to std (although it makes sense to upstream bits of stdx)I don’t usually work on projects with a lot of dependencies, but I imagine the pattern extends to large foundational frameworks, so something like
railsx
oraxumx
might make sense.Related patterns in this area:
I’ve been using the -X pattern a lot in Go. My flagx.Func managed to get upstreamed into the std! :-)
I’ve found that
util
hiearchies tend to fall into two categories:The latter is I think just a fact of life; there’s always going to be random programming goop that you need that isn’t conceptually specific to your project. But thinking of it as stuff that “should” be in a library somewhere leads one to the sub-hierarchy approach the article discusses, which makes a ton of sense.
At a certain point I ended up maintaining my util out of the tree of any one project, since there were a few things I kept copying into new projects: https://github.com/zenhack/go-util
Much of what’s in there I hope will someday be obviated by the stdlib, but right now pretty much anything that requires generics to implement doesn’t exist, so…
I didn’t even allow a util directory on my teams. Put the code with the correct domain, if it’s shared, move it up a level and give it’s own domain relative to it’s use. Otherwise it can live in it’s feature package.
I do think this
mailutil
syntax is interesting but it honestly seems like you could then just striputil
from the name and again as stated above apply it to the mail domain (portion of the code).I wish my work codebase was as organized as this. There are almost 300 util classes! I guess it’s time to clean some of those up…
That’s what I made my team do, if someone needed an extra ticket pick up some tech debt clean-up. Before all of that we had a team session working through some of the obvious util code so everyone would know what to do.
“In the end, what defines a module is what pieces of the system it will never responsible for, rather what it is currently responsible for.” —Tef
I am fond of all of that post of Tef’s, but that particular quote has stuck with me the most. As for this post by Brandur, very neat! A good way to prevent omnimodules. In the knapsack it goes, for it will come in handy someday soon.
Can’t find the exact quote, but this reminds me of someone saying that any large enough organisation system without a “miscellaneous” category is useless, because of a massive long tail of single-item categories.
I like the middle ground that accepts it and tries to create some order in “utils”, but not too strictly.
What I landed on, and have seen on many projects, is having
util
packages within other packages. You don’t get a top-levelutil
, bututil
things will be landing places for helper functions that are useful in a specific domain without being “about” something.I’ve never seen these grow too big, as generally there’s a willingness to pull out bigger chunks into their own files just out of pragmatism.
This is why I’ve done too. I move common code within a package down into
util
, which might be exported and if usable documented, orinternal
, which are implementation details and should never be used.A function for checking if a string exists in a slice of strings appears in almost all of my projects. It’s too small and simple to have a separate project and package and is clearer and cleaner as a function than inline. Ideally, Go should have an
in
keyword, like in Python, but for now it lives inutil.go
in several of my projects.util/stringutil.go
might be an improvement, but juststringutil.go
orutil/string.go
would have less repetition of words in the selected path + filename.Go 1.21 will have
slices.Contains(ss, s)
in the stdlib.That’s an improvement! It’s more cumbersome and has a bit more cognitive overhead compared to
in
, though. For example:vs
And:
vs
I guess I just really like the
in
keyword. But thanks for the news about the Go standard library!Yes, Go could be a little shorter. :-) OTOH, I need to use
enumerate
the other day, and I found myself annoyed at how long of a word it is:That’s a good point. Go excels at enumeration, syntax wise. But there is also dictionaries:
vs
An overall win for the
in
keyword, unless list enumeration is the main concern. Alsorange
is a word that does not add anything, when shouting the line out loud, for the code heralds among us.Go’s rule against circular dependencies at the package level makes splitting by dependency particularly important.
If one utility depends on any package in your app and another independently useful utility doesn’t, it’s possible someone will want the no-deps utility but be unable to use it because there’d be a circular dep with the other utility’s dependency. Besides the hard rule, avoiding circularity just tends to make things easier to think about.
I also sort of agree with another comment that I’d locate e.g. email utilities ‘near’ any other email code that lives in your world if possible, e.g.
email/emailutil
oremail
itself instead ofutil/emailutil
. If someone writing future code knows they’re trying to do something with email they know the ‘email’ part for sure, and only might know that the particular thing they want is a ‘util’. However, if your utils are related to code outside your app (stdlib or otherwise third-party), “put the utils with the other code” may just not apply and thestdx
sort of approach seems appealing.This post boils down to “if you’re gonna
util
at least organize it by types of functions.” This makes me kind of sad because the “junk drawer” is still a junk drawer, even if it’s more organized.I wish it were easier to reexport functions and types in Go. In that way, you could define your own standard library and colocate these “utils” in their proper place, along with the curated “best practices” for how to use the underlying stdlib.
At what point has a
util
package ever been a problem worth caring for? It’s no elegant but it’s like the least problematic thing ever. I always end up with a util package, and if I ever need to reorganize it because it gets too big I just do it, and if some parts become big enough that they’d be better extracted I just do it, what’s wrong with that?