Your talks and podcast appearances are both informative and entertaining, and you’ve made me interested in low level topics that I’d otherwise not be exposed to (I’m thinking of your trilogy on BSD Now where you talked about epoll weirdness, if I’m remembering correctly).
I’m not exaggerating when I say that I’m going to watch everything listed here.
Nice timing. I’m on about year 4 (or is it 5?) of Arch + xmonad. I’ve always used Thinkpads, so there haven’t been any hardware support issues, but I do miss the OS X + Macbook battery life and it seems my wifi is always more spotty than coworkers/family using the same networks on a Mac.
I was considering switching back to a Mac recently, hoping that one of the tiling “WMs” on OS X was decent these days. Are they all bad? I assume I’d just ran an Arch VM for actual development needs and use OS X as a “skin with good battery life” + iTerm (to the VM). Am I crazy?
Is the battery life that bad? I guess I can’t compare since I only ever used a 15" MBP, but the batter life on that one is comparable to the one in my 13" Thinkpad.
Though using a VM on OS X might eat more battery than if you just used Linux on the metal.
None of the alternate window management add-ons on OS X are really worth a damn. They really can’t be, because they always end up fighting the platform. This doesn’t bother me, but if you’re looking for something like ion but able to control native windows, you’re going to be disappointed. I use Optimal Layout, which allows me to easily resize windows, but it’s a far cry from when I used to use FreeBSD and X-Windows.
I’ve just started using NixOS on a Macbook Pro at work. I highly recommend using it rather than OS X. I only have two problems for features I don’t use anymore:
No Thunderbolt (I had kernel panics under Linux 3.17 which has Thunderbolt patches) - solution was to swap a Thunderbolt Display for some HDMI display
My OS configuration is specified and declared in files. I can use all of Nix for development, xmonad rather than Amethyst, Docker without boot2docker, ZFS, etc. Definitely worth the switch.
I’d love to hear more about how day-to-day operations under NixOS work for you. Functional management seems like one of the most radical rethinks of “systems” in a long time; but also, unlike many such radical changes, one that could be really broadly useful.
I do a lot of Haskell development at work and instead of using cabal sandboxes I can just use nix-shell, which shares prebuilt common libraries. No more compiling lens for each project.
It’s easy to work on the OS itself, you just clone the repo, make a configuration (e.g. vmtest.nix) and run:
NIXOS_CONFIG=$PWD/vmtest.nix nix-build -A vm nixos
I can make installation scripts, create services and try them out before running them locally. It’s a very nice way to work on any part of the OS.
I’ve put a chunk of my Macbook configuration on GitHub for people to check out:
I worked on osxmonad but backing X11 into Quartz Compositor is pretty bad. I started working on https://github.com/puffnfresh/iridium which abstracts away window management and has a partial Cocoa backend. I want to start an X11 one now.
Amethyst is alright but there’s no scripting, no custom layouts, can be buggy and sometimes just stops working.
It' more like “Why Go has different design principles” rather than why it’s not good. Go’s designed to be simple, practical and enjoyable to program in. Haskell’s designed for theoretical purity and Rust’s designed to do everything you could think of plus a lot more. Each of these approaches has their niche. I use all three for different things.
Go’s designed to be simple, practical and enjoyable to program in.
Unfortunately, the authors of Go have been done an excellent job convincing people that there is a contradiction between a more powerful type system and being simple and practical. Adding basic concepts such as ADTs and generics does not significantly complicate the language or implementation.
Go is really cheating a whole generation of developers out of progress. We already know not having generics sucks. Dropping down to a single interface type and casting back out sucks. Java went through this already and everyone is both happy to be out of it and sad that it has a lasting impact on the language.
Adding basic concepts such as ADTs and generics does not significantly complicate the language or implementation
Can you point to a language with the tiny spec and compositional simplicity of Go that also has an advanced (Milner-Hindley or otherwise) type system and generics?
We already know not having generics sucks. Dropping down to a single interface type and casting back out sucks.
Like the overwhelming majority of people who criticize Go’s lack of generics, you seem to operate from the position that generic programming is programming, programming qua programming, and you move forward from there and assume that Go programs are full of unsafe interface{} type assertions, and that Go consequently sucks.
But—and I can’t say I understand why this is so difficult to digest—but that’s not how Go works. Idiomatic, performant Go doesn’t leverage generic programming techniques in the way that C++ programmers (for example) might expect it to. Go’s “version” of generic programming is behavior-based functional composition: programming with real interfaces. I think you can successfully argue it’s less powerful than generic programming, for certain definitions of powerful, but I think you can’t argue it’s a regression in expressivity, productivity, or safety.
Go likely requires you to unlearn your default techniques, and saturate yourself in a different set of idioms and paradigms. It’s equally a challenge for folks used to strict OO, to FP folks, to folks used to expressing their domain invariants with a type system. Go doesn’t walk any of these lines deliberately; it has made a deliberate choice to provide fewer features in service of a coherent composite whole, viz. less is exponentially more.
Can you point to a language with the tiny spec and compositional simplicity of Go that also has an advanced (Milner-Hindley or otherwise) type system and generics?
I spend most of my time in Ocaml and find its spec to be quite readable. Of course, my statement hinges on what the reader believes “significantly” means.
you seem to operate from the position that generic programming is programming, programming qua programming, and you move forward from there and assume that Go programs are full of unsafe interface{} type assertions
Actually, no, I don’t believe this. In the future I would appreciate if you would ask me for clarification on my position rather than making assumptions. And please drop the condescending attitude.
My statement was simply this: proponents of Go, and specifically its lack of generics, often sight that implementing them well in the type system and/or runtime is somehow an unsolved mystery and Go is doing us a favor explicitly deciding to not be innovative. I contend this is false. There are multiple runtimes that implement generics efficiently. One does not need a H-M type system to get them either. I think the claims that yourself, and Rob Pike, make in the link you posted are either ignorant or disingenuous.
I especially think Rob Pike’s post is misleading. The argument effectively comes down to C++ vs Go. I would choose Go over C++ any day of the week. But he leaves out the middle ground languages which are rather simple but still contain powerful and expressive type systems (SML, F#, and Ocaml come to mind). He presents a false dichotomy and of course his solution wins, the odds are in his favor.
but I think you can’t argue it’s a regression in expressivity, productivity, or safety.
I have no opinion on the impact on productivity but it is a fact that the lack of a type system capable of expressing parametric polymorphism is less expressive and less safe. Having a single interface{} cast is, by definition, less safe. In practice it may be irrelevant but that is a different argument. Perhaps you’re right, and it doesn’t matter. I think that’s a shame because having parametric polymorphism in your pocket is very powerful tool, I use it almost every day. And in my opinion it simplifies code considerably. I think powerful collection libraries takes a few years for communities to decide are important so perhaps we’ll see that in Go in the future, or maybe I’m completely wrong and nobody will care. If I were a betting man I would put money on that Go will eventually get generics. Languages simply move in that direction. Even C has a weak form of generic macros now.
proponents of Go, and specifically its lack of generics, often sight that implementing them well in the type system and/or runtime is somehow an unsolved mystery and Go is doing us a favor explicitly deciding to not be innovative.
Nobody cites this at all. The Go team, and everyone active in the language and ecosystem, go out of their way to make clear at every opportunity that generics would be welcomed, as soon as a suitable implementation is proposed: one which doesn’t complect other dimensions of the language, which composes with existing idioms, which brings overall value to the language, beyond of the value of generics in and of themselves. (That’s the standard that all features in Go must meet, mind.)
If I were a betting man I would put money on that Go will eventually get generics.
Consequently re: the above, I agree.
Having a single interface{} cast is, by definition, less safe.
I agree. If that were how Go programs were structured, your point would carry weight. Since it isn't—since Go programs don’t idiomatically rely on interface{} type assertions to accomplish goals—the point is pragmatically moot.
IMO, what you said is just a longer rephrasing of what I said. You said that the Go team does not want them until the complexity vs value trade off is better. My point is: it’s already really good and this claim that it isn’t is ridiculous. We know how to build decent type systems with generics that would fit into Go’s model. Please provide explicit technical details as to what is lacking.
Since it isn't—since Go programs don’t idiomatically rely on interface{} type assertions to accomplish goals—the point is pragmatically moot
I downloaded the top 8 repositories from the Trending Repositories[1] on Github. They were: cayley, docker, gin, gwitter, peco, pongo2, ui.
I then issued the following query: grep -R 'interface{}' $repo | grep -v test | wc -l
A crude query, for sure.
Of the results, 7 of the 8 repos have interface{} in it (peco). cayley and docker have the most, they are also the largest projects. In total I had 268 lines with interface{} in them across all 7 repos that had hits. The largest projects also have more lines with interface{}. I suspect that is as a project grows the need for code reuse increases.
We know how to build decent type systems with generics that would fit into Go’s model. Please provide explicit technical details as to what is lacking.
On the contrary: nobody, to the best of my knowledge, has proposed a generics implementation that would fit nicely into Go’s model. Please provide an explicit reference to such a proposal.
(An implementation is not a high-level description of a generics system, abstract from the host language spec. It’s a specific, detailed description of a generics system, including all of its interaction points with the host language.)
I downloaded the top 8 repositories from the Trending Repositories[1] on Github.
All of those were recent HN clickbait. None of them are particularly good Go. And the presence of interface{} as a string in the source is totally independent of its use in the API of a module or package.
I suspect that is as a project grows the need for code reuse increases.
Sure. But interface{} as a generic type specifier is not an idiomatic mechanism for code reuse in Go. I don’t know how many times I need to say this.
All of those were recent HN clickbait. None of them are particularly good Go.
This amounts to a No True Scotsman. And even if it is true, your response lacks evidence.
Your precise statement was:
Since it isn't—since Go programs don’t idiomatically rely on interface{} type assertions to accomplish goals—the point is pragmatically moot.
You said nothing about APIs. And it’s unclear to me if being an API function or not is actually a useful distinction to make in this conversation. A bug is a bug, my claim is parametric polymorphism reduces a class of bugs with little expense to the programming, language, or implementation, my evidence being implementations for languages such as SML, Ocaml, Java, C# and Ada. Note: I am not saying Go needs to look like these languages, I am saying these languages demonstrate efficient runtimes with generics.
proponents of Go, and specifically its lack of generics, often sight that implementing them well in the type system and/or runtime is somehow an unsolved mystery and Go is doing us a favor explicitly deciding to not be innovative.
Nobody has said implementing generics is mysterious. People have said that implementing generics with the right trade offs is difficult.
Do you acknowledge that implementing generics of some sort is not free?
If so, could you please articulate what “not free” means? What are the costs?
Given those costs, can you imagine any reasonable person unwilling to pay them for some variety of use cases?
If I were a betting man I would put money on that Go will eventually get generics. Languages simply move in that direction.
One key difference that Go detractors often forget about is that there is some amount of blessed parametric polymorphism in the language. (The only time this is brought up by detractors is to poo-poo it because they feel cheated.) But it’s just another trade off. One can get a lot of mileage out of generic slice and map types (and their associated parametric functions built into the language). It’s not a pretty solution, but it very well could be the difference maker.
Consider, for example, that pre-generics Java was painful enough to write that the designers decided to add generics to the language. What does “painful enough” mean? There’s a line somewhere and methinks Go is in uncharted territory.
But he leaves out the middle ground languages which are rather simple but still contain powerful and expressive type systems (SML, F#, and Ocaml come to mind).
Standard ML is effectively dead. F# is effectively Microsoft only. Ocaml is an interesting choice considering its compiler is so slow (as is mlton for SML). The entire reason Go was even invented was because the creators were getting sick of compile times.
Are those languages options? I guess in some technical sense they probably are. But they’re also easily dismissed given the context. (And that’s not to say that I don’t like those languages. I <3 SML. It’s such a simple language.)
Nobody has said implementing generics is mysterious. People have said that implementing generics with the right trade offs is difficult.
I claim this is false. My evidence is the slew of languages with generics.
Do you acknowledge that implementing generics of some sort is not free?
I contend that not having generics is more expensive. And just to be clear, I am not talking about C++ style generics, I apologize if I should have stated this earlier, but I am really talking about parametric polymorphism.
peterbourgon argues here that if you need a linked list, you’re better off reimplementing the algorithm yourself. If you agree with this perspective then we are unfortunately going to just be talking past each other forever.
Standard ML is effectively dead. F# is effectively Microsoft only. Ocaml is an interesting choice considering its compiler is so slow (as is mlton for SML).
I was unclear in my phrasing: I was not arguing that Go should not exist and we should use one of those languages instead. I was arguing that those languages represent, what I consider to be, a more appropriate line between simplicity and expressiveness. Giving significantly more bang for the buck. You even state SML is a simple language.
Also, the ocaml compiler is actually quite fast, I’m not sure why you think it is otherwise. It is slower than Go’s but if that is the only acceptable speed then there isn’t much to talk about.
I claim this is false. My evidence is the slew of languages with generics.
According to you, what are the languages that have an efficient implementation of generics and that could be used as an inspiration for implementing generics in Go?
By “efficient implementation”, I mean:
Avoid code bloat caused by an exaggerated code specialization (the compiler generates one version of each function for each possible parametric type) which is bad for CPU cache (this is often an issue with C++ templates)
Avoid slow execution by having only one version of each function with everything passed as a pointer, even when the value fits in a 64 bit register (this is often an issue in Java)
Avoid slow compilation (is it possible to compile a package containing generic function without already knowing the types that will be passed to this function?)
Avoid making the language specification too much complex (type constraints, rules for type inference which is almost necessary to make generics usable, covariance and contravariance – https://blogs.janestreet.com/a-and-a/)
I claim this is false. My evidence is the slew of languages with generics.
That what is false? Existence doesn’t mean anything. The Go people want an implementation that meets their criteria, which includes keeping their compilers fast and their language simple.
I contend that not having generics is more expensive.
Is it? I don’t know. I’ve written a lot of Go and a lot of Haskell and SML and Rust, and when I need an abstract data type in Go, yes it is annoying. Nobody is saying otherwise. How much annoyance are you willing to put up with?
I contend that it is not that much in the Go world, probably because blessed generic types cover a fairly large number of use cases in practice.
I don’t care about C++.
peterbourgon argues here that if you need a linked list, you’re better off reimplementing the algorithm yourself. If you agree with this perspective then we are unfortunately going to just be talking past each other forever.
It depends on what your trade offs are. I don’t think there is one right solution for all cases in Go programs. It is true that writing a generic linked list in Go without sacrificing type safety and performance is basically impossible.
And what is this supposed to demonstrate? That writing Go programs is annoying? It certainly would be if you needed a performant generic linked list everywhere and were unwilling to sacrifice type safety. If that’s the case, then Go simply does not meet your requirements.
I was arguing that those languages represent, what I consider to be, a more appropriate line between simplicity and expressiveness.
Then just say that: “For my particular tastes, Go’s type system is not expressive enough.” That’s cool and fine by me. Not everyone’s tastes are the same! (That’s precisely my point.)
You even state SML is a simple language.
So what? Is it some kind of ultimate paradox to think that both Go and SML are simple languages?
Also, the ocaml compiler is actually quite fast, I’m not sure why you think it is otherwise. It is slower than Go’s but if that is the only acceptable speed then there isn’t much to talk about.
It’s not just slower than Go’s. It’s much slower than Go’s. All compilers for sophisticated type systems that I’ve used are much slower than Go’s (ghc, rustc, g++, mlton to name a few).
The entire reason Go exists was because of slow compile times. So I have no idea why you think Rob Pike should be proposing languages with very slow compilers (by comparison).
Go’s “version” of generic programming is behavior-based functional composition: programming with real interfaces.
I’m curious about this statement. In your opinion, is there a more better way in Go to write the LinkedList example from the OP? (genuinely curious here, not trying to attack Go, which I don’t really know anything about)
is there a more better way in Go to write the LinkedList example from the OP?
Almost certainly, you’d never see the code the OP produced in a typical/idiomatic Go program. (The presence of interface{} in an API is almost always an immediate bad smell and red flag.)
Rather, Go would respond by challenging the assumptions the OP baked into his example. One (typically) doesn’t ever write a linked list for the sake of writing a linked list. You use a linked list, in the service of some other goal. So let’s talk about that goal. What’s the actual value type? What API do you actually need, for a collection of those value types? Implement that: a specific expression of your concrete problem, rather than a generic expression of a class of problems that resembles the one you have.
What if I just want an IntMap (a map optimized for integer keys), am I supposed to reimplement it every time I want to store a different type of value in it? What’s the golang way to implement something that’s actually generic like that?
What if I just want an IntMap (a map optimized for integer keys), am I supposed to reimplement it every time I want to store a different type of value in it?
Your suggestion here is literally to keep rewriting a linked list implementation for every concrete type that you want in a linked list. This is the exact problem that generics (or parametric polymorphism) solves without the code duplication.
Your suggestion here is literally to keep rewriting a linked list implementation for every concrete type that you want in a linked list.
My suggestion is that “linked list” is a means, not an end, and that you should think about what semantics you actually need out of your container API, what performance characteristics for each operation, and so on, and implement exactly that, for your concrete data types. If you need exactly the same linked list API for N concrete data types, yes, it’s N concrete implementations (in the base case). But, in my experience, that’s so rare a situation in the development of an actual service or application as to not really be worth considering.
Go definitely takes the opinionated stance that code duplication is not a fundamental evil, and that DRY and its consequents (e.g. [premature] abstraction) are overused and overvalued. Idiomatic Go probably wouldn’t generalize a block of code until it’s used in several places, where several > 2, and until that generalization yields some other benefits beyond simply eliminating SLOC.
My suggestion is that “linked list” is a means, not an end, and that you should think about what semantics you actually need out of your container API, what performance characteristics for each operation, and so on, and implement exactly that, for your concrete data types.
The last 60 years of research in containers has demonstrated that:
Containers are generally more complicated than the values they contain.
Performance characteristics generally depend very little on what they contain, but rather the semantics of how one wants to use it.
Given that, your suggestion to reimplement container types every time sounds absolutely ridiculous to me. Even the authors of Go disagree with this to some degree, providing a few generic containers implemented in the language.
If your argument is that having to reimplement containers all the time sucks but Go simply does not need interesting containers, then that is completely different and I have no opinion on that.
The added file/daemon system looks like an eventually consistent replication wrapper around zookeeper, which they’ve added as a way to get higher availability. The consistency of the composite system (zk+wrapper) is only eventual (unlike the zookeeper subsystem) because the cache files won’t be updated and may be out of sync with each other when zk is down.
If this is right, why not just replace zookeeper with a system that was designed to be eventually consistent and highly available (riak, maybe), and not have to worry about the possible bugs introduced by the new replication system: the daemons and the client libs for reading the files? Does zk still have an advantage if you are accessing it through this layer?
The corruption safeguards are another matter. Those are probably worthwhile regardless of the choice of distributed data store. Human error being what it is…
The only thing I can think of is ZK’s support for watching Znodes. Their daemon could keep the conf file updated immediately upon changes when ZK is up. Other systems would generally require polling to attempt the same.
That said, Serf is an eventually consistent system that supports “run this thing when key X changes” (iirc), so that may be a good fit, like you said.
Good call about serf. Pinterest have engineered their system to tolerate temporary and intermittent inconsistency in things like lists of nodes and services, so they could choose AP rather than CP. Plus (I’m guessing) the gossip protocol might be more efficient than updating those files and daemons from the ZK cluster. Plus, “5 to 10 MB of resident memory” per process (serf, Go) vs jvm.
I’ve been starting to dive into Cassandra in the past couple weeks so this is aptly timed.
My question:
How can you provide ordering using Cassandra if wall clocks aren’t trustworthy? I understand how to do this with vclocks but don’t know how to get started building a system that needs ordering and convergent data structures while using Cassandra.
A simple example is using a unique column for each write. Imagine a row “user:brett:friends” and I do two writes from two servers.
// Bad way:
write1: "user:brett:friends" set column "data" to "[5]"
write2: "user:brett:friends" set column "data" to "[7]"
read1: "user:brett:friends" get column "data"
// Clearly only one of those writes will win, so you'll get [5] or [7]
// Better:
write1: "user:brett:friends" set column "68e0f266-89c1-11e3-96fa-cd5de4cf87ee" to "5"
write2: "user:brett:friends" set column "723ba1a8-89c1-11e3-94ad-82121e36f60d" to "7"
read1: "user:brett:friends" get all columns, treat items like siblings and do merge/dupe handling in your app
// Final result will be [5, 7]
Also, Cassandra 2.0 comes with it’s own consensus implementation so you can actually hold a lock on a key to do a write without having to the UUID dance. I can’t elaborate much because I don’t use it (almost all of my data is event logs so TimeUUID for columns is all I need).
Would that work? Wouldn’t you need to call git-pushall not git pushall? Because at least for me, I don’t have a git-push or git-pull command, which makes me think they’re just nice ways to separate sections of the manual.
You could write an alias, which would let you use git pushall:
It works, give it a try. The main git binary will look for a git-FOO executable for you.
Because at least for me, I don’t have a git-push or git-pull command
You actually do, it’s just not on your standard path. git knows where to look for it, on my system for example there is /usr/lib/git-core/git-pull and that git-core dir is full of the same.
I’d just like to say thank you.
Your talks and podcast appearances are both informative and entertaining, and you’ve made me interested in low level topics that I’d otherwise not be exposed to (I’m thinking of your trilogy on BSD Now where you talked about epoll weirdness, if I’m remembering correctly).
I’m not exaggerating when I say that I’m going to watch everything listed here.
Could you link to (some or all) of these podcast appearances you talk about? I’d love to listen.
Here’s some BSD Now episodes: https://www.youtube.com/results?search_query=bsdnow+cantrill
The BSD Now episodes (a subset of @bretthoerner’s link):
The three episodes are available as one merged episode:
There are links to audio versions and RSS feeds above the show notes at the bottom of the page.
Nice timing. I’m on about year 4 (or is it 5?) of Arch + xmonad. I’ve always used Thinkpads, so there haven’t been any hardware support issues, but I do miss the OS X + Macbook battery life and it seems my wifi is always more spotty than coworkers/family using the same networks on a Mac.
I was considering switching back to a Mac recently, hoping that one of the tiling “WMs” on OS X was decent these days. Are they all bad? I assume I’d just ran an Arch VM for actual development needs and use OS X as a “skin with good battery life” + iTerm (to the VM). Am I crazy?
Yeah, I just tried Yosemite on my Macbook Pro before reinstalling Ubuntu+xmonad. They’re all bad. Mostly don’t even really work properly.
Is the battery life that bad? I guess I can’t compare since I only ever used a 15" MBP, but the batter life on that one is comparable to the one in my 13" Thinkpad.
Though using a VM on OS X might eat more battery than if you just used Linux on the metal.
None of the alternate window management add-ons on OS X are really worth a damn. They really can’t be, because they always end up fighting the platform. This doesn’t bother me, but if you’re looking for something like ion but able to control native windows, you’re going to be disappointed. I use Optimal Layout, which allows me to easily resize windows, but it’s a far cry from when I used to use FreeBSD and X-Windows.
I’ve just started using NixOS on a Macbook Pro at work. I highly recommend using it rather than OS X. I only have two problems for features I don’t use anymore:
My OS configuration is specified and declared in files. I can use all of Nix for development, xmonad rather than Amethyst, Docker without boot2docker, ZFS, etc. Definitely worth the switch.
I’d love to hear more about how day-to-day operations under NixOS work for you. Functional management seems like one of the most radical rethinks of “systems” in a long time; but also, unlike many such radical changes, one that could be really broadly useful.
I do a lot of Haskell development at work and instead of using cabal sandboxes I can just use nix-shell, which shares prebuilt common libraries. No more compiling lens for each project.
It’s easy to work on the OS itself, you just clone the repo, make a configuration (e.g. vmtest.nix) and run:
I can make installation scripts, create services and try them out before running them locally. It’s a very nice way to work on any part of the OS.
I’ve put a chunk of my Macbook configuration on GitHub for people to check out:
https://github.com/puffnfresh/nix-files/blob/master/configuration.nix
Is Amethyst at least OK? I guess you’ve moved o NixOS, but I thought you were working on xmonad for OS X or something like that?
I worked on osxmonad but backing X11 into Quartz Compositor is pretty bad. I started working on https://github.com/puffnfresh/iridium which abstracts away window management and has a partial Cocoa backend. I want to start an X11 one now.
Amethyst is alright but there’s no scripting, no custom layouts, can be buggy and sometimes just stops working.
It' more like “Why Go has different design principles” rather than why it’s not good. Go’s designed to be simple, practical and enjoyable to program in. Haskell’s designed for theoretical purity and Rust’s designed to do everything you could think of plus a lot more. Each of these approaches has their niche. I use all three for different things.
Unfortunately, the authors of Go have been done an excellent job convincing people that there is a contradiction between a more powerful type system and being simple and practical. Adding basic concepts such as ADTs and generics does not significantly complicate the language or implementation.
Go is really cheating a whole generation of developers out of progress. We already know not having generics sucks. Dropping down to a single interface type and casting back out sucks. Java went through this already and everyone is both happy to be out of it and sad that it has a lasting impact on the language.
Can you point to a language with the tiny spec and compositional simplicity of Go that also has an advanced (Milner-Hindley or otherwise) type system and generics?
Like the overwhelming majority of people who criticize Go’s lack of generics, you seem to operate from the position that generic programming is programming, programming qua programming, and you move forward from there and assume that Go programs are full of unsafe
interface{}type assertions, and that Go consequently sucks.But—and I can’t say I understand why this is so difficult to digest—but that’s not how Go works. Idiomatic, performant Go doesn’t leverage generic programming techniques in the way that C++ programmers (for example) might expect it to. Go’s “version” of generic programming is behavior-based functional composition: programming with real interfaces. I think you can successfully argue it’s less powerful than generic programming, for certain definitions of powerful, but I think you can’t argue it’s a regression in expressivity, productivity, or safety.
Go likely requires you to unlearn your default techniques, and saturate yourself in a different set of idioms and paradigms. It’s equally a challenge for folks used to strict OO, to FP folks, to folks used to expressing their domain invariants with a type system. Go doesn’t walk any of these lines deliberately; it has made a deliberate choice to provide fewer features in service of a coherent composite whole, viz. less is exponentially more.
I spend most of my time in Ocaml and find its spec to be quite readable. Of course, my statement hinges on what the reader believes “significantly” means.
Actually, no, I don’t believe this. In the future I would appreciate if you would ask me for clarification on my position rather than making assumptions. And please drop the condescending attitude.
My statement was simply this: proponents of Go, and specifically its lack of generics, often sight that implementing them well in the type system and/or runtime is somehow an unsolved mystery and Go is doing us a favor explicitly deciding to not be innovative. I contend this is false. There are multiple runtimes that implement generics efficiently. One does not need a H-M type system to get them either. I think the claims that yourself, and Rob Pike, make in the link you posted are either ignorant or disingenuous.
I especially think Rob Pike’s post is misleading. The argument effectively comes down to C++ vs Go. I would choose Go over C++ any day of the week. But he leaves out the middle ground languages which are rather simple but still contain powerful and expressive type systems (SML, F#, and Ocaml come to mind). He presents a false dichotomy and of course his solution wins, the odds are in his favor.
I have no opinion on the impact on productivity but it is a fact that the lack of a type system capable of expressing parametric polymorphism is less expressive and less safe. Having a single
interface{}cast is, by definition, less safe. In practice it may be irrelevant but that is a different argument. Perhaps you’re right, and it doesn’t matter. I think that’s a shame because having parametric polymorphism in your pocket is very powerful tool, I use it almost every day. And in my opinion it simplifies code considerably. I think powerful collection libraries takes a few years for communities to decide are important so perhaps we’ll see that in Go in the future, or maybe I’m completely wrong and nobody will care. If I were a betting man I would put money on that Go will eventually get generics. Languages simply move in that direction. Even C has a weak form of generic macros now.Nobody cites this at all. The Go team, and everyone active in the language and ecosystem, go out of their way to make clear at every opportunity that generics would be welcomed, as soon as a suitable implementation is proposed: one which doesn’t complect other dimensions of the language, which composes with existing idioms, which brings overall value to the language, beyond of the value of generics in and of themselves. (That’s the standard that all features in Go must meet, mind.)
Consequently re: the above, I agree.
I agree. If that were how Go programs were structured, your point would carry weight. Since it isn't—since Go programs don’t idiomatically rely on
interface{}type assertions to accomplish goals—the point is pragmatically moot.IMO, what you said is just a longer rephrasing of what I said. You said that the Go team does not want them until the complexity vs value trade off is better. My point is: it’s already really good and this claim that it isn’t is ridiculous. We know how to build decent type systems with generics that would fit into Go’s model. Please provide explicit technical details as to what is lacking.
I downloaded the top 8 repositories from the Trending Repositories[1] on Github. They were: cayley, docker, gin, gwitter, peco, pongo2, ui.
I then issued the following query:
grep -R 'interface{}' $repo | grep -v test | wc -lA crude query, for sure.
Of the results, 7 of the 8 repos have
interface{}in it (peco). cayley and docker have the most, they are also the largest projects. In total I had 268 lines withinterface{}in them across all 7 repos that had hits. The largest projects also have more lines withinterface{}. I suspect that is as a project grows the need for code reuse increases.Can please explain this? What am I missing?
On the contrary: nobody, to the best of my knowledge, has proposed a generics implementation that would fit nicely into Go’s model. Please provide an explicit reference to such a proposal.
(An implementation is not a high-level description of a generics system, abstract from the host language spec. It’s a specific, detailed description of a generics system, including all of its interaction points with the host language.)
All of those were recent HN clickbait. None of them are particularly good Go. And the presence of
interface{}as a string in the source is totally independent of its use in the API of a module or package.Sure. But
interface{}as a generic type specifier is not an idiomatic mechanism for code reuse in Go. I don’t know how many times I need to say this.This amounts to a No True Scotsman. And even if it is true, your response lacks evidence.
Your precise statement was:
You said nothing about APIs. And it’s unclear to me if being an API function or not is actually a useful distinction to make in this conversation. A bug is a bug, my claim is parametric polymorphism reduces a class of bugs with little expense to the programming, language, or implementation, my evidence being implementations for languages such as SML, Ocaml, Java, C# and Ada. Note: I am not saying Go needs to look like these languages, I am saying these languages demonstrate efficient runtimes with generics.
Nobody has said implementing generics is mysterious. People have said that implementing generics with the right trade offs is difficult.
Do you acknowledge that implementing generics of some sort is not free?
If so, could you please articulate what “not free” means? What are the costs?
Given those costs, can you imagine any reasonable person unwilling to pay them for some variety of use cases?
One key difference that Go detractors often forget about is that there is some amount of blessed parametric polymorphism in the language. (The only time this is brought up by detractors is to poo-poo it because they feel cheated.) But it’s just another trade off. One can get a lot of mileage out of generic slice and map types (and their associated parametric functions built into the language). It’s not a pretty solution, but it very well could be the difference maker.
Consider, for example, that pre-generics Java was painful enough to write that the designers decided to add generics to the language. What does “painful enough” mean? There’s a line somewhere and methinks Go is in uncharted territory.
Standard ML is effectively dead. F# is effectively Microsoft only. Ocaml is an interesting choice considering its compiler is so slow (as is
mltonfor SML). The entire reason Go was even invented was because the creators were getting sick of compile times.Are those languages options? I guess in some technical sense they probably are. But they’re also easily dismissed given the context. (And that’s not to say that I don’t like those languages. I <3 SML. It’s such a simple language.)
Don’t let that stop you: SML# Unless you mean dead as it has a very small community
I claim this is false. My evidence is the slew of languages with generics.
I contend that not having generics is more expensive. And just to be clear, I am not talking about C++ style generics, I apologize if I should have stated this earlier, but I am really talking about parametric polymorphism.
peterbourgon argues here that if you need a linked list, you’re better off reimplementing the algorithm yourself. If you agree with this perspective then we are unfortunately going to just be talking past each other forever.
I was unclear in my phrasing: I was not arguing that Go should not exist and we should use one of those languages instead. I was arguing that those languages represent, what I consider to be, a more appropriate line between simplicity and expressiveness. Giving significantly more bang for the buck. You even state SML is a simple language.
Also, the ocaml compiler is actually quite fast, I’m not sure why you think it is otherwise. It is slower than Go’s but if that is the only acceptable speed then there isn’t much to talk about.
According to you, what are the languages that have an efficient implementation of generics and that could be used as an inspiration for implementing generics in Go?
By “efficient implementation”, I mean:
That what is false? Existence doesn’t mean anything. The Go people want an implementation that meets their criteria, which includes keeping their compilers fast and their language simple.
Is it? I don’t know. I’ve written a lot of Go and a lot of Haskell and SML and Rust, and when I need an abstract data type in Go, yes it is annoying. Nobody is saying otherwise. How much annoyance are you willing to put up with?
I contend that it is not that much in the Go world, probably because blessed generic types cover a fairly large number of use cases in practice.
I don’t care about C++.
It depends on what your trade offs are. I don’t think there is one right solution for all cases in Go programs. It is true that writing a generic linked list in Go without sacrificing type safety and performance is basically impossible.
And what is this supposed to demonstrate? That writing Go programs is annoying? It certainly would be if you needed a performant generic linked list everywhere and were unwilling to sacrifice type safety. If that’s the case, then Go simply does not meet your requirements.
Then just say that: “For my particular tastes, Go’s type system is not expressive enough.” That’s cool and fine by me. Not everyone’s tastes are the same! (That’s precisely my point.)
So what? Is it some kind of ultimate paradox to think that both Go and SML are simple languages?
It’s not just slower than Go’s. It’s much slower than Go’s. All compilers for sophisticated type systems that I’ve used are much slower than Go’s (
ghc,rustc,g++,mltonto name a few).The entire reason Go exists was because of slow compile times. So I have no idea why you think Rob Pike should be proposing languages with very slow compilers (by comparison).
Source? ocamlc compiles really quickly, and I don’t notice ocamlopt being much slower than other compilers I use (granted I use mostly javac and gcc).
We’re talking about Go here. Ocaml’s compiler is much much slower compared to Go’s.
And yes, it’s slower for very good reasons. And that’s exactly my point.
I’m curious about this statement. In your opinion, is there a more better way in Go to write the LinkedList example from the OP? (genuinely curious here, not trying to attack Go, which I don’t really know anything about)
Almost certainly, you’d never see the code the OP produced in a typical/idiomatic Go program. (The presence of
interface{}in an API is almost always an immediate bad smell and red flag.)Rather, Go would respond by challenging the assumptions the OP baked into his example. One (typically) doesn’t ever write a linked list for the sake of writing a linked list. You use a linked list, in the service of some other goal. So let’s talk about that goal. What’s the actual value type? What API do you actually need, for a collection of those value types? Implement that: a specific expression of your concrete problem, rather than a generic expression of a class of problems that resembles the one you have.
What if I just want an IntMap (a map optimized for integer keys), am I supposed to reimplement it every time I want to store a different type of value in it? What’s the golang way to implement something that’s actually generic like that?
I’m not sure to understand your question. If you want to implement a map optimized for integer keys, then it’s not generic by definition.
It’s generic with respect to what you put in it.
Yes.
Your suggestion here is literally to keep rewriting a linked list implementation for every concrete type that you want in a linked list. This is the exact problem that generics (or parametric polymorphism) solves without the code duplication.
My suggestion is that “linked list” is a means, not an end, and that you should think about what semantics you actually need out of your container API, what performance characteristics for each operation, and so on, and implement exactly that, for your concrete data types. If you need exactly the same linked list API for N concrete data types, yes, it’s N concrete implementations (in the base case). But, in my experience, that’s so rare a situation in the development of an actual service or application as to not really be worth considering.
Go definitely takes the opinionated stance that code duplication is not a fundamental evil, and that DRY and its consequents (e.g. [premature] abstraction) are overused and overvalued. Idiomatic Go probably wouldn’t generalize a block of code until it’s used in several places, where several > 2, and until that generalization yields some other benefits beyond simply eliminating SLOC.
The last 60 years of research in containers has demonstrated that:
Given that, your suggestion to reimplement container types every time sounds absolutely ridiculous to me. Even the authors of Go disagree with this to some degree, providing a few generic containers implemented in the language.
If your argument is that having to reimplement containers all the time sucks but Go simply does not need interesting containers, then that is completely different and I have no opinion on that.
Interesting, thanks!
Can you explain why you think this?
Rust is a very feature-rich language. I use it when I need those kinds of features.
The added file/daemon system looks like an eventually consistent replication wrapper around zookeeper, which they’ve added as a way to get higher availability. The consistency of the composite system (zk+wrapper) is only eventual (unlike the zookeeper subsystem) because the cache files won’t be updated and may be out of sync with each other when zk is down.
If this is right, why not just replace zookeeper with a system that was designed to be eventually consistent and highly available (riak, maybe), and not have to worry about the possible bugs introduced by the new replication system: the daemons and the client libs for reading the files? Does zk still have an advantage if you are accessing it through this layer?
The corruption safeguards are another matter. Those are probably worthwhile regardless of the choice of distributed data store. Human error being what it is…
The only thing I can think of is ZK’s support for watching Znodes. Their daemon could keep the conf file updated immediately upon changes when ZK is up. Other systems would generally require polling to attempt the same.
That said, Serf is an eventually consistent system that supports “run this thing when key X changes” (iirc), so that may be a good fit, like you said.
Good call about serf. Pinterest have engineered their system to tolerate temporary and intermittent inconsistency in things like lists of nodes and services, so they could choose AP rather than CP. Plus (I’m guessing) the gossip protocol might be more efficient than updating those files and daemons from the ZK cluster. Plus, “5 to 10 MB of resident memory” per process (serf, Go) vs jvm.
I’ve been starting to dive into Cassandra in the past couple weeks so this is aptly timed.
My question:
How can you provide ordering using Cassandra if wall clocks aren’t trustworthy? I understand how to do this with vclocks but don’t know how to get started building a system that needs ordering and convergent data structures while using Cassandra.
A simple example is using a unique column for each write. Imagine a row “user:brett:friends” and I do two writes from two servers.
Also, Cassandra 2.0 comes with it’s own consensus implementation so you can actually hold a lock on a key to do a write without having to the UUID dance. I can’t elaborate much because I don’t use it (almost all of my data is event logs so TimeUUID for columns is all I need).
Thanks. That is generally in line with what I expected.
I use this (in my ~/.gitconfig) to prevent myself having to write out the remote names:
If you named it “git-pushall” or something then you could use “git pushall” like any other git command, fwiw.
Would that work? Wouldn’t you need to call
git-pushallnotgit pushall? Because at least for me, I don’t have agit-pushorgit-pullcommand, which makes me think they’re just nice ways to separate sections of the manual.You could write an alias, which would let you use
git pushall:It works, give it a try. The main
gitbinary will look for agit-FOOexecutable for you.You actually do, it’s just not on your standard path.
gitknows where to look for it, on my system for example there is/usr/lib/git-core/git-pulland thatgit-coredir is full of the same.