The more I think about it, the more I believe it’s really impressive that Go managed to add generics to the base language without needing to jump to 2.0. How long will they continue with 1.x?
Java added generics in a “minor” release… (1.5) and maintained backwards compatibility, too. With the caveat that I’ve never written a useful type system from scratch, I’d posit that the simplicity of Go’s (and Java’s) type systems meant that bolting on generics is not much different than adding, say, a new primitive column based array-struct. That’s not to say it isn’t hard to get exactly right, just that a basic type system’s implementation is mostly a giant switch statement, and generics are just a net new case to deal with.
It’s definitely more complicated than just a new case; the implementation involves the use of unification to infer type parameters, for one.
Generics are a cross-cutting concern that’s more disruptive than many type system features; you have to think about the way it interacts with most things.
Illustratively, the Java developers did mess this up – but before general-purpose generics, when designing the special-purpose generics for arrays that were already in the language. This program compiles:
public class Unsound {
public static void main(String[] args) {
Integer[] ints = new Integer[1];
Object[] objs = ints; // Points to the same backing storage as ints
objs[0] = "oops!"; // Puts a String in your array of Integers!
}
}
The second statement really should be rejected, but isn’t, because in java there is a rule that if A is a subtype of B, then A[] is a subtype of B[] – which works fine for reads, but is broken in the presence of writes.
When this was discovered, to preserve backwards compatibility, the java developers introduced a special purpose exception just for this hole:
Exception in thread "main" java.lang.ArrayStoreException: java.lang.String
at Unsound.main(Unsound.java:5)
By the time they introduced generics proper, they knew about this pitfall and avoided it; the following does not type check:
import java.util.ArrayList;
public class Sound {
public static void main(String[] args) {
ArrayList<Integer> ints = new ArrayList<Integer>(1);
ArrayList<Object> objs = ints; // Points to the same backing array as ints
objs.set(0, "oops!"); // Puts a String in your array of Integers!
}
}
$ javac Sound.java
Sound.java:6: error: incompatible types: ArrayList<Integer> cannot be converted to ArrayList<Object>
ArrayList<Object> objs = ints; // Points to the same backing array as ints
^
1 error
…so this stuff is pretty subtle. But it is also a well-studied problem, and you’re right that starting from a very simple type system like Go’s makes it not really all that hard, if you’re careful and stand on the shoulders of giants where appropriate.
Java’s versioning system is/was weird. Java went from 1.4 to 5, but also kept the old versioning convention. Java 5 was also java 1.5; This convention stuck around until Java 9.
not really weird if you know Sun. One is the internal release number, the other one the marketing release number. They did the same with Solaris: 8 was really 5.8, 9 was 5.9 etc.
Even Microsoft did that with Windows 7, which confusingly is NT 6.1, not 7
Using the new GC memory limit and increasing GOGC to 400 is likely to be a substantial win if you’re running a service in an environment like Kubernetes where you’re the sole user of a fixed-size memory reservation.
And, doing a little bit of optimization for GC friendliness can get you some really substantial tail latency improvements, both before and after 1.19. That means reducing unnecessary heap allocations, reducing the number of pointers in things that will be retained for a long time, and even re-ordering structs to reduce slack. Looking for places where you can replace interfaces with generics can help here too, because an interface is a “box” that contains a pointer to something on the heap; if you eliminate the interface you eliminate the pointer.
On Unix operating systems, Go programs that import package os now automatically increase the open file limit (RLIMIT_NOFILE) to the maximum allowed value; … One impact of this change is that Go programs that in turn execute very old C programs in child processes may run those programs with too high a limit.
So they implemented half of the dance and left the other half as a footgun to users? Why didn’t they just set the limit back down for execed programs for users? Then they wouldn’t have to worry about it. As implemented this is going to be a repeating bug.
It is a stupid dance to maintain compatibility with select but you have to play the part (at least until select is removed from libc).
Does this fix the split DNS issues that exist in Go when using the Go DNS resolver on macOS? It’s the major frustrating part of using Go binaries on macOS.
See also my blog and Go Time.
The more I think about it, the more I believe it’s really impressive that Go managed to add generics to the base language without needing to jump to 2.0. How long will they continue with 1.x?
Java added generics in a “minor” release… (1.5) and maintained backwards compatibility, too. With the caveat that I’ve never written a useful type system from scratch, I’d posit that the simplicity of Go’s (and Java’s) type systems meant that bolting on generics is not much different than adding, say, a new primitive column based array-struct. That’s not to say it isn’t hard to get exactly right, just that a basic type system’s implementation is mostly a giant switch statement, and generics are just a net new case to deal with.
It’s definitely more complicated than just a new case; the implementation involves the use of unification to infer type parameters, for one.
Generics are a cross-cutting concern that’s more disruptive than many type system features; you have to think about the way it interacts with most things.
Illustratively, the Java developers did mess this up – but before general-purpose generics, when designing the special-purpose generics for arrays that were already in the language. This program compiles:
The second statement really should be rejected, but isn’t, because in java there is a rule that if
Ais a subtype ofB, thenA[]is a subtype ofB[]– which works fine for reads, but is broken in the presence of writes.When this was discovered, to preserve backwards compatibility, the java developers introduced a special purpose exception just for this hole:
By the time they introduced generics proper, they knew about this pitfall and avoided it; the following does not type check:
…so this stuff is pretty subtle. But it is also a well-studied problem, and you’re right that starting from a very simple type system like Go’s makes it not really all that hard, if you’re careful and stand on the shoulders of giants where appropriate.
Do you have any links to back this up?
In general, covariant arrays are a trade off, rather than a mistake: you might deliberately include them into the language, like TypeScript does.
The difference in treatment of arrays and generics in Java might be explained via two facts:
I don’t know what’s the story for Java: was it a mistake, or a deliberate design?
EDIT:
ArrayStoreExceptionwas a part of 1.0 specification: http://titanium.cs.berkeley.edu/doc/java-langspec-1.0/10.doc.html#11430You’re right; I was making assumptions about the reasoning process, but I don’t actually know.
The link is a 404.
Java’s versioning system is/was weird. Java went from 1.4 to 5, but also kept the old versioning convention. Java 5 was also java 1.5; This convention stuck around until Java 9.
not really weird if you know Sun. One is the internal release number, the other one the marketing release number. They did the same with Solaris: 8 was really 5.8, 9 was 5.9 etc.
Even Microsoft did that with Windows 7, which confusingly is NT 6.1, not 7
Using the new GC memory limit and increasing GOGC to 400 is likely to be a substantial win if you’re running a service in an environment like Kubernetes where you’re the sole user of a fixed-size memory reservation.
And, doing a little bit of optimization for GC friendliness can get you some really substantial tail latency improvements, both before and after 1.19. That means reducing unnecessary heap allocations, reducing the number of pointers in things that will be retained for a long time, and even re-ordering structs to reduce slack. Looking for places where you can replace interfaces with generics can help here too, because an interface is a “box” that contains a pointer to something on the heap; if you eliminate the interface you eliminate the pointer.
I run a web service on Lambda, and I am interested to see if the memory limit helps.
So they implemented half of the dance and left the other half as a footgun to users? Why didn’t they just set the limit back down for
execed programs for users? Then they wouldn’t have to worry about it. As implemented this is going to be a repeating bug.It is a stupid dance to maintain compatibility with
selectbut you have to play the part (at least until select is removed from libc).Does this fix the split DNS issues that exist in Go when using the Go DNS resolver on macOS? It’s the major frustrating part of using Go binaries on macOS.