These kinds of ultra-strong backwards-compatibility guarantees for programming languages and their standard libraries tend to always lead to the same few outcomes (or some combination of these outcomes):
The compatibility guarantee devolves into arguments from upstream about how “well, technically…” they didn’t break compatibility even though your code got broken, or
The language accumulates huge amounts of “here be dragons” cruft where APIs that are actively bad and wrong and harmful to use are still kept around, and everyone has to just learn lots of “oh, don’t use that, or that, or especially that, and whatever you do don’t use this…”, or
They give up and relax the backwards-compatibility policy.
So far, Go seems to do a fair amount of (1) and probably is getting to the age where (2) will be a significant factor soon.
The language accumulates huge amounts of “here be dragons” cruft where APIs that are actively bad and wrong and harmful to use are still kept around,
I’ve been thinking about this recently, and it seems that the practical perception of this problem is too much influenced by Python. Python is always touted as an example why batteries included might go wrong with time, but I think Python’s stdlib was unique randomly assembled in the first place. Like, stdlib modules like unittest don’t follow language’s naming convention! This has nothing to do with compatibility, and everything to do with the practice of deciding what goes into stdlib. I don’t know how that was decided in Python, but I have a strong suspicion that “anything goes” was pretty close to the truth.
Now, when I update back from that example, I sort-of feel that a properly curated stdlib would age well? If you are careful with what you accept, it shouldn’t be too hard to maintain compat, and it’s unlikely that things go sour with time.
I predict that in 2032 Go stdlib would be fine (I don’t know how good it actually is today, but, given that Deno modeled its API on Go, I have a high prior that it is), and in 2035 Rust stdlib would be fine, and in 2035 people would still be joking about unittest naming convention, heapq procedural APIs and three argument parsers in Python’s stdlib.
My go-to example of a “here be dragons” stdlib is actually Java.
Python pared down some of the older stuff in the 2->3 transition, and has a plan in place for how to gradually deprecate and remove things from its stdlib. But Java… the list of “don’t use that, even though it’s the most immediately obvious thing in the stdlib for what you want to do” stuff in Java is getting pretty long, and it’s entirely due to Java’s absolutist stance on compatibility.
Yeah, Java is a good example here, it’s much more of “that seemed like a good idea at that time” (eg, mutable DateTime objects). This actually makes me think that I was foolish with my Go prediction — there was a fair amount of Java API churn with the addition of generics…
there was a fair amount of Java API churn with the addition of generics…
Was there? Java’s generics are gimped specifically to avoid it, much of the churn predates the addition of generics, and instead dates back to Java 1.2’s Collections Framework which soft-deprecated a bunch of types e.g. Vector (for ArrayList), Hashtable (for HashMap), Dictionary (for Map), Enumeration (for Iterator), … With generics the existing class hierarchy just got “informative” generic parameters tacked on.
C# was the one which had a lot of churn with generics, because it uses reified generics and thus the non-generic classes and the generic classes were incompatible. Hence System.Collections and System.Collections.Generic.
Oh wow, looks like I didn’t do my Java history homework! I was thinking that Vector -> List was due to generics, but of course you are right, thanks for correction!
My go-to example of a “here be dragons” stdlib is actually Java.
Java 2 improved this a lot, and 1.1 a bit, but 1.0 was full of classes with totally different naming conventions. Java kept the old ones for quite a while with deprecated labels on them (I think some are still there, but I haven’t really looked at Java since Java 8) but made those wrappers around the consistently-named versions.
C++ is up there, too. Learning C++ is a process of figuring out which of the six ways to do any given thing is the correct way, and which five are old ways from the ’90s and ’00s that are now known to cause memory leaks or run exponentially slower. Maintaining your C++ knowledge is a process of regularly converting all of your code to the new seventh way that got added to the standard library last week because the last way was also bad.
I think any non-trivial code, including the stdlib, will include bad-in-retrospect parts. Bugs, mistakes in design, and stuff that can only improve as the rest of the world changes (e.g. supporting popular but bad protocols, like TLS 1.1).
If you want to change things and realise that however hard you try you’re going to break backwards compat sometimes for some people then I think the polite thing is to let people select the version of each dependency at a good level of granularity so that folks can upgrade components separately.
I think Rust does this well by having a small standard library with lots of core stuff available in crates. Julia did this less well by having quite a large standard library, but now you have to explicitly note your standard library imports in your project.toml, same as 3rd party imports, so you can version them.
If you let users pick and choose more versions then you get new problems, especially if breaking changes are common: like a single program depending on multiple versions of the same library, which can be confusing and bloats binary size; or requirements clashes if your language doesn’t allow multiple versions of the same library to be used together.
I think those are better problems to have than forcing users to adapt to all of the changes in the language and stdlib at once if they want to upgrade or alternatively having pseudo versioning where nothing is ever removed but you end up with a cluttered stdlib because things get replicated (urllib2, pathlib, strncat, etc).
I like the approach Go is going for here with the new godebug behaviour and features. It’s granular versioning for the stdlib (and maybe base language in future?) but with different trade-offs versus handling it all through semantic versioning and the package manager.
Python I think made a couple of really bad, but in my opinion also fairly obvious design decisions. Things that made me avoid Python both for the language and standard library for really long, until I first used it. It always baffled me that Python was advertised as that language being simple and minimal and we’ll and clearly designed when even your hello world is enough to discredit the claim with print not being a function.
As for the standard library for example dealing with any kind of timestamps were horrible.
And then how to deal with packages.
For all of these there has been better prior art.
The good thing is that most of this was fixed, but I don’t think Python is a good benchmark.
There is unfortunately no way to assert whether a given change to a user-facing API is or isn’t a breaking change. (Hyrum’s Law makes this clear.) Every change exists at some point on the spectrum of impact, so every change policy is necessarily subjective, and your option (1) is always going to apply, no matter what. The only question is where to draw that (subjective) line in the sand.
There have definitely been changes to Go which have broken user programs. But AFAIK they’ve almost exclusively been security-critical fixes to e.g. crypto packages, or enforcement of stuff asserted by the spec but not necessarily provided by the implementation(s). In all cases, AFAIK, the net impact has been very minimal, statistically zero.
There have definitely been changes to Go which have broken user programs. But AFAIK they’ve almost exclusively been security-critical fixes to e.g. crypto packages, or enforcement of stuff asserted by the spec but not necessarily provided by the implementation(s)
From the article:
ParseInt. For example, Go 1.13 added support for underscores in large numbers for readability. Along with the language change, we made strconv.ParseInt accept the new syntax. This change didn’t break anything inside Google, but much later we heard from an external user whose code did break. Their program used numbers separated by underscores as a data format. It tried ParseInt first and only fell back to checking for underscores if ParseInt failed. When ParseInt stopped failing, the underscore-handling code stopped running.
To me that’s a breaking change. As far as I’m aware, the spec for ParseInt was not ambiguous, and its prior rejection of values containing underscores was neither a bug nor a critical security vulnerability. So to argue that this didn’t violate the compatibility policy is interesting.
For sure that’s a breaking change, in the boolean/Hyrum’s Law sense. But I certainly consider it a narrow edge case. Code which depended on ParseInt rejecting strings with underscores was acting at least somewhat outside of the documented guarantees of the function.
Again, the compatibility policy – any compatibility policy – is a subjective judgment, not a boolean assessment.
Again, the compatibility policy – any compatibility policy – is a subjective judgment
To put it mildly, this drastically decreases the utility of the policy. In fact, would argue that it makes the policy effectively meaningless – if it’s possible to retroactively and subjectively declare that a given break is acceptable because it was “an edge case” or “Hyrum’s Law!” or whatever, then the policy fundamentally cannot be trusted, because if someone can subjectively argue their way into justifying breaking that guy over there’s code, they can just as well subjectively argue their way into justifying breaking my code.
And this is not some sort of way-super-out-there-ultra-hyper-mega-fringe case. It relied on the fact that Go’s definition of an integer literal didn’t include certain characters. Then, Go’s definition of an integer literal changed to include more characters. The correct backwards-compatible approach would be either to introduce a new flag to ParseInt allowing code to opt in to the new character set (keeping the old as the default), or to introduce a new function which would only support the new character set.
So this is a break, plain and simple, and I don’t see how Go can continue to pride itself on its backwards compatibility policy when it’s clear that policy is meaningless since it will just always be creatively interpreted as “sure, it was a break, but that one doesn’t count, nor that one, nor that one either…”
To put it mildly, this definition of a breaking change isn’t useful, because it captures, literally, any and every change to an implementation.
It relied on the fact that Go’s definition of an integer literal didn’t include certain characters
Go never defined an integer literal as not including certain characters. The strconv.ParseInt function expressed, via documentation, a certain set of (subjective) constraints on what it considered to be a valid input string. Over time, those constraints were narrowed.
Were those changes breaking changes? Strictly speaking, yes. Practically speaking, no. The changes didn’t violate invariants asserted by earlier versions of the function, they introduced new invariants over properties that were previously undefined.
Code that called ParseInt expecting that it would always reject strings containing _ made assumptions that were never guaranteed. And if a behavior isn’t guaranteed – by the type system, or by documentation, or by some mechanism which is understood to represent stability – then you can’t treat that behavior as a guarantee.
An API provides only the guarantees which it explicitly declares.
Code that called ParseInt expecting that it would always reject strings containing _ made assumptions that were never guaranteed. And if a behavior isn’t guaranteed – by the type system, or by documentation, or by some mechanism which is understood to represent stability – then you can’t treat that behavior as a guarantee.
ParseInt interprets a string s in the given base (0, 2 to 36) and bit size (0 to 64) and returns the corresponding value i.
The string may begin with a leading sign: “+” or “-”.
If the base argument is 0, the true base is implied by the string’s prefix following the sign (if present): 2 for “0b”, 8 for “0” or “0o”, 16 for “0x”, and 10 otherwise. Also, for argument base 0 only, underscore characters are permitted as defined by the Go syntax for integer literals.
The bitSize argument specifies the integer type that the result must fit into. Bit sizes 0, 8, 16, 32, and 64 correspond to int, int8, int16, int32, and int64. If bitSize is below 0 or above 64, an error is returned.
The errors that ParseInt returns have concrete type *NumError and include err.Num = s. If s is empty or contains invalid digits, err.Err = ErrSyntax and the returned value is 0; if the value corresponding to s cannot be represented by a signed integer of the given size, err.Err = ErrRange and the returned value is the maximum magnitude integer of the appropriate bitSize and sign.
This is under-defined enough that your approach effectively allows ParseInt to do some wild things, because it doesn’t provide an exhaustive list of what is and is not valid for each possible base, or how each specific potential invalid character might be handled. For example, a future Go version could declare that any ASCII alphanumeric that isn’t a digit of the base is ignored (“1a2b3c” -> 123), or replaced with a zero à la HTML5 color parsing (“1a2b3c” -> 102030). And by your argument this would not violate any backwards-compatibility guarantee, since the current documentation does not promise to reject those (it just says what the error will look like if there are invalid digits, without defining the set of invalid digits – the same exact loophole you seem to be using to argue that the underscore change was non-breaking).
Personally I would find that to be an absurdity, and when an argument leads to absurdity I reject it on those grounds. As I reject your argument.
The ParseInt change was a breaking one. Yelling “Hyrum’s Law!” over and over doesn’t change that. Making up one-off excuses doesn’t change that. Arguing that we have to subjectively judge whether a use case is important enough to count as breaking doesn’t change that. Arguing that ParseInt can’t break compatibility because it’s so underspecified doesn’t change that.
This is under-defined enough that your approach effectively allows ParseInt to do some wild things, because it doesn’t provide an exhaustive list of what is and is not valid for each possible base, or how each specific potential invalid character might be handled. For example, a future Go version could declare that any ASCII alphanumeric that isn’t a digit of the base is ignored (“1a2b3c” -> 123), or replaced with a zero à la HTML5 color parsing (“1a2b3c” -> 102030). And by your argument this would not violate any backwards-compatibility guarantee, since the current documentation does not promise to reject those (it just says what the error will look like if there are invalid digits, without defining the set of invalid digits – the same exact loophole you seem to be using to argue that the underscore change was non-breaking).
Correct!
Arguing that ParseInt can’t break compatibility because it’s so underspecified doesn’t change that [this change was breaking]
I mean, it literally does mean that this change isn’t breaking, right? Whether or not a change is breaking depends on the guarantees asserted by the relevant API, and the guarantees of an API are determined by it’s specification, not it’s behavior.
not sure why your tone is so contemptuous. i’ve been using golang almost exclusively at home & professionally for 3-4 years, and i’ve never experienced a breaking change.
i also maintain several python projects and… well. the difference is so obvious and palpable that it’s hard to take your argument seriously.
I don’t have problems with Python, and I’m a heavy user of it. But I also know that other people have different experiences than I do, and I try to understand that and respect it and offer constructive advice when appropriate.
I just wish Go people would show the same respect and understanding to people who run into problems with Go, rather than endlessly parroting “well I don’t have that problem” and then going back to bashing whatever language they’re looking down their noses at this week. The overwhelming response of this thread to someone whose perfectly reasonable code was broken by a change in Go has been to dismiss it and blame the programmer for writing such code, and that’s not a good look for Go.
This isn’t a useful definition of a breaking change. In an open ecosystem, literally every change has the potential to break programs. Compatibility necessarily has a narrower definition. How much narrower is up for discussion.
Functions likeParseInt are often used as a way seeing if some input is numeric. That’s a super common idiom in lots of languages. The exact flow varies, of course; in some languages you get an exception, in others you get a boolean and the parse is stored in an out param, etc. In Go, ParseInt returns an error when the value passed in contains “invalid digits”.
Except that, in Go, ParseInt has already changed its definition of valid and invalid at least once, and as a result caused a break in someone’s code because inputs that used to be rejected no longer were.
That is not some sort of crazy way-out-there fringe spacebar-heating type thing. Like it or not, whoever wrote that code was doing something perfectly reasonable – there was no bug or critical security vulnerability in ParseInt as a result of which things it did or didn’t consider to be valid “digits”, and so there was no reason to suspect that the definition of valid/invalid might actually be a fluid thing that could be changed on a whim.
As I already pointed out, the right thing – if Go were actually committed to compatibility to the degree its fans like to claim it is – would have been to either introduce a new flag to ParseInt to opt into an expanded valid character set, or keep the old ParseInt as-is forever and introduce a new function which would always use the expanded valid character set.
Instead of which, Go chose to break people’s code. And, again, not because of any bug or critical security issue, but so that numeric literals could be slightly easier on the eyes.
If Python – which doesn’t even make the kinds of guarantees Go tries to – had broken end-user code for something that frivolous we’d never hear the end of it. In fact, we still haven’t heard the end of it with the print versus print() change. But when Go does this, there are endless excuses and rationalizations and justifications to try to explain that it was good for Go to arbitrarily break someone’s code and that it was the programmer’s fault for naïvely not reading all the fine print in the backwards-compatibility policy and realizing it contained loopholes big enough to drive a fleet of trucks through.
If Python had broken end-user code for something that frivolous we’d never hear the end of it.
FWIW it did in exactly the same manner: PEP 515 applied to both source literals and int parameters.
I’d think it’s either that there was never such super hard BC promise on new convenience so if people got hit they just went “oh well” orint is not super convenient to validate decimal integers in the first place so it’s rarely used that way. For instance why would you use int to parse an underscore-separated format when you can use csv with a custom separator, or split things out with re.
I assert that if someone had reported that Python broke their code with this change, and the response from the Python core team and from Python fans on programming forums had been exactly the response being put out by Go and its fans, we would never hear the end of it, and it would be held up endlessly as evidence that Python makes frivolous breaking changes and cannot be trusted (unlike Go, which takes compatibility seriously).
And I don’t think anyone can deny that’s exactly what would happen.
Except that, in Go, ParseInt has already changed its definition of valid and invalid at least once, and as a result caused a break in someone’s code because inputs that used to be rejected no longer were.
You simply can’t rely on properties of ParseInt which aren’t guaranteed by the docs for ParseInt, or the language spec.
I would even say Go has one of the best “compatibility stories”, but it’s still on the spectrum, so to speak. To claim Go 2 will “never” break Go 1 code just reminds me of Gary promising he “will never die” in Team America.
The changes are, strictly speaking, “compatible” in the sense of the Go 1 document, but they still break programs.
How could that be “compatible”, though? Unless we just redefine the word?
I guess it’s going to be like Windows, where there’s a culture of insisting it has perfect backwards compatibility (for reasons, to be sure), yet I can’t run old EXEs.
I’m really arguing that I think the ParseInt change was a breaking change.
The linked article dismisses this because the author(s) seem to feel that this wasn’t important enough, or didn’t affect Google enough, to count as a break. People here seem to be rolling with a combination of it not being important enough, and not being specified enough, to count as a break.
Personally I think it was entirely reasonable that a string containing characters illegal for integer literals would be a string that someone would assume ParseInt should reject, and Go should just admit it was a violation of backwards compatibility. And if such breaks are to be allowed, I think Go should stop pretending to have such an absolute ironclad unbreakable policy of absoluteness.
Python doesn’t make those kinds of promises. In fact, quite the opposite: Python has always had deprecations and removals. The Python 2 -> 3 transition was just an especially big all-at-once break, rather than a one-time thing, and what came out of that was not a promise never again to do backwards-incompatible changes, but a promise never again to do it on that scale/in that way (i.e., a Python 3 -> 4 of comparable scope to the 2 -> 3 change). You can go look at any recent Python version’s release notes and find a list of what’s newly deprecated in that version, what’s due for removal in the near future, and what was removed in that version.
So I do prefer Python, because Python strikes me as much more honest; it tells me when something’s deprecated, and how long I have until that thing gets removed/broken. But Go insists that it will never break, then does break and gaslights me about it afterward.
The context for this thread is Go’s ParseInt errored on strings containing characters illegal for a Go integer literal, and people relied on that behavior, but then Go decided to change the set of legal characters for integer literals, which broke real-world code. And the overwhelming response in this thread from people who like Go is to blame the people whose code was broken, while the official article that started all this just says that the change didn’t break anything at Google, as if that’s somehow consolation.
I think that’s a bad way to do things and a bad way to treat people and is probably one of the reasons for the reputation Go and its dev team have.
The assumption was that the ParseInt change wouldn’t break anything. But we’re just going in circles now. Enjoy being “treated like an adult” while dealing with breaking changes every minor release.
Enjoy being “treated like an adult” while dealing with breaking changes every minor release.
Kind of my whole point is that Go imposes breaking changes on people, and that people have to deal with those breaking changes. Using Go does not magically get you out of having to deal with breaking changes! It just puts you in a world where on top of still having to deal with breaking changes, everyone tells you that it’s your fault for writing code that worked on version N and not in Version N+1, because you didn’t study the fine print of the compatibility policy well enough.
You’ve clearly never actually used Go so this might be hard for you to imagine. But the vast majority of users have never experienced any compatibility issues when updating their Go version.
And the vast majority of Python users don’t experience compatibility issues when updating their Python version.
But the people who do experience issues aren’t any less important because of that, and their issues shouldn’t be dismissed (and they tend to be quite loud on programming forums). And I don’t think you’d let a Python fan dismiss such issues by claiming that most people don’t experience them. So I’m not going to let a Go fan do it, either.
(also, I have tinkered with Go, and maintain enough working knowledge to be able to read and review code since some people I work with write Go, but for many reasons I do not personally choose to write it or build things in it)
It’s arguable that this one is a breaking change. It’s only for the “base 0” case, and the strconv.ParseInt docs say “for argument base 0 only, underscore characters are permitted as defined by the Go syntax for integer literals”. I think one should read that as, “if the Go spec changes / becomes more permissive, ParseInt with base 0 will too”. Notably, ParseInt does not accept underscores (and hasn’t changed behaviour from 1.0) when you give an explicit base.
every change policy is necessarily subjective, and your option (1) is always going to apply, no matter what.
Would you consider a policy “Never change an existing, stabilized API or its implementation” as a counterexample to this assertion, assuming that whether an API is stabilized is clearly marked and unambiguous?
If not, would you consider “Issue a stable release of the language and library and never afterward change the language or library in any way” as a counterexample?
Would you consider a policy “Never change an existing, stabilized API or its implementation” as a counterexample to this assertion, assuming that whether an API is stabilized is clearly marked and unambiguous?
Never change an existing stabilized API? Sure. Never change its implementation? Definitely not. An API provides only those guarantees which it asserts, consumers cannot rely on any arbitrary behavior which they observe.
If not, would you consider “Issue a stable release of the language and library and never afterward change the language or library in any way” as a counterexample?
Again, stability is not a boolean, it’s a spectrum. Software producers choose a point on that spectrum.
Perhaps I was unclear. You asserted that “every change policy is necessarily subjective”, and I meant to ask whether you would accept “Never change an existing, stabilized API or its implementation” as an example of a change policy that is not subjective.
I added “or its implementation” to head off any subjectivity in whether changing the implementation counts as changing the interface.
Never change an existing stabilized API? Sure. Never change its implementation? Definitely not.
I assume you do not really mean that adding “or its implementation” turns the policy from objective to subjective?
Again, stability is not a boolean, it’s a spectrum. Software producers choose a point on that spectrum.
Sure, I don’t think I said anything to the contrary, but I was proposing as another example of an objective “change policy” a very specific point on that spectrum: the total cessation of development activity. I’m not suggesting this is necessarily a useful policy, though it can be in some cases.
I see. Yes, my claim that “every change policy is necessarily subjective” does indeed assume a definition of a change policy that permits changes. If you define a change policy which does not permit changes – as is the case for your example – then my claim would not apply.
The GODEBUG flags also seem really smelly to me, especially when multiple flags will inevitably interact with each other in funny ways someday. The Rust editions approach seems much better, with major checkpoints of behavior bundled together all at once.
I also like the minimal standard library, deferring to an ecosystem of well-known of crates for common tasks. How “well-known” these crates are is up for debate, but there are good resources like blessed.rs.
Go import also makes adding new libraries extremely easy, so I’m not really sure why the standard library needed to be so big. I think Go could have had a lot more packages in golang.org/x and kept stuff out of the standard library. That way you wouldn’t be stuck with those packages once they have been eclipsed by packages in the wider ecosystem.
Go’s authors believe that third-party dependencies are inherently costly and should be minimized as much as is feasible. This motivates a relatively large standard library, even if it comes at the expense of needing to support APIs that are “eclipsed by packages in the wider ecosystem”. Forward stability of user programs is judged to be the most important property, everything else is secondary.
Needing to temporarily turn off an feature in the built-in HTTPS library is a thing I want configurable via an env var. I expect to need to turn those on or off in arbitrary combinations in production.
For things like “nondeterministic is a keyword now”, I think I would definitely prefer language editions.
APIs that are actively bad and wrong and harmful to use are still kept around, and everyone has to just learn lots of “oh, don’t use that, or that, or especially that, and whatever you do don’t use this…”
Would you consider including a prominent warning in an API’s documentation that “It’s bad, and here’s why, so don’t use it” to save users from “[having] to just learn” not to use it?
If not, what about (and I would prefer this latter) a machine-readable annotation in the source code that, in addition to affecting the documentation (example), makes the compiler (or interpreter) emit a warning if the API is used?
2. The language accumulates huge amounts of “here be dragons” cruft where APIs that are actively bad and wrong and harmful to use are still kept around, and everyone has to just learn lots of “oh, don’t use that, or that, or especially that, and whatever you do don’t use this…”, or
I don’t feel that this is something unique to “ultra-strong backwards-compatibility” languages, this happens to every language out there that is used by a big audience.
Go decided to include batteries, which over time will cause more dragons to pop up. But it’s not a net negative imo, as it also “strongarms” in certain idioms that would be much harder to convey with libraries on the side. For example, a thing that annoys me with OCaml is the LWT/Async bifurcation, and even if something subpar was shipped with OCaml, it would’ve made the concurrency story a bit more palatable for newcomers.
Also, I think deprecated functions and types is more of a documentation and tooling issue than anything else. Hide deprecated functions or put them in a separate section, and let linters or editors scream whenever you see on in the code.
New struct fields are a problem in languages with structural typing (including Go) for reasons that are ignored here. If you add a new field to a struct, you increase the set of interfaces that it conforms to. Imagine if I have a standard library struct with two fields and a user program with a struct that has a third. If I add a field to the standard library struct, user code that relies on pattern matching (casting to interface type in Go) will send the standard-library type down the code paths intended for the user program’s type, just because the names match. If they happen to be using the third field for the same thing, that might be right, but if they’re using it for different purposes then it will be a problem.
This was a problem for Objective-C. In GNUstep, we decided that adding methods to a class was not something that needed an SONAME bump, but that decision was controversial. Some people argued that it changed the behaviour of any code that relied on -responseToSelector: and so may have been a breaking change.
In general, I think (per Hyrum’s Law) that avoiding breaking changes entirely is impossible. I’d love to see something more quantitive in the guarantees that people provide. It’s possible to provide a mechanical checker that tells you that your program contains an interface that a previous version of a class didn’t conform to but a new one does, for example. You should (in theory - in practice it’s hard) be able to quantify the amount of churn that’s required to update code and how much of it can be mechanical.
The same applies to adding methods to types. I tend to think of these as equivalent (a field is an optimisation over a pair of a setter and a getter) but languages often surface them as different.
It’s considered a breaking change by the Go compatibility rules to add a method to a public interface. There are a couple of places in the standard library where there are two interfaces, like Fooer and FooBarer, because you can’t add a method to an interface without breaking it. One thing you can do in Go is test for an extended interface. So if you have a Fooer, you can test if it’s also a FooBarer at runtime and then call .Bar().
Struct fields aren’t part of an interface and can’t be accessed on an interface directly, so they don’t apply to backwards compatibility rules around interfaces. You either have to do I.(concreteType) to get the concrete type and then get the field or use reflection, which is pretty ugly and slow. In either case, adding a field to a concrete type wouldn’t change the interface.
I don’t think that parent commenter was talking about what you write.
Say you have a struct/object returned by a public API that has an A() and B() functions. Adding a C() function to it could potentially change type-based dispatch based on which methods it implements.
I see. I think it would be unusual for that to cause a problem in practice.
In theory, Go has a problem with methods accidentally implementing some interface with the same signature, but in practice, I’ve never seen it happen even once.
I agree that it could in theory, but in a decade of using Go, I’ve never seen a case of accidentally implementing an interface, so it’s a bit more of a stretch. I think it would have to be that you added an optimized path, and then it turned out that skipping the slow path was bad for some reason. It happens, but not very often.
This is pretty much what COM does, though because adding a function changes the layout in memory, and possibly reflection (IDispatch) cases too. The usual convention is use QueryInterface to see if your IHober also implements IHober2.
For example, it makes sense that if your program depends on a buggy behavior and we fix the bug, your program will break. But we try very hard to break as little as possible and keep Go boring.
This is a really hard thing to do well. I’ve gone back and forth a lot when deciding a semver version for a release under the assumption that “Well, this is a fix, but will break things if people depend on it”.
Ultimately I tend to be explicit about what exactly constitutes a “breaking” change and saying that I don’t guarantee undocumented behavior.
Technically, the SemVer spec requires the software to define a public API that is covered by the versioning.
Software using Semantic Versioning MUST declare a public API.
Version 1.0.0 defines the public API. The way in which the version number is incremented after this release is dependent on this public API and how it changes.
That ParseInt change seems like both a frivolous backwards incompatibility and poor API design to me. Parsing an integer and parsing an integer as if it were a golang integer literal are very different tasks. And having any decimal string with leading zeroes be interpreted as octal seems like a footgun (I know it’s traditional, but it’s still a pain).
I’d separate these so that, idk, Meta.ParseInt parses golang string literals and regular ParseInt defaults to base 10 always, but you can provide extra args to change the base, opt in to base auto-detection or digit separators.
Admire their commitment to maintaining compatibility. It’s rare you hear of painful upgrades with Go.
Would be nice to get an insight into how much effort is required to keep this up and whether this effects the overall code quality / readability of the go code base. I remember lost of the go stdlib code being super easy to grok.
How bad would things have to get to make a breaking change, is it ever justifiable?
Honestly, I wish Go would break backwards compatibility more often and make it more predictable. I wonder over time how many more foot guns in the stdlib will emerge, like net/http
These kinds of ultra-strong backwards-compatibility guarantees for programming languages and their standard libraries tend to always lead to the same few outcomes (or some combination of these outcomes):
So far, Go seems to do a fair amount of (1) and probably is getting to the age where (2) will be a significant factor soon.
I’ve been thinking about this recently, and it seems that the practical perception of this problem is too much influenced by Python. Python is always touted as an example why batteries included might go wrong with time, but I think Python’s stdlib was unique randomly assembled in the first place. Like, stdlib modules like
unittestdon’t follow language’s naming convention! This has nothing to do with compatibility, and everything to do with the practice of deciding what goes into stdlib. I don’t know how that was decided in Python, but I have a strong suspicion that “anything goes” was pretty close to the truth.Now, when I update back from that example, I sort-of feel that a properly curated stdlib would age well? If you are careful with what you accept, it shouldn’t be too hard to maintain compat, and it’s unlikely that things go sour with time.
I predict that in 2032 Go stdlib would be fine (I don’t know how good it actually is today, but, given that Deno modeled its API on Go, I have a high prior that it is), and in 2035 Rust stdlib would be fine, and in 2035 people would still be joking about unittest naming convention, heapq procedural APIs and three argument parsers in Python’s stdlib.
My go-to example of a “here be dragons” stdlib is actually Java.
Python pared down some of the older stuff in the 2->3 transition, and has a plan in place for how to gradually deprecate and remove things from its stdlib. But Java… the list of “don’t use that, even though it’s the most immediately obvious thing in the stdlib for what you want to do” stuff in Java is getting pretty long, and it’s entirely due to Java’s absolutist stance on compatibility.
Yeah, Java is a good example here, it’s much more of “that seemed like a good idea at that time” (eg, mutable DateTime objects). This actually makes me think that I was foolish with my Go prediction — there was a fair amount of Java API churn with the addition of generics…
Was there? Java’s generics are gimped specifically to avoid it, much of the churn predates the addition of generics, and instead dates back to Java 1.2’s Collections Framework which soft-deprecated a bunch of types e.g. Vector (for ArrayList), Hashtable (for HashMap), Dictionary (for Map), Enumeration (for Iterator), … With generics the existing class hierarchy just got “informative” generic parameters tacked on.
C# was the one which had a lot of churn with generics, because it uses reified generics and thus the non-generic classes and the generic classes were incompatible. Hence
System.CollectionsandSystem.Collections.Generic.Oh wow, looks like I didn’t do my Java history homework! I was thinking that Vector -> List was due to generics, but of course you are right, thanks for correction!
Java 2 improved this a lot, and 1.1 a bit, but 1.0 was full of classes with totally different naming conventions. Java kept the old ones for quite a while with deprecated labels on them (I think some are still there, but I haven’t really looked at Java since Java 8) but made those wrappers around the consistently-named versions.
C++ is up there, too. Learning C++ is a process of figuring out which of the six ways to do any given thing is the correct way, and which five are old ways from the ’90s and ’00s that are now known to cause memory leaks or run exponentially slower. Maintaining your C++ knowledge is a process of regularly converting all of your code to the new seventh way that got added to the standard library last week because the last way was also bad.
I think any non-trivial code, including the stdlib, will include bad-in-retrospect parts. Bugs, mistakes in design, and stuff that can only improve as the rest of the world changes (e.g. supporting popular but bad protocols, like TLS 1.1).
If you want to change things and realise that however hard you try you’re going to break backwards compat sometimes for some people then I think the polite thing is to let people select the version of each dependency at a good level of granularity so that folks can upgrade components separately.
I think Rust does this well by having a small standard library with lots of core stuff available in crates. Julia did this less well by having quite a large standard library, but now you have to explicitly note your standard library imports in your project.toml, same as 3rd party imports, so you can version them.
If you let users pick and choose more versions then you get new problems, especially if breaking changes are common: like a single program depending on multiple versions of the same library, which can be confusing and bloats binary size; or requirements clashes if your language doesn’t allow multiple versions of the same library to be used together.
I think those are better problems to have than forcing users to adapt to all of the changes in the language and stdlib at once if they want to upgrade or alternatively having pseudo versioning where nothing is ever removed but you end up with a cluttered stdlib because things get replicated (urllib2, pathlib, strncat, etc).
I like the approach Go is going for here with the new godebug behaviour and features. It’s granular versioning for the stdlib (and maybe base language in future?) but with different trade-offs versus handling it all through semantic versioning and the package manager.
Python I think made a couple of really bad, but in my opinion also fairly obvious design decisions. Things that made me avoid Python both for the language and standard library for really long, until I first used it. It always baffled me that Python was advertised as that language being simple and minimal and we’ll and clearly designed when even your hello world is enough to discredit the claim with print not being a function.
As for the standard library for example dealing with any kind of timestamps were horrible.
And then how to deal with packages.
For all of these there has been better prior art.
The good thing is that most of this was fixed, but I don’t think Python is a good benchmark.
There is unfortunately no way to assert whether a given change to a user-facing API is or isn’t a breaking change. (Hyrum’s Law makes this clear.) Every change exists at some point on the spectrum of impact, so every change policy is necessarily subjective, and your option (1) is always going to apply, no matter what. The only question is where to draw that (subjective) line in the sand.
There have definitely been changes to Go which have broken user programs. But AFAIK they’ve almost exclusively been security-critical fixes to e.g. crypto packages, or enforcement of stuff asserted by the spec but not necessarily provided by the implementation(s). In all cases, AFAIK, the net impact has been very minimal, statistically zero.
From the article:
To me that’s a breaking change. As far as I’m aware, the spec for
ParseIntwas not ambiguous, and its prior rejection of values containing underscores was neither a bug nor a critical security vulnerability. So to argue that this didn’t violate the compatibility policy is interesting.For sure that’s a breaking change, in the boolean/Hyrum’s Law sense. But I certainly consider it a narrow edge case. Code which depended on ParseInt rejecting strings with underscores was acting at least somewhat outside of the documented guarantees of the function.
Again, the compatibility policy – any compatibility policy – is a subjective judgment, not a boolean assessment.
To put it mildly, this drastically decreases the utility of the policy. In fact, would argue that it makes the policy effectively meaningless – if it’s possible to retroactively and subjectively declare that a given break is acceptable because it was “an edge case” or “Hyrum’s Law!” or whatever, then the policy fundamentally cannot be trusted, because if someone can subjectively argue their way into justifying breaking that guy over there’s code, they can just as well subjectively argue their way into justifying breaking my code.
And this is not some sort of way-super-out-there-ultra-hyper-mega-fringe case. It relied on the fact that Go’s definition of an integer literal didn’t include certain characters. Then, Go’s definition of an integer literal changed to include more characters. The correct backwards-compatible approach would be either to introduce a new flag to
ParseIntallowing code to opt in to the new character set (keeping the old as the default), or to introduce a new function which would only support the new character set.So this is a break, plain and simple, and I don’t see how Go can continue to pride itself on its backwards compatibility policy when it’s clear that policy is meaningless since it will just always be creatively interpreted as “sure, it was a break, but that one doesn’t count, nor that one, nor that one either…”
To put it mildly, this definition of a breaking change isn’t useful, because it captures, literally, any and every change to an implementation.
Go never defined an integer literal as not including certain characters. The strconv.ParseInt function expressed, via documentation, a certain set of (subjective) constraints on what it considered to be a valid input string. Over time, those constraints were narrowed.
Were those changes breaking changes? Strictly speaking, yes. Practically speaking, no. The changes didn’t violate invariants asserted by earlier versions of the function, they introduced new invariants over properties that were previously undefined.
Code that called ParseInt expecting that it would always reject strings containing
_made assumptions that were never guaranteed. And if a behavior isn’t guaranteed – by the type system, or by documentation, or by some mechanism which is understood to represent stability – then you can’t treat that behavior as a guarantee.An API provides only the guarantees which it explicitly declares.
Here is the entirety of the current documentation of
ParseInt:This is under-defined enough that your approach effectively allows
ParseIntto do some wild things, because it doesn’t provide an exhaustive list of what is and is not valid for each possible base, or how each specific potential invalid character might be handled. For example, a future Go version could declare that any ASCII alphanumeric that isn’t a digit of the base is ignored (“1a2b3c” -> 123), or replaced with a zero à la HTML5 color parsing (“1a2b3c” -> 102030). And by your argument this would not violate any backwards-compatibility guarantee, since the current documentation does not promise to reject those (it just says what the error will look like if there are invalid digits, without defining the set of invalid digits – the same exact loophole you seem to be using to argue that the underscore change was non-breaking).Personally I would find that to be an absurdity, and when an argument leads to absurdity I reject it on those grounds. As I reject your argument.
The
ParseIntchange was a breaking one. Yelling “Hyrum’s Law!” over and over doesn’t change that. Making up one-off excuses doesn’t change that. Arguing that we have to subjectively judge whether a use case is important enough to count as breaking doesn’t change that. Arguing thatParseIntcan’t break compatibility because it’s so underspecified doesn’t change that.Correct!
I mean, it literally does mean that this change isn’t breaking, right? Whether or not a change is breaking depends on the guarantees asserted by the relevant API, and the guarantees of an API are determined by it’s specification, not it’s behavior.
“Go: the language that promises nothing, so that it can break your code and blame you” is probably not the catchy slogan you want.
not sure why your tone is so contemptuous. i’ve been using golang almost exclusively at home & professionally for 3-4 years, and i’ve never experienced a breaking change.
i also maintain several python projects and… well. the difference is so obvious and palpable that it’s hard to take your argument seriously.
I don’t have problems with Python, and I’m a heavy user of it. But I also know that other people have different experiences than I do, and I try to understand that and respect it and offer constructive advice when appropriate.
I just wish Go people would show the same respect and understanding to people who run into problems with Go, rather than endlessly parroting “well I don’t have that problem” and then going back to bashing whatever language they’re looking down their noses at this week. The overwhelming response of this thread to someone whose perfectly reasonable code was broken by a change in Go has been to dismiss it and blame the programmer for writing such code, and that’s not a good look for Go.
This isn’t a useful definition of a breaking change. In an open ecosystem, literally every change has the potential to break programs. Compatibility necessarily has a narrower definition. How much narrower is up for discussion.
Functions like
ParseIntare often used as a way seeing if some input is numeric. That’s a super common idiom in lots of languages. The exact flow varies, of course; in some languages you get an exception, in others you get a boolean and the parse is stored in an out param, etc. In Go,ParseIntreturns an error when the value passed in contains “invalid digits”.Except that, in Go,
ParseInthas already changed its definition of valid and invalid at least once, and as a result caused a break in someone’s code because inputs that used to be rejected no longer were.That is not some sort of crazy way-out-there fringe spacebar-heating type thing. Like it or not, whoever wrote that code was doing something perfectly reasonable – there was no bug or critical security vulnerability in
ParseIntas a result of which things it did or didn’t consider to be valid “digits”, and so there was no reason to suspect that the definition of valid/invalid might actually be a fluid thing that could be changed on a whim.As I already pointed out, the right thing – if Go were actually committed to compatibility to the degree its fans like to claim it is – would have been to either introduce a new flag to
ParseIntto opt into an expanded valid character set, or keep the oldParseIntas-is forever and introduce a new function which would always use the expanded valid character set.Instead of which, Go chose to break people’s code. And, again, not because of any bug or critical security issue, but so that numeric literals could be slightly easier on the eyes.
If Python – which doesn’t even make the kinds of guarantees Go tries to – had broken end-user code for something that frivolous we’d never hear the end of it. In fact, we still haven’t heard the end of it with the
printversusprint()change. But when Go does this, there are endless excuses and rationalizations and justifications to try to explain that it was good for Go to arbitrarily break someone’s code and that it was the programmer’s fault for naïvely not reading all the fine print in the backwards-compatibility policy and realizing it contained loopholes big enough to drive a fleet of trucks through.This isn’t the first time this has happened. It probably, sadly, won’t be the last. I remember pretty clearly the time when someone literally got harassed and insulted so much by the Go partisans on this site they just gave up and quit trying to make their point.
I urge you to reflect on this and let it lead to changes in how you interact with people about Go.
FWIW it did in exactly the same manner: PEP 515 applied to both source literals and
intparameters.I’d think it’s either that there was never such super hard BC promise on new convenience so if people got hit they just went “oh well” or
intis not super convenient to validate decimal integers in the first place so it’s rarely used that way. For instance why would you useintto parse an underscore-separated format when you can usecsvwith a custom separator, or split things out withre.Then I’ll rephrase:
I assert that if someone had reported that Python broke their code with this change, and the response from the Python core team and from Python fans on programming forums had been exactly the response being put out by Go and its fans, we would never hear the end of it, and it would be held up endlessly as evidence that Python makes frivolous breaking changes and cannot be trusted (unlike Go, which takes compatibility seriously).
And I don’t think anyone can deny that’s exactly what would happen.
You simply can’t rely on properties of ParseInt which aren’t guaranteed by the docs for ParseInt, or the language spec.
Honestly, you’re not wrong.
I would even say Go has one of the best “compatibility stories”, but it’s still on the spectrum, so to speak. To claim Go 2 will “never” break Go 1 code just reminds me of Gary promising he “will never die” in Team America.
How could that be “compatible”, though? Unless we just redefine the word?
I guess it’s going to be like Windows, where there’s a culture of insisting it has perfect backwards compatibility (for reasons, to be sure), yet I can’t run old EXEs.
Are you really arguing that any observable change in a public API means you can’t claim backwards compatibility?
(Which is the definition of Hyrum’s Law)
I’m really arguing that I think the
ParseIntchange was a breaking change.The linked article dismisses this because the author(s) seem to feel that this wasn’t important enough, or didn’t affect Google enough, to count as a break. People here seem to be rolling with a combination of it not being important enough, and not being specified enough, to count as a break.
Personally I think it was entirely reasonable that a string containing characters illegal for integer literals would be a string that someone would assume
ParseIntshould reject, and Go should just admit it was a violation of backwards compatibility. And if such breaks are to be allowed, I think Go should stop pretending to have such an absolute ironclad unbreakable policy of absoluteness.Ok .. so if you don’t consider Go to have solid backwards compatibility, where does Python stand for you?
I think Go promises something it can’t deliver.
Python doesn’t make those kinds of promises. In fact, quite the opposite: Python has always had deprecations and removals. The Python 2 -> 3 transition was just an especially big all-at-once break, rather than a one-time thing, and what came out of that was not a promise never again to do backwards-incompatible changes, but a promise never again to do it on that scale/in that way (i.e., a Python 3 -> 4 of comparable scope to the 2 -> 3 change). You can go look at any recent Python version’s release notes and find a list of what’s newly deprecated in that version, what’s due for removal in the near future, and what was removed in that version.
So I do prefer Python, because Python strikes me as much more honest; it tells me when something’s deprecated, and how long I have until that thing gets removed/broken. But Go insists that it will never break, then does break and gaslights me about it afterward.
I think you’ve setup a bit of straw man for Go’s compatibility guarantee. Maybe you should read it https://go.dev/doc/go1compat
But I think it’s pretty telling that you’d prefer pythons backwards compatibility story. Maybe tone down the copium.
The context for this thread is Go’s
ParseInterrored on strings containing characters illegal for a Go integer literal, and people relied on that behavior, but then Go decided to change the set of legal characters for integer literals, which broke real-world code. And the overwhelming response in this thread from people who like Go is to blame the people whose code was broken, while the official article that started all this just says that the change didn’t break anything at Google, as if that’s somehow consolation.I think that’s a bad way to do things and a bad way to treat people and is probably one of the reasons for the reputation Go and its dev team have.
The assumption was that the ParseInt change wouldn’t break anything. But we’re just going in circles now. Enjoy being “treated like an adult” while dealing with breaking changes every minor release.
Kind of my whole point is that Go imposes breaking changes on people, and that people have to deal with those breaking changes. Using Go does not magically get you out of having to deal with breaking changes! It just puts you in a world where on top of still having to deal with breaking changes, everyone tells you that it’s your fault for writing code that worked on version N and not in Version N+1, because you didn’t study the fine print of the compatibility policy well enough.
You’ve clearly never actually used Go so this might be hard for you to imagine. But the vast majority of users have never experienced any compatibility issues when updating their Go version.
And the vast majority of Python users don’t experience compatibility issues when updating their Python version.
But the people who do experience issues aren’t any less important because of that, and their issues shouldn’t be dismissed (and they tend to be quite loud on programming forums). And I don’t think you’d let a Python fan dismiss such issues by claiming that most people don’t experience them. So I’m not going to let a Go fan do it, either.
(also, I have tinkered with Go, and maintain enough working knowledge to be able to read and review code since some people I work with write Go, but for many reasons I do not personally choose to write it or build things in it)
[Comment removed by author]
It’s arguable that this one is a breaking change. It’s only for the “base 0” case, and the strconv.ParseInt docs say “for argument base 0 only, underscore characters are permitted as defined by the Go syntax for integer literals”. I think one should read that as, “if the Go spec changes / becomes more permissive, ParseInt with base 0 will too”. Notably,
ParseIntdoes not accept underscores (and hasn’t changed behaviour from 1.0) when you give an explicit base.Would you consider a policy “Never change an existing, stabilized API or its implementation” as a counterexample to this assertion, assuming that whether an API is stabilized is clearly marked and unambiguous?
If not, would you consider “Issue a stable release of the language and library and never afterward change the language or library in any way” as a counterexample?
Never change an existing stabilized API? Sure. Never change its implementation? Definitely not. An API provides only those guarantees which it asserts, consumers cannot rely on any arbitrary behavior which they observe.
Again, stability is not a boolean, it’s a spectrum. Software producers choose a point on that spectrum.
Perhaps I was unclear. You asserted that “every change policy is necessarily subjective”, and I meant to ask whether you would accept “Never change an existing, stabilized API or its implementation” as an example of a change policy that is not subjective.
I added “or its implementation” to head off any subjectivity in whether changing the implementation counts as changing the interface.
I assume you do not really mean that adding “or its implementation” turns the policy from objective to subjective?
Sure, I don’t think I said anything to the contrary, but I was proposing as another example of an objective “change policy” a very specific point on that spectrum: the total cessation of development activity. I’m not suggesting this is necessarily a useful policy, though it can be in some cases.
I see. Yes, my claim that “every change policy is necessarily subjective” does indeed assume a definition of a change policy that permits changes. If you define a change policy which does not permit changes – as is the case for your example – then my claim would not apply.
The GODEBUG flags also seem really smelly to me, especially when multiple flags will inevitably interact with each other in funny ways someday. The Rust editions approach seems much better, with major checkpoints of behavior bundled together all at once.
I also like the minimal standard library, deferring to an ecosystem of well-known of crates for common tasks. How “well-known” these crates are is up for debate, but there are good resources like blessed.rs.
Go import also makes adding new libraries extremely easy, so I’m not really sure why the standard library needed to be so big. I think Go could have had a lot more packages in golang.org/x and kept stuff out of the standard library. That way you wouldn’t be stuck with those packages once they have been eclipsed by packages in the wider ecosystem.
Go’s authors believe that third-party dependencies are inherently costly and should be minimized as much as is feasible. This motivates a relatively large standard library, even if it comes at the expense of needing to support APIs that are “eclipsed by packages in the wider ecosystem”. Forward stability of user programs is judged to be the most important property, everything else is secondary.
They seem good for different problems I think.
Needing to temporarily turn off an feature in the built-in HTTPS library is a thing I want configurable via an env var. I expect to need to turn those on or off in arbitrary combinations in production.
For things like “
nondeterministicis a keyword now”, I think I would definitely prefer language editions.Would you consider including a prominent warning in an API’s documentation that “It’s bad, and here’s why, so don’t use it” to save users from “[having] to just learn” not to use it?
If not, what about (and I would prefer this latter) a machine-readable annotation in the source code that, in addition to affecting the documentation (example), makes the compiler (or interpreter) emit a warning if the API is used?
I don’t feel that this is something unique to “ultra-strong backwards-compatibility” languages, this happens to every language out there that is used by a big audience.
Go decided to include batteries, which over time will cause more dragons to pop up. But it’s not a net negative imo, as it also “strongarms” in certain idioms that would be much harder to convey with libraries on the side. For example, a thing that annoys me with OCaml is the LWT/Async bifurcation, and even if something subpar was shipped with OCaml, it would’ve made the concurrency story a bit more palatable for newcomers.
Also, I think deprecated functions and types is more of a documentation and tooling issue than anything else. Hide deprecated functions or put them in a separate section, and let linters or editors scream whenever you see on in the code.
Now that’s an announcement! I had already heard about Go 2 not happening, but I loved the “twist” to announce it.
New struct fields are a problem in languages with structural typing (including Go) for reasons that are ignored here. If you add a new field to a struct, you increase the set of interfaces that it conforms to. Imagine if I have a standard library struct with two fields and a user program with a struct that has a third. If I add a field to the standard library struct, user code that relies on pattern matching (casting to interface type in Go) will send the standard-library type down the code paths intended for the user program’s type, just because the names match. If they happen to be using the third field for the same thing, that might be right, but if they’re using it for different purposes then it will be a problem.
This was a problem for Objective-C. In GNUstep, we decided that adding methods to a class was not something that needed an SONAME bump, but that decision was controversial. Some people argued that it changed the behaviour of any code that relied on
-responseToSelector:and so may have been a breaking change.In general, I think (per Hyrum’s Law) that avoiding breaking changes entirely is impossible. I’d love to see something more quantitive in the guarantees that people provide. It’s possible to provide a mechanical checker that tells you that your program contains an interface that a previous version of a class didn’t conform to but a new one does, for example. You should (in theory - in practice it’s hard) be able to quantify the amount of churn that’s required to update code and how much of it can be mechanical.
Go interfaces only allow specifying methods. Struct fields are not permitted.
The same applies to adding methods to types. I tend to think of these as equivalent (a field is an optimisation over a pair of a setter and a getter) but languages often surface them as different.
It’s considered a breaking change by the Go compatibility rules to add a method to a public interface. There are a couple of places in the standard library where there are two interfaces, like Fooer and FooBarer, because you can’t add a method to an interface without breaking it. One thing you can do in Go is test for an extended interface. So if you have a Fooer, you can test if it’s also a FooBarer at runtime and then call .Bar().
Struct fields aren’t part of an interface and can’t be accessed on an interface directly, so they don’t apply to backwards compatibility rules around interfaces. You either have to do I.(concreteType) to get the concrete type and then get the field or use reflection, which is pretty ugly and slow. In either case, adding a field to a concrete type wouldn’t change the interface.
I don’t think that parent commenter was talking about what you write.
Say you have a struct/object returned by a public API that has an A() and B() functions. Adding a C() function to it could potentially change type-based dispatch based on which methods it implements.
I see. I think it would be unusual for that to cause a problem in practice.
In theory, Go has a problem with methods accidentally implementing some interface with the same signature, but in practice, I’ve never seen it happen even once.
It can definitely break some program that depends on it in a finicky way similarly to the int parse function debated in the sister thread.
Also, I find this data point about generic structural type systems very interesting.
I agree that it could in theory, but in a decade of using Go, I’ve never seen a case of accidentally implementing an interface, so it’s a bit more of a stretch. I think it would have to be that you added an optimized path, and then it turned out that skipping the slow path was bad for some reason. It happens, but not very often.
This is pretty much what COM does, though because adding a function changes the layout in memory, and possibly reflection (IDispatch) cases too. The usual convention is use QueryInterface to see if your IHober also implements IHober2.
This is a really hard thing to do well. I’ve gone back and forth a lot when deciding a semver version for a release under the assumption that “Well, this is a fix, but will break things if people depend on it”.
Ultimately I tend to be explicit about what exactly constitutes a “breaking” change and saying that I don’t guarantee undocumented behavior.
Technically, the SemVer spec requires the software to define a public API that is covered by the versioning.
That
ParseIntchange seems like both a frivolous backwards incompatibility and poor API design to me. Parsing an integer and parsing an integer as if it were a golang integer literal are very different tasks. And having any decimal string with leading zeroes be interpreted as octal seems like a footgun (I know it’s traditional, but it’s still a pain).I’d separate these so that, idk, Meta.ParseInt parses golang string literals and regular ParseInt defaults to base 10 always, but you can provide extra args to change the base, opt in to base auto-detection or digit separators.
Admire their commitment to maintaining compatibility. It’s rare you hear of painful upgrades with Go.
Would be nice to get an insight into how much effort is required to keep this up and whether this effects the overall code quality / readability of the go code base. I remember lost of the go stdlib code being super easy to grok.
How bad would things have to get to make a breaking change, is it ever justifiable?
Honestly, I wish Go would break backwards compatibility more often and make it more predictable. I wonder over time how many more foot guns in the stdlib will emerge, like
net/http