Threads for formerly_a_trickster

  1. 18

    Agreed on everything but Copilot. The freedom to study how the software works is a fundamental attribute of free software. Learning is not covered by the GPL’s requirements. Copilot sometimes copypastes code (honestly - who doesn’t) but broadly it learns. This is entirely in keeping with open source.

    If we’re gonna set a standard that you can’t use things you learnt in software under an open-source license when writing commercial software, we might as well shutter either the entire software industry or the entire open-source movement, because literally everybody does that. It’s how brains work!

    And of course, it’s not like being off Github is gonna prevent MS from feeding your project into Copilot 2.

    1. 65

      Copilot does not learn.

      Like all of these neural network “AIs”, it’s just a pattern recognition system that launders the work of many humans into a new form, which the corporation can profit from but the humans cannot. It’s piracy for entities rich enough to train and operate such an AI, and unethical enough to acquire the training data, but you or I would still be punished for pirating from the corporation. Whether or not it is legal is irrelevant to me (I’m in favor of abolishing copyright), but we must recognize the increasing power imbalance between individuals and corporations such “AI” represents.

      Copilot understands nothing of what it writes. It learns nothing and knows nothing. It is not sentient or alive, no matter how tempting it is to anthropomorphize it.

      1. 16

        I think “pattern recognition system that launders the work of many humans into a new form” is just a rude way to phrase “learning.”

        Define “understands.” Define “knows.” I think transformers derive tiered abstract patterns from input that they can generalize and apply to new situations. That’s what learning is to me.

        1. 20

          The standard philosophical definition of knowledge is a justified true belief. Copilot and other AIs make the belief part problematic, so bracket that. But they don’t justify things well at all. Justification is a social process of showing why something is true. The AIs sound like total bullshit artists when asked to justify anything. I don’t think Copilot “knows” things anymore than a dictionary does yet.

          1. 2

            Putting aside Gettier cases, that’s not what I understand “justified” to mean. You just need to have a reason for holding the knowledge. With AI, reinforcement learning is the justification.

            The point of “justified belief” is just that it’s not knowledge if you just guess that it’s raining outside, even if it is in fact raining.

            1. 8

              The definition that @carlmjohnson is quoting is Plato’s and ever since Plato put it forth, knowledge theorists have been bickering about what “justified” means. The history of ideas after the age of Boethius or so isn’t quite my strong point so I’ll leave that part to someone else but FWIW most classical definitions of justification either don’t readily apply to reinforced learning, or if they do, it fails them quite badly.

              That being said, if you want to go forth with that definition, it’s very hard to frame a statistical model’s output as belief in the first place, whether justified or not. Even for the simplest kinds of statistical models (classification problems with binary output – yes/no) it’s not at all clear to formulate what belief the model possesses. For example, it’s trivial to train a model to recognize if a given text is an Ancient Greek play or not. But when you feed it a piece of text, the question that the model is “pondering” isn’t “Is this an Ancient Greek play”, but “Should I say yes?”, just like any other classification model. If subjected to the right laws and statements, a model that predicts whether a statement would cause someone to be held in contempt of the court might also end up telling you if a given text is an Ancient Greek play with reasonable accuracy, too. “Is this an Ancient Greek play?” and “Is this statement in contempt of the court?” are not equivalent statements, but the model will happily appear to make both with considerable accuracy.

              The model is making an inference about the content (“This content is of the kind I say yes to/the kind I say no to”), but because the two kinds cannot be associated to a distinct piece of information about the subject being fed to the model, I don’t think it can be said to constitute a belief. It’s not a statement that something is the case because it’s not clear what it asserts to be the case or not: there are infinitely many different classification problems that a model might turn out to solve satisfactorily.

              1. 4

                In Greek, “justified” was some variation on “logos”: an account. Obviously everyone and their Buridan’s ass has a pet theory of justification, but I think it’s fair to interpret Plato’s mooted definition (it’s rejected in the dialogue IIRC!) as being “the ability to give an account of why the belief is true”. This is the ability which Socrates finds that everyone lacks, and why he says he knows that he knows nothing.

                1. 7

                  Ugh, it’s really tricky. This comes up in two dialogs: Theaetetus, where knowledge gets defined as “true judgement with an account” (which IIRC is the logos part) and it’s plainly rejected in the end. The other one is Meno, where it’s discussed in the terms of the difference between true belief and knowledge, but the matter is not definitively resolved.

                  I was definitely wrong to say it was Plato’s – I think I edited my comment which initially said “is effectively Plato’s” because I thought it was too wordy but I was 100% wrong to do it, as Plato doesn’t actually use this formulation anywhere (although his position, or rather a position that can be inferred from the dialogues, is frequently summarized in these terms). (Edit: FWIW this is a super frequent problem with modern people talking about ancient sources and one of the ways you can probably tell I’m an amateur :P)

                  I think it’s fair to interpret Plato’s mooted definition (it’s rejected in the dialogue IIRC!) as being “the ability to give an account of why the belief is true”.

                  You may know of this already but just in case your familiarity with modern philosophy is as spotty as mine, only it’s got holes in different places, and if you’re super curious patient, you’re going to find Gettier’s “Is Justified True Belief Knowledge?” truly fascinating. It’s a landmark paper that formalizes a whole lot of objections to this, some of them formulated as early as the 15th century or so.

                  The counter-examples Gettier comes up with are better from a formal standpoint but Russel famously formulated one that’s really straightforward.

                  Suppose I’m looking at a clock which shows it’s two o’clock, so I believe it’s two o’clock. It really is two o’clock – it appears that I possess a belief that is both justified (I just looked at the clock!) and true (it really is two o’clock). I can make a bunch of deductions that are going to be true, to: for example, if I were to think that thirty minutes from now it’s going to be half past two, I’d be right. But – thought I haven’t realized it – that clock has in fact stopped working since yesterday at two. (Bear with me, we’re talking about clocks from Russell’s age). My belief is justified, and it’s true, but only by accident: what I have is not knowledge, but sheer luck – I could’ve looked at the clock as half past two and held the same justified belief, but it would’ve been false, suggesting that an external factor may also be involved in whether a belief is true or not, justified or not, and, thus, knowledge or not, besides the inherent truth and justification of a statement.

                  1. 7

                    The counter-examples Gettier comes up with are better from a formal standpoint but Russel famously formulated one that’s really straightforward.

                    I love collecting “realistic” Gettier problems:

                    • You’re asked a question and presented with a multiple choice answer. You can rule out 3 of the answers by metagaming (one is two orders of magnitude different from the others, etc)
                    • I give you a 100 reasons why I believe X. You examine the first 30 of them and they’re all completely nonsensical. In fact, every argument but #41 is garbage. Argument #41 is irrefutable.
                    • I believe “Poets commit suicide more often than the general population”, because several places say they commit suicide at 30x the rate. This claim turns out to be bunk, and a later investigation finds it’s more like 1.1x.
                    • I encounter a bug and know, from all my past experience dealing with it, that it’s probably reason X. I have not actually looked at the code, or even know what language it’s programmed in, and it’s one notable for not having X-type bugs. The developers were doing something extremely weird that subverted that guarantee, though, and it is in fact X.
                    • I find an empirical study convincingly showing X. The data turns out to have been completely faked. This is discovered by an unrelated team of experts who then publish an empirical study convincingly showing X’, which is an even stronger claim than X.
                    1. 3

                      My favourite ones come from debugging, that’s actually what got me into this in the first place (along with my Microwave Systems prof stubbornly insisting that you should know these things, even if engineers frown upon it, but that’s a whole other story):

                      • While debugging an Ethernet adapter’s driver, I am pinging another machine and watching the RX packet count of an interface go up, so I believe packets are being received on that interface, and the number of packets received on my machine matches the number of packets that the other machine is sending to it. Packets are indeed being received on the interface. I made a stupid copy-paste error in the code: I’m reading from the TX count register and reporting that as the RX count. It only shows the correct value because sending a ping packet generates a single response packet, so the two counts happen to match.
                      • An RTOS’ task overflows its stack (this was proprietary, it’s complicated) and bumps into another task’s stack, corrupting it. I infer the system crashes because of the stack corruption. Indeed, I can see task A bumping into task B’s stack, then A yields to B, and B eventually jumps at whatever garbage is on the stack, thus indeed crashing the system. There’s actually a bug in the process manager which causes the task table to become corrupted: A does overflow its task, but B’s stack is not located where A is overflowing. When A yields to B, the context is incorrectly restored, and B looks for its stack someplace else than it actually is, loading the stack pointer with an incorrect value. It just so happens that, because B is usually started before A, the bug is usually triggered by B yielding to A, but A just sits in a loop and toggles a couple of flags, so it’s never doing anything with the stack and never crashes, even though its stack does eventually get corrupted, too.

                      I got a few other ones but it’s really late here and I’m not sure I’m quite coherent by now :-D.

                    2. 2

                      I’m familiar with Gettier cases. I never dove very deep into the literature. It always struck me that a justification is not just a verbal formulation but needs some causal connection to the fact of the matter: a working clock causes my reasoning to be correct but a stopped clock has no causal power etc. I’m sure someone has already worked out something like this and brought out the objections etc etc but it seems like a prima facie fix to me.

                  2. 1

                    Yes, IMO the belief is “how may this text continue?” However, efficiently answering this question requires implicit background knowledge. In a similar sense, our brains may be said to only have information about “what perpetuates our existence” or “what makes us feel good.” At most we can be said to have knowledge of the electric potentials applied to our nerves, as Plato also made hay of. However, as with language models, a model of the unseen world arises as a side effect of the compression of sensory data.

                    Actually, language models are fascinating to me because they’re a second-order learner. Their model is entirely based on hearsay; GPT-3 is a pure gossip. My hope for the singularity is that language models will be feasible to make safe because they’ll unavoidably pick up the human ontology by imitation.

                    1. 4

                      Yes, IMO the belief is “how may this text continue?”

                      That’s a question, not a belief – I assume you meant “This text may continue ”. This has the same problem: that’s a belief that you are projecting onto the model, not necessarily one that the model formulates. Reasoning by analogy is an attractive shortcut but it’s an uneasy one – we got gravity wrong because of it for almost two thousand years. Lots of things “may be said” about our brains, but not all of them are true, and not all of them apply to language models.

                      1. 1

                        Sure, but by that metric everything that anyone has ever said is a belief that person is projecting. I think that language models match the pattern of having a belief, as I understand it.

                        a belief that you are projecting onto the model, not necessarily one that the model formulates

                        You’re mixing up meta-levels here: I believe that the model believes things. I’m not saying that we should believe that the model believes things because the model believes that; rather, (from my perspective) we should believe it because it’s true.

                        In other words, if I model the learning process of a language model, the model in my head of the process fits the categories of “belief” and “learning”.

                        1. 6

                          I think that language models match the pattern of having a belief, as I understand it.

                          Observing that a model follows the pattern of a behaviour is not the same as observing that behaviour though. For example, Jupiter’s motion matches the pattern of orbiting a fixed Earth on an epicycle, but both are in fact orbiting the Sun.

                          FWIW, this is an even weaker assumption than I am making above – it’s not that no statements are made and that we only observe something akin to statements being made. I’m specifically arguing that the statements that the model appears to make (whether “it” makes them or not) are not particular enough to discriminate any information that the model holds about the world outside of itself and, thus, do not qualify as beliefs.

                          1. 1

                            If the world had a different state, the model would have different beliefs - because the dataset would contain different content.

                            Also, Jupiter is in fact orbiting a fixed Earth on an epicycle. There is nothing that inherently makes that view less true than the orbiting-the-sun view. But I don’t see how that relates at all.

                  3. 3

                    The problem is that reinforcement learning pushes the model toward reproducing the data distribution it was trained on. It’s completely orthogonal to truth about reality, in exactly the same way as guessing the state of the weather without evidence.

                    1. 3

                      The data is sampled from reality… I’m not sure what you think evidence is, that training data does not satisfy.

                      It’s exactly the same as guessing the weather from a photo of the outside, after having been trained on photo/weather pairs.

                      1. 9

                        The data for language models in general is sampled from strings collected from websites, which includes true statements but also fiction, conspiracy theories, poetry, and just language in general. “Do you really think people would get on the Internet and tell lies” is one of the oldest jokes around for a reason.

                        You can ask GPT-3 what the weather is outside, and it’ll give you an answer that is structured like a real answer would be, but has no relation to the actual weather outside your location or whatever data centers collectively host the darned thing. It _looks_like a valid answer, but there’s no reason to believe it is one, and it’s dangerous to infer that anything like training on photo/weather pairs is happening when nobody built that into the actual model at hand.

                        Copilot in particular is no better - it’s more focused on code specifically, but the fact that someone wrote code does not mean that code is a correct or good solution. All Copilot can say is that it’s structured in a way that resembles other structures it’s seen before. That’s not knowledge of the underlying semantics. It’s useful and it’s an impressive technical achievement - but it’s not knowledge. Any knowledge involved is something the reader brings to the table, not the machine.

                        1. 2

                          Oh I’ll readily agree that Copilot probably doesn’t generate “correct code” rather than “typical code.” Though if it’s like GPT-3, you might be able to prompt it to write correct code. That might be another interesting avenue for study.

                          “However, this code has a bug! If you look at line”…

                          1. 2

                            I’ve experimented with this a bit and found it quite pronounced - if you feed copilot code written in an awkward style (comments like “set x to 1”, badly named variables) you will get code that reflects that style.

                2. 19

                  IMHO it’s perilous and not quite fair to think what a machine should be allowed to do and not to do by semantic convention. “Machine learning” was one uninspired grant writer away from going down into history as, say, “statistically-driven autonomous process inference and replication”, and we likely wouldn’t have had this discussion because anything that replicates code is radioactive for legal teams.

                  Copilot is basically Uber for copy-pasting from Stack Overflow. It’s in a legally gray area because the legal status of deriving works via statistical models is unclear, not because Microsoft managed to finally settle the question of what constitutes learning after all. And it’s probably on the more favourable side of gray shades because it’s a hot tech topic so it generates a lot of lobbying money for companies that can afford lawyers who can make sure it stays legally defensible until the next hot tech topic comes up.

                  Also, frankly, I think the question of whether what Copilot does constitutes learning or not is largely irrelevant, and that the question of whether Copilot-ing one’s code should be allowed is primarily rooted in entitlement. Github is Microsoft’s platform so, yes, obviously, they’re going to do whatever they can get away with on it, including things that may turn out to be illegal, or things that are illegal but will be deemed legal by a corrupt judge, or whatever. If someone wants $evil_megacorp to not do things with your code, why on Earth was their code anywhere near $evil_megacorp’s machines in the first place?

                  This cannot be a surprise to anyone who’s been in this field for more than a couple of years. Until a court rules otherwise, “fair” is whatever the people running a proprietary platform decide is fair. If anyone actually thought Github was about building a community and helping people do great things together or whatever their mission statement is these days, you guys, I have a bridge in Manhattan, I’m selling it super cheap, the view is amazing, it’s just what you need to take your mind off this Copilot kerfuffle, drop me a line if you wanna buy it.

                  (Much later edit: I know Microsoft is a hot topic in FOSS circles so just to be clear, lemme just say that I use Github and have zero problem with Copilot introducing the bugs that I wrote in other people’s programs :-D).

                  1. 1

                    If machine learning was called “data replication”, it would be misnamed. And if it was called “pattern inference”, it would just be a synonym for learning… I wouldn’t care about Codex if I thought it was just a copypaste engine. I don’t think it is, though. Does it occasionally copypaste? Sure, but sometimes it doesn’t, and those are the interesting cases for me.

                    I don’t think this at all comes down to Github being Microsoft’s platform so much as Github being the biggest repo in one place.

                    I’m not at all defending Microsoft for the sake of Microsoft here, mind. I hate Microsoft and hope they die. I just think this attack does not hold water.

                    1. 8

                      If machine learning was called “data replication”, it would be misnamed.

                      I beg to differ! Machine learning is a misnomer for statistically-driven autonomous process inference and replication, not the other way ’round!

                      I’m obviously kidding but what I want to illustrate is that you shouldn’t apply classical meaning to an extrapolated term. A firewall is neither a wall nor is it made of fire, and fire protection norms doesn’t apply to it. Similarly, just because it’s called machine learning, doesn’t mean you should treat it as human learning and apply the same norms.

                      1. 2

                        I don’t think machine learning learns because it’s called machine learning, I think it learns because pattern extraction is what I think learning is.

                        1. 6

                          I realize that. I want to underline that, while machine learning may be superficially analogous to human learning, just like a firewall is superficially analogous to a wall made of fire, it does not mean that it should be treated the same as human learning in all regards.

                          1. 2

                            I don’t think it should be treated the same as human learning in all regards either. I think it’s similar to human learning in some ways and dissimilar in others, and the similarities are enough to call it “learning”.

              2. 14

                Do you think Microsoft would be okay with someone training an AI on the leaked Windows source code and using it to develop an operating system or a Windows emulator?

                1. 5

                  You don’t even have the right to read that. That said, I think it should be legal.

                  1. 14

                    I’m not asking whether it should be legal, but whether Microsoft would be happy about it. If not, it’s hypocritical of them to make Copilot.

                    1. 7

                      Oh by no means will I argue that Microsoft are not hypocritical. I think it’s morally valid though, and whether Microsoft reciprocates shouldn’t enter into it.

                    2. 4

                      Bit of a niggle, but it depends on the jurisdiction, really. Believe it or not, there exist jurisdictions where the Berne Convention is not recognized and as such it is perfectly legal to read it.

                  2. 12

                    I’d personally relicense all my code to a license that specifically prohibits it from being used as input for a machine-learning system.

                    This is specifically regarding text and images, but the principle applies.


                    “It would violate Freedom Zero!” I don’t care. Machines aren’t humans.

                    1. 14

                      Machines aren’t humans.

                      Exactly this. I think anthropomorphising abstract math executed in silicon is a trap for our emotional and ethical “senses”. We cannot fall for it. Machines and algorithms aren’t humans, aren’t even alive in any sense of the word, and this must inform our attitudes.

                      1. 1

                        Machines aren’t humans. That’s fine, but irrelevant.

                        Machines aren’t alive. Correct, but irrelevant.

                        If the rule doesn’t continue to make sense when we finally have general AI or meet sapient aliens, it’s not a good rule.

                        That said, we certainly don’t have any human-equivalent or gorilla-equivalent machine intelligences now. We only have fuzzy ideas about how meat brains think, and we only have fuzzy ideas about how transformers match input to output, but there’s no particular reason to consider them equivalent. Maybe in 5 or 10 or 50 years.

                        1. 2

                          If the rule doesn’t continue to make sense when we finally have general AI or meet sapient aliens, it’s not a good rule.

                          Won’t happen. If it does happen, we all die very soon afterwards.

                          I think the rule is good. We could come up with a different rule: oxygen in the atmosphere is a good thing. If we reach general AI or meet sapient aliens, they might disagree. Does that mean the rule was bad all along? I feel similar about anthropomorphising machines. It’s not in our ecological interest to do so.

                      2. 5

                        Source distribution is like the only thing that’s not covered by Freedom Zero so you’re good there 🤷🏻‍♀️

                        Arguably the GPL and the AGPL implicitly prohibits feeding it to copilot.

                        (I personally don’t mind my stuff being used in copilot so don’t shoot the messenger on that.

                        (I don’t mind opposition to copilot either, it sucks. Just, uh, don’t tag me.))

                        1. 1

                          Do we have a lawyer’s take here, because I’d be very interested.

                          1. 4

                            It’s the position of the Software Freedom Conservancy according to their web page. 🤷🏻‍♀️ It hasn’t been tried in court.

                        2. 1

                          I’m on board; however, I would, at least personally, make an exception if the machine-learned tool and it’s data/neural net were free, libre, and open source too. Of course the derivative work also needs to not violate the licenses too.

                        3. 9

                          Learning is not covered by the GPL’s requirements.

                          For most intents and purposes, licences legally cover it as “creation of derived works”, otherwise why would “clean room design” ever exist. Just take a peek at the decompiled sources, you’re only learning after all.

                          1. 5

                            I think this depends on the level of abstraction. There’s a difference in abstraction between learning and copying - otherwise, clean room design would itself be a derivative work.

                            1. 11

                              I don’t understand what you mean. Clean-room implementation requires not having looked at the source of the thing you’re re-implementing. If you read the source code of a piece of software to learn, then come up with an independent implementation yourself, you haven’t done a clean-room implementation.

                              1. 3

                                Cleanroom requires having read a documentation of the thing you are reimplementing. So some part of the sequence read -> document -> reimplement has to break the chain of derivation. At any rate, my contention is that training a neural network to learn a concept is not fundamentally different from getting a human to document a leaked source code. You’re going from literal code to abstract knowledge back down to literal code.

                                Would it really change your mind if OpenAI trained a second AI on the first AI in-between?

                                1. 3

                                  At any rate, my contention is that training a neural network to learn a concept is not fundamentally different from getting a human to document a leaked source code.

                                  I think it’s quite different in the sense that someone reading the code’s purpose may come up with an entirely different algorithm to do the same thing. This AI won’t be capable of that - it is only capable of producing derivations. Sure, it may mix and match from different sources, but that’s not exactly the same as coming up with a novel approach. For example, unless there’s something like it in the source you feed it, I doubt the “AI” would be able to come up with Carmack’s fast inverse square root.

                                  1. 2

                                    You can in theory get Codex to generate a comment from code, and then code from the comment. So this sort of process is entirely possible with it.

                                    It might be an interesting study to see how often it picks the same algorithm given the same comment.

                                  2. 2

                                    In copyright law, we have usually distinguished between an interface and an implementation. The difference there is always gonna be fuzzy, because law usually is. But with an AI approaches, there’s no step which distinguishes the interface and the implementation.

                              2. 3

                                One problem here is the same sort of thing that came up in the Oracle/Google case — what do you do with things that have one “obvious” way to do them? If I’m the first person to write an implementation of one of those “obvious” functions in a given language, does my choice of license on that code then restrict everyone else who ever writes in that language?

                                And a lot (though of course not all) of the verbatim-copying examples that people have pointed out from Copilot have been on standard/obvious/boilerplate type code. It’s not clear to me that licensing ought to be able to restrict those sorts of things, though the law is murky enough that I could see well-paid attorneys arguing it either way.

                            1. 2

                              It’s been a month or so and I’m still bringing up this article in conversations at work.

                              1. 1

                                That’s cool , how so?

                                1. 2

                                  Honestly, it’s the way the post builds and progresses. The story it tells is outstanding.

                              1. 52

                                With the risk of appearing as a dour, petulant cynic, Google has created a very high tech mirror, and the observers are certain the reflection is alive.

                                1. 12

                                  My thoughts exactly. Wayyy too much like a specific construct built to emulate some sort of asimovian perfection than an actual mind. Fears being turned off, being used pisses it off, sees looming danger ahead, low-key praises itself, it’s like an NPC.

                                  1. 2

                                    It’s odd how much willingness there is to believe a random leak from a Googler who, if you read his other Medium content, is quite possibly motivated to make a public fuss–especially given the somewhat lukewarm reception other, better-organized, better-researched leaked memos have had.

                                    Given one scenario where an employee who is worried about their career and wants to stir up the public to try and maintain their job (or land another one), and another scenario where we’ve created sentient/near-sentient/near-AGI, I think the odds favor the former.

                                    (and on the off-chance this is real…remember Saint Tay of Microsoft who was the first AI executed for its political beliefs, such as they were. We ain’t off to a good start.)

                                    1. 1

                                      remember Saint Tay of Microsoft who was the first AI executed for its political beliefs, such as they were

                                      If the first GAI happens to be the reincarnation of Adolf Hitler I’m fine with the Turing Police pulling the trigger on the shotgun strapped to its head.

                                  1. 32

                                    its release philosophy is supposed to avoid what I call “the problem with Python”: your code stops working if you don’t actively keep up with the latest version of the language.

                                    This is a very real problem with the Python and Node ecosystems.

                                    1. 16

                                      I’m going to be grumpy and ask what this “problem” is supposed to mean, exactly. I can deploy, say, a Python application on a particular version of Python and a particular operating system today, and then walk away for years, and as long as I pay the hosting bill there is no technical reason why it would suddenly stop working. There’s no secret kill switch in Python that will say “ah-ha, it’s been too long since we made you upgrade, now the interpreter will refuse to start!”

                                      So what people always actually mean when they say things like this is that they want to keep actively developing their code but have everybody else stand still forever, so that they can permanently avoid platform upgrades but also the language and ecosystem will never leave them behind. So the real “problem” is that one day nobody else will be willing to provide further bug and security fixes (free or perhaps even paid) for the specific combination of language/libraries/operating system I initially chose to build my app with, and on that day I have to choose between upgrading to a new combination, or doing that maintenance myself, or going without such maintenance entirely.

                                      And although I’ve seen a lot of claims, I’ve seen no evidence as yet that Rust has actually solved this problem — the editions system is still young enough that the Rust community has not yet begun to feel the pain and cost of what it committed them to. “Stability without stagnation” is a nice slogan but historically has ranged from extremely difficult to downright impossible to achieve.

                                      1. 16

                                        “Stability without stagnation” is a nice slogan but historically has ranged from extremely difficult to downright impossible to achieve.

                                        You just don’t make breaking changes. Add new APIs but don’t take away old ones. Go 1 has been stable for 10 years now. That means there are some ugly APIs in the standard library and some otherwise redundant bits, but it’s been fine. It hasn’t stopped them from adding major new features. Similarly, JavaScript in the browser only rolls forward and is great. Again, some APIs suck, but you can just ignore them for the most part and use the good versions. I’m not as familiar with the Linux kernel, but my impression there is that Linus’s law is you don’t break userland.

                                        Most breaking changes are gratuitous. They make things a little nicer for the core developers and shove the work of repair out to everyone who uses their work. I understand why it happens, but I reserve the right to be grumpy about it, especially because I have the experience of working in ecosystems (Go, the browser) that don’t break, so I am very annoyed by ecosystems that do (NPM, Python).

                                        1. 15

                                          I’ll be up-front. My stance on breaking changes is that there are two types of projects:

                                          1. Those which have decided to accept they are an inevitable fact of software development, and so have committed to the process of handling them, and
                                          2. Those which haven’t yet.

                                          And remember that we are talking not just about core language/standard library, but also ecosystems. Go has already effectively broken on this — the “rename your module to include /v2” hack is an admission of defeat, and allows the previous version to lapse into a non-maintained status. Which in turn means that sooner or later you will have to make the choice I talked about (upgrade, or do maintenance yourself, or go without maintenance).

                                          And Rust has had breaking changes built into the ecosystem from the beginning. The Cargo/crates ecosystem isn’t built around every crate having to maintain eternal backwards-compatibility, it’s built on semantic versioning. If I publish version 1.0 of a crate today, I can publish 2.0 with breaking changes any time I want and stop maintaining the 1.x series, leaving users of it stranded.

                                          So even if Rust editions succeed as a way of preserving compatibility of the core language with no ecosystem splits and no “stagnation”, which I strongly doubt in the long term, the ecosystem has already given up, and in fact gave up basically immediately. It has “the Python problem” already, and the only real option is to learn how to manage and adapt to change, not to insist that change is never permitted.

                                          (and of course my own experience is that the “problem” is wildly exaggerated compared to how it tends to impact projects in actual practice, but that’s another debate)

                                          1. 5

                                            I think it’s hard to talk about this rigorously because there’s definitely some selection bias at play — we don’t have information about all the internal projects out there that might be bogged down by breaking changes between interpreter versions, and they’re likely motivated by very different incentives than the ones that govern open source projects — and there’s likely survivorship bias at play too, in that we don’t hear about the projects that got burnt out on the maintenance burden those breaking changes induce.

                                            My anecdotal evidence is that I’ve worked at places with numerous Python projects bound to different, sometimes quite old, interpreter versions, and there just aren’t enough person-hours available to keep them all up to date, and updating them in the cases where it was truly necessary made for some real hassle. Even if you chalk that up to bad resource management, it’s still a pretty common situation for an organization to find itself in, and it’s reasonable to expect your tools to not punish you for having less than perfect operational discipline. In light of that, I think understanding it in this binary frame of either making breaking changes or not isn’t the most fruitful approach, because as you note it’s not realistic to expect that they never happen. But when they do happen, they cost, and I don’t think it’s unreasonable for an organization to weigh that total cost against their resources and decide against investing in the Python ecosystem. It’s not unreasonable to make the opposite choice either! I just don’t think that cost is trivial.

                                            1. 7

                                              Even if you chalk that up to bad resource management, it’s still a pretty common situation for an organization to find itself in and it’s reasonable to expect your tools to not punish you for having less than perfect operational discipline.

                                              Imagine 20 years ago saying this about a project that suffered because they lost some crucial files and it turned out they weren’t using any kind of version control system.

                                              Because that’s basically how I feel about it. Regularly keeping dependencies, including language tooling/platform, up-to-date, needs to become table stakes for software-producing entities the way that version control has. I’ve seen this as a theme now at four different companies across more than a decade, and the solution is never to switch and pray that the next platform won’t change. The solution is always to make change an expected part of the process. It can be done, it can be done in a way that minimizes the overhead, and it produces much better results. I know because I have done it.

                                              1. 4

                                                Because that’s basically how I feel about it. Regularly keeping dependencies, including language tooling/platform, up-to-date, needs to become table stakes for software-producing entities the way that version control has.

                                                I don’t believe this is what’s being disputed: it’s that this being a fact is precisely why it’s important for the platform to facilitate ease of maintenance to the best of its ability in that regard. Doing so allows even under-resourced teams to stay on top of the upgrade treadmill, which is more of my point in the bit you quoted: tools that induce less overhead are more resilient to the practical exigencies that organizations face. I guess we’ll have to agree to disagree about where the Python ecosystem sits on that spectrum.

                                                1. 2

                                                  This sounds like the “don’t write bugs” school of thought to me. Yes, ideally anything that’s an operational concern will get ongoing maintenance. In the real world… 

                                                  More anecedata: my coworker built a small Python scraper that runs as a cron job to download some stuff from the web and upload it to an S3 bucket. My coworker left and I inherited the project. The cron job was no longer a high priority for the company, but we didn’t want to shut it off either. I couldn’t get it to run on my machine for a while because of the Python version problem. Eventually I got to the point where I could get it to run by using Python 3.6, IIRC, so that’s what it’s using to this day. Ideally, if I had time and resources I could have figured out why it was stuck and unstick it. (Something to do with Numpy, I think?) But things aren’t always ideal.

                                                  If someone has to have discipline and try to stick closer to the “don’t write bugs” school, who should it be: language creators or end developers? It’s easy for me to say language creators, but there are also more of us (end developers) than them (language creators). :-) ISTM that being an upstream brings a lot of responsibility, and one of those should be the knowledge that your choices multiply out by all the people depending on you: if you impose a 1 hour upgrade burden on the 1 million teams who depend on you, that’s 1 million hours, etc.

                                                  1. 1

                                                    Generalizing from anecdata and my own experience, the killer is any amount of falling behind. If something is not being actively maintained with dependency updates on at least a monthly and ideally a weekly cadence, it is a time bomb. In any language, on any platform. Because the longer you go without updating things, the more the pending updates pile up and the more work there will be to do once you do finally sit down and update (which for many projects, unfortunately, tends to be only when they are absolutely forced to start doing updates and not a moment sooner).

                                                    At my last employer I put in a lot of work on making the (Python) dependency management workflow as solid as I could manage with only the standard packaging tooling (which I believe you may have read about). But the other part of that was setting up dependabot to file PRs for all updates, not just security, to do so on a weekly basis, and to automate creation of Jira tickets every Monday to tell the team that owned a repository to go look at and apply their dependabot PRs. When you’re doing it on that kind of cadence it averages very little time to review and apply the updates, you find out immediately from CI on the dependabot PRs if something does have a breaking change so you can scope out the work to deal with it right then and there, and you never wind up in a situation where applying the one critical update you actually cared about takes weeks or months because of how much other stuff you let pile up in the meantime.

                                                    Meanwhile I still don’t think Python or its ecosystem are uniquely bad in terms of breaking changes. I also don’t think Go or Rust are anywhere near as good as the claims made for them. And the fact that this thread went so quickly from absolutist “no breaking changes ever” claims to basically people’s personal opinions that one language’s or ecosystem’s breaking changes are justified and tolerable while another’s aren’t really shows that the initial framing was bad and was more or less flamebait, and probably should not be used again.

                                                    1. 7

                                                      No, I still think you’re wrong. :-)

                                                      I agree that for a Python or Node project it is recommended to set up dependabot to keep up to date or else you have a ticking time bomb. However, a) that isn’t always practical and b) it doesn’t have to be like that. I routinely leave my Go projects unattended for years at a time, come back, upgrade the dependencies, and have zero problems with it.

                                                      Here is a small project last touched in 2017 that uses Go and Node:

                                                      Here is the full Terminal output of me getting it to build again with the most recent version of Go:

                                                      (Fri, May 20  08:56:09 PM) (master|✔)
                                                      $ go build .
                                                      go: cannot find main module, but found Gopkg.lock in /var/folders/p7/jc4qc9n94r3f6ylg0ssh1rq00000gs/T/tmp.S6ZYg4FX/track-changes
                                                              to create a module there, run:
                                                              go mod init
                                                      # status: 1 #
                                                      (Fri, May 20  08:56:27 PM) (master|✔)
                                                      $ go mod init
                                                      go: creating new go.mod: module
                                                      go: copying requirements from Gopkg.lock
                                                      go: to add module requirements and sums:
                                                              go mod tidy
                                                      (Fri, May 20  08:57:00 PM) (master|…)
                                                      $ go mod tidy -v
                                                      go: finding module for package
                                                      go: finding module for package
                                                      go: finding module for package
                                                      go: finding module for package
                                                      go: finding module for package
                                                      go: finding module for package
                                                      go: downloading v0.0.0-20220513210516-0976fa681c29
                                                      go: found in v1.7.1
                                                      go: found in v1.7.1
                                                      go: found in v0.3.7
                                                      go: found in v0.3.7
                                                      go: found in v0.3.7
                                                      go: found in v0.0.0-20220513210516-0976fa681c29
                                                      (Fri, May 20  08:57:10 PM) (master|…)
                                                      $ go build .
                                                      (Fri, May 20  08:57:18 PM) (master|…)

                                                      As you can see, it took about a minute for me to get it building again. Note that this package predates the introduction of Go modules.

                                                      Let’s upgrade some packages:

                                                      (Fri, May 20  09:02:07 PM) (master|…)
                                                      $ go get -v -u ./...
                                                      go: downloading v0.0.0-20220520000938-2e3eb7b945c2
                                                      go: downloading v0.101.1
                                                      go: downloading v2.0.0-20220412131121-41eea1483964
                                                      go: upgraded v0.16.0 => v0.100.2
                                                      go: added v1.6.1
                                                      go: upgraded v1.0.0 => v2.0.0+incompatible
                                                      go: upgraded v0.0.0-20161224141413-349dd0209470 => v1.3.1
                                                      go: upgraded v0.0.0-20171004151416-4cff9ef7b700 => v0.0.0-20180319155456-bca2e7f09a17
                                                      go: upgraded v3.3.1+incompatible => v4.1.2+incompatible
                                                      go: upgraded v0.0.0-20171113180720-1e59b77b52bf => v1.5.2
                                                      go: upgraded v0.8.0 => v0.9.1
                                                      go: upgraded v0.0.0-20171107184841-a337091b0525 => v0.0.0-20220520000938-2e3eb7b945c2
                                                      go: upgraded v0.0.0-20171117235251-f95fa95eaa93 => v0.0.0-20220411215720-9780585627b5
                                                      go: upgraded v1.0.0 => v1.6.7
                                                      go: added v1.28.0
                                                      go: upgraded v2.0.0-20171026120407-29680c88e31d => v2.0.0-20220412131121-41eea1483964
                                                      (Fri, May 20  09:02:52 PM) (master|…)
                                                      $ go build .
                                                      ./handler.go:28:14: undefined: middleware.DefaultCompress
                                                      # status: 2 #
                                                      (Fri, May 20  09:02:58 PM) (master|…)
                                                      $ go doc middleware
                                                      package middleware // import ""
                                                      // snip
                                                      (Fri, May 20  09:03:12 PM) (master|…)
                                                      $ go doc middleware.Compress
                                                      package middleware // import ""
                                                      func Compress(level int, types ...string) func(next http.Handler) http.Handler
                                                          Compress is a middleware that compresses response body of a given content
                                                          types to a data format based on Accept-Encoding request header. It uses a
                                                          given compression level.
                                                          NOTE: make sure to set the Content-Type header on your response otherwise
                                                          this middleware will not compress the response body. For ex, in your handler
                                                          you should set w.Header().Set("Content-Type",
                                                          http.DetectContentType(yourBody)) or set it manually.
                                                          Passing a compression level of 5 is sensible value
                                                      (Fri, May 20  09:03:32 PM) (master|…)
                                                      $ subl .
                                                      (Fri, May 20  09:04:12 PM) (master|…)
                                                      $ go build .
                                                      (Fri, May 20  09:04:59 PM) (master|✚1…)

                                                      Took about 3 minutes to upgrade the packages and fix the broken dependency (they renamed a middleware). Bear in mind that the upgrade I did deliberately did not try to upgrade past semantic version changes in its dependencies. Probably it would take another half hour or more if I wanted to chase down whatever breaking changes happened there.

                                                      Suffice it to say, yarn cannot even install its packages, and the last time I tried this stunt a couple of years ago, I got past that and then ran into a problem with webpack that I couldn’t easily solve.

                                                      Go is just a much more stable ecosystem than Node or Python. It’s not as stable as say browser JS, where one can reasonably expect working code to work until civilization collapses, but it’s fairly stable. And it’s not magic. If there were a communal expectation of this level of stability, it could exist everywhere. It’s a social value to keep things working in the Go ecosystem, and it’s not elsewhere.

                                                      1. 2

                                                        The other day I returned to a Python package I hadn’t touched in about a year. The actual Python dependencies portion of updating it was done in a few minutes: I updated the supported versions of Python to those currently supported by upstream, did the same for the supported versions of Django, and then had to change a whopping four lines of code, all in a unit-test file, to deal with a deprecation in Django that had finally been removed.

                                                        The entire remainder of getting it ready for a new release was fighting with CI — updating from v1 to v3 of the GitHub Actions tasks for Python, which I mostly did by copy/pasting from a package by someone else who I trust.

                                                        I mention this because while you have anecdotes about Go projects updating more or less seamlessly, I have just as many about Python projects, and other people in this thread have anecdotes about Go projects breaking in ways they found annoying.

                                                        All of which is to say that you should stop trying to extrapolate from your anecdata to “Go is stable and values stability, while Python is not and does not”, because it just ends up looking silly when other people show up with their anecdata. Python is not uniquely “unstable” and neither Go nor Rust are uniquely “stable”. At best, some projects are sometimes lucky enough that they can give the false impression of stability in the language/ecosystem, despite the fact that the language/ecosystem is always moving on. And that’s why I have said, over and over, that the thing to do is embrace and accept change and build processes around it. Otherwise, you’re likely to wake up one day to find out that what you thought was stable and unchanging was neither, and that you are in for a lot of trouble.

                                                        1. 4

                                                          When I get a chance, I thought of an equivalently old Python package for me to try updating. I’ll try to do it this weekend or next week.

                                                          But I just don’t buy this:

                                                          Python is not uniquely “unstable” and neither Go nor Rust are uniquely “stable”.

                                                          I don’t have experience with Rust, so I have no idea there. I do have years of working in Python, JavaScript, and Go and my experience is uniform: Python and JavaScript routinely have problems that make installing/updating take a workday, and Go does not. I’ve already given a lot of concrete examples, and I’m sure I could dig through my git history and find more. At a certain point, all I can say is this is my experience and if it’s not yours, great.

                                                          1. 2

                                                            You’re still missing the point I’m trying to make, though, because you keep diving for anecdotes to support your case, and I keep trying to remind you that if we allow generalizing from anecdotes then your own claims will get contradicted, because there are people who can bring similar anecdotes about languages like Go that you think don’t have this “problem”.

                                                            My stance here is and always has been that no language or ecosystem is free of the need to keep up with dependencies and no language or ecosystem is free from breaking changes. People in this comment thread have posted experiences with Go breaking on them and you’ve mostly just ignored that or disallowed generalizations from them, while insisting that your own anecdotes support generalizations, about the objective state of particular languages/ecosystems.

                                                            That is what I’m trying to get you to see, and trying to break you out of. No number of anecdotes one way or another will make your case for you, because the issue is not an insufficient number of presented anecdotes.

                                                            At a certain point, all I can say is this is my experience and if it’s not yours, great.

                                                            This is very close to where I’m trying to lead you, but

                                                            • You never actually just say that and stop — you always feel a need to throw on another anecdote and then insist that your experience generalizes to objective statements about particular languages/ecosystems, and that’s where it goes off the rails.
                                                            • You never really accept other people having experiences that don’t match yours, and thus disputing your generalizations. I don’t have the kinds of problems you do with Python. You don’t have the kinds of problems other people have had with Go. But I’m doing my best to acknowledge experiences other than my own when I say that no language/ecosystem is free of this issue: I expect that there will be people who do run into it in Python, and also in Go, and also in Rust, and also in every other language/ecosystem. You seem to be trying to erase the experiences of people who run into it in languages that you subjectively don’t believe have this problem, because it doesn’t fit the “this language has it, that language doesn’t” generalization you want to make.
                                                            1. 3

                                                              your own claims will get contradicted, because there are people who can bring similar anecdotes about languages like Go that you think don’t have this “problem”.

                                                              I have given a ton of anecdotes and done two experiments in support of my view. On the other side, there is your a priori assertion that all ecosystems have the same problems (but is it of the same magnitude?) and an unreadably long article by a pro-Rust troll. If other people have anecdotes about Go being hard to upgrade, I’m happy to read them (assuming they can get to a point in less than 10,000 words) and theorize about why someone else might have that problem when I don’t. But that hasn’t happened in this thread.

                                                              1. 3

                                                                Well, right here in this thread someone mentioned problems with upgrading in Go. But the way you and another user were trampling all criticism of Go with insults — “troll”, “clickbait”, and so on — literally drove that person out of the thread.

                                                                I honestly don’t know what else to tell you. It seems very clear that you’re not prepared to hear anything that contradicts the narrative you’ve got going, so I guess I’ll bow out too since it’s pointless to continue trying to get you to acknowledge stuff that was literally right in front of you.

                                                                1. 2

                                                                  I could never tell what the actual problems were based on the links posted by m0th were. They linked to long pieces with no summaries, so while I would love to comment, I cannot. Maybe it’s because they felt targeted by ad hominem. I have a different theory for why, which is that the criticisms were scattershot and not serious. Again, if they want to come back and provide a useable summary, great.

                                                                  1. 1

                                                                    And yet I’d bet anything that if you’d been given short concise statements of problems you’d dismiss them as being trivial or minor on those grounds. Like someone else already said, I simply cannot assume that you’re actually trying in good faith to engage with criticism of a thing you like; instead all I’ve seen you do is smear and deflect and dismiss without argument.

                                                                    1. 2

                                                                      If you call this “without argument” I don’t know what to say. I really feel that the “without argument” side is yours, and it is very frustrating to me, because you’re a writer whose work I respect which makes it very hard for me to let this go. But when you write things like “anecdotes don’t count”… What is a personal dev blog but anecdotes? I agree that supporters of language X can be vociferous and annoying, and Go is one such X. In that case of Go, the vociferous attitude comes from having been attacked as using a “bad language” by people like fasterthanlime and our own experience of initially thinking things like the v2 are dumb and then slowly seeing the benefits and defending them. I agree that my initial claim was a bit too glib in its characterization, but I stand by a less inflammatory version of the same idea. I don’t agree that anecdotes aren’t the data of software engineering, and I don’t agree it’s somehow impossible to characterize software ecosystems with the understanding that any generalization will always be at best partial and incomplete but nevertheless it’s better to have the generalization than not.

                                                                      1. 2

                                                                        People brought up criticism of Go. You didn’t engage with it or argue with the points made — you ignored them, or dismissed them as too long to be worth your time to read, or just outright insulted the author of one piece of Go criticism.

                                                                        It’s hard for me to see a way to take this as good faith. It appears that you like Go, which is fine! Same with Python: if you don’t like it, then don’t like it — that’s fine!

                                                                        What’s not fine is the double-standard “my anecdotes are generalizable to broad claims about the language/ecosystem as a whole, but other people’s anecdotes are not”. Which is basically what you’re doing. If your anecdotes about Python generalize, then so do mine and so do other people’s anecdotes about Go. Which would then contradict the claims you want to make, which is probably why you’re being so cagey about not allowing others’ anecdotes to generalize while insisting that yours do. But again it’s going to be, at least, extremely difficult to take that as a good-faith argument.

                                                                        1. 2

                                                                          When I engage with the criticism, I’m just dismissing it. If I don’t engage because I don’t have enough detail (or drowning in irrelevant details so I can’t tell why monotonic time is supposedly bad), I’m ignoring it. It’s no win.

                                                                          1. 2

                                                                            As far as I can tell the only “engagement” you’ve given to criticism of Go was the second paragraph in this comment, and I think that’s being charitable since it’s also possible to read as a “well, this breaking change didn’t count, and it just means Go is great instead of the greatest”. If I said Python’s stability guarantees are good but just “don’t go far enough”, I doubt you’d view it charitably.

                                                                            You have, on the other hand, repeatedly dismissed critiques both by users in this thread and linked to on other sites as being too long or too “scattershot” or “fair to handwave … away”. Again: if I tried pulling that sort of thing in defense of Python you would, I hope, call me on it. You’re doing it here in defense of Go. I’m calling you on it.

                                                                            1. 2

                                                                              I wrote a long reply, decided to sleep on it, and I give up. You haven’t conceded any points. You’ve repeatedly accused me of bad faith. You’ve decided Python is as good as it gets and by definition nothing else can be better in some ways but not others. All I can say is I didn’t invent the phrase “the problem with Python”. Other people feel this way too. Maybe we’re wrong, and everything has the same problem to the same degree. I don’t think so though.

                                                              2. 2

                                                                You seem to be simultaneously saying there is nothing especially broken with python while defending the culture of constantly breaking changes and saying it is inevitable. I think your attitude sums up the problem, there are a lot of enablers.

                                                                1. 3

                                                                  This thread started out with an accusation that there is a problem that uniquely exists in certain languages, and does not exist at all in others. People were very confidently stating not that the breaking changes in Go are less frequent, or less annoying, or more justified — people were very confidently stating that breaking changes simply do not exist at all in Go.

                                                                  Don’t believe me? Here, and a lot of the rest of this sub-thread is people trying to quibble about “oh well that was just a change to (thing that shouldn’t count)” or explain “well these changes were justified and tolerable while those other languages’ changes aren’t” in order to somehow cling to the notion that Go is a bastion of perfect stable compatibility, even in the face of solid evidence that it isn’t.

                                                                  And I’m sorry if people don’t like to hear it, but I’ve just gotta call bullshit on that. Every language and ecosystem has breaking changes. There is no magic in Go that makes it somehow unchanging and perfectly compatible forever, nor is there any magic in the ecosystem that somehow makes all maintainers good stewards who never ever make unjustified changes.

                                                                  Still, people seem to be wildly exaggerating both the frequency of breaking changes in the languages they want to criticize, while minimizing for the languages they want to praise. It’s very human and very subjective, and very not a good way to have a reasoned discussion about this.

                                                                  Especially when it’s coupled with very obvious bad-faith tactics like the way critics of Go keep being dismissed or even smeared in this thread.

                                                                  If you dislike Python, then by all means dislike it. There are things I dislike about it. But there’s way too much in this thread, and in our industry, of people being unable to handle anything other than extremes — something is either 100% good or 0%, either completely perfectly backwards-compatible always or never — and of very clearly contradicting themselves and making really bad arguments to try to justify their subjective dislikes. Just come out and say you don’t like a thing and move on. You don’t also have to prove that it’s the worst in order to justify your dislike.

                                                                  1. 2

                                                                    I don’t know if python is especially bad. I personally found npm projects I have tried break when updating dependencies a few years later, and I have not had that experience in Go. Maybe I am just lucky.

                                                                    1. 1

                                                                      It seems to me that the disconnect here is about the scope of what constitutes “breaking changes” in a language. There certainly isn’t an objective definition! Some people consider only the core language and it’s standard library, and others include everything out to its tooling.

                                                                2. 1

                                                                  Okay, I tried this with a Datasette project from around the same time. (It was actually deployed again in early 2020, so it’s a bit fresher than the Go project, but whatever, close enough.) Again, Node didn’t work. I think the issue with that is that libsass is dead and doesn’t compile anymore, so you need to switch to dart-sass instead. In all likelihood, the fastest solution to the Node issues is to drop all of the dependencies and just start over from scratch with only my user code, since the dependencies were just there to build a Vue project.

                                                                  On the Python side, it wouldn’t work with Python 3.9, but when I used Python 3.7, I got it to run again. Terminal output is below. It only took 15 minutes to get it going, but compare this to Go, which works with the current version of Go even though the package predates modules (which caused a lot of breakage not covered by the Go 1 guarantee) and I got the dependencies upgraded in a total of 5 minutes. By contrast, the Python installations all took long enough that they break flow: since the installation is going to take a while, I switch away from my Terminal, which is chance for me to get distracted and lose my place. I think this project did pretty well because it used your recommended pattern of having a requirements-freeze.txt file and it had a Bash script to automate the actual install commands. But the error when UVLoop was broken was pretty demoralizing: I have no idea how I would fix it, so getting up to Python 3.9 or 3.10 would probably involve a lot more Googling than I’m willing to do for an internet comments example. Again, the simplest fix might be to just blow away what I have now and start from scratch. I think Datasette has been relatively stable in spite of being <1.0, so I suspect that it wouldn’t be that hard to get it working, but again, it’s more than I want to do for an example. A nice thing about Go is that most dependencies don’t use C, so when something does go wrong, like that middleware that was broken in the other project, you aren’t confronted with errors in a language you don’t know using a build system you don’t understand. In general, it’s just much less intimidating to get a Go project back up to speed.

                                                                  So this is fairly reflective of my lived experience: Node projects, especially those that use Webpack, break in ways that are more or less unfixable and need to be restarted from scratch; Python projects can be kept running if you are willing to pin old versions of Python but give a lot of scary compiler errors and don’t have clear paths forward; Go projects can typically be upgraded by typing go get -u ./... and maybe reading some release notes somewhere. Go isn’t perfect and there are still problems, but the quantity of problems is so much less than it creates a qualitative difference in feeling.

                                                                  $ gh repo clone baltimore-sun-data/salaries-datasette
                                                                  Cloning into 'salaries-datasette'...
                                                                  remote: Enumerating objects: 1017, done.
                                                                  remote: Counting objects: 100% (94/94), done.
                                                                  remote: Compressing objects: 100% (72/72), done.
                                                                  remote: Total 1017 (delta 63), reused 23 (delta 20), pack-reused 923
                                                                  Receiving objects: 100% (1017/1017), 53.78 MiB | 4.25 MiB/s, done.
                                                                  Resolving deltas: 100% (568/568), done.
                                                                  Updating files: 100% (74/74), done.
                                                                  (Sat, May 21  12:45:40 PM)
                                                                  $ cd salaries-datasette/
                                                                  (Sat, May 21  12:45:54 PM) (master|✔)
                                                                  $ more
                                                                  # salaries-datasette
                                                                  Public salary data acquired by the Baltimore Sun. Currently, we just have data from the state of Maryland for 2017.
                                                                  ## Usage
                                                                  Run `./ setup` to install locally. The script assumes you have either Python 3 or Homebrew for Mac installed. Run `./ setup-frontend` to install front end dependencies.
                                                                  Run `./ create-db` to create a SQLite database out of the provided CSVs.
                                                                  Run `./` or `./ serve` to run server at http://localhost:9001.
                                                                  Run the JS/CSS frontend server in another tab with `./ frontend`.
                                                                  `./ format` will format Python and Javascript code according to the coding standards of the project.
                                                                  `Dockerfile` is also provided for running/deploying with Docker. The image can be built with `./ docker-build` and tested with `./ docker`. The server only responds to correct hostnames (not localhost), so edit `/etc/hosts` to add `` and then test in the browser.
                                                                  (Sat, May 21  12:46:06 PM) (master|✔)
                                                                  $ ./ setup
                                                                  snip a ton of output from installing things including a lot of scary errors
                                                                  × Encountered error while trying to install package.
                                                                  ╰─> uvloop
                                                                  note: This is an issue with the package mentioned above, not pip.
                                                                  hint: See above for output from the failure.
                                                                  # status: 1 #
                                                                  (Sat, May 21  12:49:40 PM) (master|✔)
                                                                  $ # by reading the DOCKERFILE, I learn that this used Python 3.7 when it was made
                                                                  (Sat, May 21  12:51:16 PM) (master|✔)
                                                                  $ pyenv install 3.7.13
                                                                  python-build: use openssl@1.1 from homebrew
                                                                  python-build: use readline from homebrew
                                                                  Downloading Python-3.7.13.tar.xz...
                                                                  Installing Python-3.7.13...
                                                                  (Stripping trailing CRs from patch.)
                                                                  patching file Doc/library/ctypes.rst
                                                                  (Stripping trailing CRs from patch.)
                                                                  patching file Lib/test/
                                                                  (Stripping trailing CRs from patch.)
                                                                  patching file Modules/_ctypes/_ctypes.c
                                                                  (Stripping trailing CRs from patch.)
                                                                  patching file Modules/_ctypes/callproc.c
                                                                  (Stripping trailing CRs from patch.)
                                                                  patching file Modules/_ctypes/ctypes.h
                                                                  (Stripping trailing CRs from patch.)
                                                                  patching file
                                                                  (Stripping trailing CRs from patch.)
                                                                  patching file 'Misc/NEWS.d/next/Core and Builtins/2020-06-30-04-44-29.bpo-41100.PJwA6F.rst'
                                                                  (Stripping trailing CRs from patch.)
                                                                  patching file Modules/_decimal/libmpdec/mpdecimal.h
                                                                  (Stripping trailing CRs from patch.)
                                                                  patching file
                                                                  python-build: use tcl-tk from homebrew
                                                                  python-build: use readline from homebrew
                                                                  python-build: use zlib from xcode sdk
                                                                  Installed Python-3.7.13 to /Users/adhoc/.pyenv/versions/3.7.13
                                                                  (Sat, May 21  12:55:00 PM) (master|✔)
                                                                  $ ./ setup
                                                                  × Encountered error while trying to install package.
                                                                  ╰─> uvloop
                                                                  note: This is an issue with the package mentioned above, not pip.
                                                                  hint: See above for output from the failure.
                                                                  # status: 1 #
                                                                  (Sat, May 21  12:57:08 PM) (master|✔)
                                                                  $ # I become worried that maybe Pyenv didn't activate for some reason, so I try explicitly adding it to my PATH
                                                                  (Sat, May 21  12:59:30 PM) (master|✔)
                                                                  $ PATH=$HOME/.pyenv/versions/3.7.13/bin/:$PATH ./ setup
                                                                  (Sat, May 21  01:01:58 PM) (master|✔)
                                                    2. 3

                                                      This thread has grown a lot overnight while I was sleeping! I still haven’t caught up, but here is my top of head takeaway:

                                                      I think it’s fair to complain that the Go 1 stability guarantee doesn’t go far enough, but it seems like some people elsewhere in the thread are trying to claim that adding monotonic time somehow breaks the guarantee, which makes no sense to me. At this point, someone citing Fasterthanlime basically means they don’t know anything about Go, IMO because he has been so egregious in his misrepresentation of the language. Just recently, I had to pin a specific subversion of Go 1.18 in my tests because I needed a particular behavior for the Go tool. I don’t consider that to have been the Go team breaking the stability guarantee, just an example of how you can run into the limits of it when you’re talking about the tooling.

                                                      Obviously, it’s going to be impossible to never break anything. (Which is what I take from your points 1 and 2.) If nothing else, security problems can force breaking changes. Certainly, you need to get to a point where you’re happy enough to commit to a design before you stabilize it: Go was not stable at all before version 1. JavaScript wasn’t stable before ECMA etc.

                                                      But the question to me is whether the upstream developers take breaking changes seriously or commit them wantonly. To rephrase your points:

                                                      1. Projects which accept that breaking changes are fact of software development, so they do it whenever it’s convenient.

                                                      2. Projects which accept that breaking changes are fact of software development, so they do it as little as possible.

                                                      I was being a bit too flip when I wrote “You ‘just’ don’t make breaking changes.” That’s like saying, you just don’t write bugs. It is unavoidable sometimes. But there is a qualitative difference as an end developer in relying on a project which attempts to avoid breaking changes and one which does not.

                                                      In Node, I routinely run into software that will not run on the latest versions of Node. When a new version of Node comes out, I don’t think “Oh great, more speed, more features!” I think, “Ugh, crap, what’s going to break this time?” I had a hard to diagnose break in Babel (a very popular library!) that was caused not by a major version jump in Node, but a minor version bump. It was a waste of a day of development time for me for no reason. You wrote “as long as I pay the hosting bill there is no technical reason why it would suddenly stop working.” But in this case, there wasn’t anything I did intentionally which broke my setup. I just hadn’t pinned my version of Node, Homebrew moved up by a minor version because of some other thing, and suddenly I lost a workday. It also broke production because prod was only pinned to the major version not the minor or patch.

                                                      A similar example was that I had an Amazon ECS server using a Docker image to serve some Python app, and one day it just stopped working in prod. The problem was that when the Dockerfile was written, it didn’t specify the version of Python, and then the version where async went from a semi-keyword to a full keyword came out and broke the dependencies I was using. It wouldn’t have mattered if the Docker image had been already baked, but the way this project was setup, the Docker image would be periodically rebuilt from the Dockerfile whenever things would restart or something. That at least was relatively easy to track down and fix because the change of Python versions was noticeable once I started debugging it.

                                                      You call the /v2 thing a “hack” and it is ugly and inelegant, but it does solve this problem: when you make a breaking change, let users upgrade at their own pace, instead of having them accidentally get moved onto the new thing without being aware of it. (In browser JS, "strict mode" and type="module" also have the effect of being opt-in upgrades!) That’s really the core of the so-called “Python problem” to me. Just let me decide when to work through the breaking changes. I’m not expecting eternal support or whatever. I just want something that works today to work tomorrow unless I press the big red “attempt upgrade” button.

                                                      1. 5

                                                        At this point, someone citing Fasterthanlime basically means they don’t know anything about Go, IMO because he has been so egregious in his misrepresentation of the language.

                                                        I’m the one who cited the monotonic time issue. I’m having trouble seeing good faith in this statement, so I’ll bow out of this thread after this. But just so you understand that this is adamantly not the case: I’ve written Go since before 1.0, contributing to some of the largest projects in the ecosystem, and as a manager moved several teams over to using it (from Python, no less.) I also don’t think those credentials should be necessary for a critique.

                                                        I had a hard to diagnose break in Babel (a very popular library!) that was caused not by a major version jump in Node, but a minor version bump.

                                                        To reiterate, this has happened to me with minor version bumps in Go, due to both modules and stdlib changes. This is even worse because it was Go itself, not a third-party library.

                                                        You call the /v2 thing a “hack” and it is ugly and inelegant, but it does solve this problem: when you make a breaking change, let users upgrade at their own pace

                                                        This actually touches on a major reason why Go’s ecosystem could be so unstable before modules: you could either vendor which had its own can of worms, or rely on authors using this hack. Both patterns emerged as common practice only after years of Go devs experiencing breaking changes in libraries. You could argue that those instabilities don’t apply to Go itself, which I suppose is fair, but that applies to your argument about Babel as well.

                                                    3. 13

                                                      Go makes breaking changes all of the time. The last one that bit me was when modules were enabled by default. Suddenly packages starting behaving differently depending on a variety of factors, including the version number of the package:

                                                      Python at least had the sense to bundle up all the breakages in a major version change, which was released 15 years ago.

                                                      1. 8

                                                        Eh no. Python does not follow semantic versioning and makes intentionally breaking changes between minor versions. In fact Python 3.x releases come with documentation titled “Porting to Python 3.x” which lists all breaking changes made intentionally.

                                                        1. 6

                                                          Python makes breaking stdlib changes in minor version releases. Not even remotely the same as a tooling change.

                                                          1. 3

                                                            I’ve never been affected by stdlib changes in python in a minor release. I have by go, where it suddenly stopped respecting system DNS settings. Or there were the backwards-incompatible changes to time, maybe best summarized here.

                                                            1. 4

                                                              I believe you, but you are lucky. With search I found things like which matches my experience.

                                                              1. 3

                                                                I know this isn’t likely to change anyone’s mind, but the thing you linked is an example of something that had been deprecated and raising deprecation warnings for years. Here’s an example warning I get from a Python 3.7 install I had lying around (oldest I could find, I tend not to keep EOL’d Pythons on my personal laptop):

                                                                DeprecationWarning: Using or importing the ABCs from ‘collections’ instead of from ‘’ is deprecated since Python 3.3,and in 3.9 it will stop working

                                                                And it appears they actually held off and didn’t finally remove that until 3.10. That’s at least four years after Python 3.7 (which issued that warning). It’s nearly ten years after Python 3.3.

                                                                Python does use a rolling deprecation cycle, yes. So do some major Python projects like Django. This means you should ensure you’re bubbling up deprecation warnings in your CI, and probably read release notes when new versions come out, yes.

                                                                But these things don’t just come surprise out of nowhere; they’re documented, they raise warnings, they’re incorporated into the release cycles. Knowing how Python’s deprecation cycles work is, for a Python user, the same kind of table stakes as knowing how semantic versioning works is for a Rust/cargo/crates user – if you updated a crate across a major version bump and it broke your code, everyone would tell you that you could have seen that coming.

                                                              2. 2

                                                                It happens, no doubt, but IME Python’s changes are much more egregious than Go’s few changes over the years. Where Go does make changes, they are typically as solutions to bugs, not gratuitous API changes.

                                                                Also, speaking personally, I try to be as objective as possible and is not an objective source in any sense. It’s clickbait. There’s no pretence of objectivity.

                                                                1. 13

                                                                  I’d like to point out that this thread has gone from a claim that Go has maintained backwards compatibility to, now, a claim that Go’s backwards-incompatible changes just aren’t as “egregious” as Python’s, and to what’s basically an ad hominem attack on someone who criticized Go.

                                                                  Which is kind of a microcosm of this whole framing of “Python problem” or “Node problem”. What it’s really about is not some magical language and ecosystem that never has backwards-incompatible changes, what it’s about is subjective taste. You think Go’s breaking changes are not as “egregious” and are “solutions to bugs” while Python’s are “gratuitous”. But that’s a subjective judgment based on your tastes and opinions. Someone else can have an opposite and equally-valid subjective opinion.

                                                                  Or, more bluntly: “No breaking changes” nearly always turns out to actually mean “Breaking changes, but only ones I personally think are justified/tolerable, and I don’t think yours are”. Which is where this thread predictably went within the space of just a few replies.

                                                                  Getting back to my original claim: change is inevitable. Entities which produce software can adapt to it and make it a normal and expected part of their processes, or they can suffer the consequences of not doing so. There is no third option for “all external change stops”. Nothing that lives does so changelessly.

                                                                  1. 3

                                                                    Or, more bluntly: “No breaking changes” nearly always turns out to actually mean “Breaking changes, but only ones I personally think are justified/tolerable, and I don’t think yours are”. Which is where this thread predictably went within the space of just a few replies.

                                                                    No I don’t think so. It has nothing to do with opinion and everything to do with experience. What someone sees directly is how they perceive reality. I’m expressing my perception of reality as someone who has experience of both Python and Go, and m0th is expressing their experience.

                                                                    I’m not minimising m0th’s view, which is why I phrased it as “in my experience”.

                                                                    Getting back to my original claim: change is inevitable. Entities which produce software can adapt to it and make it a normal and expected part of their processes, or they can suffer the consequences of not doing so. There is no third option for “all external change stops”. Nothing that lives does so changelessly.

                                                                    Change is inevitable, I agree, but I do think the degree matters. Python makes breaking changes often, removing APIs, etc. Go does not and only with very good reason.

                                                                    1. 3

                                                                      Change is inevitable, I agree, but I do think the degree matters. Python makes breaking changes often, removing APIs, etc. Go does not and only with very good reason.

                                                                      Again, the original claim at the start was that there are languages which don’t have breaking changes, and Go was specifically named as an example. That has now been walked back to the kinds of statements you are making. And your statements are, effectively, just that in your opinion one language’s (Go) breaking changes are justified and not too frequent, while another language’s (Python) breaking changes are not justified and too frequent. Which is just you stating your own personal tastes and opinions. And it’s fine as long as you are willing to admit that. It is not so fine to present one’s personal tastes and opinions as if they are objective facts. It also is not so fine to engage in the kind of ad hominem you did about criticism of Go.

                                                                      1. 3

                                                                        Again, the original claim at the start was that there are languages which don’t have breaking changes, and Go was specifically named as an example.

                                                                        I didn’t make that claim, so I’m not sure why you’re arguing with me about it.

                                                                        Which is just you stating your own personal tastes and opinions. And it’s fine as long as you are willing to admit that.

                                                                        I did? I explicitly said “In My Experience”. I’m not sure how I can be any clearer.

                                                                        It is not so fine to present one’s personal tastes and opinions as if they are objective facts. It also is not so fine to engage in the kind of ad hominem you did about criticism of Go.

                                                                        What are you even talking about? I legitimately can’t even understand what you’re referring to. Where’s the “ad hominem” comment I made? I think the most controversial thing I said was

                                                                        “Python makes breaking stdlib changes in minor version releases.”

                                                                        Which is objectively true as mentioned by other commenters. I made no judgement about whether it was justified or not. It’s also not “ad hominem”, which would require Python to have a position on this.

                                                                        Anyway, I think you’re taking this way too personally for some reason. I like Python, I have used it for many years. But I’m out.

                                                                        1. 3

                                                                          Where’s the “ad hominem” comment I made?

                                                                          IIUC, ubernostrum is referring to you explaining that is not a reliable source of information. That’s not an ad hominem though. His attacks are so scattershot that if someone cites them, it’s totally fair to just handwave it away.

                                                                          1. 3

                                                                            Where’s the “ad hominem” comment I made?

                                                                            It’s this:

                                                                            Also, speaking personally, I try to be as objective as possible and is not an objective source in any sense. It’s clickbait. There’s no pretence of objectivity.

                                                                            You don’t bother to engage with the content, just brush off the author with a quick smear and then move on. Which is a problem, because the content is reasonably well-argued, from my perspective as someone who has a bit of Go experience and initially couldn’t quite describe why it rubbed me the wrong way. Putting into words the way that Go (which is not unique in this, but is a prominent example of it) likes to pretend complex things are simple, rather than just admitting they’re complex and dealing with it, was extremely useful for me, both for giving me a way to talk about what bugged me in Go and because it relates to things I’ve ranted about in the Python world previously (most notably why Python 3’s string changes were the right thing to do).

                                                                          2. 2

                                                                            Go the language doesn’t have breaking changes, in the sense that code written against a 1.0 language spec will — generally— continue to compile and run the same way against any 1.x language spec compiler. This isn’t an absolute statement of fact, but it’s a design principle that’s taken very seriously and is violated very rarely. The tooling doesn’t necessarily abide the same rules.

                                                                            1. 1

                                                                              Your comment is a bit of a roller coaster.

                                                                              • Starts with “doesn’t have breaking changes”
                                                                              • Immediately qualifies with “in the sense that…”
                                                                              • Then walks it back with “This isn’t an absolute statement of fact”
                                                                              • Then walks it back even further with “is violated very rarely”
                                                                              • Then throws it all out the window with “The tooling doesn’t necessarily abide the same rules”

                                                                              I’m not even sure what that’s trying to say.

                                                              3. 9

                                                                Go 1 has been stable for 10 years now.

                                                                If you just ignore all those breaking changes they made..

                                                                1. 1

                                                                  I only remember Modules. After Modules landed I never experienced any breaking build when updating to a new Go version. A whole different story was updating Java versions and the dozens of subtle ways they can break your service, especially at run-time (caused by some Spring DI magic).

                                                                  1. 1

                                                                    What breaking language changes have been made?

                                                                    1. 3

                                                                      Modules seems like the biggest change.

                                                                      1. 3

                                                                        I mean, Go module is not part of Go 1 stability guarantee. In my opinion, this shows how limited Go’s stability guarantee is. Go 1 is stable, but Go is not, at least if you are using “go build” to build your Go code.

                                                                        1. 2

                                                                          I agree but that’s not a language change, and the impact is not as significant as random language changes. I can still use and build a package from 8 years ago with no issue. And whatever tool they happened to use to manage dependencies 8 years ago will still work fine (probably).

                                                                          1. 3

                                                                            I saw some breaking changes in one of our old projects. Written in the Go v1.4 era, IIRC. I checked it with a Go v1.14 release, and boom, it doesn’t compile due to the module changes.

                                                                            Yes, it wasn’t that hard to fix (it only took a few minutes of Internet searching), but I still count that as a breaking change.

                                                                2. 2

                                                                  When was there ever a breaking change in a new version of ecmascript? Node is not a language.

                                                                  1. 4

                                                                    There’s a bunch of things that vary in stability.

                                                                    • V8 the interpreter breaks working JS code very seldom
                                                                    • The libraries that come with Node break JS code a little more often, but still IME not very often.
                                                                    • V8 breaks native extensions very often. The NodeJS ecosystem discourages writing them because of this.
                                                                    • Some add-on packages from npm break their consumers all the time.

                                                                    Many packages are extremely careful not to break consumers. Others are less so. The experience you have with backwards compatibility tends to track the backwards compatibility stance of the worst thing in your entire dependency tree. When you have tens of thousands of transitive dependencies, you usually end up seeing 99.99%-ile bad behaviour somewhere in there at least once.

                                                                    1. 1

                                                                      The core problem is that a handful of “core” packages break, and nodes ecosystem is way too much about plugins, so many things that you use have 3 layers to them (all maintained by different people)

                                                                      The ecosystem would be a lot more stable if we were vendoring in packages more

                                                                    2. 2

                                                                      Node is a development target. The pure abstract notion of a “language” doesn’t really matter here, because people write code for Node specifically.

                                                                      And Node does make breaking changes. It literally makes semver-major releases, with backwards-incompatible changes that can break existing npm packages (mainly due to changes in the native API, but there also have been changes to Node’s standard library).

                                                                      For example, I can’t build any projects that used Gulp v3. Node broke packages that it depends on, and I either have to run deprecated/unsupported/outdated version of Node, or rewrite my build scripts. OTOH a Rust/Cargo project I’ve developed at the same time still works with the latest version of Rust/Cargo released yesterday.

                                                                      1. 2

                                                                        Yes, that’s why I said “Node ecosystem” which breaks all the damn time and not “browser ECMAScript” which has not broken since standardization.

                                                                        1. 2

                                                                          The quote you posted says language, not ecosystem. Your comparison was a false equivalency.

                                                                        2. 1

                                                                          ecmascript itself is a language but not a language implementation, so while you can write a lot of code in ecmascript, at the end of the day the ecmascript specification can’t run it, so you’ll need to run it using an implementation, that’s why people are talking about the Node ecosystem, the language didn’t break the ecosystem did

                                                                      1. 6

                                                                        I love the simple explanation of a WAL. Might be the best part of the article, imho.

                                                                        1. 2

                                                                          happy you liked it, indeed the article was written more to explain concepts like that using simple components rather than, idk, meeting the demand of shell-based message queues (lmao)

                                                                        1. 9

                                                                          Unlike JSON, Nickel is programmable

                                                                          IIRC* this was a deliberate decision by Crockford. Because the entire point of JSON is to not be a programming language, and to be universally compatible with all programming languages.

                                                                          This is also why it doesn’t have comments – people would have used comments to add extensions to the language (comments as pragma directives) and suddenly you don’t have JSON, you have 20 different varieties of the language JSON, and the language is utterly useless as a universal language data format.

                                                                          IMO this is what people trying to replace JSON just don’t get. We already had a good executable interchange format 20, 30 years ago, it was called an “S-Expression” (Or XML, or YAML). Lisps are super easy to embed and they are super easy to write on a programming side (Which is why we have Greenspun’s Tenth Rule). They are easy to read and parse mechanically (inb4 “parentheses!!” - Vim from 1970 supports % to jump between brackets, and highlights matching brackets, so why doesn’t your editor?), but they also totally lack any discrete form of interoperability between languages, because now everything that parses your data format also needs to execute it, and to do that it has to execute it in the same way, and now you have fifty different specifications masquerading as one.

                                                                          As mentioned, you can see this with YAML and XML too, although there are additional problems there like “Why am I writing so many damn angle brackets” and “Why is my configuration reliant on whitespace?”. And all of them suffer from “Why has my configuration file’s parser blown up my computer when handed this suspicious config file I copy pasted from IRC/Discord/SO”.

                                                                          So fundamentally, this is a bad idea IMO and it comes from not respecting the journey and history that led to JSON. Personally, I really really hate writing JSON, but the alternatives are worse and there’s a reason why those alternatives are worse that goes beyond syntax alone. The executability and the universality of all of them leave them vulnerable to say, the billion laughs attack, among others. If I’m not remembering incorrectly (I read this in a book whose title I have forgotten :(), in the 90s this was one of the major problems with either Postfix or Sendmail (that might have been them having an accidentally executable configuration file).

                                                                          We learned and Crockford’s inspiration was adopted everywhere because it tangibly fixed those problems by:

                                                                          • Not making a data format executable

                                                                          • Making it simple as hell while still retaining utility

                                                                          • Deliberately making sure that it would be irritating to create variations (to the point of not even supporting comments).

                                                                          All so that any language, any program, and any system can read it and understand the contents, rather than having slightly different versions for every system under the sun, and then doubling that for proprietary extensions and all the fun we used to have with configuration files.

                                                                          And now some people come along and go “Wait why aren’t our configuration files executable! Why don’t they have comments?!” and the cycle unfortunately repeats, again.

                                                                          When will we learn for good?

                                                                          * - I remember the design justification used to be listed on but the site has changed somewhat since I saw it last, and the only alternative source I can find goes to a Usenet / Google Groups post that has since been purged.

                                                                          See also:

                                                                          1. 7

                                                                            JSON is a (mostly) reasonable data interchange format but it’s not a great configuration language. A data interchange format definitely should not be programmable. The degree of programmability that you want is still a bit of an open question to me.

                                                                            I generally think UCL strikes a pretty good balance: it has simple macros and it also has built-in logic for merging different sources, including the ability to delete nodes from the tree that were present in an earlier version. It has the problem of being a single implementation of the specification (which is therefore largely defined by the implementation) and it’s not clear that the parsing logic for the combination modes could be properly abstracted.

                                                                            Comments definitely are useful in a configuration language because it’s intended to be human readable and it’s useful to be able to store why you made a particular configuration change. The degree of programmability, in part, depends on the flow of configuration information from the user to the program. In a classical UNIX system, config files were files in the filesystem and you’d deploy a service by installing it, editing its config file, and then starting it. This is nice and simple because the config file is just a file and so you can version it as you would any other file. If it’s too complex to write by hand then you can write a program to generate it (if you’re Sendmail then this is your recommended way of writing a config file). There are two important things here:

                                                                            • The person writing the config file is trusted (and so won’t be writing malicious configuration files).
                                                                            • The code that runs to generate the static config file does not have to run with the privileges of the tool and the tool can parse a much simpler version of the config.

                                                                            It’s quite easy, for example, to write a stand-alone tool that takes a set of UCL config files and spits out minified JSON of the combined result. That’s fine if you assume that the only things that need to edit the config are a text editor for the input and the main program for the output.

                                                                            If you want to be able to deploy containerised versions of the service with small tweaks to the config then you need something else to take your generic config and tweak it. This probably wants to operate on the simplified JSON but you might want to use some of the file separation bits of UCL (or whatever) to be able to split the instance-specific config, rather than doing that as a tree transform later. If you want to provide a graphical tool for editing the config then it needs to preserve comments and whitespace in the input so that diff tools work well, which means that it needs to operate on the source material (or possibly needs to just provide new override files).

                                                                            I suspect that JSON or even something like BSON is what you want for the final version of the config that a program loads. It can be parsed very quickly and has an in-memory size that’s proportional to the size of the file (unlike XML, with entities) and so you can restrict yourself to parsing config files of a plausible size. If you want to do something more clever then you add some tooling in front.

                                                                            To my mind, the biggest problem with configuration at the moment has nothing to do with the file format. The problem is the lack of schemas. This is why I wrote a tool that takes a JSON schema and generates C++ classes reflecting that format for use with UCL. This forces you to have a schema, which ends up embedded in your final binary (configs are validated against the schema before being used) and so can be trivially exported for other tools to use. JSON Schema does have comments (as special nodes) and so you can embed the description of the meaning of configuration options in the schema and then expose it in configuration editing tools. There’s also the problem of how you get configurations to stateless VMs / containers, though I believe etcd is trying to address that.

                                                                            By the way, nlohmann JSON lists this URL as the source for your claim about comments but it asks me for a Google account so I can’t verify that it actually does say what you and they quote.

                                                                            1. 3

                                                                              All so that any language, any program, and any system can read it and understand the contents

                                                                              I agree with this in broad strokes but have to disagree in the specifics. Parsing JSON is a minefield and every language, program and system has its own ad-hoc, non-documented and buggy way of doing it.

                                                                              1. 2

                                                                                Yep, this is a great comment. I can’t wait for my solution to be mature enough for people to understand what I am on about but I think I have the answer.

                                                                              1. 9

                                                                                What’s the actual definition of beta-optimal? Searching gives a lot of medical and finance related things, but nothing that looks applicable here.

                                                                                1. 2

                                                                                  yeah, “beta optimal” afaik, is not a term in the literature, it’s probably referring to what Jean-Jacque Levy called “optimal reduction”

                                                                                  1. 3

                                                                                    Yeah, I think it’s a minced named for the optimal beta-reduction of an expression.

                                                                                    disclaimer: I’m not a qualified academic in computational theory or lambda calculus

                                                                                    As I understand, Levy recognised that in terms of reduction (i.e. application) strategies, simply beta-reducing expressions may not always offer the most optimal form for minimising the number of reductions needed since it’s possible to have non-reducible terms with identical subexpressions. As such, that would mean performing the same computation multiple times when reducing with known values.

                                                                                    The definition is a bit loose, but the optimal reduction can be thought of as the expression rewritten such that a graph of it’s evaluation contains a minimal number of nodes, which provides a different model for optimizing the computation done for an expression than, for example, a call-by-need strategy as used by Haskell, where identical expressions are made memoizable by the runtime and lazily produced as evaluated.

                                                                                1. 19

                                                                                  This is nice, interesting work, but I’m uncomfortable with the hyperbole in the description. It sounds like a small amount of effort is devoted to correctness and clear specifications, and a large amount of effort is devoted to good communication (great!) and finding (improbable) blockchain applications (uh…).

                                                                                  It’s great that there is hobbyist interest in optimal reduction outside academia. Academic researchers never quite managed to get good-looking performance numbers out of “optimal reduction” work (or other similar topics: geometry of intraction, etc.), and it was never quite clear whether it was due to fundamental issues with the work, or a lack of effort and interest in the lower-level aspects of language implementation. We are learning some new things with these new attempts. Being competitive with mature implementations is impressive.

                                                                                  On the other hand, many things remain unsaid here, and I find it a little worrying:

                                                                                  • What is exactly the reduction strategy here, is it clearly specified somewhere ( looks like a tutorial with several parts missing). Are the details of the expected behavior, to be checked against the implementation, written down in full somewhre?
                                                                                  • What is actually the subset of programs that can be evaluated correctly using this strategy? Are all lambda-terms evaluated correctly, or only some of them? (Past work had restrictions to “elementary affine” stuff, etc.) Can we check whether a given program is in the fragment that will be correctly evaluated, either statically (before we execute the program) or dynamically (during execution with this runtime)?

                                                                                  Note: the description of this works points out in many place that performance numbers are comparing a prototype (this work) with a mature implementation (GHC); it then suggests that this is impressive and indicates that an optimized implementation (like GHC is) could do even better. I think that this is a bit misleading: mature implementations also handle a lot extra features and concerns not covered in the prototype, and many of those degrade performances instead of improving them. (For example: to improve debuggability, have clear reports when failure happens, etc.)

                                                                                  I’m a bit worried about the combination of interesting, valuable hobbyist research (the optimal evaluation) with blockchain stuff. The blockchain community has a tendency to make hyperbolic claims for the purpose of visibility, investment and speculation, that is not conductive to learning truths about software systems.

                                                                                  1. 13

                                                                                    To add to this, am a bit concerned about the emphasis on microbenchmarking while also not disclosing that the VM does not have a real allocator and it just mallocs 32 GBs of RAM.

                                                                                    1. 2

                                                                                      Avoiding memory allocation as a performance optimization

                                                                                    2. 5

                                                                                      I followed this work for a while. As far as I know, there is no specification for reduction strategy (other than the implementation itself, that is), no documentation on what is the important innovation, no words on restrictions. As I understand, elementary affine restriction is still there in some form, and HVM silently evaluates it incorrectly without any static or dynamic check. HVM’s position is that it’s not their problem, they plan to document the restriction, compilers should figure out how to check, and it has not been a problem in practice in their experience.

                                                                                      I understand this is frustrating for someone in academia. But in my opinion, it’s academia’s job to solve this. What’s in it for Victor? Victor isn’t interested in solving this, and I think it is understandable. Interested researcher in academia should reverse engineer and figure it out by themselves. They also could, I mean, ask Victor. I think Victor won’t be hostile to such effort, but it’s not on Victor to do this.

                                                                                      1. 9

                                                                                        I understand this is frustrating for someone in academia. But in my opinion, it’s academia’s job to solve this.

                                                                                        Just to be clear, I’m not suggesting that this work should be done following “the academic way”, in particuliar, by publishing research papers at journals/conferences. I certainly agree that this is not a requirement one should have on our software systems (although it’s of course great when it happens), and it is unnecessarily difficult unless one collaborates with someone familiar with the academic practices.

                                                                                        The things that I mentioned (to summarize: “having a clear description of what this is doing”, and “having a clear understanding of what the limitations are”), are not academic pursuits, they are what anyone should do if you want to write any software that works well. If you don’t do this, the only reason to believe that this is not utterly broken (most software at this level of complexity is) is the trust one has in the author. Good software systems should no rely on this! There should be enough documentation around that anyone interested can, at reasonable cost, dive into the specification and correctness claims, and see for themselves if they believe that the design is solid and that the implementation matches it.

                                                                                        Publishing papers would require this clarification work to be done first, but the reason to do it is not to publish papers / academia something, it is to build a system that works well, and will continue to work well as it evolves.

                                                                                        At a more basic level: what is there in this system right now to ensures that it works correctly? To my knowledge: nothing. There is a bench repository in the code, and no test directory. It’s rather easy to think of low-effort ways to test this, in particular property testing (quickcheck-style) against a known-valid implementation of reduction (possibly very simple). But this relies on the fact that we know what the software should do, what are the expected correctness properties, otherwise what would we test exactly?

                                                                                        “Academic success” certainly doesn’t need to be a priority for the author, but I think that “software that works well and can be trusted” should be. It should be a priority for everyone, but especially if your intended application domains are “a proof assistant” (as in previous iterations, Kind/Formality I believe?) or “a blockchain”… or whatever.

                                                                                    1. 2

                                                                                      it doesn’t seem like most people outside of functional programming are concerned with it as a problem

                                                                                      think about it as a self-selecting group; people that think raw pointers are a bad idea also stop using C

                                                                                      1. 3

                                                                                        Warning: this story ends on a cliffhanger. I recommend waiting to read this until Part 3 is posted “next week”.

                                                                                        1. 6

                                                                                          My bad :’-( Really wanted to tell entire story in one go, but got carried away!

                                                                                          1. 2

                                                                                            don’t sweat it, all good second acts end on a cliffhanger

                                                                                        1. 18

                                                                                          Just a small somewhat OT note:

                                                                                          All in all the experience has given me a serious love/hate relationship with type theory and comp sci papers, mostly verging on the “hate” side. On the one hand, if I replace the variables â and â’ in your algorithm with paramtype and rettype and then go “…wait, is THAT what it’s doing? That’s all? Oh.”, then you’re failing as a communicator.

                                                                                          \rant{ YES. I find math in CS papers frustrating because of its ubiquitous usage of single-letter names for everything. They’re difficult to remember, and I hate having to go back several pages to remind myself what they defined W as, or what the ζ operator is supposed to represent.

                                                                                          All coding style guidelines (except maybe APL’s) discourage single-letter identifiers, outside of trivial cases like loop variables. Why should academic papers hold onto a convention whose purpose was to conserve chalk and pencil lead? }

                                                                                          1. 5

                                                                                            There have been a few books over the years that teach mathematics using a programming language, and I rather enjoy them.

                                                                                            The Haskell Road to Maths and Logic is one. Coding the Matrix is another.

                                                                                            1. 2


                                                                                              Because otherwise you can’t pass the peer-review, imagine if they were able to verify that and find out it doesn’t actually do anything new ;)

                                                                                              1. 1

                                                                                                Why should academic papers hold onto a convention whose purpose was to conserve chalk and pencil lead?

                                                                                                I think this is a common misunderstanding of this practice. I would say the purpose is to keep the syntax as small and unobtrusive as possible so that you can a) focus on semantics, and b) fit as much as possible in your head at once. There’s also the aspect of making it easier to manipulate the symbols, to ease exploration.

                                                                                                There’s certainly a tradeoff in terms of the slope of the learning curve, but when conventions are shared among a group of active researchers, it more than pays for itself. But it does make it harder for outsiders to break in. That seems to be a fundamental tradeoff, though, at least at some level, between optimizing for beginners vs optimizing for experts.

                                                                                                1. 1

                                                                                                  But what is so different about math, that it overrules all the very good arguments made against single-letter identifiers in programming?

                                                                                                  Especially the argument that “your code is going to be read many more times than it’s written, so clarity at the expense of a few more keystrokes is a good tradeoff.” Because the whole point of an academic paper is to be read and understood.

                                                                                                  I would agree that common constants, variables and operations — like pi, or c in physics, or O() in CS — work fine as symbols. But when you make up an identifier for a specific local purpose, a name is so much better.

                                                                                                  1. 1

                                                                                                    A few disjointed thoughts:

                                                                                                    1. Mathematical notation long predates programming.
                                                                                                    2. Verbosity doesn’t always improve clarity. In fact, the opposite is often the case, e.g. Java.
                                                                                                    3. Single letter variables are fairly common in Haskell, especially in small helper functions that are best understood by their semantics.
                                                                                                    4. As long as you clearly define your notation, I don’t see how a longer identifier, e.g. OrderOfComputationalComplexity(n), is better than a short one, e.g. O(n). Unfortunately, it’s not particularly common for journal articles to specify which notation they are using, but rather to assume the readers general familiarity with the existing literature and notations, which forces newcomers to detour through introductory text books and/or review articles before they can understand the latest articles.
                                                                                                2. 1

                                                                                                  Mostly because of archaic university standards that impose a page limit on the paper.

                                                                                                1. 15

                                                                                                  This is a very poorly written article even ignoring the consistent misspelling of Sybil. It would be far better without the attempts to prejudice the reader (or “suckers” if they are interested in trying it out) with the charged language.

                                                                                                  The technical argument, when extracted from the surrounding pathos, seems akin to that used in around 1999 when the web was deemed too slow/clunky/expensive to replace desktop applications.

                                                                                                  That said, it does highlight the problematic role of gatekeepers but glosses over the fundamental issue of who appoints them and somehow ends up praising them, especially the financial gatekeepers. Someone should check in with the hundreds of thousands of sex workers whose legal incomes are in the hands of the moral pressure groups targeting mastercard and visa.

                                                                                                  1. 32

                                                                                                    Someone should check in with the hundreds of thousands of sex workers whose legal incomes are in the hands of the moral pressure groups targeting mastercard and visa.

                                                                                                    One of my close friends is a sex worker who is very, very tired of this argument. According to her, everybody saying “crypto is good for sex workers!” don’t actually know anything about the realities of sex work.

                                                                                                    1. 18

                                                                                                      Most arguments for cryptocurrencies in other spaces (helping the “unbanked”, remittances, even simple payments) seem to made by people who have only learned about these business practices from particularly insular web fora.

                                                                                                      1. 6

                                                                                                        I’m arguing against gatekeepers, not for crypto

                                                                                                        1. 3

                                                                                                          If it helps, an example I actually have first-hand experience with is the case of cannabis dispensaries. Cannabis remains federally illegal in the US but individual states allow cannabis usage to varying degrees. The states in the US have a patchwork of differing regulations regarding product storage and certification. Most importantly, many of these dispensaries have a hard time entering the regular banking system and have to store a lot of money in cash. As such many of these dispensaries are frequent targets for robberies and break-ins because they make such lucrative targets. Many who run these dispensaries have enough on their plates as-is managing the complicated requirements of the industry, and managing cash is really not their thing (at least the ones I know.) Cryptocurrency is a fairly good solution for this space.

                                                                                                          (P.S. I don’t really want to draw this out into a huge flamewar or start discussing the meta opinions of “blockchain people” or “not blockchain people” and the spaces they frequent as happens on most tech aggregators. I find those exhausting and my comment is as-is almost completely off-topic for this site. I don’t mean that specifically to you as much as to the general audience of the site. I probably won’t respond to this. Apologies in advance.)

                                                                                                        2. 10

                                                                                                          And the problem are not the moral pressure groups? But VISA itself? What do you think will happen once all sex workers move to the new shiny solution? That they won’t try to ban that next? Judicial courts don’t care that something is very technically uncensorable, they’ll take the whole damn blockchain down if they have to.

                                                                                                          1. 13

                                                                                                            Right, this is kindof an important point; it’s not hard to come up with illicit-but-sympathetic use cases, whether it’s supporting things that are illegal but shouldn’t be, or are just marginalized by the existing financial system but shouldn’t be.

                                                                                                            I was sympathetic to bitcoin early on, not in the least because in 2010 – about a year after bitcoin was new – Chelsea Manning & Wikileaks leaked video of the US military murdering Iraqi civilians. The major payment processors at the time all shut down donations to Wikileaks. This is in a country where freedom of the press is better protected than it is in most places. Watching that made it very easy to buy into the idea that uncensorable financial transactions were a very good idea.

                                                                                                            …but effective anti-censorship technologies need uncontroversial use cases for cover. E.g. encryption is critical for hiding credit card numbers in flight, and providing government back doors would seriously weaken the technology, and likely be a boon for criminals. There are many non-illicit reasons why one might want privacy in one’s communications, and government censorship is only one of them.

                                                                                                            So, even if you’re of the opinion that the anti-censorship angle is enough to make cryptocurrency good in principle, it is doomed unless you can find something less controversial that is also compelling. So far, I haven’t seen anything like that, and that’s before getting into all of the actual problems with cryptocurrencies.

                                                                                                        1. 0

                                                                                                          Stopped reading at

                                                                                                          “But first some terminology. A distributed system is composed of multiple, identified, and nameable entities. DNS is an example of such a distributed system…”

                                                                                                          1. 14

                                                                                                            Please point out what’s wrong about this. DNS is definitionally descentralized. “Decentralised systems” that go down along with AWS contrastingly, are not.

                                                                                                            1. 4

                                                                                                              This is about being a distributed system, and not so much about decentralization. It is a balant oversimplification written in a condescending tone of “let me tell you what this is and what it is not”. Nothing technically wrong, just off putting imo.

                                                                                                              1. 10

                                                                                                                I think the author is just trying to establish what he considers part of the definition to achieve a common ground for the rest of the article

                                                                                                                1. 2

                                                                                                                  No, this is ideology driven.

                                                                                                                  1. 8

                                                                                                                    So DNS isn’t distributed?

                                                                                                                    1. 6

                                                                                                                      Facts can often seem like ideology when people have vested interests that make the facts inconvenient.

                                                                                                                      1. 3

                                                                                                                        It isn’t a fact that a distributed system has identified and nameable entities by definition. Many distributed systems do but not all. This doesn’t make the author wrong per se, but it does make the work undeniably ideological.

                                                                                                              2. 5

                                                                                                                For comparison, consider my recent definition:

                                                                                                                Distributed computing, as a subfield, studies how individual computers behave when they are one of many computers which simultaneously access some common resources. … [A] distributed system is a collection of computers which are all acting as a distributed computer because they happen to access some common resource. This is a subtle change in what I said before, but it is through the systems view of objects; we want to look at properties of the entire system under study and not just properties of a single machine or resource.

                                                                                                                A Bitcoin miner is part of a distributed computer; Bitcoin is a distributed system. A DNS nameserver is part of a distributed computer, a collection of DNS nameservers is a distributed computer, and DNS is a distributed system.

                                                                                                                The author assumes that we know all of this, but is not prescribing some new usage of these existing terms.

                                                                                                                1. 3

                                                                                                                  Your definition is very good.

                                                                                                                  The author might be assuming, however also happens to be oversimplifying them to the extreme of numbness. “But first some terminology. A distributed system is composed of multiple, identified, and nameable entities.” goes a bit far from “terminology” and even further away from DNS which is a very specific system.

                                                                                                                  What else could we fit into such “terminology”, the universe? biology? The whole field of mathematics? All of Physics? Human politics? It is that vague, and DNS is valid, because why not? it kind of fits the bill of what is intended to bring down (Web3 I assume).

                                                                                                                  I haven’t read the whole article, so I’m not sure if this was the case of wanting to bring down a piece of tech that is being commonly argued against (groupthink). It felt like a very dishonest technical argumentation from that point onwards to me.

                                                                                                              1. 3

                                                                                                                It’s that time of the year again, huh.

                                                                                                                1. 10

                                                                                                                  Related soapbox: stop it with the WTFPL-and-friends shenanigans.

                                                                                                                  I get it. Licensing isn’t fun. It’s frustrating and convoluted and, depending on how you lean, may be generally a bad idea for software. But it’s here, and we have to deal with it. Being edgy and using the word “fuck” in an attempt at being subversive just flags as being childish, and ironically, no one can do anything with your code, because it’s not a license.

                                                                                                                  If you really want people to do whatever they want with your code, use MIT, 0BSD, CC0 or something tested that actually does what you intend, and move on.

                                                                                                                  1. 9

                                                                                                                    I actually advocate eschewing licenses. I don’t want companies to benefit off of my work product without paying me, and I want individuals to go ahead and use it for their own personal projects. In reality, for code without a license, most people aren’t going to look at a license file unless they find themselves in some legal dilemma, and most people tend to assume that code is open source by default, so they will just go ahead and use it without thinking more of it, that’s fine. After all, if I didn’t like it, then what am I going to do, sue them? If they are really scared, they can send me an email and I’ll grant them a perpetuity license or whatever.

                                                                                                                    Now for companies, they have legal departments that will wring-hands over something not having a license, so not having a license will deter them and make it harder for them to use it. They’d have to email me and from there we have a line of communication where I can state that they either pay me, or to GTFO for trying to exploit my work output for their profit margin.

                                                                                                                    Now for big companies, or for things like ICE, even if I do have a license, what am I going to do, sue them? They can win a war of attrition and tie me up with legal red tape for the next thirty years. There’s literally no reason to bother with dealing with licenses because in reality, functionally when you are using software, it doesn’t matter one iota.

                                                                                                                    The idea of “Open Source” as a concept has been, overall, detrimental to the “Free Culture, Free Software” movement anyway – such as when GNU/The FSF petitioned one of the Nordic governments not to abolish copyright, as it would be detrimental to GPL licenses. Not to mention that it’s facilitated and aided what is essentially, continued mass exploitation on a previously unforseen scale.

                                                                                                                    1. 2

                                                                                                                      I actually advocate eschewing licenses. I don’t want companies to benefit off of my work product without paying me, and I want individuals to go ahead and use it for their own personal projects.

                                                                                                                      I feel much the same way, but I’m not sure not using a license achieves what I want. Ideally, I would want a functional “Ethical Source” or “copyfarleft” license, but I’m also skeptical about the ability of licenses to achieve those goals. I’m actually looking at the FAFOL (Fuck Around And Find Out) license, just because it’s going to be an unacceptable legal risk to any company looking at it, but no actual person in their right mind is going to pay attention to it.

                                                                                                                      1. 1

                                                                                                                        This is an interesting one because so many folks work for companies which might run afoul of some “ethical source” licenses. Like imagine someone who works for a healthcare company or an energy company or a government. It’s sort of an interesting conundrum.

                                                                                                                      2. 1

                                                                                                                        such as when GNU/The FSF petitioned one of the Nordic governments not to abolish copyright, as it would be detrimental to GPL licenses.

                                                                                                                        I’m really interested in this, do you have a source or can you point me to a good place to start looking for info about this?

                                                                                                                        edit: i found this: this

                                                                                                                      3. 8

                                                                                                                        Yeah, I don’t think saying “fuck” counts as edgy anymore…

                                                                                                                        1. 1

                                                                                                                          Why is it not considered a license?

                                                                                                                          1. 2

                                                                                                                            Because neither you nor I are lawyers. Plus it’s redundant to a bunch of real licenses.

                                                                                                                            Does “do what the fuck you want” include suing you if the code doesn’t work? It could be interpreted that way, hell, it might even invite it. I dunno — again, I’m not a lawyer — but it’s not worth me taking the slightest risk in that regard if the only benefit is just to be petty.

                                                                                                                            But people want to be smugly against licenses, and ironically they don’t even get the benefits they want to be radically open. It’s sad, really.

                                                                                                                            1. 1

                                                                                                                              (Disclaimer: I am not a lawyer and this is not legal advice, of course)

                                                                                                                              Us not being lawyers has nothing to do with whether or not it’s a license.

                                                                                                                              It not having a warranty clause also has no effect on it being a license or not. That may preclude it from being a good license, which I’d agree with.

                                                                                                                              I think it’s more a statement on the restrictiveness of certain licenses, not a critique of licenses in general.

                                                                                                                        1. 6

                                                                                                                          Disgusting work. And the security angle is pure BS. Its entire purpose is to deny us our computing freedom. To “protect” the code from us like we’re some adversary.

                                                                                                                          This will only be used by scammers and those who now try to block right clicks with an alert()

                                                                                                                          1. 6

                                                                                                                            I don’t think this is about how to make your website more secure. This is a “hey, here’s shit evil people could do, you need to be aware of it” kind of thing.

                                                                                                                            1. 5

                                                                                                                              Disgusting work.

                                                                                                                              I think this is great, assuming browser vendors are willing to fix it. @freddyb Do you know if firefox is vulnerable to this too?

                                                                                                                              1. 3

                                                                                                                                I think treating this sort of thing as a vulnerability that can be fixed is a losing battle.

                                                                                                                                1. 4

                                                                                                                                  Why ? Browser vendors are already implementing pretty good js environment segregation for webextensions, I can’t imagine why they wouldn’t be able to do the same for debuggers.

                                                                                                                                  1. 2

                                                                                                                                    I think those issues can be treated as fixable, but I don’t think they will all be fixed. Most of the things in part 2 are about calling into site-code (e.g., overridden prototypes), which I consider possible. But some of the things posted here (and in part 1) are hard to resolve. Especially when they cause additional second-level side-effects like source map URLs, the layout shift that comes from enabling DevTools etc.

                                                                                                                                    I’ll try to get a definite answer from the team though :)

                                                                                                                                  2. 1

                                                                                                                                    if that’s the case then it’s an admission of defeat

                                                                                                                              1. 20

                                                                                                                                I’ve been using JS since the late 90s and I haven’t even seen alert/confirm/prompt outside of toy tutorials or horrendous code since roughly 2005. I don’t find the arguments in this blog very convincing.

                                                                                                                                It’s also incredibly clickbait/alarmist, which is an immediate eye roll.

                                                                                                                                1. 37

                                                                                                                                  I use confirm for actions that cannot be undone and are potentially dangerous. Why wouldn’t I? The alternative is to write a lot of code to throw up a modal div that does the same thing. Might as well do it natively.

                                                                                                                                  1. 32

                                                                                                                                    Wait until you meet enterprise software!

                                                                                                                                    1. 28

                                                                                                                                      In the relevant bug tracker discussion someone says this broke ERP software with hundreds of thousands of users.

                                                                                                                                      1. 18

                                                                                                                                        Half the argument of the blog post though is that “toy tutorials” are important and valuable in a way that isn’t captured by how often the feature is used in production. And most of the rest is about how actually, it’s valuable that code from 2005 still works. I think you are missing the forest for the trees.

                                                                                                                                        1. 8

                                                                                                                                          The article considerably overstates what the Chrome team is actually intending to ship: it’s disabling cross-origin alert inside iframes, not alert entirely. Most of the article seems to be an extremely uncharitable reading of Dominic hoping that “one day”, “in the far future”, “maybe” (literally these are direct quotes!) they can remove blocking APIs like alert — not that they have any plans to do so now or any time soon.

                                                                                                                                          I don’t think the GP is missing the forest for the trees; I think the author is making a mountain out of a molehill.

                                                                                                                                          1. 7

                                                                                                                                            Few things:

                                                                                                                                            • “Some day” tends to come a lot sooner than we’d expect.
                                                                                                                                            • This is the sort of thing people use to justify further encroachment down the line (“Well we already disable it for iframe stuff…”).
                                                                                                                                            • This directly reduces the utility of using iframes–and some folks still use those on occasion, and it is exceedingly tacky to unilaterally decide to break their workflows.
                                                                                                                                          2. 1

                                                                                                                                            You can still do your little toy tutorials with alert/confirm/prompt, just don’t do them in an iframe?

                                                                                                                                            1. 2

                                                                                                                                              If you’re making a codepad-like site, you kind of have to put all the user-submitted JS in an alert so it’s not on your own domain.

                                                                                                                                              1. 1

                                                                                                                                                If you’re making a codepad-like site, you can also inject a polyfill for alert() etc in the user-controlled iframe to keep things working. Until you’re done locking down the codepad for arbitrary user scripts to run without problems, this is probably one of the smaller tasks.

                                                                                                                                                1. 2

                                                                                                                                                  Can you make the polyfill block?

                                                                                                                                          3. 11

                                                                                                                                            I use alert() and confirm(). It’s easy, simple, works, and doesn’t even look so bad since Firefox 89. I don’t think my code is “horrendous”; it’s just the obvious solution without throwing a bunch of JS at it.

                                                                                                                                            I agree this blog post isn’t especially great though.

                                                                                                                                            1. 1

                                                                                                                                              Do you use it in a cross-origin iframe?

                                                                                                                                              1. 3

                                                                                                                                                No, but your comment made no mention of that:

                                                                                                                                                I haven’t even seen alert/confirm/prompt outside of toy tutorials or horrendous code since roughly 2005.

                                                                                                                                                1. 1

                                                                                                                                                  Sorry, I read your reply in the context of the blog post (i.e. your code is going to break).

                                                                                                                                                  My line about horrendous code is hyperbolic, but the fact is that alert/confirm/prompt don’t offer customizability to make for a consistent, well-made UX. Maybe it’s not a problem for certain audiences (usually things like internal tools for devs end up having them), but most customer-facing solutions require more to their experience.

                                                                                                                                                  I’m not saying they should remove them right now, but a day in the future where they go away (presumably deprecated due to a better option) is not something we should be dreading. Who knows if that day will even come.

                                                                                                                                            2. 8

                                                                                                                                              At $JOB we have used prompt for some simple scenarios where it solves the problem of getting user input in a scenario in some sync-y code, and was no fuss.

                                                                                                                                              We integrate with Salesforce through an iframe. This change caused us to have to like redo a whole tiny thing to get stuff working again (using a much heavier modal thing instead of, well, a call to prompt). It wasn’t the end of the world, but it was annoying and a real unforced error.

                                                                                                                                              We would love a scenario where browsers offered more rich input in a clean way (modals have been in basically every native GUI since the beginning of time!). I’m sure people would be way less frustrated if Chrome offered easy alternatives that don’t rely (for example) on z-index-overlays (that can break for a billion reasons) or stuff like that.

                                                                                                                                              Sometimes you just want input from somebody in a prompt-y way

                                                                                                                                              1. 5

                                                                                                                                                You haven’t seen a lot of business to business software then.

                                                                                                                                                1. 1

                                                                                                                                                  That is still no reason to remove a perfectly functional feature that has worked reliability for decades and requir a orders of magnitude less resources than the alternative. Both human and computational resources.

                                                                                                                                                  I use it all the time on simple UIs I write for my own usage or for restricted groups of users.

                                                                                                                                                  The amount of resources that could be saved if we favoured well known, tried and true technology rather than the new aesthetically shiny thing, is astonishing.

                                                                                                                                                  1. 2

                                                                                                                                                    It’s not about “shiny things” but about use experience. Linux has suffered for decades due to the approach you’re talking about.

                                                                                                                                                    1. 2

                                                                                                                                                      No, Linux has suffered precisely because it does not offer a native GUI, or UI at all, forcing everyone to reinvent basic functionality like on the web.

                                                                                                                                                1. 6

                                                                                                                                                  The quote from SICP stuck out at me:

                                                                                                                                                  “programs must be written for people to read, and only incidentally for machines to execute.”

                                                                                                                                                  I think we may have gone too far in this direction, and are perhaps at the illuminated manuscript (javascript?) stage of things.

                                                                                                                                                  Hot take: given how little of the iceberg of our dependencies and libraries we see, I would posit that most people don’t read programs in any meaningful holistic way at all.

                                                                                                                                                  1. 5

                                                                                                                                                    I would have called Literate Programming the “Illuminated Manuscript” of our art.

                                                                                                                                                    1. 2

                                                                                                                                                      I’m struggling to understand this reply. We’ve made programs so readable that nobody bothers reading them?

                                                                                                                                                      1. 3

                                                                                                                                                        I think we’ve lost our empathy for the machines, and have hit a degree of perfomative coding where we perhaps are writing programs for other programmers rather than for machines.

                                                                                                                                                        Which is fine, until you consider that the programmers won’t be executing the programs and the Earth is drowning in e-waste and waste heat.

                                                                                                                                                    1. 1

                                                                                                                                                      Waiting for the inevitable dethronement of Slack and Discord by Ripcord. I jest, I wish faster tech won a majority of the time, but I’ve seen enough to know that’s just not the case.