Threads for kornel

  1. 1

    It’s weird that the first thing you criticize in a Critical Retrospective is something syntax-related that you yourself call superficial. It makes it hard to take the rest of the post seriously

    1. 27

      If syntax impacts understandability, is it actually superficial?

      1. 19

        Because I don’t think that’s the fault of the syntax. Huge part of criticism is expectations/preferences and lack of understanding of the trade-offs that made it the way it is. When Rust is different than whatever other language someone is used to, they compare familiar with unfamiliar (see Stroustrup’s Rule). But it’s like saying the Korean alphabet is unreadable, because you can’t read any of it.

        People who don’t like Rust’s syntax usually can’t propose anything better than a bikeshed-level tweak that has other downsides that someone else would equally strongly dislike.

        For example, <> for generics is an eyesore. But if Rust used [] for generics, it’d make array syntax either ambiguous (objectively a big problem) or seem pointlessly weird to anyone used to C-family languages. Whatever else you pick is either ambiguous, clashes with meaning in other languages, or isn’t available in all keyboard layouts.

        The closure syntax || expr may seem like line noise, but in practice it’s important for closures to be easy to write and make it easy to focus on their body. JS went from function { return expr } to () => expr. Double arrow closures aren’t objectively better, and JS users criticize them too. A real serious failure of Rust regarding closures is that they have lifetime elision rules surprisingly different than standalone functions, and that is a problem deeper than the syntax.

        Rust initially didn’t have the ? shortcut for if err != nil { return nil, err } pattern, and it had a problem of a low signal-to-noise ratio. Rust then tried removing boilerplate with a try!() macro, but it worked poorly with chains of fallible function calls (you’d have a line starting with try!(try!(try!(… and then have to figure out where each of them have the other paren). Syntax has lots of trade-offs, and even if the current one isn’t ideal in all aspects, it doesn’t mean alternatives would be better.

        And there are lots of things that Rust got right about the syntax. if doesn’t have a “goto fail” problem. Function definitions are greppable. Syntax of nested types is easy to follow, especially compared to C’s “spiral rule” types.

        1. 13

          I think a lot of criticism about syntax is oblique. People complain about “syntax” because it’s just… the most convenient way to express “I find it hard to learn how to write correct programs, and I find it hard to interpret written programs, even after substantial practice”.

          Lots of people complain that Common Lisp syntax is hard. Lisp syntax is so easy that you can write a parser in a few dozen lines. Common Lisp has a few extra things but, realistically, the syntax is absolutely trivial. But reading programs written in it is not, even after substantial practice, and I get that (as in, I like Common Lisp, and I have the practice, and I get that).

          Same thing here. A lot of thought went into Rust’s syntax, probably more than in, say, C’s syntax, if only because there was a lot more prior art for Rust to consider. So there’s probably not much that can be done to improve Rust’s syntax while not basically inventing another language. That doesn’t take away from the fact that the language is huge, so it has a syntax that’s unambiguous and efficient but also huge, so it’s just a whole lot of it to learn and keep in your head at once. I get it, I’ve been writing Rust on and off but pretty much weekly for more than an year now and I still regularly need to go back to the book when reading existing code. Hell, I still need it when reading existing code that I wrote. You pay a cognitive price for that.

          1.  

            “I find it hard to learn how to write correct programs . . .

            Do you believe “correctness” is a boolean property of a program?

            1.  

              I do, as in, I think you can always procure a “correctness oracle” that will tell you if a program’s output is the correct one and which, given a log of the program’s operations, can even tell you if the method through which it achieved the result is the correct one (so it can distinguish between correct code and buggy code that happens to produce correct output). That oracle can be the person writing the program or, in commercial settings, a product manager or even a collective – a focus group, for example. However:

              • That oracle works by decree. Not everyone may agree with its edicts, especially with user-facing software. IMHO that’s inherent to producing things according to man-made specs. There’s always an “objective” test to the correctness of physics simulation programs, for example, but the correctness of a billing program is obviously tied to whatever the person in charge of billings thinks is correct.
              • The oracle’s answer may not be immediately comprehensible, and they are not necessarily repeatable (like the Oracle in Delphi, it’s probably best to consider the fact that its answers do come from someone who’s high as a kite). IMHO that’s because not all the factors that determine a program’s correctness are inherent to the program’s source code, and presumably, some of them may even escape our quantitative grasp (e.g. “that warm fuzzy feeling” in games). Consequently, not all the knowledge that determines if a program is correct may reside with the programmer at the time of writing the code.

              More to the point, I think it’s always possible to say if something is a bug or a feature, yes :-D.

            2.  

              Well, there was a lot of prior art even when C was created, and they actively chose to disregard it. They also chose to disregard discoveries in C itself in the 70s and 80s, freezing the language far too early considering the impact it would have in the following decades.

            3. 4

              it’s like saying the Korean alphabet is unreadable, because you can’t read any of it.

              But like there is a reasonably objective language difficulty ranking index (from the perspective of English-native speakers) and Korean is squarely in the most-difficult tranche, I guess in no small part due to its complex symbology, at least in comparison to Roman alphabets. Are you saying that this dimension of complexity is, net, irrelevant?

              1. 11

                Korean uses the Hangul alphabet, which is very easy to learn. It’s much simpler than our alphabet. You can learn Hangul in a day or two. You’re thinking of Japanese, which is a nightmare based on people writing Chinese characters in cursive and italics while drinking a bottle of sake.

                1. 1

                  some simplified hanzi does look like kanji, but I would appreciate an example of a japanese character looking like a cursive or italic version of a chinese glyph before I go on to tell your analogy to everyone at parties.

                  1. 1

                    It’s not an analogy. It’s the historical truth of kana: https://en.wikipedia.org/wiki/Kana. Japanese kanji and hanzi are mostly the same modulo some font changes and simplification in the 20th c.

                    1.  

                      I meant the drunken japanese people part.

                      1.  

                        We can’t prove they weren’t drunk. :-)

                2. 10

                  from the perspective of English-native speakers

                  I think that’s what they were getting at; there’s nothing inherently difficult about it but your background as an English speaker makes it look hard to read when objectively speaking it’s dramatically simpler than English due to its regularity and internal logic.

                  1.  

                    I guess I would say that there is no “objectively speaking” in this domain? Like, there is no superhuman who can look at things invariant of a language background.

                    1.  

                      If you’re talking about “easy to learn” then I agree.

                      If you’re talking about simplicity, then I disagree. The number of rules, consistency, and prevalence of exceptions can be measured without reference to your background.

                  2. 6

                    I’ve specifically mentioned the Hangul alphabet (a syllabary, strictly speaking), not the language. The Korean language (vocabulary, grammar, spoken communication) may be hard to learn, but the alphabet itself is actually very simple and logical. It’s modern, and it has been specifically designed to be easy to learn and a good fit for the Korean language, rather than being a millennia-old historical borrowed mash-up like many other writing systems.

                    I think it’s a very fitting analogy to having an excellent simple syntax for a complex programming language. You may not understand the syntax/alphabet at all, but it doesn’t mean it’s bad. And the syntax/alphabet may be great, but the language it expresses may still be difficult to learn for other reasons.

                    With Rust I think people complaining about the syntax are shooting the messenger. For example, T: for<'a> Fn(&'a) makes lifetime subtyping contravariant for the loan in the argument of a function item trait in a generic trait bound. Is it really hard because of the syntax? No. Even when it’s expressed in plain English (with no computer language syntax at all) it’s an unintelligible techno-babble you wouldn’t know how to use unless you understand several language features it touches. That for<'a> syntax is obscure even by Rust’s standards, but syntactically it’s not hard. What’s hard is knowing when it needs to be used.

                  3. 4

                    People who don’t like Rust’s syntax usually can’t propose anything better than a bikeshed-level tweak that has other downsides that someone else would equally strongly dislike.

                    The problem with Rust’s syntax isn’t that they made this or that wrong choice for expressing certain features; it’s that there’s simply far too much of it. “Too many notes,” as Joseph II supposedly said.

                    1.  

                      I agree with this, which is why I object to blaming the syntax for it. For a language that needs to express so many features, Rust’s syntax is doing well.

                      Rust chose to be a language that aims to have strong compile-time safety, low-level control, and nearly zero run-time overhead, while still having higher-level abstractions. Rust could drop a ton of features if it offered less control and/or moved checks to run-time or relaxed safety guarantees, but there are already plenty of languages that do that. Novelty of Rust is in not compromising in any of these, and this came at a cost of having lots of features to control all of these aspects.

                      1.  

                        You can have many features without a lot of syntax. See Lisp.

                  4. 10

                    The syntax complexity of Rust is actually a big factor in why I abandoned my effort to learn it. I was only learning on my own time, and came to the realization I had a long way to go before I’d be able to pick apart a line like the author’s example.

                    So for me, it wasn’t just superficial.

                    1. 4

                      The syntax complexity of Rust is actually a big factor in why I abandoned my effort to learn it.

                      Same.

                    2. 3

                      If syntax impacts understandability, is it actually superficial?

                      I’d say so.

                      The problem is that “this syntax is ugly” is a completely subjective judgement largely influenced by the peculiarities of ones’ own background. Coming from Perl and Ruby, I happen to find Rust pleasant to look at and easy to read, whereas I find both Python and Go (which many other people prefer) unreasonably frustrating to read and just generally odd-looking. It’s not that Python and Go are doing anything objectively less understandable, per-se, but they’re certainly have an unfamiliar look, and people react to unfamiliarity as if it were objectively incorrect rather than just, well, making unfamiliar choices with unfamiliar tradeoffs.

                      It’s pure personal preference, and framing ones’ personal preferences as something that has objective reality outside oneself and which some other party is doing “wrong” is, to me, the definition of a superficial complaint.

                      1. 7

                        It’s pure personal preference

                        Is it pure personal preference? I dunno. Personal preference is a part of it, but I don’t think it’s controversial to say that Python is in general easier to understand than the q language, for example. Human cognition and coherence actually abides pretty well-defined rules, at the macro scale. Sigils are harder to grok than words. And so on.

                        1. 11

                          Personal preference is a part of it, but I don’t think it’s controversial to say that Python is in general easier to understand than the q language, for example.

                          Maybe, maybe not. What I do think is that if we’re going to try to make objective claims, we need some real objective measures and measurements. These conversations tend to be nothing but pseudoscience-y assertions and anecdata masquerading as irrefutable facts.

                          Human cognition and coherence actually abides pretty well-defined rules, at the macro scale. Sigils are harder to grok than words.

                          (In no way am I trying to pick on you, but) Case in point: “Sigils are harder to grok than words” feels like a strong objective claim but… is this actually in any way true? 馬 is a much more complicated symbol than $ or @ or ->, but we have something like 1.5 billion people in the world happily reading and writing in languages that require a knowledge of thousands of such symbols to achieve literacy, and they turn out to somehow show lower rates of dyslexia than in alphabet based languages while doing so!

                          Sigil-y writing systems are indeed actually quite common throughout history, so again we have this thing where what feels like a perfectly simple fact actually looks a heck of a lot like a simple case of familiarity when you scratch it just a little bit. The dominance of a few alphabetic writing systems outside of Asia could simply be a historical accident for all we know – there are no strong results from cognitive science supporting any claim that it’s objectively more fit to “human cognition”. We really don’t have any idea whether words are simpler or more efficient than symbols, or whether python is a global maxima of readability, a local minima, or anything in between. There are almost no good studies proving out any of this, just a lot of handwaving and poorly supported claims based on what people happen to like or be most familiar with.

                          1.  

                            馬 is a word. It happens to be written as a single character, but that doesn’t make it punctuation.

                            1.  

                              When I speak about “understandability” or whatever I’m not making a claim against an abstract Ur-human raised in a vacuum, I’m speaking about humans as they exist today, including cultural and historical influences, and measured on a demographic (macro) scale, rather than an individual (micro) scale. That is, I’m making a descriptive argument, not a normative one. In this context, “familiarity” is I guess a totally reasonable thing to account for! People understand better the things they are familiar with. Right?

                              1.  

                                That is, I’m making a descriptive argument, not a normative one.

                                It’s not a very good descriptive argument, though, insofar as you’re really failing to describe a lot of things in order to make your argument fit the conclusion that “sigils are harder to grok than words”.

                                Even if we confine ourselves to Western English speakers… what about mathematics? Why does almost everyone prefer y = x+1 to Cobol’s ADD 1 TO X GIVING Y? It’s more familiar, right? There doesn’t seem to be any long-term push to make Mathematics more wordy over time (most of the established symbols have hung around for hundreds of years and had ample opportunity to get out-competed by more grokkable approaches, if word-based approaches were found by people to be any more grokkable), so if we’re describing the long-term pressures on artificial languages I don’t think “sigils are harder to grok than words” is an accurate descriptive statement.

                                In this context, “familiarity” is I guess a totally reasonable thing to account for! People understand better the things they are familiar with. Right?

                                Well, sure. But “in some contexts words are more familiar than sigils to western audiences” is a much different claim than “sigils are harder to grok than words” in any sense, and it leaves a lot more room to talk about sigils in programming languages in a rational way. Things like “dereferencing pointers” aren’t really familiar to anyone in words or sigils, so it’s not obvious to me that x = valueat y is any more or less “correct”/“intuitive”/“grokable” than x = *y.

                                If anything, given the relative unpopularity of the Pascal/Ada & Cobol language families, a certain amount of “unfamiliar concepts compressed into sigils” seems to be appreciated by programmers at large. But other people disagree, which seems to point at this mostly being a superficial argument over tastes and perhaps how much maths background one has, rather than some kind of concrete and objective variation in any measurable metric of “understandability”.

                                1.  

                                  what about mathematics?

                                  Well, I think this substantiates my point? In the sense that way more people can read prose than can understand nontrivial math. Right?

                                  “in some contexts words are more familiar than sigils to western audiences” is a much different claim than “sigils are harder to grok than words”

                                  Not some but most or even almost all, depending on just how many sigils we’re talking about.

                                  Authors generally don’t invent new languages in order to express their literary works; they take the language(s) they already know, with all their capabilities and constraints, and work within those rules. They do this because their goal is generally not to produce the most precise representation of their vision, but instead to produce something which can be effectively consumed by other humans. The same is true of programming.

                                  1.  

                                    Well, I think this substantiates my point? In the sense that way more people can read prose than can understand nontrivial math. Right?

                                    More people can read prose (in general) than the prose portions of an advanced Mathematics text (in specific). It’s not the orthography of mathematics that’s the limiting factor here.

                                    Authors generally don’t invent new languages in order to express their literary works; they take the language(s) they already know, with all their capabilities and constraints, and work within those rules. They do this because their goal is generally not to produce the most precise representation of their vision, but instead to produce something which can be effectively consumed by other humans. The same is true of programming.

                                    Which speaks to my point. Programming uses “sigils” because in many cases these sigils are already familiar to the audience, or are at least no less familiar to the audience for the concepts involved than anything else would be, and audiences seem to show some marked preference for sigils like { … } vs begin … end, y = x + 1 seems pretty definitely preferred for effective consumption by audiences over ADD 1 TO X GIVING Y, etc.

                                    At any rate, we seem to have wandered totally away from “sigils are objectively less readable” and fully into “it’s all about familiarity”, which was my original point.

                                    1.  

                                      I’m not claiming that sigils are objectively less readable than prose. I’m objecting to the notion that syntax is a superficial aspect of comprehension.

                      2. 22

                        I would warmly suggest making an effort to hit Page Down twice to get past the syntax bit and read the rest of the post though, because it’s a pretty good and pragmatic take, based on the author’s experience writing and maintaining a kernel. Xous is a pretty cool microkernel which runs on actual hardware, it’s actually a pretty good test of Rust’s promises in terms of safety and security.

                        1. 9

                          It’s interesting but also has the weird dichotomy that the only two choices for systems programming are C or Rust. C++ also has a lot of the strengths that the author likes about Rust (easy to write rich generic data structures, for example), and has a bunch of other things that are useful in a kernel, such as support in the standard library for pluggable memory allocators, mechanisms for handling allocation failure, a stable standard library API, and so on.

                          1. 5

                            I had exactly the same thought. C++ burnt through a lot of good will in the C++98 era where it was admittedly a hot mess (and all the compilers where buggy dumpster fires). Now on one hand we have people who publicly and loudly swore off touching C++ ever again based on this experience (and even more people parroting the “C++ is a mess” statement without any experience) and on the other the excitement of Rust with all the hype making people invest a large amount of effort into learning it. But the result, as this article shows, is often not all roses. I believe oftentimes the result would have been better if people invested the same amount of time into learning modern C++. Oh well.

                            1.  

                              Writing C++ is like writing Rust but with your whole program wrapped in unsafe{}. You have to manage your memory and hope you did it right.

                              1.  

                                As I hope this article clearly demonstrates, there is a lot more to a language chice than memory safety. Also, FWIW, I write fairly large programs and I don’t find memory management particularly challenging in modern C++. At the same time, I highly doubt that these programs can be rewritten in Rust with the result having comparable performance, compilation times, and portability properties.

                              2.  

                                string_view of temporaries makes dangling pointers instead of compilation errors. optional allows unchecked dereferencing without warnings, adding more UB to the modern C++. I haven’t met a C++ user who agrees these are fatal design errors. Sorry, but this is not up to safe Rust’s standards. From Rust perspective modern C++ continues to add footguns that Rust was designed to prevent.

                                1.  

                                  I haven’t met a C++ user who agrees these are fatal design errors.

                                  I haven’t used string_view much so can’t categorically say it’s not a design error (it very well may be). But for optional I can certainly say it is a trade-off: you have the choice of checked access (optional::value()) or unchecked and you decide what to use. I personally always use unchecked and never had any problems. Probably because I pay attention to what I am writing.

                                  1.  

                                    This is the difference in approaches of the two languages. In C++ if the code is vulnerable, the blame is on the programmer. In Rust if the code is vulnerable, Rust considers it a failure of the language, and takes responsibility to stop even “bad” programmers from writing vulnerable code. I can’t stress enough how awesome it is that I can be a careless fool, and still write perfectly robust highly multi-threaded code that never crashes.

                                    In terms of capabilities, Rust’s Option is identical, but the the default behavior is safe, and there’s a lot of syntax sugar (match, if let, tons of helper methods) to make the safe usage the preferred option even for “lazy” programmers. The UB-causing version is written unsafe { o.unwrap_unchecked() }, which is deliberately verbose and clunky, so that the dangerous version stands out in code reviews, unlike subtle * or -> that are commonly used everywhere.

                                    Rust’s equivalent of string_view is &str, and it’s practically impossible to use the language without embracing it, and it’s borrow-checked, so it won’t compile if you misuse it.

                              3. 2

                                Eh, maybe the author just didn’t write that much low-level/kernel code in C++. I try not to read too much into these things. If I were to start learning F# tomorrow, then tried to write a similar piece two years from now, I’d probably end up with something that would have the weird dichotomy that the only two choices for functional programming are Scheme and F#.

                                1.  

                                  Scheme is honestly so hard to do functional in. It’s shockingly imperitive by nature given the reputation.

                              4. 3

                                I did read the entire post, but I wanted to voice that focusing on the wrong thing first makes people not take you seriously, especially when the author expresses it doesn’t matter, but they still decided to make it first?

                                1. 3

                                  I may not be interpreting this correctly but I didn’t take the author qualifying it as a superficial complaint to mean that it doesn’t matter. Based on the issues he mentions regarding the readability of Rust macros, for example, I think it’s superficial as in “superficial velocity”, i.e. occurring or characterising something that occurs at the surface.

                                  (But note that I may be reading too much into it because reviewing and auditing Rust code that uses macros is really not fun so maybe I’m projecting here…)

                              5. 18

                                The final sentence of that section said, in summary, “Rust just has a steep learning curve in terms of syntax”. A critical retrospective that does not mention the horrendous syntax or its learning curve would lack credibility.

                                1.  

                                  I find Rust’s syntax perfectly clear and sensible. I am not the only one.

                                2. 9

                                  I liked that it starts with that TBH. Rust’s dense syntax is probably the first impression of the language for many people—it was for me at least. And putting the author’s first impression first in the article makes it read more like a person telling a story, rather then a list of technical observations sorted by importance.

                                  I like to read stories by humans; i feel it easier to connect with the author and therefore to retain some of what they say. YMMV of course.

                                  1. 2

                                    And if they think rust is hard to read, wait until they discover lisp!

                                    (I know this author probably is already familiar with lisp and many other things, but the comparison stands.)

                                    1. 6

                                      I find it the other way around. If you temporarily put aside the issues of special forms and macros, the syntax of Lisp is extremely minimal and regular (it’s almost all lists and atoms). So Lisp stands at kind of an opposite extreme from Rust, with more familiar languages somewhere in between.

                                      1. 5

                                        Nim still has a dumpLisp routine to show you the shape of an AST you may want to manipulate.

                                        Syntax can be very personal, but I strongly prefer Nim’s to Rust’s and see no compelling language feature of Rust to tempt me away, though Nim is not without its own issues.

                                        1. 2

                                          Nim isn’t really comparable is it? More like Go with a GC etc?

                                          1.  

                                            “How comparable” mostly depends upon what you mean by “a GC etc”. Nim’s (AutomaticRC/OptimizedRC) memory management seems fairly similar to Rust, but I am no Rust expert and most PLs have quite a few choices either directly or ecosystem-wide. (Even C has Boehm.) There is no “GC thread” like Java/Go. The ORC part is for cycle collection. You can statically specify {.acyclic.}, sink, lent, etc. in Nim to help run-time perf. Some links that go into more detail are: https://nim-lang.org/blog/2020/10/15/introduction-to-arc-orc-in-nim.html https://nim-lang.org/blog/2020/12/08/introducing-orc.html

                                            1.  

                                              “Go with a GC” is Go.

                                              1.  

                                                Yes, that’s why I said it

                                          2. 2

                                            The complaint in the article is about noisy hard to read though, and lisp is definitely that, even if it is simple and regular that simplicity leads everything to look the same.

                                            1.  

                                              I always wondered why indentation-based reader macros (SRFI-49 is a simple one) never became popular. I can see “whys” for big macro writer types since they often want to pick apart parse trees and this adds friction there. Most programmers are not that (in any language). My best guess is a kind of community dynamic where tastes of village elders make beginners adapt more. Or wishful promises/hopes for beginners to become elders? Or a bit of both/etc.?

                                              Of course, part of the regularity is “prefix notation” which can remain a complaint.

                                        2. 1

                                          It makes it hard to take the rest of the post seriously

                                          As x64k said the post is pretty well made I think and some honest criticism. If anything you can criticize the bad blog layout, which has big white bars on mobile and desktop, giving you a hard time reading it from any device.

                                        1. 22

                                          RAII is far from perfect… here are a few complaints:

                                          1. Drop without error checking is also wrong by default. It may not be a big issue for closing files, but in the truly general case of external resources to-be-freed, you definitely need to handle errors. Consider if you wanted to use RAII for cloud resources, for which you need to use CRUD APIs to create/destroy. If you fail to destroy a resource, you need to remember that so that you can try again or alert.

                                          2. High-performance resource management utilizes arenas and other bulk acquisition and release patterns. When you bundle a deconstructor/dispose/drop/whatever into some structure, you have to bend over backwards to decouple it later if you wish to avoid the overhead of piecemeal freeing as things go out of scope.

                                          3. The Unix file API is a terrible example. Entire categories of usage of that API amount to open/write/close and could trivially be replaced by a bulk-write API that is correct-by-default regardless of whether it uses RAII internally. But beyond the common, trivial bulk case, most consumers of the file API actually care about file paths, not file descriptors. Using a descriptor is essentially an optimization to avoid path resolution. Considering the overhead of kernel calls, file system access, etc, this optimization is rarely valuable & can be minimized with a simple TTL cache on the kernel side. Unlike a descriptor-based API, a path-based API doesn’t need a close operation at all – save for some cases where files are being abused as locks or other abstractions that would be better served by their own interfaces.

                                          4. It encourages bad design in which too many types get tangled up with resource management. To be fair, this is still a cultural problem in Go. I see a lot of func NewFoo() (*Foo, error) which then of course is followed by an error check and potentially a .Close() call. Much more often than not, Foo has no need to manage its resources and could instead have those passed in: foo := &Foo{SomeService: svc} and now you never need to init or cleanup the Foo, nor check any initialization errors. I’ve worked on several services where I have systematically made this change and the result was a substantial reduction in code, ultimately centralizing all resource acquisition and release into essentially one main place where it’s pretty obvious whether or not cleanup is happening.

                                          1. 3

                                            This is super informative, thanks! Probably worth it to turn this comment into a post of its own.

                                            1. 3

                                              The error handling question for RAII is a very good point! This is honestly where I’m pretty glad with Python’s exception story (is there a bad error? Just blow up! And there’s a good-enough error handling story that you can wrap up your top level to alert nicely). As the code writer, you have no excuse to, at least, just throw an exception if there’s an issue that the user really needs to handle.

                                              I’ll quibble with 2 though. I don’t think RAII and arenas conflict too much? So many libraries are actually handlers to managed memory elsewhere, so you don’t have to release memory the instant you destruct your object if you don’t want to. Classically, reference counted references could just decrement a number by 1! I think there’s a a lot of case-by-case analysis here but I feel like common patterns don’t conflict with RAII that much?

                                              EDIT: sorry, I guess your point was more about decoupling entirely. I know there are libs that parametrize by allocator, but maybe you meant something even a bit more general

                                              1. 2

                                                It may not be a big issue for closing files

                                                from open(2):

                                                A careful programmer will check the return value of close(), since it is quite possible that errors on a previous write(2) operation are reported only on the final close() that releases the open file description. Failing to check the return value when closing a file may lead to silent loss of data. This can especially be observed with NFS and with disk quota.

                                                so it’s actually quite important to handle errors from close!

                                                The reason for this is that (AFAIK) write is just a request to do an actual write at some later time. This lets Linux coalesce writes/reschedule them without making your code block. As I understand it this is important for performance (and OSs have a long history of lying to applications about when data is written). A path-based API without close would make it difficult to add these kinds of optimizations.

                                                1. 1

                                                  My comment about close not being a big issue is with respect to disposing of the file descriptor resource. The actual contents of the file is another matter entirely.

                                                  Failing to check the return value when closing a file may lead to silent loss of data.

                                                  Is that still true if you call fsync first?

                                                  A path-based API without close would make it difficult to add these kinds of optimizations.

                                                  Again, I think fsync is relevant. The errors returned from write calls (to either paths or file descriptors) are about things like whether or not you have access to a file that actually exists, not whether or not the transfer to disk was successful.

                                                  This is also related to a more general set of problems with distributed systems (which includes kernel vs userland) that can be addressed with something like Promise Pipelining.

                                                2. 1
                                                  1. Note that in the context of comparison with defer, it doesn’t improve the defaults that much. The common pattern of defer file.close() doesn’t handle errors either. You’d need to manually set up a bit of shared mutable state to replace the outer return value in the defer callback before the outer function returns. OTOH you could throw from a destructor by default.

                                                  2. I disagree about “bend over backwards”, because destructors are called automatically, so you have no extra code to refactor. It’s even less work than finding and changing all relevant defers that were releasing resources piecemeal. When resources are owned by a pool, its use looks like your fourth point, and the pool can release them in bulk.

                                                  3/4 are general API/architecture concerns about resource management, which may be valid, but not really specific to RAII vs defer, which from perspective of these issues are just an implementation detail.

                                                  1. 1

                                                    I disagree about “bend over backwards”, because destructors are called automatically, so you have no extra code to refactor

                                                    You’re assuming you control the library that provides the RAII-based resource. Forget refactoring, thinking only about the initial code being written: If you don’t control the resource providing library, you need to do something unsavory in order to prevent deconstructors from running.

                                                    1.  

                                                      Rust has a ManuallyDrop type wrapper if you need it. It prevents destructors from running on any type, without changing it.

                                                      Additionally, using types via references never runs destructors when the reference goes out of scope, so if you refactor T to be &T coming from a pool, it just works.

                                                    2. 1

                                                      The common pattern of defer doesn’t handle errors either.

                                                      In Zig you have to handle errors or explicitly discard them, in defer and everywhere else.

                                                    3. 1

                                                      Using a descriptor is essentially an optimization to avoid path resolution.

                                                      I believe it also the point at which permissions are validated.

                                                      Entire categories of usage of that API amount to open/write/close and could trivially be replaced by a bulk-write API

                                                      Not quite sure I understand what you mean… Something like Ruby IO.write()? https://ruby-doc.org/core-3.1.2/IO.html#method-c-write

                                                      1. 3

                                                        also the point at which permissions are validated

                                                        I think that’s right, though the same “essentially an optimization” comment applies, although the TTL cache option solution is less applicable.

                                                        Something like Ruby IO.write()

                                                        Yeah, pretty much.

                                                    1. 28

                                                      its release philosophy is supposed to avoid what I call “the problem with Python”: your code stops working if you don’t actively keep up with the latest version of the language.

                                                      This is a very real problem with the Python and Node ecosystems.

                                                      1. 14

                                                        I’m going to be grumpy and ask what this “problem” is supposed to mean, exactly. I can deploy, say, a Python application on a particular version of Python and a particular operating system today, and then walk away for years, and as long as I pay the hosting bill there is no technical reason why it would suddenly stop working. There’s no secret kill switch in Python that will say “ah-ha, it’s been too long since we made you upgrade, now the interpreter will refuse to start!”

                                                        So what people always actually mean when they say things like this is that they want to keep actively developing their code but have everybody else stand still forever, so that they can permanently avoid platform upgrades but also the language and ecosystem will never leave them behind. So the real “problem” is that one day nobody else will be willing to provide further bug and security fixes (free or perhaps even paid) for the specific combination of language/libraries/operating system I initially chose to build my app with, and on that day I have to choose between upgrading to a new combination, or doing that maintenance myself, or going without such maintenance entirely.

                                                        And although I’ve seen a lot of claims, I’ve seen no evidence as yet that Rust has actually solved this problem — the editions system is still young enough that the Rust community has not yet begun to feel the pain and cost of what it committed them to. “Stability without stagnation” is a nice slogan but historically has ranged from extremely difficult to downright impossible to achieve.

                                                        1. 15

                                                          “Stability without stagnation” is a nice slogan but historically has ranged from extremely difficult to downright impossible to achieve.

                                                          You just don’t make breaking changes. Add new APIs but don’t take away old ones. Go 1 has been stable for 10 years now. That means there are some ugly APIs in the standard library and some otherwise redundant bits, but it’s been fine. It hasn’t stopped them from adding major new features. Similarly, JavaScript in the browser only rolls forward and is great. Again, some APIs suck, but you can just ignore them for the most part and use the good versions. I’m not as familiar with the Linux kernel, but my impression there is that Linus’s law is you don’t break userland.

                                                          Most breaking changes are gratuitous. They make things a little nicer for the core developers and shove the work of repair out to everyone who uses their work. I understand why it happens, but I reserve the right to be grumpy about it, especially because I have the experience of working in ecosystems (Go, the browser) that don’t break, so I am very annoyed by ecosystems that do (NPM, Python).

                                                          1. 14

                                                            I’ll be up-front. My stance on breaking changes is that there are two types of projects:

                                                            1. Those which have decided to accept they are an inevitable fact of software development, and so have committed to the process of handling them, and
                                                            2. Those which haven’t yet.

                                                            And remember that we are talking not just about core language/standard library, but also ecosystems. Go has already effectively broken on this — the “rename your module to include /v2” hack is an admission of defeat, and allows the previous version to lapse into a non-maintained status. Which in turn means that sooner or later you will have to make the choice I talked about (upgrade, or do maintenance yourself, or go without maintenance).

                                                            And Rust has had breaking changes built into the ecosystem from the beginning. The Cargo/crates ecosystem isn’t built around every crate having to maintain eternal backwards-compatibility, it’s built on semantic versioning. If I publish version 1.0 of a crate today, I can publish 2.0 with breaking changes any time I want and stop maintaining the 1.x series, leaving users of it stranded.

                                                            So even if Rust editions succeed as a way of preserving compatibility of the core language with no ecosystem splits and no “stagnation”, which I strongly doubt in the long term, the ecosystem has already given up, and in fact gave up basically immediately. It has “the Python problem” already, and the only real option is to learn how to manage and adapt to change, not to insist that change is never permitted.

                                                            (and of course my own experience is that the “problem” is wildly exaggerated compared to how it tends to impact projects in actual practice, but that’s another debate)

                                                            1.  

                                                              I think it’s hard to talk about this rigorously because there’s definitely some selection bias at play — we don’t have information about all the internal projects out there that might be bogged down by breaking changes between interpreter versions, and they’re likely motivated by very different incentives than the ones that govern open source projects — and there’s likely survivorship bias at play too, in that we don’t hear about the projects that got burnt out on the maintenance burden those breaking changes induce.

                                                              My anecdotal evidence is that I’ve worked at places with numerous Python projects bound to different, sometimes quite old, interpreter versions, and there just aren’t enough person-hours available to keep them all up to date, and updating them in the cases where it was truly necessary made for some real hassle. Even if you chalk that up to bad resource management, it’s still a pretty common situation for an organization to find itself in, and it’s reasonable to expect your tools to not punish you for having less than perfect operational discipline. In light of that, I think understanding it in this binary frame of either making breaking changes or not isn’t the most fruitful approach, because as you note it’s not realistic to expect that they never happen. But when they do happen, they cost, and I don’t think it’s unreasonable for an organization to weigh that total cost against their resources and decide against investing in the Python ecosystem. It’s not unreasonable to make the opposite choice either! I just don’t think that cost is trivial.

                                                              1. 7

                                                                Even if you chalk that up to bad resource management, it’s still a pretty common situation for an organization to find itself in and it’s reasonable to expect your tools to not punish you for having less than perfect operational discipline.

                                                                Imagine 20 years ago saying this about a project that suffered because they lost some crucial files and it turned out they weren’t using any kind of version control system.

                                                                Because that’s basically how I feel about it. Regularly keeping dependencies, including language tooling/platform, up-to-date, needs to become table stakes for software-producing entities the way that version control has. I’ve seen this as a theme now at four different companies across more than a decade, and the solution is never to switch and pray that the next platform won’t change. The solution is always to make change an expected part of the process. It can be done, it can be done in a way that minimizes the overhead, and it produces much better results. I know because I have done it.

                                                                1.  

                                                                  Because that’s basically how I feel about it. Regularly keeping dependencies, including language tooling/platform, up-to-date, needs to become table stakes for software-producing entities the way that version control has.

                                                                  I don’t believe this is what’s being disputed: it’s that this being a fact is precisely why it’s important for the platform to facilitate ease of maintenance to the best of its ability in that regard. Doing so allows even under-resourced teams to stay on top of the upgrade treadmill, which is more of my point in the bit you quoted: tools that induce less overhead are more resilient to the practical exigencies that organizations face. I guess we’ll have to agree to disagree about where the Python ecosystem sits on that spectrum.

                                                                  1.  

                                                                    This sounds like the “don’t write bugs” school of thought to me. Yes, ideally anything that’s an operational concern will get ongoing maintenance. In the real world… 

                                                                    More anecedata: my coworker built a small Python scraper that runs as a cron job to download some stuff from the web and upload it to an S3 bucket. My coworker left and I inherited the project. The cron job was no longer a high priority for the company, but we didn’t want to shut it off either. I couldn’t get it to run on my machine for a while because of the Python version problem. Eventually I got to the point where I could get it to run by using Python 3.6, IIRC, so that’s what it’s using to this day. Ideally, if I had time and resources I could have figured out why it was stuck and unstick it. (Something to do with Numpy, I think?) But things aren’t always ideal.

                                                                    If someone has to have discipline and try to stick closer to the “don’t write bugs” school, who should it be: language creators or end developers? It’s easy for me to say language creators, but there are also more of us (end developers) than them (language creators). :-) ISTM that being an upstream brings a lot of responsibility, and one of those should be the knowledge that your choices multiply out by all the people depending on you: if you impose a 1 hour upgrade burden on the 1 million teams who depend on you, that’s 1 million hours, etc.

                                                                    1.  

                                                                      Generalizing from anecdata and my own experience, the killer is any amount of falling behind. If something is not being actively maintained with dependency updates on at least a monthly and ideally a weekly cadence, it is a time bomb. In any language, on any platform. Because the longer you go without updating things, the more the pending updates pile up and the more work there will be to do once you do finally sit down and update (which for many projects, unfortunately, tends to be only when they are absolutely forced to start doing updates and not a moment sooner).

                                                                      At my last employer I put in a lot of work on making the (Python) dependency management workflow as solid as I could manage with only the standard packaging tooling (which I believe you may have read about). But the other part of that was setting up dependabot to file PRs for all updates, not just security, to do so on a weekly basis, and to automate creation of Jira tickets every Monday to tell the team that owned a repository to go look at and apply their dependabot PRs. When you’re doing it on that kind of cadence it averages very little time to review and apply the updates, you find out immediately from CI on the dependabot PRs if something does have a breaking change so you can scope out the work to deal with it right then and there, and you never wind up in a situation where applying the one critical update you actually cared about takes weeks or months because of how much other stuff you let pile up in the meantime.

                                                                      Meanwhile I still don’t think Python or its ecosystem are uniquely bad in terms of breaking changes. I also don’t think Go or Rust are anywhere near as good as the claims made for them. And the fact that this thread went so quickly from absolutist “no breaking changes ever” claims to basically people’s personal opinions that one language’s or ecosystem’s breaking changes are justified and tolerable while another’s aren’t really shows that the initial framing was bad and was more or less flamebait, and probably should not be used again.

                                                                      1.  

                                                                        No, I still think you’re wrong. :-)

                                                                        I agree that for a Python or Node project it is recommended to set up dependabot to keep up to date or else you have a ticking time bomb. However, a) that isn’t always practical and b) it doesn’t have to be like that. I routinely leave my Go projects unattended for years at a time, come back, upgrade the dependencies, and have zero problems with it.

                                                                        Here is a small project last touched in 2017 that uses Go and Node: https://github.com/baltimore-sun-data/track-changes

                                                                        Here is the full Terminal output of me getting it to build again with the most recent version of Go:

                                                                        (Fri, May 20  08:56:09 PM) (master|✔)
                                                                        $ go build .
                                                                        go: cannot find main module, but found Gopkg.lock in /var/folders/p7/jc4qc9n94r3f6ylg0ssh1rq00000gs/T/tmp.S6ZYg4FX/track-changes
                                                                                to create a module there, run:
                                                                                go mod init
                                                                        # status: 1 #
                                                                        (Fri, May 20  08:56:27 PM) (master|✔)
                                                                        $ go mod init github.com/baltimore-sun-data/track-changes
                                                                        go: creating new go.mod: module github.com/baltimore-sun-data/track-changes
                                                                        go: copying requirements from Gopkg.lock
                                                                        go: to add module requirements and sums:
                                                                                go mod tidy
                                                                        (Fri, May 20  08:57:00 PM) (master|…)
                                                                        $ go mod tidy -v
                                                                        go: finding module for package github.com/stretchr/testify/assert
                                                                        go: finding module for package golang.org/x/text/unicode/norm
                                                                        go: finding module for package golang.org/x/text/secure/bidirule
                                                                        go: finding module for package golang.org/x/text/unicode/bidi
                                                                        go: finding module for package golang.org/x/sync/errgroup
                                                                        go: finding module for package github.com/stretchr/testify/suite
                                                                        go: downloading golang.org/x/sync v0.0.0-20220513210516-0976fa681c29
                                                                        go: found github.com/stretchr/testify/assert in github.com/stretchr/testify v1.7.1
                                                                        go: found github.com/stretchr/testify/suite in github.com/stretchr/testify v1.7.1
                                                                        go: found golang.org/x/text/secure/bidirule in golang.org/x/text v0.3.7
                                                                        go: found golang.org/x/text/unicode/bidi in golang.org/x/text v0.3.7
                                                                        go: found golang.org/x/text/unicode/norm in golang.org/x/text v0.3.7
                                                                        go: found golang.org/x/sync/errgroup in golang.org/x/sync v0.0.0-20220513210516-0976fa681c29
                                                                        (Fri, May 20  08:57:10 PM) (master|…)
                                                                        $ go build .
                                                                        (Fri, May 20  08:57:18 PM) (master|…)
                                                                        $ 
                                                                        

                                                                        As you can see, it took about a minute for me to get it building again. Note that this package predates the introduction of Go modules.

                                                                        Let’s upgrade some packages:

                                                                        (Fri, May 20  09:02:07 PM) (master|…)
                                                                        $ go get -v -u ./...
                                                                        go: downloading golang.org/x/net v0.0.0-20220520000938-2e3eb7b945c2
                                                                        go: downloading cloud.google.com/go v0.101.1
                                                                        go: downloading gopkg.in/Iwark/spreadsheet.v2 v2.0.0-20220412131121-41eea1483964
                                                                        go: upgraded cloud.google.com/go v0.16.0 => v0.100.2
                                                                        go: added cloud.google.com/go/compute v1.6.1
                                                                        go: upgraded github.com/ChimeraCoder/anaconda v1.0.0 => v2.0.0+incompatible
                                                                        go: upgraded github.com/andybalholm/cascadia v0.0.0-20161224141413-349dd0209470 => v1.3.1
                                                                        go: upgraded github.com/garyburd/go-oauth v0.0.0-20171004151416-4cff9ef7b700 => v0.0.0-20180319155456-bca2e7f09a17
                                                                        go: upgraded github.com/go-chi/chi v3.3.1+incompatible => v4.1.2+incompatible
                                                                        go: upgraded github.com/golang/protobuf v0.0.0-20171113180720-1e59b77b52bf => v1.5.2
                                                                        go: upgraded github.com/pkg/errors v0.8.0 => v0.9.1
                                                                        go: upgraded golang.org/x/net v0.0.0-20171107184841-a337091b0525 => v0.0.0-20220520000938-2e3eb7b945c2
                                                                        go: upgraded golang.org/x/oauth2 v0.0.0-20171117235251-f95fa95eaa93 => v0.0.0-20220411215720-9780585627b5
                                                                        go: upgraded google.golang.org/appengine v1.0.0 => v1.6.7
                                                                        go: added google.golang.org/protobuf v1.28.0
                                                                        go: upgraded gopkg.in/Iwark/spreadsheet.v2 v2.0.0-20171026120407-29680c88e31d => v2.0.0-20220412131121-41eea1483964
                                                                        (Fri, May 20  09:02:52 PM) (master|…)
                                                                        $ go build .
                                                                        # github.com/baltimore-sun-data/track-changes
                                                                        ./handler.go:28:14: undefined: middleware.DefaultCompress
                                                                        # status: 2 #
                                                                        (Fri, May 20  09:02:58 PM) (master|…)
                                                                        $ go doc middleware
                                                                        package middleware // import "github.com/go-chi/chi/middleware"
                                                                        
                                                                        // snip
                                                                        
                                                                        (Fri, May 20  09:03:12 PM) (master|…)
                                                                        $ go doc middleware.Compress
                                                                        package middleware // import "github.com/go-chi/chi/middleware"
                                                                        
                                                                        func Compress(level int, types ...string) func(next http.Handler) http.Handler
                                                                            Compress is a middleware that compresses response body of a given content
                                                                            types to a data format based on Accept-Encoding request header. It uses a
                                                                            given compression level.
                                                                        
                                                                            NOTE: make sure to set the Content-Type header on your response otherwise
                                                                            this middleware will not compress the response body. For ex, in your handler
                                                                            you should set w.Header().Set("Content-Type",
                                                                            http.DetectContentType(yourBody)) or set it manually.
                                                                        
                                                                            Passing a compression level of 5 is sensible value
                                                                        
                                                                        (Fri, May 20  09:03:32 PM) (master|…)
                                                                        $ subl .
                                                                        (Fri, May 20  09:04:12 PM) (master|…)
                                                                        $ go build .
                                                                        (Fri, May 20  09:04:59 PM) (master|✚1…)
                                                                        $
                                                                        

                                                                        Took about 3 minutes to upgrade the packages and fix the broken dependency (they renamed a middleware). Bear in mind that the upgrade I did deliberately did not try to upgrade past semantic version changes in its dependencies. Probably it would take another half hour or more if I wanted to chase down whatever breaking changes happened there.

                                                                        Suffice it to say, yarn cannot even install its packages, and the last time I tried this stunt a couple of years ago, I got past that and then ran into a problem with webpack that I couldn’t easily solve.

                                                                        Go is just a much more stable ecosystem than Node or Python. It’s not as stable as say browser JS, where one can reasonably expect working code to work until civilization collapses, but it’s fairly stable. And it’s not magic. If there were a communal expectation of this level of stability, it could exist everywhere. It’s a social value to keep things working in the Go ecosystem, and it’s not elsewhere.

                                                                        1.  

                                                                          The other day I returned to a Python package I hadn’t touched in about a year. The actual Python dependencies portion of updating it was done in a few minutes: I updated the supported versions of Python to those currently supported by upstream, did the same for the supported versions of Django, and then had to change a whopping four lines of code, all in a unit-test file, to deal with a deprecation in Django that had finally been removed.

                                                                          The entire remainder of getting it ready for a new release was fighting with CI — updating from v1 to v3 of the GitHub Actions tasks for Python, which I mostly did by copy/pasting from a package by someone else who I trust.

                                                                          I mention this because while you have anecdotes about Go projects updating more or less seamlessly, I have just as many about Python projects, and other people in this thread have anecdotes about Go projects breaking in ways they found annoying.

                                                                          All of which is to say that you should stop trying to extrapolate from your anecdata to “Go is stable and values stability, while Python is not and does not”, because it just ends up looking silly when other people show up with their anecdata. Python is not uniquely “unstable” and neither Go nor Rust are uniquely “stable”. At best, some projects are sometimes lucky enough that they can give the false impression of stability in the language/ecosystem, despite the fact that the language/ecosystem is always moving on. And that’s why I have said, over and over, that the thing to do is embrace and accept change and build processes around it. Otherwise, you’re likely to wake up one day to find out that what you thought was stable and unchanging was neither, and that you are in for a lot of trouble.

                                                                2.  

                                                                  This thread has grown a lot overnight while I was sleeping! I still haven’t caught up, but here is my top of head takeaway:

                                                                  I think it’s fair to complain that the Go 1 stability guarantee doesn’t go far enough, but it seems like some people elsewhere in the thread are trying to claim that adding monotonic time somehow breaks the guarantee, which makes no sense to me. At this point, someone citing Fasterthanlime basically means they don’t know anything about Go, IMO because he has been so egregious in his misrepresentation of the language. Just recently, I had to pin a specific subversion of Go 1.18 in my tests because I needed a particular behavior for the Go tool. I don’t consider that to have been the Go team breaking the stability guarantee, just an example of how you can run into the limits of it when you’re talking about the tooling.

                                                                  Obviously, it’s going to be impossible to never break anything. (Which is what I take from your points 1 and 2.) If nothing else, security problems can force breaking changes. Certainly, you need to get to a point where you’re happy enough to commit to a design before you stabilize it: Go was not stable at all before version 1. JavaScript wasn’t stable before ECMA etc.

                                                                  But the question to me is whether the upstream developers take breaking changes seriously or commit them wantonly. To rephrase your points:

                                                                  1. Projects which accept that breaking changes are fact of software development, so they do it whenever it’s convenient.

                                                                  2. Projects which accept that breaking changes are fact of software development, so they do it as little as possible.

                                                                  I was being a bit too flip when I wrote “You ‘just’ don’t make breaking changes.” That’s like saying, you just don’t write bugs. It is unavoidable sometimes. But there is a qualitative difference as an end developer in relying on a project which attempts to avoid breaking changes and one which does not.

                                                                  In Node, I routinely run into software that will not run on the latest versions of Node. When a new version of Node comes out, I don’t think “Oh great, more speed, more features!” I think, “Ugh, crap, what’s going to break this time?” I had a hard to diagnose break in Babel (a very popular library!) that was caused not by a major version jump in Node, but a minor version bump. It was a waste of a day of development time for me for no reason. You wrote “as long as I pay the hosting bill there is no technical reason why it would suddenly stop working.” But in this case, there wasn’t anything I did intentionally which broke my setup. I just hadn’t pinned my version of Node, Homebrew moved up by a minor version because of some other thing, and suddenly I lost a workday. It also broke production because prod was only pinned to the major version not the minor or patch.

                                                                  A similar example was that I had an Amazon ECS server using a Docker image to serve some Python app, and one day it just stopped working in prod. The problem was that when the Dockerfile was written, it didn’t specify the version of Python, and then the version where async went from a semi-keyword to a full keyword came out and broke the dependencies I was using. It wouldn’t have mattered if the Docker image had been already baked, but the way this project was setup, the Docker image would be periodically rebuilt from the Dockerfile whenever things would restart or something. That at least was relatively easy to track down and fix because the change of Python versions was noticeable once I started debugging it.

                                                                  You call the /v2 thing a “hack” and it is ugly and inelegant, but it does solve this problem: when you make a breaking change, let users upgrade at their own pace, instead of having them accidentally get moved onto the new thing without being aware of it. (In browser JS, "strict mode" and type="module" also have the effect of being opt-in upgrades!) That’s really the core of the so-called “Python problem” to me. Just let me decide when to work through the breaking changes. I’m not expecting eternal support or whatever. I just want something that works today to work tomorrow unless I press the big red “attempt upgrade” button.

                                                                  1. 5

                                                                    At this point, someone citing Fasterthanlime basically means they don’t know anything about Go, IMO because he has been so egregious in his misrepresentation of the language.

                                                                    I’m the one who cited the monotonic time issue. I’m having trouble seeing good faith in this statement, so I’ll bow out of this thread after this. But just so you understand that this is adamantly not the case: I’ve written Go since before 1.0, contributing to some of the largest projects in the ecosystem, and as a manager moved several teams over to using it (from Python, no less.) I also don’t think those credentials should be necessary for a critique.

                                                                    I had a hard to diagnose break in Babel (a very popular library!) that was caused not by a major version jump in Node, but a minor version bump.

                                                                    To reiterate, this has happened to me with minor version bumps in Go, due to both modules and stdlib changes. This is even worse because it was Go itself, not a third-party library.

                                                                    You call the /v2 thing a “hack” and it is ugly and inelegant, but it does solve this problem: when you make a breaking change, let users upgrade at their own pace

                                                                    This actually touches on a major reason why Go’s ecosystem could be so unstable before modules: you could either vendor which had its own can of worms, or rely on authors using this hack. Both patterns emerged as common practice only after years of Go devs experiencing breaking changes in libraries. You could argue that those instabilities don’t apply to Go itself, which I suppose is fair, but that applies to your argument about Babel as well.

                                                                3. 13

                                                                  Go makes breaking changes all of the time. The last one that bit me was when modules were enabled by default. Suddenly packages starting behaving differently depending on a variety of factors, including the version number of the package: https://stackoverflow.com/questions/57355929/what-does-incompatible-in-go-mod-mean-will-it-cause-harm/57372286#57372286

                                                                  Python at least had the sense to bundle up all the breakages in a major version change, which was released 15 years ago.

                                                                  1. 5

                                                                    Eh no. Python does not follow semantic versioning and makes intentionally breaking changes between minor versions. In fact Python 3.x releases come with documentation titled “Porting to Python 3.x” which lists all breaking changes made intentionally.

                                                                    1.  

                                                                      Python makes breaking stdlib changes in minor version releases. Not even remotely the same as a tooling change.

                                                                      1.  

                                                                        I’ve never been affected by stdlib changes in python in a minor release. I have by go, where it suddenly stopped respecting system DNS settings. Or there were the backwards-incompatible changes to time, maybe best summarized here.

                                                                        1.  

                                                                          I believe you, but you are lucky. With search I found things like https://github.com/wazuh/wazuh/issues/13365 which matches my experience.

                                                                          1.  

                                                                            It happens, no doubt, but IME Python’s changes are much more egregious than Go’s few changes over the years. Where Go does make changes, they are typically as solutions to bugs, not gratuitous API changes.

                                                                            Also, speaking personally, I try to be as objective as possible and fasterthanli.me is not an objective source in any sense. It’s clickbait. There’s no pretence of objectivity.

                                                                            1. 11

                                                                              I’d like to point out that this thread has gone from a claim that Go has maintained backwards compatibility to, now, a claim that Go’s backwards-incompatible changes just aren’t as “egregious” as Python’s, and to what’s basically an ad hominem attack on someone who criticized Go.

                                                                              Which is kind of a microcosm of this whole framing of “Python problem” or “Node problem”. What it’s really about is not some magical language and ecosystem that never has backwards-incompatible changes, what it’s about is subjective taste. You think Go’s breaking changes are not as “egregious” and are “solutions to bugs” while Python’s are “gratuitous”. But that’s a subjective judgment based on your tastes and opinions. Someone else can have an opposite and equally-valid subjective opinion.

                                                                              Or, more bluntly: “No breaking changes” nearly always turns out to actually mean “Breaking changes, but only ones I personally think are justified/tolerable, and I don’t think yours are”. Which is where this thread predictably went within the space of just a few replies.

                                                                              Getting back to my original claim: change is inevitable. Entities which produce software can adapt to it and make it a normal and expected part of their processes, or they can suffer the consequences of not doing so. There is no third option for “all external change stops”. Nothing that lives does so changelessly.

                                                                              1.  

                                                                                Or, more bluntly: “No breaking changes” nearly always turns out to actually mean “Breaking changes, but only ones I personally think are justified/tolerable, and I don’t think yours are”. Which is where this thread predictably went within the space of just a few replies.

                                                                                No I don’t think so. It has nothing to do with opinion and everything to do with experience. What someone sees directly is how they perceive reality. I’m expressing my perception of reality as someone who has experience of both Python and Go, and m0th is expressing their experience.

                                                                                I’m not minimising m0th’s view, which is why I phrased it as “in my experience”.

                                                                                Getting back to my original claim: change is inevitable. Entities which produce software can adapt to it and make it a normal and expected part of their processes, or they can suffer the consequences of not doing so. There is no third option for “all external change stops”. Nothing that lives does so changelessly.

                                                                                Change is inevitable, I agree, but I do think the degree matters. Python makes breaking changes often, removing APIs, etc. Go does not and only with very good reason.

                                                                                1.  

                                                                                  Change is inevitable, I agree, but I do think the degree matters. Python makes breaking changes often, removing APIs, etc. Go does not and only with very good reason.

                                                                                  Again, the original claim at the start was that there are languages which don’t have breaking changes, and Go was specifically named as an example. That has now been walked back to the kinds of statements you are making. And your statements are, effectively, just that in your opinion one language’s (Go) breaking changes are justified and not too frequent, while another language’s (Python) breaking changes are not justified and too frequent. Which is just you stating your own personal tastes and opinions. And it’s fine as long as you are willing to admit that. It is not so fine to present one’s personal tastes and opinions as if they are objective facts. It also is not so fine to engage in the kind of ad hominem you did about criticism of Go.

                                                                                  1.  

                                                                                    Again, the original claim at the start was that there are languages which don’t have breaking changes, and Go was specifically named as an example.

                                                                                    I didn’t make that claim, so I’m not sure why you’re arguing with me about it.

                                                                                    Which is just you stating your own personal tastes and opinions. And it’s fine as long as you are willing to admit that.

                                                                                    I did? I explicitly said “In My Experience”. I’m not sure how I can be any clearer.

                                                                                    It is not so fine to present one’s personal tastes and opinions as if they are objective facts. It also is not so fine to engage in the kind of ad hominem you did about criticism of Go.

                                                                                    What are you even talking about? I legitimately can’t even understand what you’re referring to. Where’s the “ad hominem” comment I made? I think the most controversial thing I said was

                                                                                    “Python makes breaking stdlib changes in minor version releases.”

                                                                                    Which is objectively true as mentioned by other commenters. I made no judgement about whether it was justified or not. It’s also not “ad hominem”, which would require Python to have a position on this.

                                                                                    Anyway, I think you’re taking this way too personally for some reason. I like Python, I have used it for many years. But I’m out.

                                                                                    1.  

                                                                                      Where’s the “ad hominem” comment I made?

                                                                                      IIUC, ubernostrum is referring to you explaining that fasterthanli.me is not a reliable source of information. That’s not an ad hominem though. His attacks are so scattershot that if someone cites them, it’s totally fair to just handwave it away.

                                                                                      1.  

                                                                                        Where’s the “ad hominem” comment I made?

                                                                                        It’s this:

                                                                                        Also, speaking personally, I try to be as objective as possible and fasterthanli.me is not an objective source in any sense. It’s clickbait. There’s no pretence of objectivity.

                                                                                        You don’t bother to engage with the content, just brush off the author with a quick smear and then move on. Which is a problem, because the content is reasonably well-argued, from my perspective as someone who has a bit of Go experience and initially couldn’t quite describe why it rubbed me the wrong way. Putting into words the way that Go (which is not unique in this, but is a prominent example of it) likes to pretend complex things are simple, rather than just admitting they’re complex and dealing with it, was extremely useful for me, both for giving me a way to talk about what bugged me in Go and because it relates to things I’ve ranted about in the Python world previously (most notably why Python 3’s string changes were the right thing to do).

                                                                        2. 9

                                                                          Go 1 has been stable for 10 years now.

                                                                          If you just ignore all those breaking changes they made..

                                                                          1.  

                                                                            I only remember Modules. After Modules landed I never experienced any breaking build when updating to a new Go version. A whole different story was updating Java versions and the dozens of subtle ways they can break your service, especially at run-time (caused by some Spring DI magic).

                                                                            1.  

                                                                              What breaking language changes have been made?

                                                                              1.  

                                                                                Modules seems like the biggest change.

                                                                                1.  

                                                                                  I mean, Go module is not part of Go 1 stability guarantee. In my opinion, this shows how limited Go’s stability guarantee is. Go 1 is stable, but Go is not, at least if you are using “go build” to build your Go code.

                                                                                  1.  

                                                                                    I agree but that’s not a language change, and the impact is not as significant as random language changes. I can still use and build a package from 8 years ago with no issue. And whatever tool they happened to use to manage dependencies 8 years ago will still work fine (probably).

                                                                                    1.  

                                                                                      I saw some breaking changes in one of our old projects. Written in the Go v1.4 era, IIRC. I checked it with a Go v1.14 release, and boom, it doesn’t compile due to the module changes.

                                                                                      Yes, it wasn’t that hard to fix (it only took a few minutes of Internet searching), but I still count that as a breaking change.

                                                                          2.  

                                                                            When was there ever a breaking change in a new version of ecmascript? Node is not a language.

                                                                            1.  

                                                                              There’s a bunch of things that vary in stability.

                                                                              • V8 the interpreter breaks working JS code very seldom
                                                                              • The libraries that come with Node break JS code a little more often, but still IME not very often.
                                                                              • V8 breaks native extensions very often. The NodeJS ecosystem discourages writing them because of this.
                                                                              • Some add-on packages from npm break their consumers all the time.

                                                                              Many packages are extremely careful not to break consumers. Others are less so. The experience you have with backwards compatibility tends to track the backwards compatibility stance of the worst thing in your entire dependency tree. When you have tens of thousands of transitive dependencies, you usually end up seeing 99.99%-ile bad behaviour somewhere in there at least once.

                                                                              1.  

                                                                                The core problem is that a handful of “core” packages break, and nodes ecosystem is way too much about plugins, so many things that you use have 3 layers to them (all maintained by different people)

                                                                                The ecosystem would be a lot more stable if we were vendoring in packages more

                                                                              2.  

                                                                                Node is a development target. The pure abstract notion of a “language” doesn’t really matter here, because people write code for Node specifically.

                                                                                And Node does make breaking changes. It literally makes semver-major releases, with backwards-incompatible changes that can break existing npm packages (mainly due to changes in the native API, but there also have been changes to Node’s standard library).

                                                                                For example, I can’t build any projects that used Gulp v3. Node broke packages that it depends on, and I either have to run deprecated/unsupported/outdated version of Node, or rewrite my build scripts. OTOH a Rust/Cargo project I’ve developed at the same time still works with the latest version of Rust/Cargo released yesterday.

                                                                                1.  

                                                                                  Yes, that’s why I said “Node ecosystem” which breaks all the damn time and not “browser ECMAScript” which has not broken since standardization.

                                                                                  1.  

                                                                                    The quote you posted says language, not ecosystem. Your comparison was a false equivalency.

                                                                              1. 46

                                                                                I think you need to consider the Turing Tarpit. It’s true you can write anything in C in theory. However, programming languages are for humans, and you need to consider the human factor.

                                                                                There are types of “simple” programs (e.g. the ones we use scripting languages for) that don’t need performance or low-level features. For these cases C is an overkill, and therefore needlessly hard or tedious. The problem isn’t that C can’t do it, but that the skill and effort required needlessly increases development time and cost. C can do everything that Excel can, but when you have an Excel-sized problem, it’s cheaper to solve it in Excel.

                                                                                And on the other end of the spectrum there are some complex problems for which C is not helping enough. You can do multi-threading in C, but doing so requires a very high level of diligence or limiting yourself to only simplest coarse-grained constructs. Same for security — in theory there isn’t anything in C that makes it insecure if you write it properly, but somehow we struggle to find programmers who never write bugs.

                                                                                I think size and complexity of programs changes what language is needed. If you can hold the whole program in your head, then you can use a small/simple language. OTOH if the program is so large it’s maintained by a whole team of programmers, over many years, with people coming and going, then you start benefiting from higher-level abstractions and the language knowing the program better than you do. At such scale C is doable, but far from convenient.

                                                                                1. 28

                                                                                  I think you need to consider the Turing Tarpit. It’s true you can write anything in C in theory. However, programming languages are for humans, and you need to consider the human factor.

                                                                                  It is worth remembering that there is an infinitely large set of programs but a very small set of useful programs. A programming language is a form of ad-hoc compression algorithm, it is intended to sort the set of useful programs to have shorter encodings than the undesirable ones. Programs with certain categories of error or security vulnerability should be harder or impossible to express. C does very poorly when considered in this way.

                                                                                  1. 4

                                                                                    The abstract view of programming languages as ad-hoc compression algorithms does not motivate any particular criteria for which categories of error should be made difficult to express, or how difficult. So there is a lot missing to get to your conclusion.

                                                                                    1. 5

                                                                                      This compression perspective doesn’t explain everything in language design, but I think it’s roughly correct. BTW, it is the basis of Kolmogorov complexity of programs.

                                                                                      We don’t compress code in a literal byte-saving way (apart from code golfing), but we do like to reduce repetitiveness of code (the DRY principle), eliminate “boilerplate”, and replace detailed low-level constructs with terser high-level ones. All of these happen to reduce the number of symbols/constructs used to build the program.

                                                                                      When we discover “good” programming patterns, we invent shorter syntax for them and make languages based on that. When you look from assembly perspective, even basic constructs like structs and functions with a stack are syntax sugar to codify common patterns and eliminate mistakes. When structs of function pointers became popular, we’ve slapped a syntax on it and called it OOP. Some languages decided that callbacks are good for network code, but there are too many ways to mess up the callbacks, so there’s async/await. Even type systems are an example of the principle that good code is shorter: casts require extra code.

                                                                                      In case of Rust making safe syntax terse and unsafe syntax verbose is an explicit guiding principle. That’s why single-char . operator performs automagic dereference on smart pointers, but the same on “raw” pointers is an eyesore: unsafe { (*x).foo() }.

                                                                                      1. 2

                                                                                        All of these happen to reduce the number of symbols/constructs used to build the program.

                                                                                        I feel like this would deserve more importance tha it gets. All these discussions quickly turn into bikeshedding around type safety, type anotations, casts, type systems, etc.

                                                                                        Yet a terse simple syntax that minimizes the amount of code that needs to be written to perform a task, is often ignored.

                                                                                        The popularity of python, shellscripts or even JavaScript is much due to their ergonomics. Getting the usual tasks done requires writting little code and therefore little effort. None of these languages has strong enforced type checking on compile time. I personally would like some level of type safety in these languages, but that is still secondary. And also, type declarations have to be written sometimes that slows one down.

                                                                                        1. 3

                                                                                          The most popular language on the planet by a huge margin, with over a billion users, is the Excel formula language, for exactly the reasons you describe. Users can start with some data and write a program to do something useful with it in a few seconds.

                                                                                        2. 1

                                                                                          All true. But the abstract view doesn’t justify any particular notion of “safe” or “unsafe,” or any criteria for when a language is doing a good job. C requires extra syntax to use a value as a different type without warnings: (unsigned int) x;

                                                                                          1. 1

                                                                                            I think it does: we’re interested in writing programs that don’t crash, so we optimize languages for not crashing, instead of giving equal importance to crashing and not crashing.

                                                                                            Since we’re unable to express this so directly, the actual syntax is a proxy of that via safe/not-sure-if-safe. For example, an incorrectly placed free can crash the program. So we make syntax for a correctly placed free shorter (e.g. zero in case of GC/ARC/RAII) and then from the pigeonhole principle the syntax for an incorrectly placed free is either impossible or has to be longer.

                                                                                            The C cast is an example where language makes the safe syntax terser than the unsafe syntax. If the types are compatible (which C thinks is good and safe) you just use x, and when the types are not compatible (which may be unsafe), you have to use (type)x. C isn’t very good at it (some bad implicit conversions are allowed), but clearly it’s trying.

                                                                                            1. 1

                                                                                              The abstract view is that a programming language should let you write useful programs more easily/concisely than useless ones. That much is uncontroversial.

                                                                                              From that premise I don’t think you can say that avoiding crashing is the most important criterion. What if “useful” means a program runs as fast as possible using a minimum of resources? Then it’s impossible to express certain desirable programs in languages with automatic memory management, which could be expressed in C.

                                                                                              1. 1

                                                                                                Crashing is never a goal or a desirable property (apart from xkcd 1172), therefore languages never optimize for it.

                                                                                                Programs that are too slow are not useful, so optimization for useful (not too slow) programs may have to make compromises if safer constructs are impossible/not invented yet.

                                                                                                1. 1

                                                                                                  Yes, obviously.

                                                                                    2. 2

                                                                                      Thank you for your thoughts - all good stuff!

                                                                                    1. 2

                                                                                      How would you notice an error returned by the close() syscall for a file?

                                                                                      1. 1

                                                                                        Exceptions, maybe?

                                                                                        1. 2

                                                                                          In Rust?

                                                                                          1. 1

                                                                                            Panic, probably.

                                                                                            1. 3

                                                                                              Error handling in drop is problematic indeed. Some libraries provide close(self) -> Result for handling this the hard way when you really care about the result.

                                                                                              1. 2

                                                                                                std::file::File chooses to ignore those errors.

                                                                                                https://doc.rust-lang.org/stable/std/fs/struct.File.html

                                                                                                1. 2

                                                                                                  Ah, but importantly, it gives you the option to explicitly handle them and also explicitly documents what happens in the default case:

                                                                                                  https://doc.rust-lang.org/stable/std/fs/struct.File.html#method.sync_all

                                                                                          2. 1

                                                                                            To be honest how would you like to handle that situation in your program? Terminate it completely? Retry closing? What if you can’t close the file at all? This is one of those scenarios where error handling isn’t obvious.

                                                                                            1. 1

                                                                                              I agree that there is no obvious right answer. But hiding the error is obviously wrong.

                                                                                          1. 6

                                                                                            There are multiple points here I disagree with:

                                                                                            1. Go’s and Zig’s defer are rather different beasts Go runs defered statements at the end of the function, Zig at the end of scope. ant to lock a mutex insife a loop? Can’t use Go defer for that..
                                                                                            2. destructors can’t take arguments or return values While most destructions only release acquired resources, passing an argument to a defered call can be very useful in many cases
                                                                                            3. hidden code all defer code is visible in the scope. Look for all lines starting with defer in the current scope and you have all the calls. Looking for destructors means looking how drop is implemented for all the types in the scopes.
                                                                                            1. 11

                                                                                              Go’s and Zig’s defer are rather different beasts Go runs defered statements at the end of the function, Zig at the end of scope. ant to lock a mutex insife a loop? Can’t use Go defer for that..

                                                                                              This distinction doesn’t really matter in a language with first-class lambdas. If you want to unlock a mutex at the end of a loop iteration with Go, create and call a lambda in the loop that uses defer internally.

                                                                                              destructors can’t take arguments or return values

                                                                                              But constructors can. If you implement a Defer class to use RAII, it takes a lambda in the constructor and calls it in the destructor.

                                                                                              hidden code all defer code is visible in the scope

                                                                                              I’m not sure I buy that argument, given that the code in defer is almost always calling another function. The code inside the constructor for the object whose cleanup you are defering is also not visible in the calling function.

                                                                                              1. 4

                                                                                                hidden code all defer code is visible in the scope

                                                                                                I’m not sure I buy that argument, given that the code in defer is almost always calling another function. The code inside the constructor for the object whose cleanup you are defering is also not visible in the calling function.

                                                                                                The point is that as a reader of zig, you can look at the function and see all the code which can be executed. You can see the call and breakpoint that line. As a reader of c++, it’s a bit more convoluted to breakpoint on destructors.

                                                                                                1. 2

                                                                                                  you can look at the function and see all the code which can be executed.

                                                                                                  As someone that works daily with several hundred lines functions, that sounds like a con way more than a pro.

                                                                                                2. 1

                                                                                                  But constructors can.

                                                                                                  This can work sometimes, but other times packing pointers in a struct just so you can drop it later is wasteful. This happens a lot with for example the Vulkan API where a lot of the vkDestroy* functions take multiple arguments. I’m a big fan of RAII but it’s not strictly better.

                                                                                                  1. 1

                                                                                                    At least in C++, most of this all goes away after inlining. First the constructor and destructor are both inlined in the enclosing scope. This turns the capture of the arguments in the constructor into local assignments in a structure in the current stack frame. Then scalar replacement of aggregates runs and splits the structure into individual allocas in the first phase and then into SSA values in the second. At this point, the ‘captured’ values are just propagated directly into the code from the destructor.

                                                                                                  2. 1

                                                                                                    If you want to unlock a mutex at the end of a loop iteration with Go, create and call a lambda in the loop that uses defer internally.

                                                                                                    Note that Go uses function scope for defer. So this will actually acquire locks slowly then release them all at the end of function. This is very likely not what you want and can even risk deadlocks.

                                                                                                    1. 1

                                                                                                      Is a lambda not a function in Go? I wouldn’t expect defer in a lambda to release the lock at the end of the enclosing scope, because what happens if the lambda outlives the function?

                                                                                                      1. 1

                                                                                                        Sorry, I misread what you said. I was thinking defer func() { ... }() not func() { defer ... }().

                                                                                                        1. 2

                                                                                                          Sorry, I should have put some code - it’s much clearer what I meant from your post.

                                                                                                  3. 5

                                                                                                    The first point is minor, and not really changing the overall picture of leaking by default.

                                                                                                    Destruction with arguments is sometimes useful indeed, but there are workarounds. Sometimes you can take arguments when constructing the object. In the worst case you can require an explicit function call to drop with arguments (just like defer does), but still use the default drop to either catch bugs (log or panic when the right drop has been forgotten) or provide a sensible default, e.g. delete a temporary file if temp_file.keep() hasn’t been called.

                                                                                                    Automatic drop code is indeed implicit and can’t be grepped for, but you have to consider the trade-off: a forgotten defer is also invisible and can’t be grepped for either. This is the change in default: by default there may be drop code you may not be aware of, instead of by default there may be a leak you may not be aware of.

                                                                                                    1. 3

                                                                                                      destructors can’t take arguments or return values. While most destructions only release acquired resources, passing an argument to a deferred call can be very useful in many cases.

                                                                                                      Yes, more than useful:

                                                                                                      • Zero-cost abstraction in terms of state: A deferred call doesn’t artificially require objects to contain all state needed by their destructors. State is generally bad, especially references, and especially long lived objects that secretly know about each other.
                                                                                                      • Dependencies are better when they are explicit: If one function needs to run before another, letting it show (in terms of what arguments they require) is a good thing: It makes wrong code look wrong (yes, destruction order is a common problem in C++) and prevents it from compiling if you have lifetimes like Rust.
                                                                                                      • Expressiveness: In the harsh reality we live in, destructors can fail.

                                                                                                      I think the right solution is explicit destructors: Instead of the compiler inserting invisible destructor calls, the compiler fails if you don’t. This would be a natural extension to an explicit language like C – it would only add safety. Not only that: It fits well with defer too – syntactic sugar doesn’t matter, because it just solves the «wrong default» problem. But more than anything, I think it would shine in a language with lifetimes, like Rust, where long lived references are precisely what you don’t want to mess with.

                                                                                                      1. 2

                                                                                                        You could run an anonymous function within a loop in Go, just to get the per-loop defer. Returning a value in a defer is also possible.

                                                                                                        func main() (retval int) {
                                                                                                            for {
                                                                                                                func() {
                                                                                                                    // do stuff per loop
                                                                                                                    defer func() {
                                                                                                                        // per loop cleanup
                                                                                                                    }()
                                                                                                                }()
                                                                                                            }
                                                                                                            defer func() {
                                                                                                                retval = 42
                                                                                                            }()
                                                                                                            return
                                                                                                        }
                                                                                                        
                                                                                                      1. 24

                                                                                                        Yeah yeah, mention Rust. Rust is too complicated to implement by one person.

                                                                                                        I’m not sure that’s a practical metric by which to judge a tool. The C compilers that provide a practical foundation for modern software development were not implemented by one person either.

                                                                                                        In general Turing completeness is necessary but not sufficient: it’s just one facet of what makes a language practically useful. There are many other properties that end up resulting in costs someone has to pay to use a language; e.g., is it memory safe, or will engineers and users alike be on the hook for an unknown number of egregious memory safety bugs?

                                                                                                        1. 12

                                                                                                          Also mrustc has been implemented mostly by one person.

                                                                                                          1. 2

                                                                                                            I knew this would be brought up; you know the effort they’ve had to do to achieve this? An incredible amount.

                                                                                                            1. 8

                                                                                                              It’s 100K lines of code, and majority of it was developed over a 2-3 year period (with ongoing development to catch up with evolution of Rust). The number of commits and lines of code happens to be close to TCC:

                                                                                                              It does take a couple of shortcuts: it’s a Rust-to-C compiler (no machine code generation) and it doesn’t perform borrow checking (the Rust language is carefully designed to make it optional. Lifetimes are purely a compile-time lint, and don’t affect generated code or its behavior).

                                                                                                              I think overall in terms of implementation difficulty Rust is somewhere between C and C++. Parsing of Rust is much simpler than C++, and Rust has fewer crufty language features than C++ (there’s one way to initialize a variable), but some features are big-ish (borrow checker, type inference).

                                                                                                              How hard it is to implement mainly depends on how good quality of implementation you want to have. For example, LLVM is 85× larger than mrustc and tcc, with over 130× more commits. It’s a 20-year collaborative effort, likely not possible to do by a single person. The main rustc project is also maximalist like that, because it isn’t merely an effort to get Rust working, but to make it fast, efficient, user-friendly, well-documented, reliable, portable, etc., so much much more work went into it beyond just the language implementation.

                                                                                                              1. 2

                                                                                                                I cannot speak for mrustc, but 100k loc for tcc is bullshit. Just counting sources and headers in the top level, I get 55k loc (the remainder is taken up by tests and win32 headers). Close to 40k is taken up by target-specific code. The core compiler is about 10k loc.

                                                                                                                1. 1

                                                                                                                  openhub stats I’ve quoted are for the whole repo, and I see 57K .c and 38K .h in there. This includes tests, so it’s indeed more than just the compiler.

                                                                                                                  1. 2

                                                                                                                    If I run a word count on everything in the ‘src’ directory of mrustc, I get about 130k loc. I therefore conclude that mrustc’s rust compiler is approximately 10x larger and more complex than tcc’s c compiler. Recall that tcc also includes assemblers and linkers, and supports many targets.

                                                                                                                2. 0

                                                                                                                  I mean if 3 years is not a lot of effort then cheers to you! You must be an absolute coding beast.

                                                                                                                  1. 15

                                                                                                                    I feel like this is a fairly disingenuous and dismissive argument - your original post stated that “Rust is too complicated to implement by one person.” The comment you were responding to was making the point that not only is there an implementation of Rust by primarily one person, but a single-contributor C implementation is a comparable size and would theoretically take a similar amount of effort to implement. People here aren’t trying say it’s not a lot of effort, but that it does exist and you may be trivializing the amount of effort needed for a C implementation.

                                                                                                                    1. 3

                                                                                                                      Sorry, I didn’t mean to dismiss anything! Isn’t the statement still true if it’s been mentioned they still got help?… Regardless the general sentiment is right. I should have said instead that it’s not reasonable!

                                                                                                                      I may very well be trivializing the effort for a C implementation. In my mind C’s type system, lack of borrow checker, and other features make its implementation maybe a magnitude easier. I could be completely wrong though and please elaborate if that’s the case!

                                                                                                                      1. 4

                                                                                                                        A non-optimizing C89 or C90 compiler is relatively simple to implement, with only minor inconveniences from the messy preprocessor, bitfields, parsing ambiguities of dangling else and typedef (did you know it can be scoped and nested and this affects syntax around it!?). The aren’t any things that are hard per-se, mostly just tedious and laborious, because there’s a lot of small quirks underneath the surface (e.g. arrays don’t always decay to pointers, sizeof evaluates things differently, there are rules around “sequence points”).

                                                                                                                        There are corners of C that most users don’t use, but compiler in theory needs to support, e.g. case doesn’t have to be at the top level of switch, but can be nested inside other arbitrary code. C can generate “irreducible” control flow, which is hard to reason about and hard to optimize. In fact, a lot of optimization is pretty hard due to aliasing, broken const, and the labyrinth of what is and isn’t UB described in the spec.

                                                                                                                        1. 3

                                                                                                                          There are corners of C that most users don’t use, but compiler in theory needs to support, e.g. case doesn’t have to be at the top level of switch, but can be nested inside other arbitrary code

                                                                                                                          It’s worth noting that, since you said ‘non-optimising’ these things are generally very easy in a non-optimising compiler. You can compile C more or less one statement at a time, including case statements, as long as you are able to insert labels after you insert a jump to them (which you can with most assembly languages). Similarly, sequence points matter only if you’re doing more than just evaluating expressions as you parse them.

                                                                                                                          The original C compiler ran on a computer that didn’t have enough memory for a full parsed AST and so the language had to support incremental code generation from a single-pass compiler.

                                                                                                            2. 9

                                                                                                              LLVM was originally just Chris Latner. I think the question isn’t “Can one person build it?” It’s “Can one person build it to the point where it has enough value for other people to work on it too?”

                                                                                                              1. 5

                                                                                                                LLVM was originally just Chris Latner

                                                                                                                Several of the folks in / formerly in Vikram Adve’s group at UIUC would be quite surprised to learn that.

                                                                                                                1. 1

                                                                                                                  I actually looked at Wikipedia first before my comment, but that made it seems like it was Latner’s project under Adve’s mentorship. I’ll take your word for it that it was a group effort from the start.

                                                                                                              2. 3

                                                                                                                This was my first thought as well. There are a lot of very useful things that are too complicated to be implemented by one person - the current state of Linux probably falls into that category, and I know that at least I wouldn’t want to go back to even a version from 5 years ago, much less back to a version that could have been implemented by a single person.

                                                                                                                1. 2

                                                                                                                  …And there are a lot of useful things that are simple enough for one person to implement! :D

                                                                                                                  1. 3

                                                                                                                    Ha, I agree with that, was mostly just highlighting that I don’t feel like “too complicated to implement by one person” is a good reason to dismiss Rust’s potential usefulness.

                                                                                                                    For myself, I originally got frustrated with Rust not allowing me to do things; eventually, I realized that it was statically removing bad habits that I’d built in the past. Now I love when it yells at me :)

                                                                                                                2. 1

                                                                                                                  [Tool] is too complicated to implement by one person.

                                                                                                                  I’m not sure that’s a practical metric by which to judge a tool

                                                                                                                  I am. Short term, that means the tool will cost much less: less time to make, fewer bugs, more opportunities for improvement. Long term it means other people will be able to rebuild it from scratch if they need to. At a lower cost.

                                                                                                                  1. 2

                                                                                                                    The flip side of this is that the tool will do much less. A wooden hammer is a tool that a single person can make. A hammer with a steel head that can drive in nails requires a lot more infrastructure (smelting the ore and casting the head are probably large enough tasks that you’ll need multiple people before you even get to adding a wooden handle). An electric screwdriver requires many different parts made in different factories. If I want to fix two pieces of wood together than a screw driven by an electric screwdriver is both easier to use and produces a much better result than a nail driven by a wooden hammer.

                                                                                                                    1. 1

                                                                                                                      Obviously I was limiting my analysis to software tools, where the ability of a single person to make it is directly tied to its complexity.

                                                                                                                      One fair point you do have is how much infrastructure the tool sits upon. Something written in Forth needs almost nothing besides the hardware itself. Something written in Haskell is a very different story. Then you need to chose what pieces of infrastructure you want to depend on. For instance, when I wrote my crypto library I chose C because of it’s ubiquity. It’s also a guarantee of fairly extreme stability. There’s a good chance that the code I write now will still work several decades from now. If I wanted to maximise safety instead, I would probably have picked Rust.

                                                                                                                      1. 3

                                                                                                                        Obviously I was limiting my analysis to software tools, where the ability of a single person to make it is directly tied to its complexity.

                                                                                                                        My point still applies. A complex software tool allows me to do more. In the case of a programming language, a more complex compiler allows me to write fewer bugs or more features. The number of bugs in the compiler may be lower for a compiler written by a single person but I would be willing to bet that the number of bugs in the ecosystem is significantly higher.

                                                                                                                        The compiler and standard library are among the best places for complexity in an ecosystem because the cost is amortised across a great many users and the benefits are shared similarly. If physical tools were, like software, zero marginal cost goods, then nail guns, pillar drills, band saws, and so on would all be ubiquitous. If you tried to make the argument that you prefer a manual screwdriver to an electric one because you could build one yourself if you needed then you’d be laughed at.

                                                                                                                        For instance, when I wrote my crypto library I chose C because of it’s ubiquity. It’s also a guarantee of fairly extreme stability

                                                                                                                        It also gives you absolutely no help in writing constant-time code, whereas a language such as Low* allows you to prove constant-time properties at the source level. The low* compiler probably depends on at least a hundred person-years of engineering but I’d consider it very likely that the EverCrypt implementations of the same algorithms would be safer to use than your C versions.

                                                                                                                        1. 2

                                                                                                                          I reckon amortized cost is a strong argument. In a world where something is build once and used a gazillion times the cost analysis is very different from something that only has a couple users. Which is why by the way I have a very different outlook for Oberon and Go: the former were used in a single system, and the cost of a more powerful compiler could easily outweigh the benefits across the rest of the system; while Go set out to be used by a gazillion semi-competent programmers, and the benefit of some conspicuously absent features would be multiplied accordingly.

                                                                                                                          Honestly, I’m not sure where I stand. For the things I make, I like to keep it very very simple. On the other hand, If I’m being honest with myself I have little qualms sitting on a mountain of complexity, provided such foundation is solid enough.

                                                                                                                          Do you have a link to Low*? My search engine is failing me.

                                                                                                                          1. 2

                                                                                                                            Do you have a link to Low*? My search engine is failing me.

                                                                                                                            This paper is probably the best place to start

                                                                                                                  2. 1

                                                                                                                    The C compilers that provide a practical foundation for modern software development were not implemented by one person either.

                                                                                                                    Right but there are many C compilers which were written by one person and still work. To me, that’s the important part. Thank you for your thoughts!

                                                                                                                    1. 2

                                                                                                                      Why is that important?

                                                                                                                      1. 1

                                                                                                                        It’s important because fast forward 300 years and no one uses your language anymore. It must be reasonable the future humans can write a compiler on their own if they want to run your program.

                                                                                                                        I’m really trying to encourage people thinking beyond their lives in the software realm lately, just as we need to do the same for the ecosystem.

                                                                                                                        1. 3

                                                                                                                          trying to build software to last 300 years seems like it would limit hardware development
                                                                                                                          and everyone implements C compatibility in their new hardware so that people will use it
                                                                                                                          if people can figure out quantum computers and computers not based on binary, they’ll probably need to figure out what the next C will be for that new architecture
                                                                                                                          if you want your software to last 300 years, write it in the most readable and easy-to-understand manner, and preserve it’s source so people can port it in the future

                                                                                                                          1. 3

                                                                                                                            And this is why C is not good for longevity, but languages which are more abstracted. Thank you for that! Completely agree with what you’re thinking here.

                                                                                                                            1. 3

                                                                                                                              i don’t think the biggest blockers to software longevity is language choices or even hardware, it’s the economy/politics of it… long lasting anything doesn’t fit in well with our throw-away society, and since it can’t be monetized, the capitalist society snubs it’s nose at it

                                                                                                                              1. 1

                                                                                                                                Hehe, an interesting thread of thought we could travel down here. I’ll just say I agree to a degree.

                                                                                                                          2. 3

                                                                                                                            It’s important because fast forward 300 years and no one uses your language anymore. It must be reasonable the future humans can write a compiler on their own if they want to run your program.

                                                                                                                            If you’re considering a person 300 years in the future then you should also consider that they will have tools 300 years more advanced than ours. 30 years ago, writing a simple game like space invaders was weeks worth of programming, now it’s something that you can do in an afternoon, with significantly better graphics. In the same time, parser generators have improved hugely, reusable back ends are common, and so on. In 300 years, it seems entirely feasible that you’d be able to generate a formal specification for a language from a spoken description and synthesise an optimising compiler directly from the operational semantics.

                                                                                                                            1. 1

                                                                                                                              You’re right, I haven’t considered this! I don’t know what to say immediately other than I think this is very important to think about. I’d like to see what others have to comment on this aspect too…!

                                                                                                                              1. 1

                                                                                                                                you should also consider that they will have tools 300 years more advanced than ours.

                                                                                                                                Unless there has been a collapse in between. With climate change and peak oil, we have some serious trouble ahead of us.

                                                                                                                                1. 4

                                                                                                                                  In which case, implementing the compiler is one of the easiest parts of the problem. I could build a simple mechanical computer that could execute one instruction every few seconds out of the kind of materials that a society with a Victorian level of technology could produce, but that society existed only because coal was readily accessible. I’ve seen one assessment that said that if the Victorians had needed to use wood instead of coal to power their technology they’d have completely deforested Britain in a year. You can smelt metals with charcoal, but the total cost is significantly higher than with coal (ignoring all of the climate-related externalities).

                                                                                                                                  Going from there to a transistor is pretty hard. A thermionic valve is easier, but it requires a lot of glass blowing (which, in turn, requires an energy-dense fuel source such as coal to reach the right temperatures) and the rest of a ‘50s-era computer required fairly pure copper, which has similar requirements. Maybe a post-collapse civilisation would be lucky here because there’s likely to be fairly pure copper lying around in various places.

                                                                                                                                  Doping silicon to produce integrated circuits requires a lot of chemical infrastructure. Once you can do that, the step up to something on the complexity of a 4004 is pretty easy but getting lithography to the point where you can produce an IC powerful enough to run even a fairly simple C program is nontrivial. Remember that C has a separate preprocessor, compiler (which traditionally had a separate assembler), and linker because it was designed for computers that couldn’t fit more than one of those in RAM at a time. Even those computers were the result of many billions of dollars of investment from a society that already had mass production, mass mining, and large-scale chemistry infrastructure.

                                                                                                                                  C code today tends to assume megabytes of RAM, at a minimum. Magnetic core storage could do something like 1 KiB in something the size of a wardrobe. Scaling up production to the point where 1 MiB is readily available requires ICs, so any non-trivial C program is going to have a dependency on at least ’80s-era computing hardware.

                                                                                                                                  TL;DR: If a society has collapsed and recovered to the point where it’s rediscovering computers, writing a compiler for a fairly complex language is going to be very low cost in comparison to building the hardware that the compiler can target.

                                                                                                                                  1. 1

                                                                                                                                    Well, I wasn’t anticipating such a hard collapse. I was imagining a situation where salvage is still a thing, or where technology doesn’t regress that far. Still, you’re making a good point.

                                                                                                                                    1. 3

                                                                                                                                      That’s an interesting middle ground. It’s hard for me to imagine a scenario in which computers are salvageable but storage is all lost to the point where a working compiler is impossible to find. At the moment, flash loses its ability to hold charge if not powered for a few years but spinning rust is still fine, as is magnetic tape, for a much longer period, so you’d need something else to be responsible for destroying them. Cheap optical storage degrades quite quickly but there are archive-quality disks that are rated for decades. If anything, processors and storage are more fragile.

                                                                                                                                      In the event of a collapse of society, I think it’s a lot more likely that copies of V8 would survive longer than any computer capable of running them. The implicit assumption in the idea that the compiler would be a bottleneck recovering from a collapse of society is that information is more easily destroyed than physical artefacts. This ignore the fact that information is infinitely copyable, whereas the physical artefacts in question are incredibly complex and have very tight manufacturing tolerances.

                                                                                                                                      Of course, this is assuming known threats. It’s possible that someone might create a worm that understands a sufficiently broad range of vulnerabilities that it propagates into all computers and erases all online data. If it also propagates into the control systems for data warehouses then it may successfully destroy a large proportion of backups. Maybe this could be combined with a mutated bacterium that ate something in optical disks and prevented recovering from backup DVDs or whatever. Possibly offline storage will completely go out of fashion and we’ll end up with all storage being some form of RAM that is susceptible to EMP and have all data erased by a solar flare.

                                                                                                                                      1. 1

                                                                                                                                        It really depends on what we can salvage, and what chips can withstand salvage operations. In a world where we stop manufacturing computers (or at least high-end chips), I’d expect chips to fail over the years, and the most complex ones will likely go first. And those that don’t will be harder to salvage for various reasons: how thin their connection pins are, ball arrays, multi-layer boards requirements, and the stupidly fast rise times that are sure to cause cross-talk and EMI problems with the hand made boards of a somewhat collapsed future.

                                                                                                                                        In the end, many of us may be stuck with fairly low-end micro controllers and very limited static memory chips (forget about controlling DRAM, it’s impossible to do even now without a whole company behind you). In that environment, physical salvage is not that horrible, but we’d have lost enough computing power that we’ll need custom software for it. Systems that optimise for simplicity, like Oberon, might be much more survivable in this environment.

                                                                                                                                        C code today tends to assume megabytes of RAM, at a minimum.

                                                                                                                                        In this hypothetical future, that is relevant indeed. Also, I believe you. But then the first serious project I wrote in C, Monocypher, requires only a couple KB of stack memory (no heap allocation) for everything save password hashing. The compiled code itself fits requires less than 40KB of memory. Thing is, I optimised it for simplicity and speed, not for memory usage (well, I did curb memory use a bit when I’ve heard I had embedded users).

                                                                                                                                        I suspect that when we optimise for simplicity, we also tend to use less resources as a side effect.


                                                                                                                                        Now sure, those simple systems will take no time to rebuild from scratch… if we have the skills. In our world of bigger and faster computers with a tower of abstraction taller than the Everest, I feel most of us simply don’t have those skills.

                                                                                                                                        1. 3

                                                                                                                                          Now sure, those simple systems will take no time to rebuild from scratch… if we have the skills. In our world of bigger and faster computers with a tower of abstraction taller than the Everest, I feel most of us simply don’t have those skills.

                                                                                                                                          While it’s an interesting thought exercise, but I think this really is the key point. The effort in salvaging a working compiler to be able to run some tuned C code in a post-apocalyptic future may be significantly higher than just rewriting it in assembly for whatever system you were able to salvage (and, if you can’t salvage an assembler, you can even assemble it by hand after writing it out on some paper. Assuming cheap paper survives - it was very expensive until a couple of hundred years ago).

                                                                                                                                          Most of us probably don’t have the skills to reproduce the massive towers of abstraction that we use today from scratch but my experience teaching children and young adults to program suggests that learning to write simple assembly routines is a skill that a large proportion of the population could pick up fairly easily if necessary. If anything, it’s easier to teach people to write assembly for microcontrollers than JavaScript for the web because they can easily build a mostly correct mental model of how everything works in the microcontroller.

                                                                                                                                          Perhaps more importantly, it’s unlikely that any software that you write now will solve an actual need for a subsistence level post-apocalyptic community. They’re likely to want computers for automating things like irrigation systems or monitoring intrusion sensors. Monocypher is a crypto library that implements cryptosystems that assume an adversary who had access to thousands of dedicated ASICs trying to crack your communications. A realistic adversary in this scenario would struggle to crack a five-wheel Enigma code and that would be something that you could implement in assembly in a few hours and then send the resulting messages in Morse code with an AM radio.

                                                                                                                                          1. 1

                                                                                                                                            Most of us probably don’t have the skills to reproduce the massive towers of abstraction that we use today from scratch but my experience teaching children and young adults to program suggests that learning to write simple assembly routines is a skill that a large proportion of the population could pick up fairly easily if necessary.

                                                                                                                                            I feel a little silly for not having thought of that. Feels obvious in retrospect. If people who have never programmed can play Human Resource Machine, they can probably learn enough assembly to be useful.

                                                                                                                                            Perhaps more importantly, it’s unlikely that any software that you write now will solve an actual need for a subsistence level post-apocalyptic community.

                                                                                                                                            Yeah, I have to agree there.

                                                                                                                              2. 2

                                                                                                                                Today’s humans were able to create Rust, so I don’t see why future humans wouldn’t. Future humans will probably just ask GPT-3000 to generate the compiler for them.

                                                                                                                                If you’re thinking about some post-apocalyptic scenario with a lone survivor rebuilding the civilisation, then our computing is already way beyond that. In the 1960’s you were able to hand-stitch RAM, but to even hold source code of modern software, let alone compile and run it, you need more technology than a single person can figure out.

                                                                                                                                C may be your point of reference, because it’s simple by contemporary standards, but it wasn’t a simple language back when the hardware was possible to comprehend by a single person. K&R C and single-pass C compilers for PDP-11 are unusable for any contemporary C programs, and C is too complex and bloated for 8-bit era computers.

                                                                                                                                1. 1

                                                                                                                                  If GPT can do that for us then hey, I will gladly gladly welcome it. I’m not thinking about a post-apocalyptic scenario but I can see the relationship to it.

                                                                                                                                2. 1

                                                                                                                                  I would also be careful about timespans here. computers haven’t been around for a century yet, so who knows what things will be like 100 years from now? I don’t even know if it’s possible to emulate an ENIAC and run old punch card code on modern hardware, that’s the sort of change we’ve seen in just 75y. maybe multicore x86 machines running windows/*nix/BSD will seem similarly arcane 300y from now.

                                                                                                                                  1. 1

                                                                                                                                    But why one person? I think we’ll still write software in teams in 2322, if we write software at all by that point instead of flying spaceships and/or farming turnips in radioactive wastelands. The software was written by teams today, and I think, if it needs to be rewritten, it will be rewritten by teams in the future.

                                                                                                                                3. 1

                                                                                                                                  Wouldn’t a published standard be more important to future programmers? Go might be a wonderful language, but is there a standards document I can read from which an implementation can be written from?

                                                                                                                              1. 4

                                                                                                                                Another thing to consider is that a cryptographic hash can reliably give you 128 bits that will uniquely and deterministically identify any piece of data. Very handy if you need to make UUIDs for something that doesn’t have them.

                                                                                                                                1. 1

                                                                                                                                  But even a cryptographic hash has collisions, however unlikely. So there is always a chance that two distinct pieces of data will end up with the same id. But probably the same can happen, for example, with random UUID (except here you rely on the quality of your randomness source rather than the quality of your hash function and the shape of your data). Somehow using a crypto hash as a long-term id always feels iffy to me.

                                                                                                                                  1. 3

                                                                                                                                    Given the size of hashes, using the hash of the content as its id is totally safe. The hashing algorithms are designed such that collisions are so unlikely that they are impossible to happen.

                                                                                                                                    If this wasn’t the case, all systems based on content adressing would be in serious trouble. Systems like git or ipfs

                                                                                                                                    1. 1

                                                                                                                                      If this wasn’t the case, all systems based on content adressing would be in serious trouble. Systems like git or ipfs

                                                                                                                                      Any system that assumes a hash of some content uniquely identifies that content is in fact in serious trouble! They work most of the time, but IPFS is absolutely unsound in this regard. So is ~every blockchain.

                                                                                                                                      1. 2

                                                                                                                                        I’ve seen several cryptographic systems rely on exactly this fact for their security. So while it’s probabilistic, you’re relying on the Birthday Paradox to ensure it’s highly unlikely.

                                                                                                                                        From that table, for a 128-bit hash function, for a 0.1% chance of a collision, you’d need to hash 8.3e17 items. In practical terms, a machine that can hash 1,000,000 items per second would need to run for just over 26 millennia to have 0.1% chance of a collision.

                                                                                                                                        For systems that use 256-bit digests (like IPFS), it would take many orders of magnitude longer.

                                                                                                                                        1. 1

                                                                                                                                          I’ve seen several cryptographic systems rely on exactly this fact for their security. So while it’s probabilistic, you’re relying on the Birthday Paradox to ensure it’s highly unlikely.

                                                                                                                                          If collisions are an acceptable risk as long as their frequency is low enough, then sure, no problem! Engineering is all about tradeoffs like this one. You just can’t assert that something which is very improbable is impossible.

                                                                                                                                          1. 1

                                                                                                                                            I can still pretend and almost certainly get away with it. If the chances of getting bitten by this pretense is ten orders of magnitude lower than the chance of a cosmic ray hitting the right transistor in a server’s RAM and cause something really bad to happen, then for all practical purposes, I’m safe to live in blissful ignorance. And what a bliss it is; Assuming a SHA-256 hash uniquely identifies a given string can immensely simplify your system architecture.

                                                                                                                                          2. 1

                                                                                                                                            You’ve misread the table. 1.5e37 is for 256 bit hashes. For 128 bits it’s 8.3e17, which is obviously a lot smaller.

                                                                                                                                            For context with IPFS, the Google search index is estimated to contain between 10 and 50 billion pages.

                                                                                                                                            1. 1

                                                                                                                                              You’ve misread the table

                                                                                                                                              Thanks. Although it’s what happens when you start with an 256bit example, then remember everyone’s talking about UUIDs, and hastily re-calculate everything. :/

                                                                                                                                      2. 3

                                                                                                                                        No, this is absolutely not a consideration. Your CPU and RAM have lower reliability than a 128-bit cryptographic hash. If you ever find a collision by chance it’ll be more likely to be false positive due to hardware failure (we’re talking 100-year timespan at a constant rate of billions of uuids per second).

                                                                                                                                        And before you mention potential cryptographic weaknesses, consider that useful attacks need a preimage attack, and the “collision” attacks known currently are useless for making such uuids collide.

                                                                                                                                        1. 2

                                                                                                                                          Whenever you map from a set of cardinality N (content) to a set of cardinality less than N (hashes of that content) by definition you will have collisions. A hash of something is just definitionally not equivalent to that thing, and doesn’t uniquely identify it.

                                                                                                                                          As an example, if I operate a multi-tenant hosting service, and a customer uploads a file F, I simply can’t assert that any hash of the content of F can be used as a globally unique reference to F. Another customer can upload a different file F’ which hashes identically.

                                                                                                                                          “Highly unlikely” isn’t equivalent to “impossible”.

                                                                                                                                          1. 4

                                                                                                                                            It’s absolutely impossible for all practical purposes. It’s a useless pedantry to consider otherwise.

                                                                                                                                            Remember we’re talking about v4 UUID here which already assumes a “risk” of collisions. Cryptographic hashes are indistinguishable from random data, and are probably more robust than your prng.

                                                                                                                                            The risk of an accidental collision is so small, you can question whether there’s enough energy available to our entire civilisation to compute enough data to ever collide in the 128-bit space in it’s lifetime.

                                                                                                                                            1. 1

                                                                                                                                              It’s absolutely impossible for all practical purposes. It’s a useless pedantry to consider otherwise.

                                                                                                                                              I mean it literally isn’t, right? “Absolutely impossible” is just factually not equivalent to “highly improbable” — or am I in the wrong timeline again? 😉 Going back to the hosting example, if you want to use UUIDs to identify customer documents that’s fine, but you can’t depend on the low risk of collisions to establish tenant isolation, you still have to namespace them.

                                                                                                                                              1. 1

                                                                                                                                                By your definition of impossible, there is literally no possible solution since it would require infinite memory. At that point you should question why using a computer at all.

                                                                                                                                                The fact is that people don’t quite understand uuid and use it everywhere in meme fashion. Most uuid usages as database keys i’ve seen, it is even stored as a string. Which creates much more serious problems than those in discussion here.

                                                                                                                                                1. 1

                                                                                                                                                  You don’t need infinite memory to uniquely identify documents among customers. You just need a coördination point to assign namespaces.

                                                                                                                                                  I agree that UUID using in the wild is…. wild. I don’t think most developers even really understand that the dashed-hex form of a UUID is actually just one of many possible encodings of what is ultimately just 16 bytes in memory.

                                                                                                                                            2. 3

                                                                                                                                              The only system behaviour that the universe guarantees is that all systems will eventually decay.

                                                                                                                                              For any other behaviour you have to accept some probability that it won’t happe (hardware failure, bugs, operator error, attacks, business failure, death, and, yes, hash collisions).

                                                                                                                                              Hash collisions with a good algorithm will often be a risk much lower than other factors that you can’t control. When they are, what sense does it make to worry about them?

                                                                                                                                              1. 1

                                                                                                                                                There is a categorical difference between hash collisions and the kind of unavoidable risk you’re describing, e.g. solar flares flipping a bit in my main memory.

                                                                                                                                            3. 1

                                                                                                                                              After doing some more thinking/reading, I would agree with you for something like SHA-256. But using a 128-bit hash still seems like a bad idea. I found this paragraph from a reply on HN summarizes it quite well:

                                                                                                                                              In cases where you can demonstrate that you only care about preimage resistance and not collision resistance, then a 128-bit hash would be sufficient. However often collision attacks crop in in unexpected places or when your protocol is used in ways you didn’t design for. Better to just double the hash size and not worry about it.

                                                                                                                                              I think Git’s messy migration from SHA-1 is a good cautionary tale. Or do you believe it’s completely unnecessary?

                                                                                                                                              1. 1

                                                                                                                                                Git’s migration is due to a known weakness in SHA-1, not due to the hash size being too small. I believe git would be perfectly fine if it used a different 128-bit cryptographic hash.

                                                                                                                                                The first sentence you’ve quoted is important. There are uses of git where this SHA-1 weakness could matter. For UUID generation it’s harder to imagine scenarios where it could be relevant. But remember you don’t need to use SHA-1 — you can use 128 bits of any not-yet-broken cryptographic hash algorithm, and you can even pepper it if you’re super paranoid about that algorithm getting cracked too.

                                                                                                                                                1. 1

                                                                                                                                                  The first sentence you’ve quoted is important. There are uses of git where this SHA-1 weakness could matter.

                                                                                                                                                  Yes, but isn’t git’s situation exactly what the rest of that paragraph warns about: SHA-1 is succeptible to a collision attack, not a preimage attack. And now everyone is trying to figure out whether this could be exploited in some way even though on the surface git is just a simple conten-addressable system where collision attacks shouldn’t matter. And as far as I can tell there is still no consensus either way.

                                                                                                                                                  And as the rest of that reply explains, 128-bit is not enough to guarantee collision resistance.

                                                                                                                                                  1. 1

                                                                                                                                                    If the 128-bit space is not enough for you, then it means you can’t use UUID v4 at all.

                                                                                                                                                    Their whole principle of these UUIDs is based on the fact that random collisions in the 128-bit space are so massively improbable that they can be safely assumed to never ever happen. I need to reiterate that outputs of a not-broken cryptographic hash are entirely indistinguishable from random.

                                                                                                                                                    Resistance of a hash algorithm to cryptographic (analytic) attacks is only slightly related to the hash size. There are other much more important factors like the number of rounds that the hash uses, and that factor is independent of the output size, so it’s inaccurate to say that 128-bit hashes are inherently weaker than hashes with a larger output.

                                                                                                                                                    Please note that you don’t need to use SHA-1. SHA-1’s weakness is unique its specific algorithm, not to 128-bit hashes in general. You can pick any other algorithm. You can use SHA-2, SHA-3, bcrypt/scrypt, or whatever else you like, maybe even a XOR of all of them together.

                                                                                                                                        1. 2

                                                                                                                                          If you’d like more control over collation, the ICU library has an API for UCA, that will give you a binary blob representing a sortable normalized version of a string. You can store it as a blob in the db alongside your text colums.

                                                                                                                                          1. 47

                                                                                                                                            At Cloudflare we have big Rust projects/teams and onboard new developers regularly.

                                                                                                                                            There is a learning curve. Rust is rigid and unforgiving, and noobs need assistance when the compiler says “no” (although error messages and Clippy do a good job for common mistakes).

                                                                                                                                            However, the big upside is that noobs can contribute safely to Rust projects. Rust limits severity of the damage an inexperienced programmer can cause. Once they manage to get the code to compile, it already has lots of correctness guarantees. “Bad” Rust code may just clone more than strictly necessary, or write 10 lines of code for something that has a helper method in the stdlib, but it won’t corrupt memory or blindly run the happy path without checking for errors. Rust prefers to be locally explicit, so it’s also easy to review.

                                                                                                                                            1. 13

                                                                                                                                              I was going to say, measured by how much time and effort it takes to be able to confidently contribute to projects written in it, Rust is easier than many popular languages, in my opinion and experience (which mostly consists of doing just that.)

                                                                                                                                            1. 6

                                                                                                                                              The cookie consent window is one of the worst things to happen to the internet. I get the idea but the execution just annoys everyone on all sides of the aisle, and I doubt people feel any safer (if they cared at all in the first place).

                                                                                                                                              Why would you consent to being tracked, other than to make the pop-up go away?

                                                                                                                                              GDPR should protect people who don’t understand or don’t care, not the 1% of users who actually read the pop-up.

                                                                                                                                              1. 21

                                                                                                                                                That’s part of the goal. Most people wouldn’t consent to being tracked, the law requires you to disclose the unethical behaviour. This article is important in light of the recent lawsuit that affirmed that most of the ‘dark patterns’ in this article are not just unethical, they are also illegal. The article says:

                                                                                                                                                When designing the cookie (tracker) window, we only need to keep one thing in mind: transparency.

                                                                                                                                                This is true from a usability perspective but it’s closely related to a critical legal concept: informed consent. Under the GDPR, you may track people if and only if they have explicitly consented and understand what they are consenting to.

                                                                                                                                                More importantly:

                                                                                                                                                Another point we need to pay attention to is the name “cookie”. The real name of these “cookies” is trackers. Their main purpose is to track you. Therefore, it would be more correct to use the name “tracker”. Better not hide it behind a sweet word.

                                                                                                                                                This is also really critical. You don’t need consent to use cookies. You need consent to track visitors. The mechanism that you use is irrelevant. If you use cookies, HTML5 local data, local cache timing from JavaScript, whatever, it’s the effect that requires consent, not the mechanism.

                                                                                                                                                Companies that rely on tracking people have had a big marketing push to try to frame these things as ‘cookie banners’. They are not. They are consent to be tracked banners. If, like GitHub, you don’t record any linked or linkable data for visitors that have not logged into the site, then you don’t need a pop-up banner at all.

                                                                                                                                                1. 11

                                                                                                                                                  A minor nit;

                                                                                                                                                  You need consent to track visitors.

                                                                                                                                                  You actually don’t even need consent to do this.

                                                                                                                                                  What is going on is that you need consent for is to transfer customer data to a third-party, when it’s not specifically required to service the users’ intentions:

                                                                                                                                                  Companies that rely on tracking people have had a big marketing push to try to frame these things as ‘cookie banners’. They are not. They are consent to be tracked banners.

                                                                                                                                                  This is exactly right.

                                                                                                                                                  The directive cited by ad networks that require their publishers hoist this machinery into the view of the user in order to get those higher-paying “targeted” impressions, is the eprivacy directive and it’s not even law yet. Maybe 2022 is the year, but the reality is that this is about the GDPR requires users actually consent to stuff that isn’t to their benefit, and the bid to the regulators is that these forms (a) actually obtain consent to do nasty things and (b) that they provide the best technological measure (another cookie!) towards communicating and distributing that consent. So far, I don’t think the regulators agree, but I also think Europe is a bit distracted at the moment, so I don’t expect it to be resolved very soon either.

                                                                                                                                                  We need to keep reminding people these forms are about their intention to do something that isn’t to their benefit, and not about complying with some stupid euro-law, and so when they see one, consider leaving, or if not that consider at least using private browsing/container browsing to keep data-sharing impact to a minimum.

                                                                                                                                                  1. 4

                                                                                                                                                    What is going on is that you need consent for is to transfer customer data to a third-party, when it’s not specifically required to service the users’ intentions:

                                                                                                                                                    You also require consent to store the data. In some cases the consent is a side effect of some other action (if I create an account on your web site, then there is implied consent for you to retain my username and profile). The missing part for a lot of sites is that you must also provide a mechanism to withdraw consent. If all personal information is tied to an account then a delete-account action is sufficient here as long as it actually deletes the data tied to the account but if you’re collecting information about visitors to your site that don’t create an account and you don’t have a mechanism to for them to request deletion, then you may also be in violation.

                                                                                                                                                    1. 6

                                                                                                                                                      It is legitimate to store traffic data to deal with security issues, app crashes and so on. You need consent to process them for other purposes not immediately related to providing the service. E.g. to target content, unless you are in fact a recommendation service.

                                                                                                                                                      Also, you cannot ask people to delete account to opt out of profiling that can be enabled separately. You must let them use the service without it, if they wish so.

                                                                                                                                                2. 9

                                                                                                                                                  The law got watered down, because there’s a multi-trillion-dollar adtech surveilance lobby fighting to it death.

                                                                                                                                                  Keep in mind that most ad cookie popups are intentionally annoying. The surveilance lobby wants you to associate privacy with annoyance, so that you will blame their malicious (non)compliance on the law and demand less privacy protections from your legislators.

                                                                                                                                                  1. 8

                                                                                                                                                    I will go further than that: having consent as a legal base for processing in GDPR is a mistake.

                                                                                                                                                    People often misunderstand GDPR as “you need consent to process user data” but that’s completely wrong.

                                                                                                                                                    Consent is one of the 6 legal bases for processing, the other 5 (and two in particular: execution of contracts and legitimate interest) are enough in most legitimate cases. So let’s just get rid of consent.

                                                                                                                                                    1. 2

                                                                                                                                                      I am not sure. It might steer people into signing risky contracts to “get better prices” at stores. Some shops might pull shady crap like periodic membership fees once it gets normalized to sign contracts online. Then after couple of years sue with interest.

                                                                                                                                                      But conceptually I agree with you.

                                                                                                                                                    2. 4

                                                                                                                                                      Instead of pop-up dialogs the browser should be able to automatically select the desired behaviour for you. Which means, that it should always deny any tracking. If one really wants to get tracked then they can override that decision manually for each domain.

                                                                                                                                                      Update Someone else already suggested the same in a comment down below.

                                                                                                                                                      1. 8

                                                                                                                                                        Instead of diddling with browsers, we should just make it illegal to track users for purposes of advertising or to sell data about users to third parties.

                                                                                                                                                        1. 3

                                                                                                                                                          I think diddling with browsers has shown itself to be much more productive at blocking trackers than the law.

                                                                                                                                                    1. 3

                                                                                                                                                      Crash-tolerant software is indeed a virtue, but crash-only software often tends to encourage programming practices that make applications fiendishly difficult to model. If every error is a terminal error then you kind of opt out of deterministic control flow.

                                                                                                                                                      1. 1

                                                                                                                                                        crash-only software often tends to encourage programming practices that make applications fiendishly difficult to model.

                                                                                                                                                        Can you expand on that a bit?

                                                                                                                                                        If every error is a terminal error then you kind of opt out of deterministic control flow.

                                                                                                                                                        Well, there’s a bit more depth to it than that. For example, within the lifecycle of an individual request, experiencing a transient error (such as a timeout) might be fatal to that request, but not to the web server as a whole. Or for example, if your message queue consumer loses it’s connection; then you’d usually only restart the consumer, rather than the process as a whole.

                                                                                                                                                        1. 1

                                                                                                                                                          Is that what crash only means? All errors are crashes? My understanding was more that there was no happy/sad path when you terminated, a normal exit is indistinguishable from an abrupt one due to a crash, so any recovery happens in the startup path (and is constantly exercised).

                                                                                                                                                          1. 6

                                                                                                                                                            Going by Wikipedia definition:

                                                                                                                                                            Crash-only software refers to computer programs that handle failures by simply restarting, without attempting any sophisticated recovery.

                                                                                                                                                            The argument usually is: sophisticated (precise) error recovery is hard, and if you’re attempting it, you already have a potentially-broken program in an inconsistent state. Throwing it all away and starting from a blank state again is easier, well-tested, and therefore more robust.

                                                                                                                                                            Take for example an out-of-memory error: if an allocation fails, you can either carefully backtrack the current operation and report the problem to the caller, or just abort() the whole program.

                                                                                                                                                            I generally agree with the approach, but it’s not a silver bullet. Crash-only systems are prone to DoS-ing themselves. A persistent bad input can put the whole system in a crash loop, instead of being carefully skipped over. Even when everything technically works (and makes incremental progress as it should), the restart may be expensive/disruptive (e.g. loses caches or performs extra work on startup) and the system can still fail due to insufficient throughput.

                                                                                                                                                            1. 1

                                                                                                                                                              In a crash-only service, if an incoming request encounters a database invariant violation, does the entire process die? Or if the database connection fails, are all in-flight requests abandoned?

                                                                                                                                                              Designing software so that it can start up and recover from a wide variety of prior states is a good idea in general, but it’s structurally impossible to write a program that can crash at any point during its execution and reliably leave underlying resources in a recoverable state. Any nontrivial DB transaction commit, for example, is a multi-stage operation. Same with network operations.

                                                                                                                                                              More generally, it’s definitely a good idea to design the layer above the individual process to be resilient, but you can’t just assert that error handling isn’t a concern of the process. The process, the program, is what human beings need to mentally model and understand. That requires deterministic control flow.

                                                                                                                                                              1. 2

                                                                                                                                                                but you can’t just assert that error handling isn’t a concern of the process.

                                                                                                                                                                I agree that is a bad idea, and I would almost say objectively so. Which is why I don’t think it is actually what the idea of “crash-only” is trying to convey.

                                                                                                                                                                Again, my understanding was that crash-only software wasn’t “we crash on all errors/we don’t care about errors”, but rather “we don’t have shutdown code/the shutdown code isn’t where all the invariants are enforced”. It’s more about not having atexit handlers than not having catch blocks if you will. All program terminations are crashes, not all errors are crashes. If you have no shutdown code do you have to put that code somewhere else (periodic autosaves say, etc.), which means when you do crash you’re much likely to be closer to a recent good state.

                                                                                                                                                                1. 1

                                                                                                                                                                  I may misunderstand what “crash-only” means. I take “crash” to mean “terminate the operating system process unceremoniously and without unwinding call stacks”, and I understand “only” not to mean all conceivable errors but definitely more than is strictly necessary.

                                                                                                                                                                2. 1

                                                                                                                                                                  if an incoming request encounters a database invariant violation, does the entire process die? The original paper “Crash Only Software” talks about it in terms of individual components that can perform micro-reboots, and references Erlang heavily.

                                                                                                                                                                  So for an example, you’d want to use tools such as transactions so you can pretend that a multi-step operation is a single one. Alternatively, make everything idempotent, and retry a lot.

                                                                                                                                                                  structurally impossible to write a program that can crash at any point during its execution and reliably leave underlying resources in a recoverable state.

                                                                                                                                                                  I’m reasonably sure that this is what database software should be designed to do. Unless I’m somehow misunderstanding you.

                                                                                                                                                                  1. 1

                                                                                                                                                                    There is no OS-level operation against e.g. disks, networks, etc. which can be interrupted at any arbitrary point, and can be reliably recovered-from. You can do your best to minimize the damage of that interruption – and maybe this is what crash-only is gunning for – but you can’t solve the problem. Every transition between layers of abstractions, between system models, represents an exchange of information over time. No matter how you design your protocol, no matter how tiny the “commit” or “sync” signal is, if your operation spans more than one layer of abstraction, and it’s possible for that operation to be interrupted, you can’t avoid the possibility of ending up in an invalid state. That’s fine! My point is only that systems at each layer of abstraction should not only do their best to recover from wonky initial state, but should also do their best to avoid making that wonky state in the first place. If you encounter an error, you should deal with it responsibly, to the best of your ability. That’s it.

                                                                                                                                                            1. 1

                                                                                                                                                              When you need dozens of dependencies even for simple things and your code rots within a few months due to API changes of said dependencies, by choosing Rust you truly build your house on sand.

                                                                                                                                                              1. 6

                                                                                                                                                                edit: I’ve just noticed that the 0.4.0 version of jwt was from 2016, so it’s upgrading 6 years worth of development. That’s almost entire lifetime of post-1.0 Rust. Changing a couple lines of code per 6 years of a nominally experimental and unstable library is really not that bad. The old dependency worked for twice as long as Debian stable release cycle.

                                                                                                                                                                1. 3

                                                                                                                                                                  JWT is not a simple thing.

                                                                                                                                                                1. 2

                                                                                                                                                                  Apple’s native code signing tooling has been a kafkaesque nightmare for me. For every combination of signing flags and commands, at most 2 out of 3 of (codesign, spctl, notarytool) agree everything is fine, but the third one doesn’t, and won’t tell why.

                                                                                                                                                                  So I’m keen to give it a try even for macOS-native workflows just to escape the half-assed underdocumented Apple tooling. Apple doesn’t seem to care about properly supporting any other use than “Upload to Mac App Store” via a GUI button in Xcode.

                                                                                                                                                                  1. 4

                                                                                                                                                                    But many people want more… flexibility. Open source developers, for example, often want to distribute cross-platform applications with minimal effort. There are entire programming language ecosystems where the operating system you are running on is abstracted away as an implementation detail for many applications. By creating a de facto requirement that macOS, iOS, etc development require the direct access to macOS and (often above market priced) Apple hardware, the distribution requirements imposed by Apple’s software ecosystems are effectively exclusionary and prevent interested parties from contributing to the ecosystem.

                                                                                                                                                                    Mildly hot take: It’s not hard to have access to a Mac. And if you don’t have access to a Mac, I wouldn’t be confident that the binaries you made would even work. I myself try to avoid cross-compiling with the issues I’ve had with i.e. MinGW on Linux.

                                                                                                                                                                    1. 14

                                                                                                                                                                      Counterpoint: I build Windows binaries for my Go software but depend on users to report any platform issues as I don’t use Windows. 🤷🏻‍♂️

                                                                                                                                                                      1. 3

                                                                                                                                                                        I wouldn’t feel comfortable shipping binaries for a platform I didn’t use. Imagine trying to figure out Windows issues without a knowledge of Windows.

                                                                                                                                                                        1. 1

                                                                                                                                                                          Yeah, I’ve been there. It can be tricky, but fortunately Go and Rust abstract away a few common pitfalls. If your project is open-source, someone else may help.

                                                                                                                                                                      2. 3

                                                                                                                                                                        Having open tooling can be extremely useful even if you do have access to Macs. Apple binds toolchain and OS compatibility pretty tightly these days - a given version of Xcode typically runs on the current major macOS version; if it’s Xcode ?.0 ~ ?.2, it’ll typically also run on the previous version (?.3+ will typically not), and usually it’ll work for about one more future major OS version. Individual tools might run outside that range, but it’s certainly all unsupported. If you need to use different toolchain versions for whatever reason, this can get quite annoying. Especially with the x86-64 -> ARM64 transition, “just spin up a VM with an older OS” is no longer a simple go-to solution. (I’m not aware of any emulators which run x86-64 macOS guests with usable speed on M1 based Macs.)

                                                                                                                                                                        1. 2

                                                                                                                                                                          Do you cross compile software for iOS? Most people do, but they’re stuck doing that on a single vendor’s OS that’s locked to specific hardware and virtualization constraints.

                                                                                                                                                                        1. 14

                                                                                                                                                                          I still think it’s more of a N+M vs N*M thing, but the critical change LSP brought was a change in thinking about the problem.

                                                                                                                                                                          Language support used to be assumed to be IDE’s job. We’ve had “Java IDEs” and “Visual Studio for $LANG”, and we were hoping someone would write a “Rust IDE”. Not a language server, not a compiler infrastructure for an IDE, but an actual editor with Rust support built-in.

                                                                                                                                                                          The problem was that it took twice as much effort to write an IDE with an integrated analyzer, than just a bare editor, or just a GUI-less analyzer. On top of that people good at writing IDEs aren’t necessarily also good at writing analyzers, so we’ve had many good editors with very shallow language support.

                                                                                                                                                                          1. 4

                                                                                                                                                                            the critical change LSP brought was a change in thinking about the problem.

                                                                                                                                                                            I see it as a Betamax/VHS situation. We had language-agnostic, editor-agnostic editor protocols before, (like nREPL) but they never saw mainstream adoption because they didn’t have a megacorp backing them.

                                                                                                                                                                            1. 3

                                                                                                                                                                              I mean, VHS beat Beta for a variety of reasons, but the big one was that you could record more than 60m of video at a time.

                                                                                                                                                                              1. 1

                                                                                                                                                                                So how did rental video on Betamax work?

                                                                                                                                                                                1. 1

                                                                                                                                                                                  Multiple tapes.

                                                                                                                                                                                  1. 1

                                                                                                                                                                                    Are you sure you’re not thinking of Philips/Grundig VCR (with the square cassettes) rather than Betamax?

                                                                                                                                                                                2. 1

                                                                                                                                                                                  I’ll admit I’m too young to have ever owned either so maybe that wasn’t the best analogy!

                                                                                                                                                                                  1. 1

                                                                                                                                                                                    It’s cool. It used to be a very common, canonical example of the “better” technology losing to the inferior one; but “better” is a multidimensional comparison, and by certain important measures, VHS was technically superior to Beta. Beta did have superior picture quality, though.

                                                                                                                                                                                    1. 3

                                                                                                                                                                                      Also note it’s not cut and dry either - while Beta “lost” the consumer market, it massively won the professional market, and variants there dominated.

                                                                                                                                                                                      I think the biggest takeaway is there’s no binary winner-loser, especially when multiple market segments exist.

                                                                                                                                                                                      1. 1

                                                                                                                                                                                        This is a very good point.

                                                                                                                                                                                3. 2

                                                                                                                                                                                  Sure, compatibility with a rapidly growing editor from a big corp was definitely a big motivation that helped LSP.

                                                                                                                                                                                  But I’m not sure if nREPL was really an alternative. From the docs it seems very REPL-centric, and beyond that quite generic and unopinionated, almost to the point of being a telnet protocol. You can do anything over telnet, but “what methods are available in line 7 col 15 after typing ‘Z’” needs a more specific protocol.

                                                                                                                                                                                  1. 3

                                                                                                                                                                                    From the docs it seems very REPL-centric, and beyond that quite generic and unopinionated

                                                                                                                                                                                    This is a fair criticism; the docs on the reference implementation’s site are not very clear on what nREPL actually is. I think this might be because that site is maintained by the same person who runs Cider, the most widely used client for nREPL, which only works for Clojure; the language-agnostic aspects which were key to the protocol’s original designer appear on that site as a bit of an afterthought because that’s not where his interest is.

                                                                                                                                                                                    But the protocol is extensible; eval is only one of the supported operations. You can use it to look up documentation with the describe op, and they recently added a complete op as well: https://github.com/nrepl/nrepl/issues/174

                                                                                                                                                                                    The fact that the docs don’t make this clear is admittedly a big problem, as is the fact that the protocol doesn’t standardize a wider range of declarative queries and operations. I think of these as both being part of the “doesn’t have megacorp resources behind it” problem but that doesn’t help.

                                                                                                                                                                                  2. 1

                                                                                                                                                                                    TextMate language definition files didn’t have a megacorp backing them, yet they’re the defacto standard for syntax highlighting.

                                                                                                                                                                                1. 3

                                                                                                                                                                                  You can add pngquant to the list.

                                                                                                                                                                                  When pricing software, keep in mind that B2C and B2B software is priced and sold very very differently. When a business adopts your software, it usually saves them from having to write and maintain something like this in-house, which would cost them some multiple of a developer’s salary. If your software reduces their overhead, risks, improves time-to-market or sales, that’s worth way more than $20. Enterprise companies will spend an order of magnitude more than this just on the administrative overhead of buying your software (e.g. lawyer checking your license, admin staff tracking POs/invoices/payments/renewals/compliance.)

                                                                                                                                                                                  1. 1

                                                                                                                                                                                    Is something going wrong for me, or is it supposed to get 1 frame a week?

                                                                                                                                                                                    1. 1

                                                                                                                                                                                      It’s smooth 60fps for me in Firefox/M1/macOS.

                                                                                                                                                                                      1. 1

                                                                                                                                                                                        Interesting. I was getting maybe 10 seconds per frame on Chrome. There was too much malware for it to even run on my Firefox.

                                                                                                                                                                                    1. 2

                                                                                                                                                                                      The comparison between self-selected survey users vs users from vscode prompt is very interesting — shows how much the data can vary based on where you advertise the survey.

                                                                                                                                                                                      1. 2

                                                                                                                                                                                        Rust’s String has reserve() method to pre-allocate as much space as you need. It makes appends as fast as memcpy, and should give perfectly linear performance.