1.  

    Another one that someone linked to in the HN thread: https://infraql.io/

    1. 30

      It’s odd to see C described as boring. How can it be boring if you’re constantly navigating a minefield and a single misstep could cause the whole thing to explode? Writing C should be exhilarating, like shoplifting or driving a motorcycle fast on a crowded freeway.

      1. 17

        Hush! We don’t need more reasons for impressionable youngsters to start experimenting with C.

        1. 10

          Something can be boring while still be trying to kill you. One example is described in Things I Won’t Work With.

          1. 1

            ‘Boring’ is I suspect the author’s wording for ‘I approve of this language based on my experiences’.

            1. 10

              I suspect “boring” is used to describe established languages whose strengths and weaknesses are well known. These are languages you don’t spend any “weirdness points” for picking.

              1. 4

                Normally I’d lean towards this interpretation, but I’ve read many other posts by this author and he strikes me as being more thoughtful than that. Perhaps a momentary lapse in judgement; happens to everyone I suppose.

                1. 4

                  ‘Boring’ is I suspect the author’s wording for ‘I approve of this language based on my experiences’.

                  I’m curious if you read the post, and if so, how you got that impression when I said things like “it feels much nicer to use an interesting language (like F#)”, “I still love F#”, etc.

                  Thanks for the feedback.

                  1. 4

                    I found your article pretty full of non-sequiturs and contradictions, actually.

                    boring languages are widely panned. … One thing I find interesting is that, in personal conversations with people, the vast majority of experienced developers I know think that most mainstream langauges are basically fine,

                    Are they widely panned or are they basically fine?

                    But when I’m doing interesting work, the boilerplate is a rounding error and I don’t mind using a boring language like Java, even if that means a huge fraction of the code I’m writing is boilerplate.

                    Is it a rounding error or is it a huge fraction? Once the code has been written down, it doesn’t matter how much effort it was to mentally wrestle with the problem. That was a one-time effort, you don’t optimize for that. The only thing that matters is clearly communicating the code to readers. And if it’s full of boilerplate, that is not great for communication. I want to optimize for clear, succinct communication.

                    Of course, neither people who are loud on the internet nor people I personally know are representative samples of programmers, but I still find it interesting.

                    I’m fairly sure, based on this, that you are just commenting based on your own experiences, and are not claiming to have an unbiased sample?

                    To me it basically seems that your argument is, ‘the languages which should be used are the ones which are already used’. The same argument was used against C, C++, Java, Python, and every other boring language you can think of.

                    1.  

                      Are they widely panned or are they basically fine?

                      I think the point is that the people who spend a lot of time panning boring languages (and advocating their favourite “interesting” one) are not representative of “experienced developers”. They’re just very loud and have an axe to grind.

                      1.  

                        Having a tough time reconciling this notion that a narrow section of loudmouths criticize ‘boring languages’, against ‘widely panned’, which to me means ‘by a wide or significant section’.

                        But it’s really quite interesting how the experienced programmers who like ‘boring languages’ are the ones being highlighted here. It begs the question, what about the experienced programmers who don’t? Are they just not experienced enough? Sounds like an unquestionable dogma to me. If you don’t like the boring languages in the list, you’re just not experienced enough to realize that languages ultimately don’t matter.

                        Another interesting thing, some essential languages of the past few decades are simply not in this list. E.g. SQL, JavaScript, shell. Want to use a relational database, make interactive web pages, or just bash out a quick script? Sorry, can’t, not boring enough 😉

                        Of course that’s a silly argument. The point is to use the right tool for the job. Sometimes that’s a low-level real-time stuff that needs C, sometimes it’s safety-critical high-perf stuff that needs Ada or Rust, sometimes you need a performant language with good domain modelling and safety properties like OCaml or F#. Having approved lists of ‘boring languages’ is a silly situation to get into.

                        1.  

                          To be honest, I don’t really see why that’s hard to reconcile at all. Take an extreme example:

                          Let’s say programming language X is used for the vast majority of real world software development. Through some strange mechanism (doesn’t matter), programmers who write language X never proselytize programming languages on the Internet. Meanwhile, among the set of people who do, they almost always have nasty things to say about X. So, all the articles you can find on the general topic are at least critical of X, and a lot of them are specifically about how X is the devil.

                          Is saying that X is “widely panned” accurate? Yes.

                          Of course that’s a silly argument.

                          Yes it is.

                          The point is to use the right tool for the job.

                          Indeed.

              1. 34

                return err is almost always the wrong thing to do. Instead of:

                if err := foo(); err != nil {
                	return err
                }
                

                Write:

                if err := foo(); err != nil {
                	return fmt.Errorf("fooing: %w", err)
                }
                

                Yes, this is even more verbose, but doing this is what makes error messages actually useful. Deciding what to put in the error message requires meaningful thought and cannot be adequately automated. Furthermore, stack traces are not adequate context for user-facing, non-programming errors. They are verbose, leak implementation details, are disrupted by any form of indirection or concurrency, etc.

                Even with proper context, lots of error paths like this is potentially a code smell. It means you probably have broader error strategy problems. I’d try to give some advice on how to improve the code the author provided, but it is too abstract in order to provide any useful insights.

                1. 18

                  I disagree on a higher level. What we really want is a stacktrace so we know where the error originated, not manually dispensed breadcrumbs…

                  1. 32

                    maybe you do, but I prefer an error chain that was designed. A Go program rarely has just one stack, because every goroutine is its own stack. Having the trace of just that one stack isn’t really a statement of the program as a whole since there’s many stacks, not one. Additionally, stack traces omit the parameters to the functions at each frame, which means that understanding the error means starting with your stack trace, and then bouncing all over your code and reading the code and running it in your head in order to understand your stack trace. This is even more annoying if you’re looking at an error several days later in a heterogeneous environment where you may need the additional complication of having to figure out which version of the code was running when that trace originated. Or you could just have an error like “failed to create a room: unable to reserve room in database ‘database-name’: request timed out” or something similar. Additionally, hand-crafted error chains have the effect that they are often much easier to understand for people who operate but don’t author something; they may have never seen the code before, so understanding what a stack trace means exactly may be difficult for them, especially if they’re not familiar with the language.

                    1. 6

                      I dunno. Erlang and related languages give you back a stack trace (with parameters) in concurrently running processes no problem

                      1. 5

                        It’s been ages since I wrote Erlang, but I remember that back then I rarely wanted a stack trace. My stack were typically 1-2 levels deep: each process had a single function that dispatched messages and did a small amount of work in each one. The thing that I wanted was the state of the process that had sent the unexpected message. I ended up with some debugging modes that attached the PID of the sending process and some other information so that I could reconstruct the state at the point where the problem occurred. This is almost the same situation as Go, where you don’t want the stack trace of the goroutine, you want to capture a stack trace of the program at the point where a goroutine was created and inspect that at the point where the goroutine failed.

                        This isn’t specific to concurrent programs, though it is more common there, it’s similar for anything written in a dataflow / pipeline style. For example, when I’m debugging something in clang’s IR generation I often wish I could go back and see what had caused that particular AST node to be constructed during parsing or semantic analysis. I can’t because all of the state associated with that stack is long gone.

                    2. 10

                      FWIW, I wrote a helper that adds tracing information.

                      I sort of have two minds about this. On the one hand, yeah, computers are good at tracking stack traces, why are we adding them manually and sporadically? OTOH, it’s nice that you can decide if you want the traces or not and it gives you the ability to do higher level things like using errors as response codes and whatnot.

                      The thing that I have read about in Zig that I wish Go had is an error trace which is different from the stack trace, which shows how the error was created, not the how the error propagates back to the execution error boundary which is not very interesting in most scenarios.

                      1. 7

                        The nice thing about those error traces is that they end where the stack trace begins, so it’s seamless to the point that you don’t even need to know that they are a thing, you just get exactly the information that otherwise you would be manually looking for.

                      2. 8

                        In a multiprocess system that’s exchanging messages: which stack?

                        1. 2

                          see: erlang

                        2. 5

                          You don’t want stack traces; you want to know what went wrong.

                          A stack trace can suggest what may have gone wrong, but an error message that declares exactly what went wrong is far more valuable, no?

                          1. 8

                            An error message is easy, we already have that: “i/o timeout”. A stack trace tells me the exact code path that lead to that error. Building up a string of breadcrumbs that led to that timeout is just a poorly implemented, ad-hoc stack trace.

                            1. 5

                              Indeed and I wouldn’t argue with that. I love a good stack trace, but I find they’re often relied upon in lieu of useful error messages and I think that’s a problem.

                              1. 2

                                Building up a string of breadcrumbs that led to that timeout is just a poorly implemented, ad-hoc stack trace.

                                That’s a bit of an over-generalization. A stack trace is inherently a story about the construction of the program that originated the error, while an error chain is a story about the events that led to an error. A stack trace can’t tell you what went wrong if you don’t have access to the program’s source code in the way that a hand crafted error chain can. A stack trace is more about where an error occurred, while an error chain is more about why an error occurred. I think they’re much more distinct than you are suggesting.

                                and of course, if people are just bubbling up errors without wrapping them, yeah you’re going to have a bad time, but I think attacking that case is like suggesting that every language that has exceptions encourages Pokémon exception handling. That’s a bad exception-handling pattern, but I don’t think that the possibility of this pattern is a fair indictment of exceptions generally. Meanwhile you’re using examples of bad error handling practices that are not usually employed by Go programmers with more than a few weeks experience to indict the entire paradigm.

                            2. 4

                              Stack traces are expensive to compute and inappropriate to display to most users. Also, errors aren’t exceptions.

                              1. 1

                                That’s why Swift throws errors instead. Exceptions immediately abort the program.

                              2. 3

                                What really is the “origin” of an error? Isn’t that somewhat arbitrary? If the error comes from a system call, isn’t the origin deeper in the kernel somewhere? What if you call in to a remote, 3rd party service. Do you want the client to get the stack trace with references to the service’s private code? If you’re using an interface, presumably the purpose is to abstract over the specific implementation. Maybe the stack trace should be truncated at the boundary like a kernel call or API call?

                                Stack traces are inherently an encapsulation violation. They can be useful for debugging your internals, but they are an anti-feature for your users debugging their own system. If your user sees a stack trace, that means your program is bugged, not theirs.

                                1. 5

                                  I get a line of logging output: error: i/o timeout. What do I do with that? With Ruby, I get a stack trace which tells me exactly where the timeout came from, giving me a huge lead on debugging the issue.

                                  1. 5

                                    I get a line of logging output: error: i/o timeout. What do I do with that?

                                    Well, that’s a problem you fix by annotating your errors properly. You don’t need stack traces.

                                    1. 3

                                      When your Ruby service returns an HTTP 500, do you send me the stack trace in the response body? What do I do with that?

                                      Go will produce stack traces on panics as well, but that’s precisely the point here: these are two different things. Panics capture stack traces as a “better than nothing” breadcrumb trail for when the programmer has failed to account for a possibility. They are for producers of code, not consumers of it.

                                    2. 2

                                      There’s definitely competing needs between different audiences and environments here.

                                      A non-technical end user doesn’t want to see anything past “something went wrong on our end, but we’re aware of it”. Well, they don’t even want to see that.

                                      A developer wants to see the entire stack trace, or at least have it available. They probably only care about frames in their own code at first, and maybe will want to delve into library code if the error truly doesn’t seem to come from their code or is hard to understand in the first place.

                                      A technical end user might want to see something in-between: they don’t want to see “something was wrong”. They might not even want to see solely the outer error of “something went wrong while persisting data” if the root cause was “I couldn’t reach this host”, because the latter is something they could actually debug within their environment.

                                  2. 9

                                    This is one reason I haven’t gone back to Go since university - There’s no right way to do anything. I think I’ve seen a thousand different right ways to return errors.

                                    1. 9

                                      Lots of pundits say lots of stuff. One good way to learn good patterns (I won’t call them “right”), is to look at real code by experienced Go developers. For instance, if you look at https://github.com/tailscale/tailscale you’ll find pervasive use of fmt.Errorf. One thing you might not see – at least not without careful study – is how to handle code with lots of error paths. That is by it’s very nature harder to see because you have to read and understand what the code is trying to do and what has to happen when something goes wrong in that specific situation.

                                      1. 6

                                        there is a right way to do most things; but it takes some context and understanding for why.

                                        the mistake is thinking go is approachable for beginners; it’s not.

                                        go is an ergonomic joy for people that spend a lot of time investing in it, or bring a ton of context from other languages.

                                        for beginners with little context, it is definitely a mess.

                                        1. 9

                                          I thought Go was for beginners, because Rob Pike doesn’t trust programmers to be good.

                                          1. 18

                                            I’d assume that Rob Pike, an industry veteran, probably has excellent insight into precisely how good the average programmer at Google is, and what kind of language will enable them to be productive at the stuff Google makes. If this makes programming languages connaisseurs sad, that’s not his problem.

                                            1. 9

                                              Here’s the actual quote:

                                              The key point here is our programmers are Googlers, they’re not researchers. They’re typically, fairly young, fresh out of school, probably learned Java, maybe learned C or C++, probably learned Python. They’re not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt.

                                              So I have to wonder who is capable of understanding a “brilliant language” …

                                              1. 8

                                                So I have to wonder who is capable of understanding a “brilliant language” …

                                                Many people. They don’t work at Google at an entry-level capacity, that’s all.

                                                There’s a subtle fallacy at work here - Google makes a lot of money, so Google can afford to employ smart people (like Rob Pike!) It does not follow that everyone who works at Google is, on average, smarter than anyone else.

                                                (edited to include quote)

                                                1. 8

                                                  Let’s say concretely we are talking about OCaml. Surely entry-level Googlers are capable of understanding OCaml. Jane Street teaches it to all new hires (devs or not) in a two-week bootcamp. I’ve heard stories of people quickly becoming productive in Elm too.

                                                  The real meaning of that quote is not ‘entry-level Googlers are not capable of it’, it’s ‘We don’t trust them with it’ and ‘We’re not willing to invest in training them in it’. They want people to start banging out code almost instantly, not take some time to ramp up.

                                                  1. 8

                                                    Let’s say concretely we are talking about OCaml. Surely entry-level Googlers are capable of understanding OCaml. Jane Street teaches it to all new hires (devs or not) in a two-week bootcamp.

                                                    I suspect that Jane Street’s hiring selects for people who are capable of understanding OCaml; I guarantee that the inverse happens and applicants interested in OCaml self select for careers at Jane Street, just like Erlang-ers used to flock towards Ericsson.

                                                    Google has two orders of magnitude more employees than Jane Street. It needs a much bigger funnel and is likely far less selective in hiring. Go is “the law of large numbers” manifest as a programming language. That’s not necessarily bad, just something that is important for a massive software company and far less important for small boutiques.

                                                    1. 2

                                                      applicants interested in OCaml self select for careers at Jane Street,

                                                      As I said, they teach it to all hires, including non-devs.

                                                      Google has two orders of magnitude more employees than Jane Street. It needs a much bigger funnel and is likely far less selective in hiring

                                                      Surely though, they are not so loose that they hire Tom Dick and Harry off the street. Why don’t we actually look at an actual listing and check? E.g. https://careers.google.com/jobs/results/115367821606560454-software-developer-intern-bachelors-summer-2022/

                                                      Job title: Software Developer Intern, Bachelors, Summer 2022 (not exactly senior level)

                                                      Minimum qualifications:

                                                      Pursuing a Bachelor’s degree program or post secondary or training experience with a focus on subjects in software development or other technical related field. Experience in Software Development and coding in a general purpose programming language. Experience coding in two of C, C++, Java, JavaScript, Python or similar.

                                                      I’m sorry but there’s no way I’m believing that these candidates would be capable of learning Go but not OCaml (e.g.). It’s not about their capability, it’s about what Google wants to invest in them. Another reply even openly admits this! https://lobste.rs/s/yjvmlh/go_ing_insane_part_one_endless_error#c_s3peh9

                                                      1. 2

                                                        And I remember when Google would require at minimum a Masters Degree before hiring.

                                                        1. 1

                                                          I had a master’s degree in engineering (though not in CS) and I couldn’t code my way out of a paper bag when I graduated. Thankfully no-one cared in Dot Com Bubble 1.0!

                                                      2. 3

                                                        They want people to start banging out code almost instantly, not take some time to ramp up.

                                                        Yes, and? The commodification of software developers is a well-known trend (and goal) of most companies. When your assets are basically servers, intangible assets like software and patents, and the people required to keep the stuff running, you naturally try to lower the costs of hiring and paying salary, just like you try to have faster servers and more efficient code.

                                                        People are mad at Rob Pike, but he just made a language for Google. It’s not his fault the rest of the industry thought “OMG this is the bee’s knees, let’s GO!” and adopted it widely.

                                                        1. 1

                                                          Yes, I agree that the commodification of software developers is prevalent today. And we can all see the result, the profession is in dire straits–hard to hire because of bonkers interview practices, hard to keep people because management refuses to compensate them properly, and cranking out bugs like no tomorrow.

                                                        2.  

                                                          on the contrary, google provides a ton of ramp up time for new hires because getting to grips with all the internal infrastructure takes a while (the language is the least part of it). indeed, when I joined a standard part of the orientation lecture was that whatever our experience level was, we should not expect to be productive any time soon.

                                                          what go (which I do not use very much) might be optimising for is a certain straightforwardness and uniformity in the code base, so that engineers can move between projects without having to learn essentially a new DSL every time they do.

                                                  2. 8

                                                    No, Rob observed, correctly, that in an organization of 10,000 programmers, the skill level trends towards the mean. And so if you’re designing a language for this environment, you have to keep that in mind.

                                                    1. 4

                                                      it’s not just that. It’s a language that has to reconcile the reality that skill level trends toward the mean, with the fact that the way that google interviews incurs a selection/survival bias towards very junior programmers who think they are the shit, and thus are very dangerous with the wrong type of power.

                                                      1. 4

                                                        As I get older and become, presumably, a better programmer, it really does occur to me just how bad I was for how long. I think because I learned how to program as a second grader, I didn’t get how much of a factor “it’s neat he can do it all” was in my self-assessment. I was pretty bad, but since I was being compared to the other kids who did zero programming, it didn’t matter that objectively I was quite awful, and I thought I was hot shit.

                                                      2. 4

                                                        Right! But the cargo-cult mentality of the industry meant that a language designed to facilitate the commodification of software development for a huge, singular organization escaped and was inflicted on the rest of us.

                                                        1. 4

                                                          But let’s be real for a moment:

                                                          a language designed to facilitate the commodification of software development

                                                          This is what matters.

                                                          It doesn’t matter if you work for a company of 12 or 120,000: if you are paid to program – that is, you are not a founder – the people who sign your paychecks are absolutely doing everything within their power to make you and your coworkers just cogs in the machine.

                                                          So I don’t think this is a case of “the little fish copying what big bad Google does” as much as it is an essential quality of being a software developer.

                                                          1. 1

                                                            Thank you, yes. But also, the cargo cult mentality is real.

                                                      3. 2

                                                        Go is for compilers, because Google builds a billion lines a day.

                                                  3. 2

                                                    return errors.Wrapf(err, "fooing %s", bar) is a bit nicer.

                                                    1. 13

                                                      That uses the non-standard errors package and has been obsolete since 1.13: https://stackoverflow.com/questions/61933650/whats-the-difference-between-errors-wrapf-errors-errorf-and-fmt-errorf

                                                      1. 1

                                                        Thanks, that’s good to know.

                                                      2. 8

                                                        return fmt.Errorf("fooing %s %w", bar, err) is idiomatic.

                                                        1. 8

                                                          Very small tweak: normally you’d include a colon between the current message and the %w, to separate error messages in the chain, like so:

                                                          return fmt.Errorf("fooing %s: %w", bar, err)
                                                          
                                                      3. 1

                                                        It makes error messages useful but if it returns a modified err then I can’t catch it further up with if err == someErr, correct?

                                                        1. 2

                                                          You can use errors.Is to check wrapped errors - https://pkg.go.dev/errors#Is

                                                          Is unwraps its first argument sequentially looking for an error that matches the second. It reports whether it finds a match. It should be used in preference to simple equality checks

                                                          1. 2

                                                            Thanks! I actually didn’t know about that.

                                                          2. 2

                                                            Yes, but you can use errors.Is and errors.As to solve that problem. These use errors.Unwrap under the hood. This error chaining mechanism was introduced in Go 1.13 after being incubated in the “errors” package for a long while before that. See https://go.dev/blog/go1.13-errors for details.

                                                        1. 3

                                                          Well-written, informative article. Thanks.

                                                          1. 6

                                                            Hm this is probably heresy, but “out params” to return values and the error being the actual return value solves a lot of these problems.

                                                            That style is idiomatic in C and some C++, which don’t really have multiple return values. I believe C# has explicit support for it.

                                                            I find myself using it more in Python even though it’s not idiomatic. And it does actually make code shorter and reduces these dependency problems. Surprisingly, the code can be more composable and refactorable.

                                                            There are a couple examples from the Oil codebase I could dig up. I started with multiple return values like Go and refactored to out params and I was surprised how much cleaner the code is. A lot of it has to do with the fact the interpreters have more intricate conditional logic than other types of code.

                                                            I also use exceptions, but the out params style is nice for specific cases where “I know exactly what error can happen and I want to do something special with it at a specific spot”. The code can also be shorter than exceptions which cause a 3-4 line boilerplate around a function.

                                                            1. 2

                                                              A sum type would be better–it forces handling two distinct cases with clear separation of concerns.

                                                              1. 1

                                                                Hm this is probably heresy, but “out params” to return values and the error being the actual return value solves a lot of these problems.

                                                                I don’t immediately see in which way this is different from multiple return values. Could you give an example?

                                                                1. 3

                                                                  This might not be that convincing / intelligible, but it’s the code I was thinking of:

                                                                  https://github.com/oilshell/oil/blob/master/osh/word_eval.py#L1332

                                                                  Basically vsub_state is an “out param” that is mutated, but there’s also a return value val.

                                                                  This is for evaluating:

                                                                  • $foo and "$@" – “simple var sub”
                                                                  • ${foo} and ${@} – “braced var sub”
                                                                  • as well as “var refs” like ${!a} where the value is @, which is yet another way to get $@

                                                                  So all of those return a “val”. It is like a typical interpreter that does a switch/match over dozens of cases and returns a different value based on the sum type.

                                                                  But then only in minority of cases do you have to convert an array to a string – this is weird shell behavior.

                                                                  The out param means you don’t have to “pollute” every code path with that logic; instead I just mutate the out param in the one place I need to. AND sometimes you pass the out param down TWO levels instead of one.

                                                                  And it composes well – I can use it from 3 different contexts.

                                                                  This is pretty deep inside baseball, but I did actually refactor it from a different style, and this ended up cleaner, shorter, and more composable. It probably looks pretty confusing without detailed knowledge of the problem domain, but I think reading the signatures of the functions now clarifies a lot of things.


                                                                  I don’t think this style applies everywhere – sum types and exceptions are more common. But when I saw the Go code with 2 return values I can definitely see it being cleaner, roughly like

                                                                  T1 x;
                                                                  if (!func1(&x) {  // x is output param, return value is success/fail
                                                                    return false;
                                                                  }
                                                                  
                                                                  T2 y;
                                                                  if (!func2(x, &y)) {  // x is input, y is output
                                                                    return false;
                                                                  }
                                                                  
                                                                  T3 z;
                                                                   if (!func3(y, &z)) {  // y is input, z is ouput
                                                                     return false;
                                                                   }
                                                                   return true;  // success, output can be read from z "out param"
                                                                  

                                                                  This is basically func3(func2(func1())) but with (non-exception) error handling (in C++ syntax).

                                                              1. 21

                                                                (context: I’ve used Go in production for about a year, and am neither a lover nor hater of the language, though I began as a hater.)

                                                                With that said, my take on the article is:

                                                                1. The “order dependence” problem is a non-problem. It doesn’t come up that often, dealing with it easy – this is simply low-priority stuff. If I wanted to mention it, it would be as an ergonomic nitpick.
                                                                2. The infamous Go error handling bloat, while annoying to look at, has the great benefit of honesty and explicitness: You have to document your errors as part of your interface, and you have to explicitly deal with any error-producing code you use. Despite personally really caring about aesthetics and hygiene – and disliking the bloat like the author – I’ll still take this tradeoff. I also work in ruby, and while raising errors allows you to avoid this boilerplate, it also introduces a hidden, implicit part of your interface, which is worse.

                                                                It’s also worth pointing out Rob Pike’s Errors are Values which offers advice for mitigating this kind of boilerplate in some situations.

                                                                1. 22

                                                                  There’s a difference between explicitness and pointless tediousness.

                                                                  Go’s error handling is more structured compared to error handling in C, and more explicit and controllable compared to unchecked exceptions in C++ and similar languages. But that’s a very low bar now.

                                                                  Once you get a taste of error handling via sum types (generic enums with values), you can see you can have the cake and eat it too. You can have very explicit error documentation (via types), errors as values, and locally explicit control flow without burdensome syntax (via the ? syntax sugar).

                                                                  1. 4

                                                                    I agree.

                                                                    But Go, eg, is not Haskell, and that’s an explicit language design decision. I think Haskell is a more beautiful language than Go, but Go has its reasons for not wanting to go that direction – Go values simple verbosity over more abstract elegance.

                                                                    1. 15

                                                                      If it’s Go’s decision then ¯\_(ツ)_/¯

                                                                      but I’ve struggled with its error handling in many ways. From annoyances where commenting out one line requires changing = to := on another, silly errors due to juggling err and err2, to app leaking temp files badly due to lack of some robust “never forget to clean up after error” feature (defer needs to be repeated in every function, there isn’t errdefer even, and there’s no RAII or deterministic destruction).

                                                                      1. 5

                                                                        Sounds like you’re fighting the language 🤷

                                                                        1. 3

                                                                          commenting out one line requires changing = to := on another

                                                                          I do not agree that this is a problem. := is an explicit and clear declaration that helps the programmer to see in which scope the variable is defined and to highlight clear boundaries between old and new declarations for a given variable name. Being forced to think about this during refactoring is a good thing.

                                                                          1.  

                                                                            Explicit binding definition by itself is good, but when it’s involved in error propagation it becomes a pointless chore.

                                                                            That’s because variable (re)definition is not the point of error handling, it’s only self-inflicted requirement go made for itself.

                                                                            1.  

                                                                              Go takes the stance that error propagation is not different than any other value propagation. You don’t have to agree that it’s a good decision, but if you internalize the notion that errors are not special and don’t get special consideration, things fall into place.

                                                                          2. 3

                                                                            there isn’t errdefer even

                                                                            I mean, it’s a pretty trivial helper func if you want it:

                                                                            func Errdefer(errp *error, f func()) {
                                                                                if (*err) != nil {
                                                                                    f()
                                                                                }
                                                                            }
                                                                            
                                                                            func whatever() (err error) {
                                                                                defer Errdefer(&err, func() {
                                                                                   // cleanup
                                                                                })
                                                                                // ...
                                                                            }
                                                                            

                                                                            In general, to have fun in Go, you have to have a high tolerance for figuring out what 3 line helper funcs would make your life easier and then just writing them. If you get into it, it’s the fun part of writing Go, but if you’re not into it, you’re going to be “why do I have to write my own flatmap!!” every fourth function.

                                                                            1. 1

                                                                              commenting out one line requires changing = to := on another

                                                                              IMHO := (outside of if, for & switch) was a mistake; I prefer a C-style var block at the top of my function.

                                                                              silly errors due to juggling err and err2

                                                                              I think that this is mostly avoidable.

                                                                            2. 6

                                                                              Yup, Go (well, presumably Rob Pike) made a lot of explicit design decisions like this, which drove me away from the language after a year or two and many thousands of LOC written.

                                                                              Beside the awfulness of error handling, other big ones were the inane way you have to rename a variable/function just to change its visibility, the lack of real inheritance, and the NIH attitude to platform ABIs that makes Go a mess to integrate with other languages. The condescending attitude of the Go team on mailing lists didn’t help either.

                                                                              1. 3

                                                                                There is no value in verbosity, though. It’s a waste of characters. The entire attitude is an apology for the bare fact that Go doesn’t have error-handling syntax.

                                                                                1. 9

                                                                                  What you label “verbosity” I see as “explicitness”. What to you is a lack of error-handling syntax is to me a simplification that normalizes execution paths.

                                                                                  It’s very clear to me that the people who dislike Go’s approach to error handling see errors as a first-class concept in language design, which deserves special accommodation at the language level. I get that. I understand the position and perspective. But this isn’t an objective truth, or something that is strictly correct. It’s just a perspective, a model, which has both costs and benefits. This much at least is hopefully noncontroversial. And Go makes the claim that, for the contexts which it targets, this model of error handling has more costs than benefits. If you want to object to that position then that’s fine. But it’s — bluntly — incorrect to claim that this is some kind of apologia, or that Go is missing things that it should objectively have.

                                                                                  1. 2

                                                                                    It often feels to me that people who complain about error handling in Go have never suffered dealing with throwing and catching exceptions in a huge codebase. At least in Go, you can be very explicit on how to handle errors (in particular, non-fatal ones) without the program trying to catapult you out of an escape hatch. Error handing is tedious in general, in any language. I don’t think Go’s approach is really any more tedious than anywhere else.

                                                                                    1. 3

                                                                                      Error handing is tedious in general, in any language. I don’t think Go’s approach is really any more tedious than anywhere else.

                                                                                      Yep, and a bit more — it brings the “tedium” forward, which is for sure a short term cost. But that cost creates disproportionate long-term benefits, as the explicitness reduces risk otherwise created by conveniences.

                                                                                  2. 3

                                                                                    The argument isn’t that verbosity has a value in itself – it doesn’t.

                                                                                    The argument is that if you have to choose between “simple, but concrete and verbose” and “more complex and abstract, but elegant”, it’s better to choose the former. It’s a statement about relative values. And you see it everywhere in Go. Think about the generics arguments:

                                                                                    People: “WTF! I have to rewrite my function for every fucking datatype!”.
                                                                                    Go: “What’s the big deal? It’s just some repeated code. Better than us bloating the language and making Go syntax more complex”

                                                                                    They caved on that one eventually, but the argument is still germane.

                                                                                    As I said, I don’t personally like all the decisions, and it’s not my favorite language, but once I got where they were coming from, I stopped hating it. The ethos has value.

                                                                                    It all stems from taking a hard line against over-engineering. The whole language is geared toward that. No inheritance. You don’t even get map! “Just use a for loop.” You only see the payoff of the philosophy in a large team setting, where you have many devs of varying experience levels working over years on something. The “Go way” isn’t so crazy there.

                                                                              2. 3

                                                                                Java included exceptions in the function signature and everyone hated those, even Kotlin made them optional. Just like how this Python developer has grown to enjoy types, I also enjoy the explicit throws declarations.

                                                                                1. 3

                                                                                  you have to explicitly deal with any error-producing code you use.

                                                                                  Except if you forget to deal with it, forget to check for the error, or just drop it.

                                                                                1. 5

                                                                                  Very interesting article. While reading the ‘Random salt on startup’ section, I was reminded of this passage in the OCaml Hashtbl module documentation:

                                                                                  A hash table that is created with ~random set to false uses a fixed hash function (Hashtbl.hash) to distribute keys among buckets. As a consequence, collisions between keys happen deterministically. In Web-facing applications or other security-sensitive applications, the deterministic collision patterns can be exploited by a malicious user to create a denial-of-service attack: the attacker sends input crafted to create many collisions in the table, slowing the application down.

                                                                                  A hash table that is created with ~random set to true uses the seeded hash function Hashtbl.seeded_hash with a seed that is randomly chosen at hash table creation time. In effect, the hash function used is randomly selected among 2^{30} different hash functions. All these hash functions have different collision patterns, rendering ineffective the denial-of-service attack described above. However, because of randomization, enumerating all elements of the hash table using Hashtbl.fold or Hashtbl.iter is no longer deterministic: elements are enumerated in different orders at different runs of the program.

                                                                                  Now, personally I don’t think JSON objects should have a guaranteed ordering of keys, but it is understandable that it’s nicer when they do. I’ve come across this in the past when logging in JSON format–you really do want the log lines to be ordered consistently. In my case I converted into an ordered format explicitly.

                                                                                  Side note, OCaml libraries don’t use hashes but rather association lists to represent JSON objects, so they don’t suffer from collision issues, but I guess memory usage issues instead.

                                                                                  1. 2

                                                                                    This is really neat, it adds 4 bytes of memory but if the table is above a few elements (where it would be a DoS) it won’t matter.

                                                                                    Thank you for sharing the OCaml docs

                                                                                    1. 4

                                                                                      It’s pretty interesting how most OCaml libraries for JSON/YAML/TOML/S-exp/etc. use algebraic types when the standard library provides mutable hashes. I always felt that aeson complicated things for no good reason.

                                                                                      I definitely prefer it that way, and I made my OTOML library work that way too, as opposed to the old To.ml that did use mutable hashes (though it was my least significant objection to its design).

                                                                                  1. 20

                                                                                    If you’re going to use (much more expensive) ref-counted heap objects instead of direct references, you might as well be using Swift. (Or Nim, or one of the other new-ish native-compiling languages.) Rust’s competitive advantage is the borrow checker and the way it lets you use direct pointers safely.

                                                                                    The author should at least have pointed out that there’s a significant runtime overhead to using their recommended technique.

                                                                                    1. 20

                                                                                      No, this a fatalistic take almost like “if you use dyn you may as well use Python”.

                                                                                      Swift’s refcounting is always atomic, but Rust can also use faster non-atomic Rc. Swift has a few local cases where it can omit redundant refcounts, but Rust can borrow Rc‘s content and avoid all refcounts within a scope, even if object’s usage is complex, and that’s a guarantee not dependent on a Sufficiently Smart Compiler.

                                                                                      Swift doesn’t mind doing implicit heap allocations, and all class instances are heap-allocated. Rust doesn’t allocate implicitly and can keep more things on the stack. Swift uses dynamic dispatch quite often, even in basic data structures like strings. In Rust direct inlineable code is the norm, and monomorphisation is a guarantee, even across libraries.

                                                                                      So there’s still a lot more to Rust, even if you need to use Arc in a few places.

                                                                                      1. 9

                                                                                        Uhu. It seems to me that there are two schools of thought here.

                                                                                        One says: .clone(), Rc and RefCell to make life easier.

                                                                                        The other says: the zen of Rust is ownership: if you express a problem as a tree with clear ownership semantics, then the architecture of your entire application becomes radically simpler. Not every problem has clean ownership mapping, but most problems do, even if it might not be obvious for the start.

                                                                                        I don’t know what approach is better for learning Rust. For writing large-scale production apps, I rather strongly feel that the second one is superior. Arcs and Mutexes make the code significantly harder to understand. The last example, a struct where every filed is an Arc, is a code smell to me: I always try to push arcs outwards in such cases, and have an Arc of struct rather than a struct of arcs.

                                                                                        It’s not that every Arc and mutex is a code smell: on the contrary, there’s usually a couple of Arcs and Mutexes at the top level which are the linch-pin of the whole architecture. Like, the whole rust-analyzer is basically an Arc<RwLock<GlobalState>> plus cancellation. But just throwing arcs and interior mutability everywhere makes it harder to note these central pieces of state management.

                                                                                        1. 3

                                                                                          I’ve always felt that the order of preference for new code is:

                                                                                          1. make it work
                                                                                          2. make it pretty
                                                                                          3. make it fast/resource-efficient

                                                                                          (some people may choose to wedge in “make it correct” somewhere there, but I think that’s either mostly a pipe dream or already part of 1.)

                                                                                          That would mean that you always use the easiest possible techniques in phases 1 and 2 and in phase 3 do something more clever but only if the easy techniques turned out to be a bottleneck.

                                                                                          I’m guessing the easiest technique in Rust terms would be copying a lot.

                                                                                          1. 2

                                                                                            I tend to agree about that ordering, but I’ve also found that heap allocation and copying is frequently a bottleneck, so much so that I keep it in mind even in steps 1-2. (Of course this applies to pretty low-level performance sensitive code, but that’s the kind of domain Rust gets used for.)

                                                                                          2. 3

                                                                                            I completely agree. If you don’t need precise control over memory, including the ability to pass around refs to memory safely, then the sane choice is to use a well-designed garbage collected language.

                                                                                            Maybe you’re building something where half needs to control memory and the other half doesn’t. I guess something like this could make sense then.

                                                                                            1. 2

                                                                                              Swift isn’t exactly “available” on many Linux distributions due to its overengineered build system. The same goes for dotnet. Both of these languages are extremely fickle and run many versions behind the latest stable release offered on the natively supported OS (macOS for Swift and Windows for dotnet).

                                                                                              To build Rust is comparatively sane and a breath of fresh air.

                                                                                              1. 1

                                                                                                Well, there is OCaml of course. On Linux with a reasonable machine, compiling it from scratch with a C toolchain should take just a handful of minutes. Of course, setting up the OCaml platform tools like opam, dune, etc., will take a few minutes more.

                                                                                            1. 3

                                                                                              I’m still looking for an argument/example for why all this abstraction carries its own weight, in a software-development context.

                                                                                              1. 3

                                                                                                Like most software engineering patterns, it exists to facilitate code reuse in a principled way. An abstraction’s utility can be measured along two axes:

                                                                                                1. Working only within the abstraction, what things can I say?
                                                                                                2. How many things embody this abstraction?

                                                                                                Monad is a useful abstraction because it applies to a surprisingly large range of types, while still allowing a broad vocabulary of functions that work over any Monad. It is also hard to understand for these reasons, which is why the most effective teaching method seems to involve finding some vaguely familiar concept (e.g., promises) that happens to be a Monad, using that to give the student a toe-hold, and then asking the student to write Monad instances for a zillion types, letting the instinctive, pattern-matching part of the student’s brain notice the underlying similarities.

                                                                                                The Monad abstraction in Haskell enables (among other things) a large family of functions (when, unless, forever, etc) that would be control structures in other languages. This is handy because a) we don’t have to petition some language steward to add new ones (contrast: Python’s with statement), and b) we can use our normal programming tools to work with them.

                                                                                                I can use the Monad abstraction when checking for missing values, to halt my program on the first error, to make an ergonomic database Transaction data type that prohibits non-database I/O, to write deserialisers, to set up eDSLs with great ergonomics, to treat lists as nondeterministic computations, to provide a good interface to software transactional memory, to build up an additional “summary” result in a computation, to pass around additional arguments, and other things I’ve surely forgotten. You could well say (as you said to /u/shapr in a sibling comment) that none of these need the theory of Monads. And they don’t. What the theory of Monads gives you is a useful toolbox to see how they’re all similar, and it’s one that’s usefully expressed only in a few programming languages. A tool like mapM works exactly as well in each of those apparently-diverse cases, and only needed writing once.

                                                                                                1. 2

                                                                                                  mapM’s functionality is trivial, though, and I expect anything you can do with an arbitrary monad would be equally trivial. In my experience, code reuse is useful when the code carries significant domain knowledge or provides a single point of reference for application details which are subject to change. Abstracting away repeated patterns for the sake of it, or simply for the sake of concision, is often not worth the cognitive load it adds.

                                                                                                2. 3

                                                                                                  I’m not a haskeller, but I’ve spent a little time with the language. One benefit is that you can write your functions for the success case and the monadic machinery will short circuit if there is a failure. This means that you don’t need to litter you code with checks for nulls or nothings.

                                                                                                  1. 2

                                                                                                    One thing I like is that I can use the same code with a fake in memory database without changing the code itself, just feeding a different value into the monad.

                                                                                                    1. 7

                                                                                                      You don’t need the theory of monads to enable that.

                                                                                                      1. 1

                                                                                                        Using a monad to separate concerns and do easy dependency injection is one of many cases where the monad abstraction carries its weight.

                                                                                                        I agree, you don’t need the theory to do those things with a monad, you just use it.

                                                                                                        1. 3

                                                                                                          I can do those things quite easily in languages which don’t even have the concept of “monad.” The abstractions I use might be representable as monads, but I see no benefit to calling them out as such.

                                                                                                          1. 1

                                                                                                            Consider Elm. Its designer has banned the word monad from all the documentation. Nevertheless, it has a very consistent and intuitive way of handling all monadic types. How can that be? Because the designer knew they were monads.

                                                                                                            Most users won’t ever have to declare a monad. They don’t need monad tutorials or even know the word, but the world would be a better place if all language designers did.

                                                                                                      2. 3

                                                                                                        that’s just an interface, isn’t it?

                                                                                                        1. 1

                                                                                                          If you mean an interface as in an API being a pattern of functions and data, then yes. A good interface can make a problem easier to think about and solve.

                                                                                                          1. 2

                                                                                                            Or an interface as in literally like a Java interface, i.e. a simple everyday programming concept that doesn’t need any theoretical mathematics to understand.

                                                                                                            1. 3

                                                                                                              That’s what I was thinking. Jdbc is the ultimate example here. You program only against interfaces and you can swap databases in tests trivially easy. All without reading complex Monad tutorials.

                                                                                                              1. 1

                                                                                                                Interface theory is very complex and mathematical. You never see blog sized tutorials about all the theory because it doesn’t fit in a blog. Monads are stupid simple in comparison, which is why there are so many blogs about it. Get over the abstract nonsense words, implement it yourself in 10 lines, then write a blog about how you finally understood it. That’s all there is to it.

                                                                                                                Designing interface rules for your language is hard to get right even if you know all the theory, cause there are many tradeoffs and you might make the wrong choice for what you’ll want later on. Getting monads wrong is only possible when you refuse to acknowledge you’re dealing with a monad, like JS’s Promises.

                                                                                                                1. 2

                                                                                                                  Interface theory is very complex and mathematical. You never see blog sized tutorials about all the theory because it doesn’t fit in a blog.

                                                                                                                  I think you never see blogs about it, because - at least Java programmers - learn about them in the very beginning and then use them. They are trivial to understand and use. Java programmers write them every single day and most of them do not have deep type theory backgrounds. I think that is what this thread is about. Pragmatic programmers using something vs. pure functional programmers exchanging complex Monad tutorials.

                                                                                                                  1. 2

                                                                                                                    Exactly, you can use monad-like things without ever learning about monads. You’ll have a better time if the language designer did know monad theory though. Same goes for interfaces.

                                                                                                                    I really don’t want to call monads complex though. That’s what leads to this “monad tutorial fallacy”, it’s always the first mythical hard thing new Haskellers run in to, so when it clicks they must write a tutorial. Haskell has other stuff much more complex, that never gets blog explanations either. I’d say GADTs are at the level of interfaces, and when a new Haskeller reaches those, suddenly they’re pragmatics again. (And then after a year you make it your master thesis and never touch Haskell again lmao.)

                                                                                                      3. 1

                                                                                                        Code reuse

                                                                                                      1. 7

                                                                                                        it takes advantage of compilation model based on proper modules (crates)

                                                                                                        Hang on a second, crates are not ‘proper’ modules. Crates are a collection of modules. If any file in the crate changes, the entire crate needs to be recompiled.

                                                                                                        ‘Proper’ modules are ones where the unit of compilation is the file, and the build can be parallelized and made much more incremental.

                                                                                                        The first advice you get when complaining about compile times in Rust is: “split the code into crates”.

                                                                                                        When files are compilation units, you split the code into files, which is standard development practice.

                                                                                                        Keeping Instantiations In Check…If you care about API ergonomics enough to use impl trait, you should use inner trick — compile times are as big part of ergonomics, as the syntax used to call the function.

                                                                                                        I think something has gone off the rails if, in the normal course of writing code, you need to use sophisticated techniques like these to contain build times. Hopefully it gets better in the future.

                                                                                                        Great article though.

                                                                                                        1. 10

                                                                                                          Yeah, agree that calling something “proper” without explaining your own personal definition of proper is wrong. What I’ve meant by “proper modules” is more-or-less two things:

                                                                                                          • code is paresd/typechecked only once (no header files)
                                                                                                          • there are explicit up-front dependencies between components, there’s no shared global namespace a-la PYTHONPATH or CLASSPATH

                                                                                                          So, I wouldn’t say that pre C++20 compilation model has “proper modules”.

                                                                                                          That being said – yes, that’s a very important point that the (old) C++ way of doing things is embarrassingly parallel, and that’s huge deal. One of the problems with Rust builds is that, unlike C++, it is not embarrassingly parallel.

                                                                                                          I’d actually be curious to learn what’s the situation with C++20 – how template compilation actually works with modules? Are builds still as parallel? I’ve tried reading a couple of articles, but I am still confused about this.

                                                                                                          And yeah, it would be better if, in addition to crates being well-defined units with specific interfaces, it would be possible to naively process every crate’s constituting module in parallel.

                                                                                                          I think something has gone off the rails if, in the normal course of writing code, you need to use sophisticated techniques like these to contain build times.

                                                                                                          To clarify, there’s an or statement, in normal application code one should write just

                                                                                                          pub fn read(path: &Path) -> io::Result<Vec<u8>> {
                                                                                                            let mut file = File::open(path)?;
                                                                                                            let mut bytes = Vec::new();
                                                                                                            file.read_to_end(&mut bytes)?;
                                                                                                            Ok(bytes)
                                                                                                          }
                                                                                                          

                                                                                                          There’s no need to make this template at all, unless you are building a library.

                                                                                                          But I kinda disagree with the broader assertion. Imo, in Rust you absolutely should care about compile times when writing application code, the same way you should care what to put in a header file when writing C++. I think we simply don’t know how to make a language which is both fast to run and fast too compile. If you chose Rust, you choose a bunch of accidental complexity, including slower built times. If you don’t care about performance that much, you probably should choose a different language.

                                                                                                          That being said, I would love to read some deeper analysis of D performance though – my understanding is that it, like C++ and Rust, chose “slow compiler” approach, but at the same time compiles as fast as go? So maybe we actually do know how to build fast fast to compile languages, just not too well?

                                                                                                          1. 2

                                                                                                            I’d actually be curious to learn what’s the situation with C++20 – how template compilation actually works with modules?

                                                                                                            I believe pretty much like in Rust: templates are compiled to some intermediate representation and then used during instantiation.

                                                                                                            Are builds still as parallel?

                                                                                                            No, again the situation is pretty much like in Rust: a module interface is compiled into BMI (binary module interface; equivalent to Rust’s crate metadata) and any translation unit that imports said module cannot start compiling before the BMI is available.

                                                                                                            I also agree that C++20 module’s approximate equivalent in Rust is a crate (and not a module).

                                                                                                            BTW, a question on monomorphization: aren’t argument’s lifetimes also turn functions into templates? My understanding is that while in C++ we have type and value template parameters, in Rust we also have lifetime template parameters which turn Rust into an “almost everything is a template” kind of language. But perhaps I am misunderstanding things.

                                                                                                            1. 5

                                                                                                              No, again the situation is pretty much like in Rust: a module interface is compiled into BMI (binary module interface; equivalent to Rust’s crate metadata) and any translation unit that imports said module cannot start compiling before the BMI is available.

                                                                                                              Thank you! This is much more helpful (and much shorter) than the articles I’ve mentioned.

                                                                                                              BTW, a question on monomorphization: aren’t argument’s lifetimes also turn functions into templates

                                                                                                              That’s an excellent question! One of the invariants of Rust compiler is lifetime parametricity – lifetimes are completely erased after type checking, and code generation doesn’t depend on lifetimes in any way. As a special case, “when the value is dropped” isn’t affected by lifetimes. Rather the opposite – the drop location is fixed, and compiler tries to find a lifetime that’s consistent with this location.

                                                                                                              So, while in the type system type parameters, value parameters and lifetimes are treated similarly, when generating machine code types work like templates in C++, and lifetimes roughly like generics in Java. That’s the reason why specialization takes so much time to ship – it’s very hard to make specialization not depend on lifetimes.

                                                                                                            2. 1

                                                                                                              Well your definition is interesting, because I actually don’t (exactly) agree.

                                                                                                              • code is paresd/typechecked only once (no header files)

                                                                                                              In OCaml you have ‘header’ or rather interface files, which are parsed/typechecked only once. They’re the equivalent of C++ precompiled headers except the compiler does it automatically.

                                                                                                              • there are explicit up-front dependencies between components, there’s no shared global namespace a-la PYTHONPATH or CLASSPATH

                                                                                                              Again in OCaml, there are dependencies between modules, but they are implicit. They just fail the build if modules are not compiled in the right order with the right dependencies. Fortunately the build system (dune) takes care of all that.

                                                                                                              Also OCaml has a shared global namespace–all source code files automatically become globally-visible modules within the project. Again fortunately the build system provides namespacing within projects to prevent name clashes.

                                                                                                              Another example of ‘proper modules’ is Pascal’s units, which actually satisfies both your above criteria (no header files, and explicit dependencies between units), and provides embarrassingly-parallel compilation.

                                                                                                              I think we simply don’t know how to make a language which is both fast to run and fast too compile.

                                                                                                              That may well be true.

                                                                                                              D

                                                                                                              From what I’ve heard, D compiles fast. And I assume it runs fast too. OCaml is pretty similar e.g. in some benchmarks it has similar single-core performance to Rust.

                                                                                                              1. 1

                                                                                                                Yeah (and here I would probably be crucified), I’d say that OCaml doesn’t have proper modules :-) Compilation in some order into a single shared namespace is not modular.

                                                                                                                Rust’s approach with an explicit DAG of crates which might contain internal circular dependencies is much more principled. Though, it’s sad that it lost crate interface files at some point.

                                                                                                                1. 1

                                                                                                                  I’d say that OCaml doesn’t have proper modules :-)

                                                                                                                  Heh, OK then ;-)

                                                                                                                  Compilation in some order into a single shared namespace is not modular.

                                                                                                                  That’s exactly what Rust crates end up doing. It just shifts the concept of ‘namespace’ into the crate names. Same with Java, C#, etc. Rust:

                                                                                                                  use serde::{Serialize, Deserialize};
                                                                                                                  

                                                                                                                  OCaml:

                                                                                                                  Serde.Serialize.blabla
                                                                                                                  
                                                                                                                  1. 2

                                                                                                                    It just shifts the concept of ‘namespace’ into the crate names

                                                                                                                    Not exactly: Rust crates don’t have names. The name is a property of the dependency edge between two crates. The same crate can be known under different names in two of its reverse dependencies, and that same name can refer to different crates in different crates.

                                                                                                                    1. 1

                                                                                                                      I think this is a distinction without a difference. Rust crates have names. They’re defined in the Cargo.toml file. E.g. https://github.com/serde-rs/serde/blob/65e1a50749938612cfbdb69b57fc4cf249f87149/serde/Cargo.toml#L2

                                                                                                                      [package]
                                                                                                                      name = "serde"
                                                                                                                      

                                                                                                                      And these names are referenced by their consumers.

                                                                                                                      The same crate can be known under different names in two of its reverse dependencies

                                                                                                                      But they have to explicitly rename the crate though, i.e. https://stackoverflow.com/a/51508848/20371 , which makes it a moot point.

                                                                                                                      and that same name can refer to different crates in different crates.

                                                                                                                      Same in OCaml. Different projects can have toplevel modules named Config and have them refer to different actual modules. If there is a conflict it will break the build.

                                                                                                                      1. 2

                                                                                                                        If there is a conflict it will break the build.

                                                                                                                        The build will work in Rust. If, eg, serde some day publishes serde version 2.0, then a project will be able to use both serdes at the same time.

                                                                                                                        So, eg, you can depend on two libraries, A and B, which both depend on serde. Then, if serde published 2.0 and A updates but but B does not, your build will continue to work just fine.

                                                                                                                        Not sure if OCaml can do that, but I think Java (at least pre modules)/C/Python can’t, without some hacks.

                                                                                                                        1. 1

                                                                                                                          That is a design choice. OCaml makes the choice to not allow duplicate dependencies with the same name.

                                                                                                            3. 4

                                                                                                              What do you mean when you say ‘proper’ modules?

                                                                                                              I understand the author mean a compilation unit, so a boundary around which you can count on compilation being separable (and therefore likely also cacheable). In C and C++, that happens to be a file (modulo some confusion about the preprocessor). In Rust, it is a collection of files called a crate. Once you accept that, everything else you say about file-based compilation units holds for Rust crates too: you can parallelize compilation of crates, you can cache compiled crates, and you get faster compile times from splitting code into crates.

                                                                                                              1. 1

                                                                                                                I defined ‘proper’ modules in my comment second paragraph.

                                                                                                            1. 22

                                                                                                              Clang and GCC both support type-checking the parameters to printf and related functions; you just have to enable the right warning flag (which I don’t have handy right now on my lawn chair.) You can even enable this checking on your own functions that take format strings + varargs, by tagging them with an attribute.

                                                                                                              This has been around for more than 10 years. Given how easy it is to make mistakes with printf, every C or C++ programmer should enable these warnings (and -Werror) if their compiler supports them.

                                                                                                              (Yeah, C++ has its own formatted output library, and I sometimes use it, but it generates very inefficient code.)

                                                                                                              1. 8

                                                                                                                -Wall -Werror. And when developing, -fsanitize=memory.

                                                                                                                1. 10

                                                                                                                  I have nothing against -Werror for CI, or for development, or for anything like that.

                                                                                                                  But please, please don’t use -Werror by default if you’re working on an open-source project where you want others to come and compile your code. Inevitably, a new compiler version comes around which introduces a new warning or makes an existing warning trigger in more situations. There are so, so many cases where I’ve cloned a project, then followed the build instructions exactly, only to realize that my compiler is slightly different than the one the author is using, and the build fails due to -Werror. It’s never fun to have to dive deep into an unknown build system to disable -Werror when you just want to compile something.

                                                                                                                  -Werror for CI and for development. -Wno-error for the default debug and release builds. Please.

                                                                                                                  Even if you’re the only developer and user, -Werror is probably gonna bite you while bisecting. Your 2 year old commits probably contain code which your current compiler will warn about.

                                                                                                                  1. 0

                                                                                                                    I mean you can just disable -Werror while you’re developing, no big deal.

                                                                                                                    1. 3

                                                                                                                      What do you mean? My main points are about the experience for a user who wants to compile your software, not the experience for the developer.

                                                                                                                      1. 0

                                                                                                                        I mean if they want to compile it then presumably they know their way around a Makefile or a config script. Users shouldn’t impose development practices on the project developers.

                                                                                                                        1. 4

                                                                                                                          What? The needs of your users shouldn’t affect development? That literally doesn’t make sense. You’re developing software for your users, no? Presumably all development should be influenced by the needs of your users, right?

                                                                                                                          If you make software where no user is going to compile your software from source, and where you’re not looking for contributors, and won’t ever ask people to make a build with debug symbols enabled to get a stack trace, and you release binaries for every conceivable system, then sure, do whatever you want. But most open source projects will have some users who want to compile from source, and you should probably adapt the software to the needs of your users. Most software acknowledges this by putting build instructions prominently in the readme.

                                                                                                                          You can at least do the slightest amount of effort to make your software nicer to use for your users. If you care at all, avoid intentionally breaking their build by default. Please.

                                                                                                                          1. 1

                                                                                                                            EDIT: btw, your point:

                                                                                                                            -Wno-error for the default debug and release builds.

                                                                                                                            Is super reasonable and I totally agree for release builds.

                                                                                                                            But for development…

                                                                                                                            The needs of your users shouldn’t affect development? That literally doesn’t make sense. You’re developing software for your users, no? Presumably all development should be influenced by the needs of your users, right?

                                                                                                                            The needs of users should influence the user experience of the software, not the development process of the software. If a user wants to compile from source it would surely be easier if the compilation disabled safety checks like say the typechecker phase. But it would also be massively inconvenient for the developers. Like I said, when it comes to development process, the needs of the developers outweigh the needs of users.

                                                                                                                            If you care at all, avoid intentionally breaking their build by default. Please.

                                                                                                                            Developers don’t intentionally break builds. At least, assuming a reasonable developer. What we don’t like is demands from users that would compromise our development practices. If a compiler warning breaks a build, the user of the open source software hopefully takes the time to file a ticket so the dev is aware and can fix it. Open source is a give-and-take, it’s not one side (developers) continuously caving to user demands.

                                                                                                                      2. 1

                                                                                                                        Use it for CI if you want it. Otherwise the point about compilers stands.

                                                                                                                        1. 1

                                                                                                                          No, it doesn’t, as I explained elsewhere in this thread. By default disabling safety checks during the normal development process just leads to more buggy and unmaintainable code over time.

                                                                                                                          1. 1

                                                                                                                            They’re warnings for a reason. If you want to ensure your checked-in code has no warnings, put them in CI. But don’t burden your users.

                                                                                                                            1. 1

                                                                                                                              Don’t optimize for the niche cases. Users typically don’t compile software themselves, they install from pre-compiled packages or download pre-built executables. In the normal course of things it’s much more likely that packagers will be the ones actually compiling/setting up builds. And if their packaging system is any good, they will lock down the versions of things like C compiler as much as possible, to ensure reproducibility.

                                                                                                                    2. 4

                                                                                                                      I would make that -Wall -Wextra -Werror=format -Werror=switch, plus -Werror and -fsanitize=… for the developer build.

                                                                                                                      -Wall is effectively the first warning level. There is also -Wextra and -Wpedantic.

                                                                                                                      I would reserve -Werror for the developer build, unless you want to break the ability to compile your current software with future compilers (since compiler writers come up with new warnings all the time). In particular, if you have users, you don’t want them to have to debug this.

                                                                                                                      The problem is of course false warnings – some warnings are pure noise in my opinion:

                                                                                                                      if(size == argc) // -Wsign-compare
                                                                                                                      

                                                                                                                      Lastly, if you care about specific warnings (who doesn’t?), it’s safer (safe enough, I suppose) to turn just those into errors, such as with -Werror=format (which catches this) and -Werror=switch (which catches unhandled switch cases). Those are some quality warnings on top of my head that are no reason to tolerate once your code is written to that standard.

                                                                                                                      Also, -Werror=sometimes-uninitialized for clang.

                                                                                                                      1. 2

                                                                                                                        At least the clang policy is for -Wextra to include things that have a lot of false positives, whereas -Wall includes things that have very low false-positive reates. You should think very carefully about enabling it by default for two reasons:

                                                                                                                        • Every false positive a developer encounters for a warning makes them more likely to just do the thing that silences the warning, rather than fix the issue.
                                                                                                                        • Your codebase will include an increasing number of things that just silence warnings, which makes the code less readable and makes it easier for bugs to sneak in.
                                                                                                                        1. 1

                                                                                                                          Are you sure you don’t mean -Weverything? Apart from -Wsign-compare (which is in -Wall in g++ and -Wextra in gcc), which I would say that about, I find -Wextra and even -Wpedantic quite livable.

                                                                                                                          1. 4

                                                                                                                            At least the last time we tried to write a policy for this, the consensus was:

                                                                                                                            • -Wall is things that are unlikely to cause false positives and we’d like to recommend everyone uses.
                                                                                                                            • -Wextra is things with too high a false positive rate for -Wall.
                                                                                                                            • -Weverything is all warnings including experimental ones that may give completely nonsense results.

                                                                                                                            I wouldn’t recommend -Weverything for anyone who is not working on the compiler. As I recall, -Wsign-compare had quite a bit of discussion because it does have a fairly high false positive rate, but it also catches a lot of very subtle bugs and it encourages people to be more consistent about their use of types, so leads to better code overall.

                                                                                                                            1. 1

                                                                                                                              Ok. My take on -Wsign-compare is that it mixes up equality vs inequaity comparisons. I wish it didn’t:

                                                                                                                              if(size < argc) // Bad: Result depends on signedness.
                                                                                                                              if(size == argc) // Ok: Result does not depend on signedness.
                                                                                                                              

                                                                                                                              Only one of these comparisons is problematic, yet -Wsign-compare complains about both! As such, half this warning is legit; half of it is never that, and the need to silence it is just detrimental.

                                                                                                                              You’re right, I wouldn’t use -Wextra: I would use -Wextra -Wno-sign-compare, because that’s the unreasonable part.

                                                                                                                              1. 2

                                                                                                                                The second is not, by itself, a problem, but it can depend on a cast that might be a problem (you must have constructed size from something, and what happens if that size is in the range that is negative for int? Is your logic still correct in this case?). It can also mask problems on some platforms because of the exciting integer promotion rules in C that are invoked here. Unfortunately, C and POSIX APIs are incredibly inconsistent about when they use signed and unsigned values (argv really should be a size_t, but that would break backwards compatibility[1]).

                                                                                                                                C also doesn’t have a good way of asking the question that you really want to ask, which is ‘are both of these values contained in the range that can be expressed in both types and, if so, are they identical?’. The integer promotion rules correctly handle that if there is a type that can express the range of both, but fails with annoying corner cases where there isn’t one. You really want a 31- or 63-bit unsigned type in a lot of cases so that you have something that can be unambiguously cast to a signed or unsigned 32- or 64-bit integer.

                                                                                                                                [1] I’d love for C++ to define a new entry point that provided a pair of iterators for the arguments and have the compiler synthesise the prologue in main, but that’s probably never going to happen.

                                                                                                                            2. 4

                                                                                                                              -Wextra has warnings for a lot of things which don’t necessarily indicate a problem. The biggest example of this is the unused parameter warning, which warns when a function has an unused parameter.

                                                                                                                              Sometimes, an unused parameter indicates a problem, sure. But a lot of the time, the parameters have to exist because the function has to implement some particular signature to be used as a function pointer (or to override a virtual method in C++), but doesn’t need to use one of the arguments. In C++, a common pattern is to comment out the name of the parameter - something like, void foo(int /*count*/, int *whatever) { ... }, but that’s illegal syntax in C.

                                                                                                                              I find things like -Wsign-compare to be a bit annoying sometimes, but mostly reasonable. It could possibly catch errors, and it’s easy enough to do the cast (or use the right signedness in for loops). But it’s certainly another example of something which has a lot of false positives.

                                                                                                                              I often use -Wall -Wextra -Wpedantic -Wno-unused-parameter in my makefiles.

                                                                                                                              1. 3

                                                                                                                                You can avoid that warning in C too. I think the commonest way is to cast it to void:

                                                                                                                                void api(int unused) {
                                                                                                                                    (void)unused;
                                                                                                                                }
                                                                                                                                
                                                                                                                                1. 3

                                                                                                                                  You can silence most of the warnings. The questions is whether the extra noise in the source code lowering the overall readability is going to cause more bugs than the warning prevents.

                                                                                                                        2. 1

                                                                                                                          Except -Wall doesn’t turn on all warnings, just “a lot”, and in my experience omits some I consider important. (Im sure this is compiler- and maybe version-dependent.)

                                                                                                                          IIRC, -Wpedantic turns on “too many” warnings in Clang but “about right” in GCC.

                                                                                                                          In Xcode, I use a curated xcconfig file that turns on nearly every warning; then in my own config file I turn off the ones that are so annoying I don’t use them, mostly super pedantic stuff like implicit sign conversions.

                                                                                                                        3. 1

                                                                                                                          (Yeah, C++ has its own formatted output library, and I sometimes use it, but it generates very inefficient code.)

                                                                                                                          Not to mention terrible ergonomics.

                                                                                                                          1. 6

                                                                                                                            In what way? The API is almost the same as printf, except that the format strings are in braces and the qualifiers can be a bit more readable and generates better code. It also provides a much richer API allowing things like compiling the format string so that you get compile-time type checking even in cases where the string and the arguments are provided in different places. You can also provide formatters for your own types that have the same type safety guarantees.

                                                                                                                            1. 2

                                                                                                                              Wow, I thought they were talking about ostream. I’m not up to date on C++20, the last I used was C++17. This is cool! It seems just like Rust’s format! macro.

                                                                                                                              1. 2

                                                                                                                                C++20 has std::format, which is basically a stable subset of the fmt APIs. It’s sufficiently easy to add fmt to a project that I’d only use it for things that are aggressively wanting to trim dependencies though.

                                                                                                                                Completely agreed on ostream. It’s not amenable to localisation, it’s hard to use in a multithreaded context, and the APIs for actually doing the stringification are horrible (depending on pushing state into the stream to control formatting). The core bits of iostream are actually quite nice: separating the buffer management from the I/O interface, for example, but the high-level bits are awful. They’re over 20 years old at this point (they’re inherited from STL, before C++98’s standard library), so also predate multithreading as a common pattern.

                                                                                                                        1. 3

                                                                                                                          On my system, g++ by default prints

                                                                                                                          pr.cc: In function ‘int main()’:
                                                                                                                          pr.cc:4:35: warning: format ‘%d’ expects argument of type ‘int’, but argument 2 has type ‘double’ [-Wformat=]
                                                                                                                               printf("%d\n", double(2101253));
                                                                                                                          

                                                                                                                          And yes, I do use -Wall (even though not needed for this example), and -fsanitize, and C++ smart pointers. Bugs caused by unsafe memory access are pretty rare, and when they happen, I get verbose runtime error messages that make the problem easy to fix. It’s easy to exaggerate the difficulties of C++ memory unsafety if you don’t regularly use the language and know the idioms and tools.

                                                                                                                          1. 3

                                                                                                                            Then why do memory unsafety CVE’s keep happening in the most heavily-used C++ projects in the world? Are their developers idiots?

                                                                                                                          1. 3

                                                                                                                            Printf is actually perfectly type-safe if you’re using the right language :-)

                                                                                                                            1. 3

                                                                                                                              I think you meant this language.

                                                                                                                              1. 1

                                                                                                                                How type-safe is it? I only looked at it briefly without knowing rust, and it seems to me they sidestep the issue by making it possible to plug in any other type in a printf? Or am I mistaken?

                                                                                                                                1. 3

                                                                                                                                  Not a rust programmer, but I know that format! is a macro. What could be happening is that at compile time the fmt string is parsed, and display specifc functions for the specifier take the positional argument. If that argument isn’t the right type for the specifier, it’s a compile time type error.

                                                                                                                                  My guess is that Rust doesn’t allow dynamic format strings, or if it does, it’s a runtime error, no different than OCaml, when the format is incompatible.

                                                                                                                                  Would love to be corrected.

                                                                                                                                  1. 3

                                                                                                                                    What could be happening is that at compile time the fmt string is parsed, and display specifc functions for the specifier take the positional argument.

                                                                                                                                    Good guess, this is exactly what happens!

                                                                                                                                    My guess is that Rust doesn’t allow dynamic format strings

                                                                                                                                    2/2, format strings must be static, and they’re checked at compile time.

                                                                                                                            1. 15

                                                                                                                              Other TL;DR: Just pay for the thing if you’re a big company.

                                                                                                                              1. 9

                                                                                                                                It’s kind of easy to say, and likely doable in the smaller end of the “big company”. But at some level of corporate, you also have to fight with “the process”. In other words, if I get a choice to spend a week migrating to podman or to deal with expense approval chains, legal, it, management, etc. then there’s a really good chance I’m going with podman. (If no critical features are missing)

                                                                                                                                1. 6

                                                                                                                                  Wouldn’t you also have to get legal and IT approval to introduce Podman in places where you were previously using Docker?

                                                                                                                                  1. 4

                                                                                                                                    Depends on the project. In some cases yes, in others, if it already comes with an approved distro then no.

                                                                                                                                    1. 1

                                                                                                                                      No doubt these places exist, but the theoretical company you describe sounds like a really miserable place to work

                                                                                                                                    2. 4

                                                                                                                                      There’s also an argument that we should try to support good open source projects when they offer paid products that we actually also use.

                                                                                                                                  1. 4

                                                                                                                                    Direct link to code (with commentary): https://play.golang.org/p/83fLiHDTSdY (and on the Go2go Playground: https://go2goplay.golang.org/p/83fLiHDTSdY)

                                                                                                                                    I think I see what he’s doing, building up an interpreter DSL, but man, I have no idea what most of his commentary means. For example:

                                                                                                                                    I exploit the duality between universal and existential quantification and encode sum types using the existential dual of the Boehm-Berarducci isomorphism.

                                                                                                                                    1. 11

                                                                                                                                      Well it’s pretty simple, really. Types are all about the composition of things, so the question is how do we compose what we have (language primitives) to get what we want. We can guess, try things out, but if that doesn’t work out we can use math to derive the result we seek, because these are all simple settled questions in math, and due to the Curry–Howard corespondence we should be able to implement them in our language.

                                                                                                                                      Alonzo Church showed us how to encode anything into (untyped) lambda calculus, and indeed, we can encode variants using Church encoding like so:

                                                                                                                                      a+b ≡ λa b f . f a b
                                                                                                                                      fst ≡ λt . t (λa b . a)
                                                                                                                                      snd ≡ λt . t (λa b . b)
                                                                                                                                      

                                                                                                                                      a+b takes three arguments, but we only supply two when we create it. Then a+b is a pending computation, and fst and snd extract the first or second component by supplying the continuation that tells the sum type what to do!

                                                                                                                                      Unfortunately, this is not quite what we want, because untyped lambda calculus is untyped, and the whole point of the exercise is to be typed so this encoding won’t cut it. But it’s ok, this captures the procedural aspect of the idea, it tells us what we have to do with the data, but not with the types. Now we have to find a way to determine a typeful encoding.

                                                                                                                                      Well, it’s pretty easy if we use math. A type like a can always be written like ∀z . (a→z) → z. But we are interested in a+b, so we just plug-in a+b, and we do some trivial algebra and we get:

                                                                                                                                      ∀z . (a+b →z) →z = ∀z . ((a→z) * (b→z)) →z
                                                                                                                                      

                                                                                                                                      This looks suspiciously similar to Church encoding, we can see the two continuations. However, this is a type equation. It doesn’t tells us anything what to do with the values, it’s just a relation between types. But we know what to do with the values from the CPS interpretation of the Church encoding, and now we can implement this in any language that has parametric polymorphism. This is the road to the Boehm-Berarducci encoding, which expands on this idea (except that it goes a step further and encodes it as ∀z . (a→z) → (b→z) →z).

                                                                                                                                      We can implement this in Haskell.

                                                                                                                                      {-# LANGUAGE ExistentialQuantification #-}
                                                                                                                                      {-# LANGUAGE RankNTypes #-}
                                                                                                                                      
                                                                                                                                      import Text.Printf
                                                                                                                                      
                                                                                                                                      data Variants a b z = Variants (a->z) (b->z)
                                                                                                                                      type Sum a b = (forall z. Variants a b z -> z)
                                                                                                                                      
                                                                                                                                      plus :: Int -> Sum Int Int
                                                                                                                                      plus x (Variants f _) = f x
                                                                                                                                      minus :: Int -> Sum Int Int
                                                                                                                                      minus x (Variants _ f) = f x
                                                                                                                                      
                                                                                                                                      neg :: Sum Int Int -> Sum Int Int
                                                                                                                                      neg v = v (Variants (\x -> minus x) (\x -> plus x))
                                                                                                                                      
                                                                                                                                      foo = plus 42
                                                                                                                                      bar = minus 16
                                                                                                                                      baz = neg foo
                                                                                                                                      
                                                                                                                                      example :: Sum Int Int -> String
                                                                                                                                      example v = v (Variants
                                                                                                                                          (\x -> printf "got a plus %d" $ x)
                                                                                                                                          (\x -> printf "got a minus %d" $ x))
                                                                                                                                      
                                                                                                                                      main = do
                                                                                                                                          putStrLn (example foo)
                                                                                                                                          putStrLn (example bar)
                                                                                                                                          putStrLn (example baz)
                                                                                                                                      

                                                                                                                                      Prints:

                                                                                                                                      : z800:aram; a
                                                                                                                                      got a plus 42
                                                                                                                                      got a minus 16
                                                                                                                                      got a minus 42
                                                                                                                                      

                                                                                                                                      However, we can’t implement this in Go. Until very recently, Go didn’t have parametric polymorphism at all, and even so, notice that the forall in Haskell is inside the type. We need rank-2 polymorphism which Go doesn’t have. But the fact that we had to enable ExistentialQuantification in Haskell should give us a hint.

                                                                                                                                      Go interfaces are existential types. In most (all?) languages based on System F/HM existential types are encoded as rank-n universals, but Go is unusual in that Go has always had existential types even before it had parametric polymorphism.

                                                                                                                                      But universal quantification in logic is dual to existential quantification! We can always switch back and forth between the two (at least in classical logic, but because of intentional limitations of the Go type system, the excluded middle axiom should always hold, so we should be ok here).

                                                                                                                                      So we can just translate any statement using universal quantifiers into its dual that uses only existential quantifiers, basically through double negation.

                                                                                                                                      ∀x P(x) ≡ ¬∃x ¬P(x)
                                                                                                                                      

                                                                                                                                      So we just do that, and we get what’s implemented in my program!

                                                                                                                                      Notice that I make the above transformation in logic, not in Go. If I’d made it in Go I would have to assume that the Curry–Howard isomorphism holds in the Go type system, which it may or it might not depending on the semantics of its generics and interfaces. I do not make this assumption. I do not assume we can use that axiom inside Go, but of course, we can use it in the meta-theory.

                                                                                                                                      I explained all of this starting from Church encoding, going though Boehm-Berarducci, etc, but that’s not how I discovered this at all. I was trying to formalize Go generics, and equations such as the ones above popped out as intermediary results. I then immediately saw the potential of this, and trying to make sure that it works, and that I understood it, I started from the existential representation, and transformed it into the Boehm-Berarducci encoding (which I wasn’t even aware it existed when I started this exercise).

                                                                                                                                      I only considered ADTs (not GADTs) in this post, but I hope this at least gives you some intuition.

                                                                                                                                      Hope this helps.

                                                                                                                                      1. 2

                                                                                                                                        I appreciate you taking time with this answer, and I did read it all, but unfortunately I just don’t have the math and computer science background to comprehend this. Back to school for me!

                                                                                                                                        1. 2

                                                                                                                                          This explanation would be more effective for me and other laypeople if you include a brief tutorial on the math notation you use.

                                                                                                                                          It’s easier to pick up the Haskell than it is to pick up the math notation.

                                                                                                                                          1. 2

                                                                                                                                            Alonzo Church showed us how to encode anything into (untyped) lambda calculus, and indeed, we can encode variants using Church encoding like so:

                                                                                                                                            a+b ≡ λa b f . f a b
                                                                                                                                            fst ≡ λt . t (λa b . a)
                                                                                                                                            snd ≡ λt . t (λa b . b)
                                                                                                                                            

                                                                                                                                            a+b takes three arguments, but we only supply two when we create it. Then a+b is a pending computation, and fst and snd extract the first or second component by supplying the continuation that tells the sum type what to do!

                                                                                                                                            Something is off here. That’s the Church encoding for products, not variants/sums.

                                                                                                                                            1. 1

                                                                                                                                              maybe because of this equivalence?

                                                                                                                                              ∀z . (a+b →z) →z = ∀z . ((a→z) * (b→z)) →z

                                                                                                                                              1. 2

                                                                                                                                                It’s true that destructing a sum amounts to providing a product of handlers, one for each case. But that’s not how OP explained it. OP said the provided Church encoding was for variants/sums, which it isn’t, and then didn’t seem to use it at all in the Haskell code, since the Variants constructor is already a product of exponentials and doesn’t need to be Church-encoded. I think it’s just a mistake. It doesn’t detract from the Go demonstration, which is really cool. But Church encoding explanation is wrong, and I can see why people in these comments are confused.

                                                                                                                                              2. 1

                                                                                                                                                Thanks! That is what happens when you didn’t sleep for 24 hours. It should have been:

                                                                                                                                                c₁ ≡ λt f₁ f₂ . f₁ t
                                                                                                                                                c₂ ≡ λt f₁ f₂ . f₂ t
                                                                                                                                                 m ≡ λt f₁ f₂ . t f₁ f₂
                                                                                                                                                

                                                                                                                                                e.g for usage: swap ≡ λt . m t c₂ c₁

                                                                                                                                                (I can’t edit the mistake in the original post.)

                                                                                                                                            2. 5

                                                                                                                                              It’s like casting a spell. ‘I invoke the elemental magics of Yogg-Zonnroth, and cast the energies of Zum-ra into the ether!’

                                                                                                                                              1. 1

                                                                                                                                                Boehm-Berarducci encoding

                                                                                                                                                Oleg has a good paper about this. Basically you can translate all A(lgebraic)DT operations (construction and destruction (aka pattern-matching)) into lambdas and applications of lambdas, meaning you can embed ADTs in languages that do not have them.

                                                                                                                                                1. 1

                                                                                                                                                  The code fails to run in both playgrounds … I’m guessing they are not using the right golang version ?

                                                                                                                                                  1. 1

                                                                                                                                                    Yes, it only works in the latest version of the compiler.

                                                                                                                                                1. 2

                                                                                                                                                  Interesting. COBOL’s string concatenation looks like it would be a better fit for Elixir’s iolists, e.g. IO.puts(["Hello ", var_Name]).

                                                                                                                                                  1. 31

                                                                                                                                                    If the author is here and open to changing the post title, Rust is for everyone, and good for professionals too! The title as is sounds like it may be exclusionary toward people who don’t identity as professionals, or feel intimidated by Rust, which is contrary to Rust’s core goal of opening up systems programming to everyone!

                                                                                                                                                    1. 39

                                                                                                                                                      I, the author, am here.

                                                                                                                                                      I appreciate the constructive feedback and can see your point. Choosing titles is hard. I too was concerned about the exclusionary potential for the title. However, I thought I had it sufficiently mitigated by the introduction paragraphs. Apparently I could have done better.

                                                                                                                                                      I’m not going to change the title of this post because I don’t want to deal with the hassle of rewriting URLs. However, I will try to consider your feedback for future posts I author.

                                                                                                                                                      1. 6

                                                                                                                                                        I appreciated the way you took the feedback. Just wanted to say thanks for listening and considering it.

                                                                                                                                                        1. 3

                                                                                                                                                          I’ve programmed in the following languages: C, C++ (only until C++11), C#, Erlang, Go, JavaScript, Java, Lua, Perl, PHP, Python, Ruby, Rust, shell, SQL, and Verilog.

                                                                                                                                                          To me, Rust introduced a number of new concepts, like match for control flow, enums as algebraic types, the borrow checker, the Option and Result types/enums and more.

                                                                                                                                                          How did you use Erlang without pattern matching?

                                                                                                                                                          1. 2

                                                                                                                                                            I think the order is alphabetical so its possible that author used Erlang after he learned Rust.

                                                                                                                                                            Have a nice day!

                                                                                                                                                            1. 1

                                                                                                                                                              Doesn’t add up, he says that Rust introduced pattern matching to him :-)

                                                                                                                                                          2. 3

                                                                                                                                                            My usual way of framing it is “Rust is an industrial programming language”. That leaves the door open for practitioners of all kinds. It’s more like a cars are (in general) a utility and not a plaything or research endeavor, but are open to enthusiasts and amateur practitioners as well.

                                                                                                                                                            1. 2

                                                                                                                                                              Thanks! I liked the disclaimer, and think predicting how people will respond can be tough. Appreciate you putting in the time to write this, and agree that Rust can be used in professional / production environments successfully, and that it can feel like a breath of fresh air for many!

                                                                                                                                                            2. 22

                                                                                                                                                              Or, alternately, when someone writes 13,000 words about how nice Rust is, how ergonomic and human-centric and welcoming and humane the language and tooling and community are, they can keep the title without someone telling them they’re maybe being exclusionary.

                                                                                                                                                              1. 16

                                                                                                                                                                It’s constructive criticism. If you post something on the public web, it’s fair game to criticize it- especially if done in a polite manner.

                                                                                                                                                                1. 23

                                                                                                                                                                  It’s criticism, but just because it’s delivered politely does not mean it is constructive. There is a cost to spending the effort to write a thirteen thousand word love letter to a language, just to have someone nipping at your ankles about how the first four words might exclude someone. Objectively speaking, this is a blogger that Rust would benefit from sticking around – but what is the signal they’ve received here?

                                                                                                                                                                  1. 11

                                                                                                                                                                    Agree 💯.

                                                                                                                                                                    Recently a friend submitted his static site generator which is optimised for math heavy sites on HN.

                                                                                                                                                                    He made the fatal mistake of calling it “beautiful” in the description and the amount of hangups that people had. It was done in a mostly polite manner, but the amount of time and energy wasted in discussing just the first sentence with barely any technical discussion of the internals, was a form of pathological bikeshedding that I was surprised to see. It was just that “nipping at his ankles” as you’ve described, which can take its toll.

                                                                                                                                                                    Vowed to never take that site seriously again.

                                                                                                                                                                    1. 1

                                                                                                                                                                      It was done in a mostly polite manner, but the amount of time and energy wasted in discussing just the first sentence with barely any technical discussion of the internals, was a form of pathological bikeshedding that I was surprised to see. It was just that “nipping at his ankles” as you’ve described, which can take its toll.

                                                                                                                                                                      Agreed. Yet here we are- arguing about arguing. I need a life… xD

                                                                                                                                                                      1. 1

                                                                                                                                                                        Right, in my experience, HN often attracts uninteresting, unhelpful, or even mean-spirited commentary. Sometimes I am surprised by the opposite, though. In any case, understanding the dynamic really helps. A lot of commentary, seems to me, are upset at some level in some way and take it out in their commentary.

                                                                                                                                                                      2. 3

                                                                                                                                                                        The author thanked them and described the feedback as constructive. So if they feel like they’ve taken something away from it, who are we to decide otherwise? My own hopefully constructive feedback here is that this subthread strikes me as projecting conflict into an interaction between two parties who themselves seem to have had a positive experience with it.

                                                                                                                                                                        1. 2

                                                                                                                                                                          Yeah, I get where you’re coming from and I don’t really disagree.

                                                                                                                                                                          But I also think that the person who offered the criticism is just trying to help the author get more views.

                                                                                                                                                                          It’s fair to suggest that we all should consider our criticisms carefully, though. Because you’re right- if the criticism isn’t that important, we may just be disincentivizing someone from future contributions.

                                                                                                                                                                          I also wonder if you’re maybe reading a little too hard into the choice of the word “exclude” used by the criticizer, though… I think that term has a little bit of extra charge and connotation in today’s social environment and I don’t believe the criticizer meant it in that way. I think they simply meant that a person who isn’t writing enterprise software might pass over the article because they believe they aren’t the target audience. I agree with that conclusion as well, because I do write enterprisey software and I thought the article was going to be about that specific point of view- I thought I would see arguments about productivity and developer costs and cross-platform support and other less-sexy stuff. But, it was really- as you said, just a love letter to everything about Rust.

                                                                                                                                                                      3. 8

                                                                                                                                                                        I felt this lead-in made it a friendly suggestion and not harsh:

                                                                                                                                                                        If the author is here and open to changing the post title

                                                                                                                                                                        1. 4

                                                                                                                                                                          FWIW first third of those thousands of words is irellevant chitchat. The First actual claim on why is rust good is rust makes me giddy. I’m not sure “giddy” describes profesionals. It’s not no, but for me definitely not a yes either - giddy is just…giddy. Then all the technical points, borrow checkers, race conditions and whatnot - none of that tells me why is it a thing for professionals.

                                                                                                                                                                          What I mean is, I don’t know if rust is for pros or not, I just think that the title is a bit click-baity, and “rust is good for everyone, even pros” like alilleybrinker suggested would work much better.

                                                                                                                                                                          1. 7

                                                                                                                                                                            The First actual claim on why is rust good is rust makes me giddy. I’m not sure “giddy” describes profesionals.

                                                                                                                                                                            There’s a layer of extra irony here in that the word “amateur” comes from a root meaning “love”; in other words, amateurs are people who do something because they love it. So taken literally, this is very unprofessional!

                                                                                                                                                                        2. 18

                                                                                                                                                                          It seems relevant to note that the author does address this in the post itself:

                                                                                                                                                                          The statement Rust is for Professionals does not imply any logical variant thereof. e.g. I am not implying Rust is not for non-professionals. Rather, the subject/thesis merely defines the audience I want to speak to: people who spend a lot of time authoring, maintaining, and supporting software and are invested in its longer-term outcomes.

                                                                                                                                                                          1. 10

                                                                                                                                                                            Yes, except the title doesn’t have a disclaimer, and will be taken in the way they say they don’t want. Rather than leaving it ambiguous, they could change the title to remove the ambiguity.

                                                                                                                                                                            1. 28

                                                                                                                                                                              This side-steps the issue “people don’t read past the title” - which is the real problem. It’s bad, and unrealistic, to expect every person who writes on the internet to tailor each individual sub-part of an article (including the title) to an audience that is refusing to read the (whole) article itself before making judgements or assumptions. That’s purely the fault of the audience, not the writer, and changing the article title is merely addressing the symptoms, instead of the root problem, which will allow it to just become worse.

                                                                                                                                                                              We have an expression “judging a book by its cover” specifically for this scenario, and in addition to that, any reasonably-thoughtful reader is aware that, once a sufficient amount of context has been stripped away, any statement loses its meaning - which logically implies that full context must be absorbed before the reader has any chance of understanding the author’s intended meaning.

                                                                                                                                                                              People should not have to write hyper-defensively on the internet, or anywhere, because civil discussion collapses when your audience isn’t acting honestly.

                                                                                                                                                                              Reading the title and judging before reading the content itself is not acting honestly.

                                                                                                                                                                              1. 4

                                                                                                                                                                                Thank you for writing this. I’m sick of how much culture is warped by toxic social media sites.

                                                                                                                                                                              2. 7

                                                                                                                                                                                Agreed. It’s a poor choice for a title, even with a disclaimer. In fact, that he knew he needed a disclaimer should have been a big clue that it wasn’t a good title.

                                                                                                                                                                                A better title would be: “Professional devs should love Rust”, or “Rust isn’t just for enthusiasts and hipsters”

                                                                                                                                                                                1. 0

                                                                                                                                                                                  I don’t disagree.

                                                                                                                                                                              3. 18

                                                                                                                                                                                Responses like this make me hesitant to try Rust, because they make me feel that I’d have to tiptoe around every word I put into a chat room, issue tracker, or mailing list.

                                                                                                                                                                                1. 10

                                                                                                                                                                                  I’m not sure this is a Rust issue as much as a general issue of recent years. Be delicate and excessively sensitive about everything because everyone’s an egg shell. It’s contrary to anti-fragility which I would suggest is a far better approach; and the opposite approach. “Rust is for professionals” would have had me targeting Rust so that I could level up, not scare me away. Show me a challenge,

                                                                                                                                                                                2. 5

                                                                                                                                                                                  I used rust for three years outside of work, but I’m afraid I stopped last year because I find it too big for a hobbyist. If I used it all day every day - no problem. It’s a great language, but I agree with the title.

                                                                                                                                                                                  1. 2

                                                                                                                                                                                    This was my first thought, as well, before even reading the article. Obviously I should read before judging, but the author may want to consider that the title could turn away potential readers.

                                                                                                                                                                                    I made the same mistake in my own My staff engineering reading list. There was no reason for me to needlessly exclude people earlier in their careers from my list.

                                                                                                                                                                                    And it’s good feedback that the author of this piece may not want to unintentionally exclude people from their article.

                                                                                                                                                                                  1. 25

                                                                                                                                                                                    How has AGPL failed? Quoting the introduction to Google’s own policies, which are the top hit for “agpl google” on DuckDuckGo:

                                                                                                                                                                                    WARNING: Code licensed under the GNU Affero General Public License (AGPL) MUST NOT be used at Google. The license places restrictions on software used over a network which are extremely difficult for Google to comply with.

                                                                                                                                                                                    This seems like a resounding success of AGPL.

                                                                                                                                                                                    Proprietary distributed systems frequently incorporate AGPL software to provide services.

                                                                                                                                                                                    Who, and which software packages? This isn’t just about naming and shaming, but ensuring that those software authors are informed and get the chance to exercise their legal rights. Similarly, please don’t talk about “the legal world” without specific references to legal opinions or cases.

                                                                                                                                                                                    I feel like this goes hand-in-hand with the fact that you use “open source” fourteen times and “Free Software” zero times. (The submitted title doesn’t line up with the headline of the page as currently written.) This shows an interest in the continued commercialization of software and exploitation of the commons, rather than in protecting Free Software from that exploitation.

                                                                                                                                                                                    1. 8

                                                                                                                                                                                      How has AGPL failed?

                                                                                                                                                                                      I said how in the article:

                                                                                                                                                                                      The AGPL was intended, in part, to guarantee this freedom to users, but it has failed. Proprietary distributed systems frequently incorporate AGPL software to provide services. The organizations implementing such systems believe that as long as the individual process that provides the service complies with the AGPL, the rest of the distributed system does not need to comply; and it appears that the legal world agrees.

                                                                                                                                                                                      The purpose of the AGPL is not to stop commercial users of the software, it’s to preserve the four freedoms. It doesn’t preserve those freedoms in practice when it’s been used, so it’s a failure.

                                                                                                                                                                                      But really AGPL doesn’t have anything to do with this. No-one claims AGPL is a license like the one I describe in the article.

                                                                                                                                                                                      Proprietary distributed systems frequently incorporate AGPL software to provide services.

                                                                                                                                                                                      Who, and which software packages?

                                                                                                                                                                                      mongodb is a high-profile example, used by Amazon.

                                                                                                                                                                                      1. 10

                                                                                                                                                                                        I’m fairly certain that the version of mongodb that Amazon based DocumentDB off of was Apache licensed, so I don’t think that applies here. From what I’m seeing, they also explicitly don’t offer hosted instances of the AGPL licensed versions of Mongo.

                                                                                                                                                                                        1. 5

                                                                                                                                                                                          But really AGPL doesn’t have anything to do with this. No-one claims AGPL is a license like the one I describe in the article.

                                                                                                                                                                                          Then your article headline is misleading, is it not?

                                                                                                                                                                                          1. 2

                                                                                                                                                                                            This is true. MongoDB changed their license, in response to Amazon using forks of their own software to compete with them.

                                                                                                                                                                                            1. 1

                                                                                                                                                                                              That’s fair, you’re probably right. There are lots of other hosted instances of MongoDB that used the AGPL version though, so MongoDB is still the highest-profile example of this. That was the motivation for MongoDB’s move to SSPL.

                                                                                                                                                                                            2. 2

                                                                                                                                                                                              I don’t see a laundry list here. I appreciate that you checked your examples beforehand and removed those which were wrong, but now there’s only one example left. The reason that I push back on this so heavily is not just because I have evidence that companies shun AGPL, but because I personally have been instructed by every employer I’ve had in the industry that AGPL code is unacceptable in their corporate environment, sometimes including AGPL developer tools! It would have been grounds for termination at three different employers, including my oldest and my most recent employments.

                                                                                                                                                                                              Regarding MongoDB, I have no evidence that AWS violated the terms of the AGPL, and they appear to have put effort into respecting it somewhat. It seems that MongoDB’s owners were unhappy that their own in-house SaaS offering was not competing enough with others, and they chose licensing as their way to fight. Neither of these companies are good, but none of them appear to be disrespecting the AGPL.

                                                                                                                                                                                            3. 4

                                                                                                                                                                                              the agpl says

                                                                                                                                                                                              Notwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software.

                                                                                                                                                                                              in other words, you cannot run your own proprietary fork of an agpl program; you have to offer users the modified sources. it says nothing about the sources of the other programs that that program communicates with, or the network infrastructure and configuration that comprises your distributed system.

                                                                                                                                                                                              1. 3

                                                                                                                                                                                                Yes. This is how we define distributed systems, in some security contexts: A distributed system consists of a patchwork network with multiple administrators, and many machines which are under the control of many different mutually-untrusting people. Running AGPL-licensed daemons on one machine under one’s control does not entitle one to any control over any other machines in the network, including control over what they execute or transmit.

                                                                                                                                                                                                Copyright cannot help here, so copyleft cannot help here. Moreover, this problematic layout seems to be required by typical asynchronous constructions; it’s good engineering practice to only assume partial control over a distributed system.

                                                                                                                                                                                                1. 1

                                                                                                                                                                                                  So, that seems fine then? What’s the problem with that?

                                                                                                                                                                                                  1. 1

                                                                                                                                                                                                    the problem, as the article says, is that we have no copyleft license that applies to entire distributed systems, and it would be nice to. not so much a problem with the agpl as an explanation of why it is not the license the OP was wishing for.

                                                                                                                                                                                                    1. 4

                                                                                                                                                                                                      I explain more upthread, but such licenses aren’t possible in typical distributed systems. Therefore we should not be so quick to shun AGPL in the hopes that some hypothetical better license is around the corner. (Without analyzing the author too much, I note that their popular work on GitHub is either MIT-licensed or using the GH default license, instead of licenses like AGPL which are known to repel corporate interests and preserve Free Software from exploitation.)

                                                                                                                                                                                                      1. 2

                                                                                                                                                                                                        We have Parity as a start. Its author says:

                                                                                                                                                                                                        “Parity notably strengthens copyleft for development tools: you can’t use a Parity-licensed tool to build closed software. “

                                                                                                                                                                                                        Its terms are copyleft for other software you “develop, operate, or analyze” with licensed software. That’s reads broad enough that anything an operator of distributed systems both owns and integrates should be open sourced.

                                                                                                                                                                                                1. 31

                                                                                                                                                                                                  I prefer to see this type of project that builds upon what it considers the good parts of systemd, instead of systemic refusal and dismissal that I’ve seen mostly.

                                                                                                                                                                                                  1. 15

                                                                                                                                                                                                    Same. Too often I see “critiques” of systemd that essentially boil down to personal antipathy against its creator.

                                                                                                                                                                                                    1. 22

                                                                                                                                                                                                      I think it makes sense to take in to account how a project is maintained. It’s not too dissimilar to how one might judge a company by the quality of their support department: will they really try to help you out if you have a problem, or will they just apathetically shrug it off and do nothing?

                                                                                                                                                                                                      In the case of systemd, real problems have been caused by the way it’s maintained. It’s not very good IMO. Of course, some people go (way) to far in this with an almost visceral hate, but you can say that about anything: there are always some nutjobs that go way too far.

                                                                                                                                                                                                      1. 3

                                                                                                                                                                                                        Disclaimer: I have not paid close attention to how systemd has been run and what kind of communication has happened around it.

                                                                                                                                                                                                        But based on observing software projects both open and closed, I’m willing to give the authors of any project (including systemd) the benefit of the doubt. It’s very probable that any offensive behaviour they might have is merely a reaction to suffering way too many hours of abuse from the users. Some people have an uncanny ability to crawl under the skin of other people just by writing things.

                                                                                                                                                                                                        1. 6

                                                                                                                                                                                                          There’s absolutely a feedback loop going on which doesn’t serve anyone’s interests. I don’t know “who started it” – I don’t think it’s a very interesting question at this point – but that doesn’t really change the outcome at the end of the day, nor does it really explain things like the casual dismissal of reasonable bug reports after incompatible changes and the like.

                                                                                                                                                                                                          1. 4

                                                                                                                                                                                                            I think that statements like “casual dismissal” and “reasonable bug reports” require some kind of example.

                                                                                                                                                                                                          2. 3

                                                                                                                                                                                                            tbf, Lennart Poettering, the person people are talking about here is a very controversial personality. He can come across as an absolutely terrible know-it-all. I don’t know if he is like this in private, but I have seen him hijacking a conference talk by someone else. He was in the audience and basically got himself a mic and challenged anything that was said. The person giving the talk did not back down, but it was really quite something to see. This was either at Fosdem or at a CCC event, I can’t remember. I think it was the latter. It was really intense and over the top to see. There are many articles and controversies around him, so I think it is fair that people take that into account, when they look at systemd.

                                                                                                                                                                                                            People are also salty because he basically broke their sound on linux so many years ago, when he made pulseaudio. ;-) Yes, that guy.

                                                                                                                                                                                                            Personally I think systemd is fine, what I don’t like about it is the eternal growth of it. I use unit files all the time, but I really don’t need a new dhcp client or ntp client or resolv.conf handler or whatever else they came up with.

                                                                                                                                                                                                            1. 4

                                                                                                                                                                                                              tbf, Lennart Poettering, the person people are talking about here is a very controversial personality.

                                                                                                                                                                                                              In my experience, most people who hate systemd also lionize and excuse “difficult” personalities like RMS, Linus pre-intervention, and Theo de Raadt.

                                                                                                                                                                                                              I think it’s fine to call out abrasive personalities. I also appreciate consistency in criticism.

                                                                                                                                                                                                      2. 4

                                                                                                                                                                                                        Why?

                                                                                                                                                                                                        1. 7

                                                                                                                                                                                                          At least because it’s statistically improbable that there are no good ideas in systemd.

                                                                                                                                                                                                          1. 1

                                                                                                                                                                                                            Seems illogical to say projects that use parts of systemd are categorically better than those that don’t, considering that there are plenty of bad ideas in systemd, and they wouldn’t be there unless some people thought they were good.

                                                                                                                                                                                                            1. 2

                                                                                                                                                                                                              Seems illogical to say projects that use parts of systemd are categorically better than those that don’t

                                                                                                                                                                                                              Where did I say that though?

                                                                                                                                                                                                              1. 2

                                                                                                                                                                                                                I prefer to see this type of project that builds upon what it considers the good parts of systemd

                                                                                                                                                                                                                Obviously any project that builds on a part of system will consider that part to be good. So I read this as a categorical preference for projects that use parts of systemd.

                                                                                                                                                                                                        2. 2

                                                                                                                                                                                                          There have been other attempts at this. uselessd (which is now abandoned) and s6 (which still seems to be maintained)

                                                                                                                                                                                                          1. 4

                                                                                                                                                                                                            I believe s6 is more styled after daemontools rather than systemd. I never looked at it too deeply, but that’s the impression I have from a quick overview, and also what the homepage says: “s6 is a process supervision suite, like its ancestor daemontools and its close cousin runit.”

                                                                                                                                                                                                            A number of key concepts are shared, but it’s not like systemd invented those.

                                                                                                                                                                                                            1. 1

                                                                                                                                                                                                              s6 I saw bunch of folks using s6 in docker, but afaik that’s one of most not user friendly software i’ve been used.

                                                                                                                                                                                                        1. 1

                                                                                                                                                                                                          We only needed to change the order of the loops, but the Rust compiler which uses the LLVM backend, did not do it. Maybe the famed sufficiently smart compiler would, but for the foreseeable future, arranging data in a way that it can be processed efficiently will not be the compiler’s job — it’ll be ours.

                                                                                                                                                                                                          Loop interchange isn’t an exotic optimization; there were compilers performing it in the 1980s. LLVM has a loop interchange pass that should do this, I’m not sure why it wasn’t triggered.

                                                                                                                                                                                                          1. 2

                                                                                                                                                                                                            That’s exactly the point. We can’t rely on compilers to do it even when we think they should. But designing the system correctly from the start frees us from having to do the guesswork of hitting the compiler’s optimization passes.

                                                                                                                                                                                                            1. 1

                                                                                                                                                                                                              I suspect that it’s because some optimisations are hard/brittle to “pattern-match” due to the low-level nature of LLVM’s IR (e.g. some front-end might elide important type/shape information when compiling to the IR or some optimisation are simply too costly to track without blowing the compile time). The goal of MLIR is to provide tools that allow creating composable IRs where each IR might contain specific information where optimisation are easy to detect and apply.

                                                                                                                                                                                                              Luckily @david_chisnall often comment here and might shed light on this.