Threads for ferd

  1. 2

    I’m still confused. Isn’t the whole point of with that you want to collapse all the error handling into a single block? Why would you use it if you didn’t want this behavior?

    1. 7

      Yeah the way I’m thinking of it, the problem here is either:

      • the error handling should be general and not necessarily say which call failed or why
      • the error handling should be specific and therefore the with construct’s else block is inadequate

      The article chose to weight on the latter but the argument for the first option is also a possibly valid one if you make it part of your validation contract to return an error reason such as {:error, step, reason} and return these as if they were part of a contract.

      Or alternatively and in the examples of the blog post, have parse_schema(...) return {:error, :schema_unparseable} as a value (and provide a wrapper if it is a library function) so that it can be generically handled.

      But either way, clarity requires a better delineation of where responsibility sits for the translation of more opaque internal validation errors into user-readable content vs. the control flow that is chosen.

      1. 1

        That’s a good way to put it, yeah.

        This is interesting to me because we recently introduced a match-try macro to Fennel with similar semantics to with (but a better name): https://fennel-lang.org/reference#match-try-for-matching-multiple-steps

        I’ve found it extremely useful for situations where you specifically want any failure that might happen in a series of steps all to be handled the same way; maybe they should always print an error message, or maybe they should always just short-circuit and return nil.

        Obviously such a construct can be abused; maybe this is a situation where the problem is that people misunderstand what it’s for, not a problem with the construct itself. (Or maybe the problem is not enough monads; who knows.)

    1. 12

      This is a good reminder that whenever you’re using TLS, you want to make sure to add a case to the test suite that checks whether it will connect to a server where the certs don’t match.

      https://wrong.host.badssl.com/ is a handy tool for this.

      1. 3

        Yep. What’s interesting is that even if we had tests for it (we don’t and tend to mock out Hex to avoid pummeling it with fake requests, but I should add one that pings a loopback host with a self-signed cert), they would need to be specifically for that code path; other uses of TLS (I.e. when fetching code updates or other types of dependencies) are actually properly working. This ends up being more areas to cover than it initially looks like, but would be useful.

        1. 1

          Taking my own advice, it looks like I may have found a bug in maven-resolver (used by Leiningen) in how it validates wildcard certificates! I had manually tested previously with some configurations of bad TLS servers, but the specific misconfiguration on https://wrong.host.badssl.com seems to uncover a new problem we weren’t aware of. (it’s not catastrophically bad, but it really looks like a legitimate if difficult-to-exploit bug to me)

      1. 11

        Huh, hadn’t expected this to pop up here. Thanks for the extra visibility and hopefully the post is informative.

        1. 1

          I remember hex SSL being flakey as hell a few years back…is this related to that?

          1. 2

            Unrelated as far as I know. Might just be the CDN setup that is now better than it used to be.

            1. 2

              Thank you! Also, thanks for all your work in the ecosystem. :)

              Edit:

              For Elixir users, this should be $ mix local.rebar, right?

              1. 6

                Yep. But for mix users there’s generally no risk because they use us for an Erlang project build interface, but take over dependency fetching on their own.

        1. 9

          I literally cannot make sense of this thread at all. Half of the messages are from someone who I would assume is a conspiracy-theory spammer. I could literally replace that person’s messages with things copy-pasted from Time Cube and get something about as coherent. Except other people in the thread appear to interact with that person and a quick glance at the group archive suggests this person is somewhat important to what they do?

          Is this some sort of Poe’s Law parody group that I’m just not deep enough into the material to get?

          1. 15

            The poster you’re talking about is Carl Hewitt. He has become a crank, and he denies various basic facts about lambda calculus, Gödel’s work, and Turing’s work. In this thread, he is complaining about Mark S. Miller’s proposed taxonomy for side channels and similar security issues, a work which simply tries to put names to certain common bug classes. Miller has written papers like this before, and just like how plan interference was first discovered in the context of capability theory, we might hope that other non-obvious bug classes can also be found via similar inquiries.

            The main thrust of my post is to rigorously show that Hewitt is wrong to claim that actors are above and beyond Turing machines; we can indeed compute with actors using standard computers. This is something that he and I have argued about before.

            I have owed this particular proof to the group for a few years; during an academic conference, I responded to an open unanswered question by noting that object graphs behave somewhat like hypergraphs, which can be lifted to categories. However, it wasn’t yet obvious how to mutate those graphs, and so it was an idle curiosity. The rest of the proof just snapped together last night.

            1. 5

              The poster you’re talking about is Carl Hewitt. He has become a crank, and he denies various basic facts about lambda calculus, Gödel’s work, and Turing’s work.

              Don’t argue with cranks, your life is worth more than that

              1. 3

                some people watch TV, some play video games… others, they have less conventional hobbies…

            2. 14

              Carl Hewitt is famous for kinda sorta inspiring the actor model of concurrency, which is used by Erlang and Pony and stuff. In the mid-00’s or so he went completely bugnuts crazy and has since been banned from ArXiv, Wikipedia, and then Wikipedia a second time. However, since he did influential work in the 70’s and 80’s people still think he’s an intellectual giant.

              I especially dislike him because he’s ruined a ton of Wikipedia and C2 pages on CS history. Every year Wikipedia admins find like another two dozen of his sockpuppets.

              1. 1

                went completely bugnuts crazy

                Do you mind sharing some details?

                1. 7

                  He’s been vandalizing Wikipedia pages for the past 15 years with falsehoods like “Gödel was wrong” and “all modern logic programming was inspired by PLANNER” and “Actors are stronger than Turing machines”.

                  If you want to learn more, best place to start is the Wikipedia talk page: https://en.wikipedia.org/wiki/User_talk:Prof._Carl_Hewitt. That doesn’t cover everything; it missed a bunch of pages he vandalized, or that, after having been banned for a decade, he was finally unbanned in October 2016 and then had to be banned again three weeks later.

                  1. 5

                    He gave a talk in Cambridge a couple of years back. I was excited to hear him speak, until about 5 minutes into the talk when it became clear that he was talking complete nonsense. It was pretty uncomfortable listening, he was making assertions that I’d expect any undergraduate to be able to disprove and no one wanted to interrupt because there’s no way of arguing usefully with someone that out of touch with reality.

                2. 5

                  it’s category theory

                  1. 2

                    I’m gonna try just because I have some time on my hands: because of this document people wondered “do we have a formal model of this?” as it would be quite useful to reason about the stuff. Then @corbin (please correct me) linked the “everything is an actor” premises to a category theory notion, there must be a category of actors (he goes on to sketch one) and it might have some properties (can this thing actually compute everything?). A category of actors could be a Turing category and this is where things are confusing, the category of actors as sketched would be typed but we know Turing categories to be untyped, contadiction.

                    Is this some sort of Poe’s Law parody group that I’m just not deep enough into the material to get?

                    I’m gonna go with maybe :)

                    1. 2

                      I think that you picked up everything. The paradox of Turing categories is that all of the different types exist, but also they can be erased.

                      Like, given some object O, and a Turing object A, the arrows O -> A freeze elements of O as code literals. For example, if O were a natural numbers object, then the elements of O might be 42 or 100, and the corresponding elements of A might be "42" or "100". Similarly, the arrows A -> O are like evaluation of code literals, with "42" being sent to 42 and e.g. "anyRandomIdentifier" being sent to failure/partiality.

                      The effect of this is that an actor could both be fully strictly statically typed in the category-theory tradition, and also have the potential to evaluate code literals which could do anything, including representing elements who don’t belong to any type at all. I have mentioned this paradox before and given examples in C++ and Haskell.

                      Another effect, which contradicts Hewitt’s claims, is that actors can be truly Turing-complete; they can diverge.

                    2. 1

                      I’m curious, who? (you can respond privately if you don’t want to say in public)

                      Jonathan Shapiro and Mark Miller are two of the main figures behind capabilites. Shapiro used to be a professor at CMU. Miller was done lots of work related to sandboxing for JS (at google).

                      Carl Hewitt is the guy behind the Actor Model.

                      Lots of Actor model stuff sounds crazy (since it seems to try awfully hard to be it’s own unique thing), but it’s definitely influential on lots of things.

                      1. 12

                        Carl Hewitt is the guy behind the Actor Model.

                        Carl Hewitt is the guy behind the Actor Model of computation, which was a CS dead end. Notably, Steele and Sussman tried to implement it in Scheme and quickly abandoned it as pointless.

                        The Actor Model of concurrency was invented by Gul Agha, one of Hewitt’s grad students. Hewitt’s been taking credit for it ever since.

                        1. 2

                          OK, this plus the “yeah, he really is that sort of crank” stuff makes things make more sense, because I was reading “Actor” as the concurrency model and not getting anywhere from that.

                        2. 4

                          I’m curious, who?

                          It’s pretty clear he’s referring to message like this, which have a typography and writing style with more than a passing resemblance to the Timecube website.

                          This entire “Universal Intelligent Systems” (also see video) thing seems to be missing some points. It looks like a mathematician thinking they can solve problems that are not mathematical in nature with mathematics.

                          1. 3

                            I’ve seen his talks in person. They are much, much worse. But he keeps getting invited to prestigious places because he once did some interesting work. It’s kinda depressing all around.

                            1. 5

                              I’ve seen his talks in person. They are much, much worse. But he keeps getting invited to prestigious places because he once did some interesting work. It’s kinda depressing all around.

                              I’ve been to one of these, and IMO the best part of the talk was when Hewitt got so caught up in his own conspiracy theory rant that he forgot where he was and literally fell off the stage.

                              1. 3

                                I feel bad, but fuck me that’s hilarious. Pure absent-minded professor.

                              2. 1

                                I really think there should be a limit on how much we tolerate cranks in the CS space. I get that some people are contrarian, but still.

                                1. 6

                                  Contrarian implies some level of expertise. In a 2018 keynote I attended, he claimed he solved the halting problem.

                                  1. 1

                                    I’m a contrarian (or maybe a partial crank) in that I think fear of the halting problem keeps us from doing a lot of really useful stuff, but it’s a pretty simple thing to grok, how could it be wrong/solved?

                                    1. 3

                                      I believe his specific claim was “actors can have timeouts, so they’re guaranteed to halt”

                              3. 2

                                Someone should do a psychological study on the people who use colors and typefaces like this

                                https://professorhewitt.blogspot.com/

                                I’m unfamiliar with SSRN.com - it’s an open-access site run by Elsevier?

                                1. 3

                                  SSRN is a pre-print site owned by elsevier.

                          1. 2

                            my own blog’s been a shitty static site generator forever. I also happen to comaintain a build tool where I eventually refactored a DAG for partial rebuilds and used its compiler plugins interface to turn my shitty site generator into a compiler hook and now I get a shitty static site generator that does minimal rebuilds with full dependency analysis across templates.

                            1. 1

                              is the source available? is the build tool rebar3?

                              1. 1

                                Yes and yes: https://github.com/ferd/blog3r

                                I’d still need to re add the demo structure (as a template for rebar3) to make it complete if anyone is interested.

                                1. 1

                                  I’ve added templates and such.

                            1. 2

                              What is the business explanation behind such moves? How can anyone trade a working and reliable system with something that is way more complex to set up and keep functional? Are people really that afraid of Erlang’s syntax and fear of not finding developers? Does one Erlang engineer cost as a team of people managing k8s cluster?

                              1. 5

                                My guess is moving a bunch of independently managed and sufficiently different systems into a generic runtime environment. That way people can use common tools to manage the environment. Nice in some cases, but seems like it discounts the runtime environment of languages, which is a particular advantage of Erlang.

                                1. 3

                                  Another thing is talent aquisition. You’re far more likely to run into a decent Java dev than a decent Erlang dev. (Come to think of it, can you meet an in-decent Erlang dev?)

                                  1. 27

                                    the best hiring decisions I was part of is when we were doing IoT development on BSD devices with Erlang. We assumed nobody would know our stack and had to set up the whole thing from the ground up with the assumption we needed to teach everyone how some things work.

                                    Never been part of a team where we had such an easy time on-boarding people and making them productive in a short time. Turns out it’s much easier to teach the local product and business logic when training each other is part of the team culture, and the benefits extend further than language.

                                    1. 10

                                      I’m sure it’s a bit self selecting too: you get people who are interested in learning about these things so they tend to be more motivated. Get a bunch of motivated people together and you have something special.

                                      1. 6

                                        Sounds like you have the topic of another blog post if you so desire :)

                                1. 4

                                  “hours of rollouts and draining and reconnection storms with state losses.”

                                  I work with a platform that’s mostly built from containers running services (no k8s here though, if that’s important), but the above isn’t familar to me.

                                  State doesn’t get lost: load balancers drain connections and new tasks are spun up and requests go to the new tasks.

                                  When there’s a failure: Retries happen.

                                  When we absolutely have to have something work (eventually) or know everything about the failure: Persistent queues.

                                  The author doesn’t specify what’s behind the time necessary for rollouts. I’ve seen some problematic services, but mostly rollout takes minutes - and a whole code/test/security scan/deploy to preprod/test/deploy to prod/test/… cycle can be done in under an hour, with the longest part being the build and security scanning.

                                  The author also talks about required - and scheduled - downtime. Again I don’t know why the platform(s) being described would necessarily force such a requirement.

                                  1. 13

                                    Here’s one example: the service may require gigabytes of state to be downloaded to work with acceptable latency on local decisions, and that state is replicated in a way that is constantly updated. These could include ML models, large routing tables, or anything of the kind. Connections could be expected to be active for many minutes at a time (not everything is a web server serving short HTTP requests that are easy to resume), and so on.

                                    Rolling the instance means having to re-transfer all of the data and re-establish all of that state, on all of the nodes. If you have a fleet with 300 instances requiring 10 minutes to shut down from connection drains, and that they take 10 minutes to come back up and re-sync their state and come back to top performance, rolling them in batches of 10 (because you want it somewhat gradual and to let times for things to rebalance) will take roughly 10 hours, longer than a working day.

                                    1. 3

                                      I do have some services which work in a similar way, come to think of it - loading some mathematical models and warming up before they’re performing adequately - along similar timescales.

                                      I think we’ve been lucky enough not to be working at the number of instances you’ve described there, or to have fast enough iteration on the models for us to need to release them as often as daily.

                                      For Erlang to be taken out of the picture where it was working nicely doing what (OTP) does best does sound painful.

                                      1. 7

                                        We have something similar, at much larger numbers than described. We cut over new traffic to the new service versions, and keep the old service versions around for 2-4 weeks as the long tail of work drains.

                                        It sucks. I really wish we had live reloading.

                                      2. 3

                                        On the other hand, the article mentioned something like “log in to repl and shoot the upgrade” seems like a manual work. I would think that having 1 hour of manual work over 10 hours of automated rollout have different tradeoffs.

                                        As for the fleet shutdown calculation, you can also deal with that differently. You can at the very least halve the time by first bringing the new instances up, then shutting down the old ones, so your batch doesn’t take 20 minutes, but 10. If you want to “let times things for things to rebalance”, you still have to do that in the system that you described in the article.

                                        Now, I’m not saying that I didn’t agree a lot with what you wrote there. But I did get a wibe that seem to be talking down on containers or k8s, and not comparing the tradeoffs. But mostly I do agree with a lot of what you’ve said.

                                        1. 1

                                          You can at the very least halve the time by first bringing the new instances up, then shutting down the old ones, so your batch doesn’t take 20 minutes, but 10.

                                          That doesn’t sound right. It takes 10 minutes to bring up new instances and 10 minutes to drain the old ones them, at least that’s my understanding. Changing around the order of these steps has the advantage of over-provisioning such that availability can be guaranteed but the trade-off is (slightly‽) higher cost short-term (10h in that example). Doing these 2 steps in parallel is of course an option and probably what you suggest.

                                    1. 14

                                      Nice article, ferd. One thing that resonated with me was

                                      To me, abandoning all these live upgrades to have only k8s is like someone is asking me to just get rid of all error and exceptions handling and reboot the computer each time a small thing goes wrong.

                                      I think this stems from how complex software has gotten and the myriad of dependencies that an average developer/team has to deal with. More often than not it’s just simple to pull the rug underneath and start afresh. Throw in some redundancy there and another layer of abstraction via a LB and your clients are mostly unaware of the shit that you’re dealing with. Where all of this breaks down is when this pattern is being applied to workloads where it makes no sense and you need fine-grained control. At this point, I feel k8s has become like a hammer and developers/evangelists are hoping to change every problem as a nail to use k8s on it.

                                      PS: Was the hot reload thing at Heroku? :D I’m at Heroku now and I wish I knew more erlang/gotten a chance to work with you.

                                      1. 10

                                        The first story was on Logplex, which I think finally got taken down / replaced earlier this year after the 3rd or 4th attempt by people to write its Go replacement (though I do not have first-hand reports on its demise). The routing layer never had live code updates during my time there: although it could have benefited from it, the imperative to preserve transient state was much lower, and instead we did rolling restarts with a disk cache and without node replacements on there (full node replacement came with CVEs, more major security updates, or unexplained bad performance where rotating to a new instance made sense as a resolution).

                                        1. 4

                                          I miss classic logplex, that thing could take a surprising amount of punishment and keep on smiling

                                          1. 6

                                            I learned so much by working on Logplex with Ferd; in particular by observing how well the early design decisions allowed it to grow and gracefully respond to whatever we threw at it. That’s probably the most I’ve ever learned at any job in the shortest time.

                                            1. 2

                                              Oh cool you’re ex herokai too?

                                              1. 1

                                                Yeah, 2011-2014; worked on codon/git, buildpacks, and then logplex at the end.

                                                https://github.com/heroku/logplex/commits?author=technomancy

                                      1. 4

                                        I read the title and thought “this is such a bad take.” I read the post and thought it made some sense.

                                        I’m taking from this that the author is considering navigation and menu items as “content” - this seems like the wrong term entirely. I think this was my initial confusion.

                                        I think of “content” as the actual meat of a blog post, page, article or even comment. Usually paragraphs or sentences, etc. Controlling uppercase with CSS seems bonkers in this context. Those writing content (as I know it) should not ever have to worry about CSS, it’s a different concern.

                                        While the author does acknowledge it’s okay to uppercase when it makes semantic sense, the title terminology says something different. Or maybe I’m crazy and “content” just means anything that is displayed on the screen?

                                        1. 2

                                          A very common case for all-caps content is going to be stuff like section titles within text. Other ones could include names in citations, figure descriptions, certain types of emphasis for technical terms, and so on.

                                          These often come down to stylistic differences in a way where a new designer reworking your choices of typefaces or design could decide that something now needs small caps, to be title-cased, and so on. For these, you want CSS to be handling things, otherwise a change in design means you now need to re-edit all your content to work with it.

                                          Semantic differences would imply that moving from uppercase to lowercase (or from lowercase to all caps). For example, “getting your PIN stolen” has a different meaning from “getting your pin stolen.” This example is also partly why you can often pick a small-caps font variant, which gives you the uppercase look without impacting the meaning.

                                        1. 4

                                          I found that generally, w3m was a much better and more compatible experience than whatever links provides.

                                          1. 1

                                            When doing something like this on a production system, are there any dangers or extra load it puts on the VM?

                                            1. 5

                                              Yes. This is why libraries such as recon have been written. See Erlang in Anger for more content on debugging production systems.

                                            1. 1

                                              I would be interested in hearing the author’s take on the RuboCop example that was discussed on here a week or so ago. That one seems to be an interesting borderline case in some respects.

                                              1. 1

                                                Personally I think changing the project name is probably a lot of work and one of the compromise routes would be the kind of thing I look into. That being said different people might have different tolerances to the amount of work that is acceptable there (we’ve seen projects be renamed after forks and it worked obviously) and I don’t want to speak for other maintainers

                                              1. 8

                                                True, but the real argument is about unnecessary complexity. I’m reminded of the syscall comparison graph between Apache and IIS https://www.schrankmonster.de/2006/06/12/iis6-and-apache-syscall-graph/

                                                1. 3

                                                  I’ll stick to my point: Accidental (unnecessary) complexity is just essential complexity that shows its age.

                                                  It would be a mistake to consider the software independently from the people who have to write and maintain it. It doesn’t exist in a vacuum. The challenges in understanding the complexity and in people working around it (leaving behind messy artifacts and bits of code) are not really avoidable except in hindsight, once people tried things, shipped some, found mistakes, and course-corrected.

                                                  At the time these things were thought to be necessary to accomplish a task within the constraints imposed on developers and their experience. It would be misguided to think that this can be avoided or prevented ahead of time. They leave traces and it’s only because of that work and experience gained over time that we can eventually consider the complexity to be unnecessary.

                                                  1. 7

                                                    are not really avoidable except in hindsight, once people tried things, shipped some, found mistakes, and course-corrected.

                                                    This is either trivially true, or empirically false. Trivially true: If you replay history with the exact same set of starting conditions. Empirically false: Why can some teams produce software of greater simplicity and higher quality than others? Why can the same team, after training, or other changes, produce software of greater simplicity than they previously had?

                                                    The argument you’re making feels both defeatist and responsibility-shirking.

                                                    1. 2

                                                      It’s actually a call for more responsibility. We need to care for it, train people, and see complexity as something to be managed, adapted to, and not just something we can hope to prevent or push outside of our purview.

                                                      1. 7

                                                        You explicitly argue the seeking simplicity is misguided.

                                                        Another quote:

                                                        A common trap we have in software design comes from focusing on how “simple” we find it to read and interpret a given piece of code. Focusing on simplicity is fraught with peril because complexity can’t be removed: it can just be shifted around.

                                                        This simply isn’t true. It happens all the time that someone realizes that a piece of code – their own or someone else’s – can be rewritten in a drastically simpler way. A shift in point of view, an insight – they can change everything. The complexity you thought was there can just vanish.

                                                        What I take issue with is that you seem to be explicitly arguing against this as a goal.

                                                        1. 3

                                                          The shift in point of view often is just people learning things. It’s knowledge moving from the world and into your head. To understand the code that uses that shift in perspective, you have to be able to teach that perspective. You shifted the complexity around. Picasso’s bulls get simpler and simpler, but only retain their meaning as long as you have the ability to mentally connect the minimal outlines with the more complete realization inside your head.

                                                          It would be a mistake to think that because the code is simple, the complexity vanished. Look at books like “Pearls of Functional Algorithm Design”, or at most papers. The simplicity of some implementations (or even their use) is done by having people who are able to understand, implement, and ultimately handle the complexity inherent to problems and their solutions. People can pick up the book, see the short code, and never get why it works. But it is “simple”.

                                                          You can’t take the software in isolation from the people working on it. Without them, no software gets written or maintained at this point.

                                                          1. 8

                                                            You can’t take the software in isolation from the people working on it. Without them, no software gets written or maintained at this point.

                                                            You are saying this as if it’s a rebuttal to something I’ve said, when in fact I agree with it. Yet I still find the way you are framing the rest of the argument to be either counterproductive or factually wrong.

                                                            The shift in point of view often is just people learning things. It’s knowledge moving from the world and into your head.

                                                            Sure, I guess you can say it like that. I don’t what to make of it until I see what you plan to do with this framing…

                                                            To understand the code that uses that shift in perspective, you have to be able to teach that perspective.

                                                            Sometimes. Sometimes not: Often you can refactor code in a way that is simpler and that the original author immediately sees, with no teaching. In fact, they may exclaim something like: “Ah, that’s what I meant! I knew I was making it too complex!” or “Ah, I’d forgotten about that trick!”

                                                            All of these are common possibilities for what happens when you make such a change:

                                                            1. The change has no cost. It’s simply free and better.
                                                            2. The code is simpler if you understand some new concept first. So there is a teaching cost. And perhaps a documentation cost, etc.
                                                            3. The code gets shorter and superficially simpler, but hasn’t actually improved. You’ve moved stuff around. 6 of one, half a dozen of the other.
                                                            4. The code gets shorter and superficially simpler, but has actually gotten more complex. What you thought was a win is in fact a loss.

                                                            It seems to me that your argument assumes we are always in case 2. Or rather: That case 1 isn’t possible.

                                                            Maybe my real question is: What do you want people to do with that argument?

                                                            What I want people to do with my argument is to learn more, press harder for simpler solutions, have humility about their own designs, realize that there’s very often a better solution than the one they’ve found, and that this remains true almost no matter how experienced you are. I think if you understand that and take it seriously, and are on a team where everyone else does too, good things will happen.

                                                            1. 0

                                                              It seems to me that your argument assumes we are always in case 2. Or rather: That case 1 isn’t possible.

                                                              I mean, there are surely cases where 1 is possible and the change has no cost, but that would usually happen when there is already enough complexity absorbed in program rules to deal with it (i.e. a linter that changes things for you), or by the people who do the work. I.e. knowing that 1+1+1+1 can be replaced by 4 is almost always a better thing and it’s so trivial even most compilers will do it without fear.

                                                              Case 2 is what I hint at with the assumption that you need to understand new concepts or new perspectives. I.e. if you go and gather statistics and find out that 95% of cases hit in a certain way, you can afford to re-structure things differently. It may not have changed much to the algorithm, but there’s a need for new awareness.

                                                              Cases 3 and 4 are more particularly interesting, and far more likely when you start working on systems at large. Changing an ID from an integer to a UUID should be self-contained (and help do things like shard data sets). But if an external services uses integer IDs to sort on them, you’ve potentially broken an unspecified contract (was the ordering ever intended or only used accidentally?)

                                                              You can make assumptions that “you must be this tall to ride”, in which case all of these ‘complexity’ cases are assumed to be self-evident, need little validation or maintenance. But that’s an assumption you make based on what? Probably personal experience, hiring standards, people self-selecting? That’s a tricky classification to make, and can be very subjective. That’s why I’m thinking of defaulting to embracing the concept of complexity.

                                                              Sure you can make the case that everyone should be fine with this change, it is equivalent, you can demonstrate it to be the same, get someone to approve it, and move on. Or do it through an automated tool that has accepted these transformations as safe and worthwhile.

                                                              But an interesting aspect there is to always consider that there could be fallibility in what we change, in what we create, and that communication and sharing of information is a necessary part of managing complexity. If I assume a thing is category 1 as a change, but it turns out that in communicating it I show a new perspective that hits category 2 for a coworker (maybe I don’t know them that well!), aren’t we winning and improving? Knowledge management is part of coping with complexity, and that’s why I make such a point of tying people and what they know, and why I’m so hesitant to clearly delineate essential and accidental knowledge.

                                                              1. 3

                                                                Case 2 is what I hint at with the assumption that you need to understand new concepts or new perspectives. I.e. if you go and gather statistics and find out that 95% of cases hit in a certain way, you can afford to re-structure things differently. It may not have changed much to the algorithm, but there’s a need for new awareness.

                                                                Or you can realize that a problem has a better way of being posed. For example, a common interview problem is validating a binary search tree. You can do this bottom up, propagating the validity of the binary tree, but the book-keeping needed to make this approach is hard and fragile, and you’re looking at probably abotu 50 lines of code to get it right.

                                                                Or you can think of placing the nodes of a tree on a number line, and the solution becomes 5 lines of code.

                                                                Neither conceptualization is complex, and the one that solves the problem better is probably simpler to most people, it just takes a bit of thought to find the right isomorphism.

                                                                I find that similar effects happen in large systems, where entire subsystems become unnecessary because of a good engineering choice at the right time. An example of this may be standardizing on one data format, instead of having different systems connected using protobuf, xml, json, csv, thrift, and. This eliminates a huge amount of glue code, as well as the bugs that come when you need to deal with the representational mismatches between the various formats chosen.

                                                                I don’t buy your nihilism.

                                                                1. 0

                                                                  So, once you’ve made that choice that saves a lot of complexity, how do you make sure the other engineers that follow after you work in line with it? How do you deal with the folks for whom JSON no longer works well because they want to transfer bytes and the overhead of base64ing them is too much?

                                                                  You have to communicate and maintain that knowledge, and it needs to be actively kept alive for the benefits to keep going (or to be revisited when they don’t make sense anymore)

                                                                  1. 3

                                                                    So, once you’ve made that choice that saves a lot of complexity, how do you make sure the other engineers that follow after you work in line with it?

                                                                    Are you seriously arguing that software can’t be maintained?

                                                                    Honestly, the way you’re writing makes me worry that you need to talk to someone about burnout.

                                                                    1. 1

                                                                      No, it’s rhethorical. Again the whole point is that complexity is fundamental and part of the maintenance and creation of software. The whole thing relates to the concepts behind complex adaptive systems, and again most of the thing is that the complexity doesn’t go away and remains essential to manage, which is more useful than ignoring it.

                                                                      1. 2

                                                                        If it’s fundamental and can’t be reduced, then why bother managing it? It’s going to be there no matter what you do.

                                                    2. 3

                                                      Accidental (unnecessary) complexity is just essential complexity that shows its age.

                                                      Your description is true if you assume that the developers intention is 100% aligned with developing the software as good and correct as possible - which might be closer to the case on a project like the development or rebar3.

                                                      But on a project like developing a boring CRUD system and being paid for it in a 9-5 - this is farther from reality.

                                                      Developers also are working in an economic framework, where certain skills are more valuable than others and they are trying to include those to their resumes to obtain higher paying, higher status or simply more interesting roles. They might also simply be bored and resent their managers and product managers and not be entirely motivated to look for the simplest solution possible, where many times that solution is on detriment of their own career.

                                                      In these situations, engineers will bring unnecessary complexity to the system, it is not the same as the unavoidable fact that some code has mistakes, have to be adjusted, have to be course-corrected and fixed, I agree this is essential complexity.

                                                      But there’s also complexity that goes beyond the software/problem at hand, they are also human, but they’re from the economical and social interactions of the people who develop that system. And I find that calling this essential is not right, because this type of economic relationship, this type of interaction, is not essential to software development!

                                                  1. 6

                                                    I want to agree with a more limited scope of this, that software has to deal with complexity from the business domain eventually and either you do or you narrow your focus until you’ve ignore swathes of the business domain.

                                                    Unfortunately, the full claim (at least as articulated here) also seems to hand wave away shitty developers and bad engineering and inexperience as another form of complexity. While I can kinda see that argument–e.g., that you have to account for your resources in writing software and that failure to do so will leak into the finished project as emergent complexity in implementation instead of developer training–it seems to me to both be too easily misunderstood and to go in the face of experience many of us have with software that is just obviously too complicated (to wit, enterprise FizzBuzz) for what it is trying to do.

                                                    1. 1

                                                      I think the most contentious part of the post is that I just simply assert that people are an inherent part of software. You often avoid the incidental complexity in code by indirectly shifting it to the people working the software.

                                                      Their mental models and their understanding of everything is not fungible, but is still real and often what lets us shift the complexity outside of the software.

                                                      The teachings of disciplines like resilience engineering and models like naturalistic decision making is that this tacit knowledge and expertise can be surfaced, trained, and given the right environment to grow and gain effectiveness. It expresses itself in the active adaptation of organizations.

                                                      But as long as you look at the software as a system of its own that is independent from the people who use, write, and maintain it, it looks like the complexity just vanishes if it’s not in the code, and can be eliminated without side effects.

                                                      1. 9

                                                        I think it’d be easier for people to have got that if you’d built the case for simple software and then explored the people side of “okay, sure, the software is simple, but how’d we get there”.

                                                        The problem with your current framing is that it seems to boil down to this conversation (:

                                                        • Me: “Software is overly complicated, we can make it simpler.”
                                                        • Thee: “No we can’t, only complexity can manage complexity!”
                                                        • Me: “Preeeeettttty sure we can. A lot of code is clearly overcomplicated for what it does. <example of, say, too much ceremony around adding a seconds offset to a time and then printing it in UTC>”
                                                        • Thee: “The code is simpler, yes, but at what cost? Can we not say that the training required to prevent that engineer from overcomplicating a time routine is itself a form of complication? Can any among us not indeed be considered but spirited complications upon the timeface of software development?”
                                                        • Me: “If we claim that all complexity in software can only be removed at the increase of complexity elsewhere, I find this conclusion uniquely unsatisfying, somewhat like the laws of thermodynamics.”
                                                        • Thee: “Indeed. Life is often unsatisfying, and one might even say, complicated.”
                                                        • Me: “…”
                                                        • Me: “I’m not going to reward that joke. Anyways, it’s easy to construct (or, sadly, to recount) cases where software was simple and then, for literally no reason other than it seemed like a giggle, somebody made it less simple without appreciable increase in value or functionality. This strikes me as not as an issue of complexity but one of poor decisions or ignorance.”
                                                        • Thee: “Dealing with poor decisions and ignorance is core to the human part of software, and thus is a complication core to software itself.”
                                                        • Me:” Right, and I kinda agree, but if we recast all malice, poor decisions, or ignorance as complexity then we might as well define complexity as anything that makes life harder than we want it. And while I am attracted to that as a way of correlating with Evil, I also think that such a redefinition of complexity conflates it with other forms of things that make life harder than we want it. Ergo, I disagree with your overly-broad usage of complexity.”
                                                        • Me: “…Erlang in anger is still a great read though, as is Learn You Some Erlang. <3
                                                        1. 2

                                                          Point taken on the article being possible to word more clearly.

                                                          The cybernetics argument for the law is that the inherent complexity you are able to represent and model in your mind is what lets you extract and cancel some of the complexity.

                                                          If you want to make a long story short, part of my blog post would probably be a neighboring idea to “if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it” – if you had to be very clever to simplify the code and reduce the problem to a simpler expression, you’ll have to have that cleverness around to be able to update it the next time around. I.e. we have to embrace that aspect of complexity as well if we want the overall enterprise to keep moving and evolving sustainably.

                                                          Let’s pick the time example you introduced, maybe the example is more workable.

                                                          Imagine the risk of simplifying too much, for example, where instead of adding a seconds offset to a time in UTC, you decide to only handle UTC, and since you only have UTC, you don’t show the timezones anymore. Also since you do time calculation, you can make it a lot simpler by using the unix epoch (ignoring leap seconds) everywhere.

                                                          This essentially forces the complexity onto the user, at the benefit of the developer. Clearly, aiming for code simplicity isn’t to anyone’s benefit once we increase our scope to include the user.

                                                          If you shift it around and always work with user-level timestamps (so it always works aligned with what the user wants), then you have to fight a lot harder when you want to calculate time differences of short duration within your codebase’s internals. This will increase the complexity of code in “unnecessary” manners, but encourages alignment with user needs throughout.

                                                          You might decide that the proper amount of complexity in code results from using both timestamps in a human referent format (year/month/day, hours, minutes, seconds, microseconds) in some cases, and then monotonic timestamps in other cases. This is a trade-off, and now you need to manage your developers’ knowledge to make sure it is applied properly within the rules. Because this higher variety sometimes increases code quality, but can lead to subtle bugs.

                                                          I think embracing the complexity of the real world is the way to go, but embracing it also means managing it, investing in training, and considering the people to write the software to be first order components. In some cases, it might be worth it to have messy ugly code that we put in one pile because it lets everyone else deal with the rest more clearly. We might decide “all the poop goes in this one hole in the ground, that way we know where it isn’t”, and sometimes we might decide not.

                                                          Also I’m glad you enjoyed the books :) (incidentally, how much should be “discoverable” in the tool, and how much should be relegated to the docs is a question in managing complexity, IMO)

                                                          1. 2

                                                            if you had to be very clever to simplify the code and reduce the problem to a simpler expression, you’ll have to have that cleverness around to be able to update it the next time around.

                                                            This ignores the fact that many times simplifying code has nothing to do with cleverness and everything to do with just wanting less code. Folding a bunch of array assignments into a simple for loop isn’t clever, for example–and if you argue that it is clever, then I fear that we’ve set such a low bar that we can’t continue to discuss this productively.

                                                            Clearly, aiming for code simplicity isn’t to anyone’s benefit once we increase our scope to include the user.

                                                            There’s nothing clear about that. It benefits the implementing engineer, it benefits the debugging engineer, and it benefits the computer that runs it. Your example about making the code as simple as possible until it’s just delivering a Unix timestamp (instead of the UTC string) is a red herring because the original deliverable spec was ignored.

                                                            1. 3

                                                              This ignores the fact that many times simplifying code has nothing to do with cleverness and everything to do with just wanting less code. Folding a bunch of array assignments into a simple for loop isn’t clever, for example–and if you argue that it is clever, then I fear that we’ve set such a low bar that we can’t continue to discuss this productively.

                                                              No I’d probably argue that it isn’t significant. Probably. On its face, it’s such a common refactoring or re-structuring that anyone with the most basic programming knowledge has already internalized enough information to do it without thinking. It fades back into the background as mostly anyone with basic knowledge is able to conceptualize and correct that.

                                                              I figure it might have been a trickier thing to do with assembler or punch cards (would it have even been relevant?) It might play differently with some macros or post-processing. Or changing to a loop folds the lines and reduces code coverage statistics to the point where a build fails (I’ve seen that happen). Or a loop implies new rules when it comes to preemptive scheduling based on your runtime, or plays with how much optimization takes place, etc.

                                                          2. 2

                                                            I love this reply. I’d just add that there is definitely no conservation law for “things that make life harder than we want it”.

                                                            It might be that we’re often living in optimization troughs that look locally flat, but when that’s true it’s usually because those are equilibria that we’ve spent effort to choose, relative to nearby possibilities that are worse.

                                                            It’s dangerous to say “everywhere we look, there are no easy wins, so we should just give up on making improvements”. To use an annoyingly topical analogy, it’s like saying “see? The coronavirus isn’t spreading anywhere near it’s initial exponential rate! We didn’t need to socially isolate after all!”

                                                      1. 34

                                                        Accidental complexity is just essential complexity that shows its age.

                                                        This is dangerously wrong, and I see the unimpeachable evidence that it’s wrong every day.

                                                        A ton of complexity comes from developers choosing suboptimal designs and overly complex tools. Two people writing the same software with the exact same set of requirements, edge cases, weird assumptions, etc, can arrive at designs with radically different complexity.

                                                        So while yes, there is essential complexity that has to live somewhere, the high-order bit is usually the complexity of the design, and the skill of the designer matters a lot.

                                                        From one of the author’s own posts:

                                                        It’s one small piece of data-centred functional design, where thinking a bit harder about the problem at hand greatly simplified what would have been a more straightforward and obvious implementation choice in a mutable language.

                                                        The other problem with the POV being argued for here is that, even while it contains some truth, as soon as it accepted it is used to justify shoddy work as “inevitable.”

                                                        1. 4

                                                          Would the two developers also have the same knowledge, same environment, context, and time pressures? Ignoring these is ignoring fundamental aspects of software design.

                                                          The idea is not that the shoddy work is inevitable, but that complexity is inevitable. You can’t control it, but you can manage it. The previous article of mine you referred to in terms of functional design is one example of doing work on an implementation that was slower (both to write and to execute), less simple/obvious in approach (it needed more documentation and training for newcomers), and was conceptually more complex and demanding, but ended up composing better to simplify some amounts of maintenance (the logical separation and how change sets would need to be applied in the future).

                                                          If we had been under intense time pressure to ship before losing major contracts? Taking the time to do it right the first time would have definitely been the wrong way to go about it. If we ever had the need or shifting requirements for logs “as soon as the request is received” instead of once it is over? We would have been retrospectively wrong. You put in the time sooner or later, and context matters. It’s why I say accidental complexity is just essential complexity that shows its age.

                                                          1. 11

                                                            but that complexity is inevitable.

                                                            Unnecessary, accidental complexity isn’t inevitable as illustrated by those that avoid it in their work. It’s a cultural and/or educational problem. An example is language compile times. C++ has a design that makes it really, really hard to compile. The Lisp’s and D can do either more than or as much as C++ in terms of features. Yet, they compile way, way faster since they were designed to be easy to compile. In different ways, they eliminated accidental complexity that hurt them on that metric.

                                                            Another example is probably SystemD. I’ll say ahead of time I’m not a zealot in any camp in that debate. I just have seen simple things to run in privileged processes, user-mode components for system management, and kernel hooks for components like that to use. The SystemD solution was a massive pile of code running with high privilege. We have decades of evidence that is bad for maintenance, reliability and security. Yet, they did it anyway for who knows what reason. Meanwhile, the modular and isolating design of QNX kept those systems just running and running and running with changes being easier to make with less breakage when they made them.

                                                            On usability side, I always thought it was strange how hard it was to set up most web servers. If one was easy, you’d have to worry about its security or something. Supporting new standards just added more complexity for both developers and users. Then, two people decide to build a usable one, Caddy, in a memory-safe language supporting modern standards. The advanced features are there if you need to put time into them. Otherwise, install, config, HTTPS, etc is super easy compared to prior systems I saw. It’s super-easy because the developers intentionally eliminated most of the accidental complexity in setup and configuration for common usage.

                                                            So, it just looks like much of the accidental complexity isn’t inevitable. Developers either don’t know how to avoid it or are intentionally adding it in for questionable reasons. If it’s inevitable, it’s for social reasons rather than anything inherent to complexity. Maybe the social sciences, not complexity science, need to be responsible for studying how to combat accidental complexity. Might get further than the tech people. ;)

                                                            1. 4

                                                              Maybe the social sciences, not complexity science, need to be responsible for studying how to combat accidental complexity.

                                                              I believe that this is much more true than commonly acknowledged. There is so much to gain by having insight in basic rules of how people behave and work together. Currently I feel like we are completely in the dark. 30 years from now, I imagine that the main topic in software development will not be which language or IDE to use but how to optimize circumstances, incentives and relationships for the most stable code.

                                                              1. 1

                                                                I think you’re right about the former, but in 30 years the incentives that produce the current obsessions will not have changed and neither will the willingness to ignore human factors while producing crap.

                                                            2. 8

                                                              “People often don’t have time to do a good job” isn’t the same thing as “Complexity is unavoidable”.

                                                              1. 4

                                                                “Resources are finite and decisions must be made accordingly” is how I would frame it.

                                                                1. 8

                                                                  Which implies that complexity is optional, and people often chose to invest elsewhere – and are then surprised when making changes is hard due to poor choices.

                                                                  1. 5

                                                                    Software isn’t written in a vacuum by robots with infinite time and energy, in an immutable environment that never evolves (nor get impacted by the software that contains it). There’s a reason most of us don’t use formal proofs to write blog engines. The complexity is an inherent part of development, the same way juggling between finite resources is not avoidable. You can’t de-couple the programs to be written from the people who will write it. That just means you’re working from a poorer model of things.

                                                                    1. 6

                                                                      You seem to have completely changed arguments. Capitalism and the resulting market pressure may breed complexity, but that’s a rather different conversation.

                                                                      1. 5

                                                                        No. Complexity can control complexity, as mentioned in the post via the law of requisite variety. To simplify software, you still have to add/shift complexity; a shift in perspective is asking the devs to gain better understanding, to absorb (and carry) that complexity in their heads to simplify the program. Someone, somewhere, whether in code or in their mind, needs to establish a more complete and complex model of the world.

                                                                        The only difference between an implementation that was done fast as required to work and an implementation that was cleaned up and made simpler is that as an organization (or as an engineer) is that we have taken the time to gain expertise (transfer knowledge “into the head” to reuse terms from the post) to extract it from the program, clarify it, and re-inject a distilled form. In order to maintain the software effectively, we have to maintain that ability to control complexity, that fancier model which now resides in people. If you don’t do that, the next modifications will also be full of “accidental complexity”.

                                                                        If you write simple software without necessarily handling complex things (i.e. there is no gain in complexity in the entire development chain whether in code or in people), the complexity gets passed on to the users, who now have to adjust to software that doesn’t handle complex case. They learn to “use the software the right way”, to enter or format data in a certain way, to save before doing operations that often crash, and so on. They end up coping with what wasn’t handled anywhere before it made it to them.

                                                                        I maintain that complexity just shifts around.

                                                                        1. 3

                                                                          I think I understand your claim. However, it’s not clear to me, once you expand the term ‘complexity’ to encompass so much, what that buys you. Could you give an example where somebody thinking about reducing complexity would act differently from somebody “embracing it, giving it the place it deserves, focusing on adapting to it”?

                                                                          Edit: Ah, looks like @jonahx asked the same thing.

                                                                2. 2

                                                                  I also think that “modern” software has a fetish for features, and, indirectly, for complexity.

                                                                  When I was programming in qbasic I could draw a graph in literally four lines of code.

                                                                  Now in any “modern” environment this will be really hard. I could probably work it out in C#.NET, but it will take me at least half an hour. Then sure, it will have a little more than the qbasic program (a window around it, which may be resizable, a lot more colors, and maybe double buffering), but none of that was a goal; I just wanted to draw a graph.

                                                            1. 4

                                                              I think complexity is sometimes (read: a lot) confused with knowledge imbalance.

                                                              X thing looks complex, because you don’t understand the set of concepts Y within it.

                                                              Many Xs look complex in software because they touch on many domains of Ys. Software intersects everything.

                                                              This is why it’s important to use common language, and if you can’t, explain it “like they are five” as people usually say. I think a better phrase is explain it like they know nothing about the domain.

                                                              You don’t need to simplify explanations; you need to define everything using common language.

                                                              1. 2

                                                                This is, I think, the difference between “knowledge in the world” and “knowledge in the head”. The argument you’re making here when it comes to defining things in a common language is essentially shifting the complexity of understanding the software as it evolved into every developer’s personal mind and knowledge. You have to be aware of how you define the language and frame things, the same way you tend to be aware of how you define programs.

                                                                Doing this well implies that you should at least be aware of this complexity, and prepare your organization for it. You’d want, for example, to provide proper training and on-boarding for people coming in, to make sure they can gain that knowledge if they don’t have it, and possibly do this even with regulars to make sure the knowledge hasn’t eroded. This can more or less be compared to doing things like incident simulations or chaos engineering, but with a focus on the domain knowledge required to maintain the software.

                                                                Shifting the complexity to people understanding things and then not caring about that complexity anymore isn’t necessarily a lot more responsible than doing the same in code. It’s just less visible.

                                                                1. 1

                                                                  1st paragraph

                                                                  What are we saying different here? From what I understand you are re-iterating my knowledge imbalance point.

                                                                  2nd paragraph

                                                                  100% agree with that’s why training and documentation are a must.

                                                                  3rd paragraph

                                                                  I didn’t mean to imply shifting anything, in fact I meant the opposite, in fact I mean almost precisely as you’ve written it heh.

                                                                  Overall, I agree.

                                                                  1. 3

                                                                    I see the agreement better now; I think I was over-focusing on “complexity is confused with knowledge imbalance” whereas I more or less conceptualize knowledge imbalance as an inevitable facet of complexity. In practice, both definitions are close to equivalent.

                                                              1. 2

                                                                https://ferd.ca – close to 10 years blogging now, mostly about functional programming stuff, maintenance, and dev practices, with an obvious big slant on erlang things.

                                                                1. 14

                                                                  Error handling also causes repetition. Many functions have more if err != nil { return err } boilerplate than interesting code.

                                                                  Whenever I see return err, I see a missed opportunity. Every error return is a chance to add additional context to the error, stringing together the exact sequence of events leading to the error directly into the error message. Done well, you end up with a lovely semantic “stack trace” that completely identifies the situation leading to the error.

                                                                  You could have logs full of ERROR: connect timed out, or you could have:

                                                                  ERROR: failed to zing the bop "abcd": failed to fetch bibble diddle: failed to initialize HTTPS connection to "https://bleep-bloop.domain": timed out waiting for DNS response

                                                                  1. 7

                                                                    yes, returning an error without wrapping it is, nine times out of ten, Doing It Wrong. At my company we have a package that is similar to this one that contains various tools for manipulating and handling errors: https://github.com/pkg/errors

                                                                    also, after 8 years of programming Go, I strongly dislike stack traces now. Stack traces do not tell me a story of how an error happened, they give me homework of how to divine that story by reading the source code and running the program in my head. If you don’t have the source code, or if you’re running many versions of many programs, the utility of the stack trace further decreases. My Go code consistently has the best error-handling semantics of any code that I actually put into production.

                                                                    1. 6

                                                                      That’s just assembling a stacktrace by hand.

                                                                      1. 3

                                                                        I’m not a go programmer, so a stupid question: How would the error handling code look then? Like this?

                                                                        return err, "timed out waiting for DNS response"
                                                                        

                                                                        Or something more complex? Would this affect the function signature of everything in the call chain?

                                                                        1. 12

                                                                          Go 1.13 added error wrapping, so you can now do this:

                                                                          return fmt.Errorf(“timed out waiting for DNS response: %w”, err)
                                                                          
                                                                          1. 3

                                                                            that’s been around since 2010; it didn’t actually take the Go team a decade to come up with that.

                                                                            https://github.com/golang/go/commit/558477eeb16aa81bc8bd7776c819cb98f96fc5c1

                                                                            1. 7

                                                                              The %w is what’s new in 1.13, permitting e.g. errors.Is.

                                                                              1. 4

                                                                                ah! Nice! Yeah, that’s a useful improvement. Wasn’t clear from the comment before how 1.13 changed it; I thought trousers was saying that 1.13 added fmt.Errorf. Thanks for the clarification :)

                                                                            2. 2

                                                                              In addition, depending on the case where this return is located, the “time out” info may already be included in err, so it might even be potentially removed from the error message; personally, I recently start to lean towards, in each function, including what “extra” info this function can add to the error, about context it controls, and the function itself; so, I might for example write instead something like:

                                                                              return fmt.Errorf("waiting for DNS response: %w", err)
                                                                              

                                                                              or maybe even:

                                                                              return fmt.Errorf("retrieving DNS record %q: waiting for server response: %w", recordName, err)
                                                                              

                                                                              This is based on how errors in stdlib are composed, e.g. IIRC an error from os.Open would often look like: "open 'foobar.txt': file not found"

                                                                          2. 2

                                                                            I haven’t yet used the “new” errors packages (e.g., github.com/pkg/errors) in anger yet. How do they work with respect to checking if an error in the chain was of a specific type (e.g., os.IsNotExist() or io.EOF or converting to a specific type)?

                                                                            1. 2

                                                                              errors.Is(err, os.ErrExist)

                                                                              There are some other helper functions so that you can quickly handle wrapped errors.

                                                                          1. 5

                                                                            In most async systems … you end up in a world where you chain a bunch of async functions together with no regard of back pressure.

                                                                            yup. Back pressure doesn’t compose in the world of callbacks / async. It does compose if designed well in coroutine world (see: erlang).

                                                                            async/await is great but it encourages writing stuff that will behave catastrophically when overloaded.

                                                                            yup. It’s very hard, in larger systems impossible, to do back pressure right with callbacks / async programming model.

                                                                            This is how I assess software projects I look at. How fast is database is one thing. What does it do when I send it 2GiB of requests not reading responses? What happens when I open a bazillion connections to it? Will a previously established connections have priority over handling new connections?

                                                                            1. 6

                                                                              Even in the Erlang world, there are technical risks at every point where you reintroduce asynchrony. See https://ferd.ca/handling-overload.html for a long post on all the ways you can find to handle overload in the Erlang ecosystem, from the simplest one to more complex systematic approaches.

                                                                              1. 2

                                                                                In Monte, where I/O is managed by callbacks, my “streamcaps” stream library has perfect backpressure. Each callback generates a promise, and each promise is waited upon before more data is generated.

                                                                                The main downside to perfect backpressure is that flow is slow and managed. Many APIs have “firehose” or “streaming” configurations that can send multiple packets of data as fast as possible, but such an API is not possible when backpressure is perfectly communicated.