1. 29

  2. 7

    Very good summary with a ton of interesting links, I’ll probably get back to them later.

    One point the author missed however, is about organizational complexity. Quite often in my experience, the decision is made to move to a distributed system to reflect the structure of the organisation. That’s a solution (with its set of trade-offs) to the problem of “how to have 50 engineers working on the same project?”.

    1. 4

      Agreed I think I touch on that briefly as one of the real reasons people currently use microservices where in theory with better tools for separately updated hot-reloaded dynamic libraries with stable interfaces could solve the problems people split services based on teams for without adding the asynchrony.

    2. 3
      1. 2

        Many of these problems are key successes of urbit, a distributed system with ubiquitous idempotency, exactly-once messaging. For example your mention of NUMA:

        Tons of effort is also expended on things like how various bottlenecks often entail a complicated hierarchy of caches that need to be kept in sync with the underlying data store.

        …reminds me of this item from https://urbit.org/blog/a-founders-farewell/ :

        20th-century computers have two brains: one fast and temporary, the other slow and permanent. When one of these two-brained computers hears a packet, and sends an acknowledgment, that only means the temporary brain heard it. Acks are not end-to-end transactions.

        This means you can’t write a command protocol where your commands get executed exactly once. You can only choose “at least once” or “at most once.” This problem is as ridiculous and frustrating as it sounds.

        You can address it in three ways: (a) make all your commands formally idempotent, so that repeating them has no additional effect; (b) insert a layer of stateful middleware — a message queue; (c) be fine with “almost always once.” Guess which approach is most popular?

        [In Urbit,] acks are end-to-end transactions — no packet is acknowledged until all its direct effects are finalized. Every message channel is its own message queue. And every message is executed once.

        Also from your post:

        one case I can remember was when I wanted to send a large data structure, but the available serialization system could only serialize it all at once rather than streaming it packet by packet

        Urbit does that transparently.

        1. 1

          Thinking out loud:

          Distributing your work across {fnlam}pipes kills productivity.

          Corollary: one process programming might net you productivity wins.

          Computers are fast; you can run StackOverflow on a handful of servers.

          Maybe you shouldn’t switch to a mainframe, but you might want to consider whether better utilizing your hardware might shrink your problem to a “one process” problem.

          From the mainframe essay linked above:

          Yet another problem is that many companies don’t want to switch [from mainframes], because it saves them money to run a small, expensive machine instead of a small, expensive cluster. In The Economist article above, Eurocontrol reported saving 50% on software costs. The NYT article reports that Radixx switched from a cluster to an IBM mainframe and saved 50%.

          1. 1

            Honestly, the title sounds like it applies to remote work as much as anything else.