1. 33
  1. 11

    This article basically sums up the last eight years or so of my career. I spend a lot of time thinking about UI and state management. While I find many of the links (eff, koka, algebraic effects) and ideas (modeling effects with linear types) novel and interesting, I have not found a one-size-fits-all solution and doubt one exists.

    I can safely say it’s definitely not message buses. The first time I needed to bind UI’s state to a WebGL scene, I used a message bus called postal.js. (This was before Redux). Ensuring that subscribers are ready before state updates are broadcasted was basically impossible. To work around that, I set up listeners for requests for some piece of the state and sent corresponding response messages back. Naturally, users of such request-response message pairs abstracted them with promises. What in any other application was a fairly straightforward synchronous operation became a perilous asynchronous one. When Redux came out, I slapped my forehead and wondered why I didn’t think of something like it.

    I like working with state containers like Redux (or Pinia, Vuex, or the one more-or-less baked into TEA). But it can be hard to justify to both junior devs who don’t see the point and senior devs who favor the YAGNI principle. I tend to favor them because they’re a lot easier to start with and not need them than to add later on. But I have encountered scenarios where they don’t make sense. So much depends on where else the data needs to go (server APIs, Apollo in-memory caches, localStorage). Sometimes you don’t want a component to hold on to any state at all.

    I’m going to go out on a limb and say “functional core, imperative shell” is upside down. React’s Virtual DOM could be said to follow this model where the functional core is the Virtual DOM and the imperative shell is hooks. The problem is that Virtual DOM reconciliation is a slow process compared with competing frameworks. Suspense in React 18 is a band-aid. React isn’t noticeably slow for run-of-the-mill software on fast hardware, but I don’t think I’m alone in saying it’s pretty bad for heavy software, especially on phones. Fan as I am of FP, the more performant solution is an imperative core. Taking Svelte for example, all the DOM mutations are precompiled into vanilla imperative JavaScript. The result is very fast. Elm also precompiles its mutations into imperative JavaScript and is also very fast. It has the additional benefit of a functional shell, which gives it some amazing runtime guarantees.

    In spite of my skepticism of a one-size-fits-all state manager, I am very curious to know what you mean by an “in-memory database with some type of restricted update semantics.” Are you planning on exploring this idea further?

    1. 2

      I am very curious to know what you mean by an “in-memory database with some type of restricted update semantics.”

      Even if you don’t use an actual database, all the mental machinery of normalization, consistency, and isolation are still useful. For example, in classic MVC you have a model (a write master) and views (read replicas) that have to be kept in sync. If you are writing a web app, you have the local model and the remote model on the server which have to be kept in sync. When you click a button you have to update the local model transactionally so that views that may query for its state (such as when they are being created) get a consistent state. Or perhaps, every program that maintains state can be viewed as containing a logical database even if there isn’t a separate SQL store.

      For web work, I have gotten a fair amount of mileage from having a JavaScript object as the model, functions that update it and then call a notify method on a set of observers, and objects that act as views which have a notify method, a draw method, and hold references to the DOM elements they control. As long as you don’t yield the event loop during a set of updates before calling notify you get isolation. It’s usually straightforward to write a particular view in such a way that a notify that has no effect on it results in nothing touching the DOM. I wrote up a set of examples as I was exploring this: https://github.com/madhadron/mvc_for_the_web

      On the other hand, if I were doing something that was rewriting most of the screen like a 3D game, this would make less sense. Then I would just want to calculate frames from state in a loop (as such games do).

      1. 1

        If you are writing a web app, you have the local model and the remote model on the server which have to be kept in sync.

        I wrote a separate post about the abstractions getting destroyed at the boundary between client/server, which you might be interested in. It’s coming from a different direction, but it seems to be leading to a similar place, where an in-memory database might make sense. I’m not sure yet, and still have work to do to figure it out.

        As long as you don’t yield the event loop during a set of updates before calling notify you get isolation.

        what does notify notify? I assume it’s the state manager in response to user or network events? Maybe it’ll be clearer after I go through your repo?

        1. 2

          I wrote a separate post

          Yup. This is basically it. The handling of data in distributed systems is a global concern, and it’s useful to logically treat it as a problem of a database.

          I don’t think this can be papered over. The best we can hope for is to have something like 0MQ, a library of data consistency abstractions the way 0MQ has a library of communication pattern abstractions. So you might have a global definition of state that has a strongly consistent field for one field and a CRDT for a list of other values and an append and interleave list of messages…

          what does notify notify?

          The observers (i.e., all the views). So, yes, the state manager.

      2. 1

        I’m going to go out on a limb and say “functional core, imperative shell” is upside down. React’s Virtual DOM could be said to follow this model where the functional core is the Virtual DOM and the imperative shell is hooks.

        Is React’s virtual DOM a functional core? It’s job is to take the declared shape of what the DOM should look like, and do a load of imperative work on the actual DOM to make it look like that?

        Edit: Or does “functional core” refer to the programmer interface to the virtual DOM? It “looks functional”?

        1. 2

          I took “shell” to mean what the user sees and “core” to mean the behavior being encapsulated or abstracted. But reading the source of the quote, I see that it’s I who am upside down:

          We review a Twitter client whose core is functional: managing tweets, syncing timelines to incoming Twitter API data, remembering cursor positions within the tweet list, and rendering tweets to text for display. This functional core is surrounded by a shell of imperative code: it manipulates stdin, stdout, the database, and the network, all based on values produced by the functional core.

          In answer to your questions:

          Is React’s virtual DOM a functional core? It’s job is to take the declared shape of what the DOM should look like, and do a load of imperative work on the actual DOM to make it look like that?

          The virtual DOM is an immutable data structure. Rather than using observables or proxies to trigger targeted changes to the real DOM as Vue does, React generates a new virtual DOM on every state change and effectively diffs each virtual DOM instance with the previous one to determine what needs to change in the real DOM. How purely functional the reconciliation process is exactly, I don’t know, but it strikes me as at least functional in spirit.

          Conceptually, I love the virtual DOM architecture because it creates very clear separations of concerns for each step of the update process. In practice, framework users don’t care how elegant the innards of the framework, especially if there’s a faster framework. The step of updating the real DOM is, of course, an inherently imperative process regardless of the framework because the DOM is an imperative API.

        2. 1

          Glad that there’s a recognition here. I was hoping to see that a description of the problem would resonate with someone.

          Ensuring that subscribers are ready before state updates are broadcasted was basically impossible.

          Postal.js didn’t allow you to set everything up before letting loose all the updates? Maybe I’m not interpreting it correctly. I’m reading it as an issue with initialization, but maybe you mean that subscribers were busy with something else so they missed the state update broadcasts?

          I’m going to go out on a limb and say “functional core, imperative shell” is upside down. React’s Virtual DOM could be said to follow this model where the functional core is the Virtual DOM and the imperative shell is hooks.

          No, you had it right. the functional core is the React virtual DOM because it allows users to just think of rendering as a pure function of state. And hooks are analogous to the imperative shell, in that they both serve the same purpose: the boundary of the program that handles side effects from the outside world. Whether its compiled implementation is imperative or not doesn’t matter to the person writing the functional core, as long as it can be treated as a pure function.

          In spite of my skepticism of a one-size-fits-all state manager, I am very curious to know what you mean by an “in-memory database with some type of restricted update semantics.” Are you planning on exploring this idea further?

          I am. I’ve been looking at the history of programming, distributed systems, as well as current front-end libraries, trying to get a sense of the thoughts in this area. I still shaping up my own thoughts about it, so it’s currently still rough in my head. But that sentence was just a drive-by attempt to guess what might address the bullet points at the end. We typically store data as data structures in-memory, and as a database on-disk. I’m not sure the distinction makes sense anymore. Queries might help for apps where the shape of state isn’t like the shape of the view. And typically databases let you update data willy-nilly. Reducers from redux are state machines. Maybe we can get the same bang for the buck with some kind of restricted update. I’ll have to think about it some more, read about previous attempts, and try some experiments to flesh it out in a different post.

          1. 2

            Postal.js didn’t allow you to set everything up before letting loose all the updates? Maybe I’m not interpreting it correctly. I’m reading it as an issue with initialization, but maybe you mean that subscribers were busy with something else so they missed the state update broadcasts?

            In the apps I was supporting, the apps’ states were all fairly complex trees of data upon which multiple, dynamically loaded components could operate. If these dynamic components get their state from a message bus but are loaded after the data upon which they operate, they have to broadcast a message saying, “Hey, give me this piece of the state.” So that’s what most of the traffic on such a message bus ends up being: lots of request/response pairs, many of which were redundant. Purpose-built state managers/containers like Redux, Pinia, and Vuex lack that overhead. They’re much faster and easier to bind components to. The browser dev tools extensions also make debugging way easier.

        3. 2

          I found this experiment https://riffle.systems/essays/prelude/ very interesting, and seems to be somewhat related to the OP last remarks. I believe it had already been posted here, yet I can’t seem to find the discussion.

          1. 2

            I think the question of ‘what is the ideal frontend architecture’ can’t be answered without thinking about the backend as well. After all, the frontend is generally just a cache of the system / backend state. So it makes sense to design the whole architecture at once.

            For example, choosing something like GraphQL changes your whole backend architecture. Or microservices vs monolith - this affects your frontend too.

            So really, full stack architecture is about building a data caching and update system with multiple layers.

            For me, I want that data caching system to be decoupled from the actual design of the appearance of views / pages. By this I mean, I prefer the full stack portion to be designed together, and to make the actual user interface a humble object which just calls into this ‘application kernel.’

            1. 1

              Interactive programs also exist outside of web pages and game engines! A couple of developers are building native apps and having some success.

              Apple’s recent SwiftUI framework (for apps on all its platforms) operates on similar principles to React et al, but I’m not well-enough versed in it or web frameworks to be able to say how it compares.

              Apple does have a lower-level data/state management framework called Combine that’s based on FRP, though again I don’t have the expertise to state how strictly it adheres to FRP principles.

              1. 1

                SwiftUI is a whole lot like React. Many concepts map one-to-one. By itself, Combine is mainly Rx Observables (“Publishers”) with some extensions to use them to subscribe to things like NSNotification. But SwiftUI can then subscribe to Publishers and use them as sources of truth for view updates.

                1. 1

                  Combine enables backpressure, which IIRC the traditional Rx model doesn’t. That stood out to me because backpressure is important when working with network data.

                2. 1

                  Definitely! Most of my experience has been in web apps, and a cursory understanding of games. I know less about native app architecture.

                  If you know of any posts or framework documentation that can elucidate its design decisions, let me know, as I’m curious what other ecosystems do. I’ll check out Combine.