1. 11

  2. 4

    I was waiting the end to scream materialize.io, pipelinedb or postgres 15, and then realized it was written in 2016.

    1. 4

      What do you see in Postgres 15 that’s relevant? I did a quick search and couldn’t find much of a roadmap.

      1. 3
        1. 2

          Wow, never knew something like this were coming to Postgres, can be a game changer for some applications.

    2. 2

      Query efficiency is a nice-to-have […]if we’re architecting for 10M+ users, it’s a must-have. For serving up unread-message counts at scale, the denormalized approach is the only approach.

      Only if you have a totally centralized architecture, where even UI-related state like an “unread” indicator is being computed server-side. In which case, don’t do that. Stop treating the supercomputer in the user’s pocket as though it were an old IBM 3270 terminal.

      If the view and controller layers are on the client, scalability is a non-issue because the number of users in the data model will be in the tens or hundreds, so the queries will be fast*.

      The best approach for this IMO is offline-first, where the client has its own database subset that holds the state it needs, and it syncs with the vastly-larger server database. The client code has no dependency on network I/O for most of its logic, and can make super low latency queries on a small quantity of local data, even while disconnected from the network. Meanwhile in the background, a replicator task ensure that the local data set is kept up to date. (This is exactly the kind of system I build in my day job.)

      * Unless you start accidentally writing code that runs in O(n^2) by repeating queries over and over, which my experience with Core Data taught me is easy to do when there’s a fancy ORM separating you from the queries.

      1. 2

        I’ve sketched out a concept that solves this problem using automatic differentiation (or something similar) for an in-language data-oriented event-driven data engine. You would probably not have a fun time doing this with SQL.