Threads for machineloquist

    1. 1

      Frank McSherry founded the company Materialize to solve this problem and build the first database capable of maintaining materialised views

      They’re a little late to be be first.

      1. 1

        Thanks for the pointer.

        Noria looks really fascinating – though appears to be a “research prototype” (in the author’s own words).

        I guess Materialize is the first production-grade database that supports materialised views for arbitrary queries. Also it’s powered by the Differential Dataflow model from MS Research’s project Naiad dating back to ~2013.

        Would love to learn about any fundamental tradeoffs Noria makes (i.e. things that can’t be simply addressed with engineering man-hours) and why it doesn’t appear to be active any more (see GitHub commits).

        For example, at Tably we’re huge fans of Materialize – but also well aware of some limitations.

        The major ones that come to mind:

        • Everything you can do with Materialize is expressible in standard SQL… but not everything you can do in standard SQL is supported by Materialize (e.g. ALTER SCHEMA)

        • It doesn’t support schema changes so you can’t insert/delete colums

        • State is volatile so you can’t persist changes on a view back to the table

        As I understand, these are inherently due to its architecture – mostly stemming from the fact that the dataflow isn’t bi-directional. (Not hating on Materialize though, it’s amazing!)

        1. 1

          ping @jonhoo - maybe you have thoughts on this, or would just like to be aware of news in the space.