1. 15

Slides: https://jon.thesquareplanet.com/slides/thesis.pdf

  1. 4

    Here’s a link to the actual paper.

    Jon also does Rust live streams that are ridiculously in-depth and educational.

    1. 3

      from the video text

      this is from Jon Gjengset who was a PhD student at MIT working on the Noria database written in Rust.

      This is my PhD dissertation presentation, which I gave at MIT (virtually) on October 22nd, 2020. It was immediately followed by my thesis defense, which I passed subject to the typical bits and pieces of revisions the committee wanted to see.

      The person who introduces me in the video is Robert Morris, my thesis advisor. The Q&A has been cut out, as I want to edit it a bit and mix in questions from the public presentation I gave on YouTube the day before. I will link the Q&A video here once it is out. You can find the slides at https://jon.thesquareplanet.com/slide… and at https://docs.google.com/presentation/…. The thesis itself is not yet published, but I will link it here once it is.

      What follows is the thesis abstract:

      This thesis proposes a practical database system that lowers latency and increases supported load for read-heavy applications by using incrementally-maintained materialized views to cache query results. As opposed to state-of-the-art materialized view systems, the presented system builds the cache on demand, and evicts cache entries in response to a shifting workload.

      The enabling technique the thesis introduces is partially stateful materialization, which allows entries in materialized views to be missing. The thesis proposes upqueries as a mechanism to fill such missing state on demand using dataflow, and implements them in the materialized view system Noria. The thesis then discusses additional mechanisms needed to establish eventual consistency for partially stateful dataflow.

      Noria with partial materialization saves application developers from implementing their own ad hoc caching mechanisms to speed up their database accesses. Instead, the caching is built into the database, and is transparent to the application. Experimental results suggest that the presented system increases supported application load by up to 20x over MySQL and performs similarly to an optimized key-value store cache. Partial state also reduces memory use by up to 2/3 compared to traditional materialized views.

      1. 1

        Since this thesis uses Lobsters as a work load generator, will someone implement this database as the new backend for this site? The evaluation looks very promising.