Probably most interesting to folks here is https://github.com/jamii/dida/blob/main/docs/why.md which explains how differential dataflow works. The code itself is not yet optimized enough to be useful.
Thanks! That’s the first clear intro to it that I’ve seen. I’m still trying to wrap my head around some of the later sections, but I’m making progress!
I’m coming at this from the perspective of wanting to watch changes to a SQLite database and update query results incrementally … or at least know when a change affects the query results and when it doesn’t. I imagine this is very difficult in general, but it might be easier for a large subset of straightforward queries…
Almost all the complexity will come from dealing with sql. Incremental updates are not that hard by comparison.
For a sense of scale, timely + differential dataflow is 16kloc. Materialize adds ~110kloc and is still discovering sql edge cases.
I love that more and more random projects I check out have shell.nix or flakes definition :)
I don’t even like nix, I just can’t live without it.