1. 25
  1. 8

    We see this at Notion too - the SQLite queries to load a page from our local cache are slower than fetching the page from network on low end devices. The problem with racing network & SQLite is that this can end up using up to 2x the IO & memory on the device and is still slower that always using network for many devices. Well, at least we still have the cache in case network is not available.

    1. 2

      That’s interesting, and I find myself surprised to read this. I don’t know too much by way of low-level database, IO, or kernel API tuning, but do you think this is mainly because of IO bottleneck, or is there room for optimization?

      1. 2

        There’s still plenty of room for optimization, but I’m not working closely with the iOS & Android teams anymore so I’m not sure what their outlook is on this problem. Our general local cache stores records in our standard normal form. A common solution we’ve used in the past is to denormalize the data needed for specific UI components into a more specialized cache. This is harder to do for page rendering since the page editor is our most general component family. We could do like a tiered boot? Or try to cut down on scans but otherwise keep stuff normalized? Tricky.

        Tuning anything like this takes a bunch of time because we either need to develop and benchmark against a slow device (which is slow to do since installing a build and attaching a debugger takes longer), or if we can’t repro since p95 situations are often corner-cases, ship improvement experiments to users and wait for data to come back.

        Being a database admin for a fleet of millions of hopefully-identical databases running on low end hardware is an interesting challenge.

    2. 1

      Have seen this more than a few times when tuning not web apps, but native Android apps. Some devices have astonishingly slow internal storage that can be be further slowed by background processes. I never implemented cache races, but I have had to explain away some very curious in app instrumentation graphs where locally cached images loaded more slowly than new ones! It usually came down to certain models of hardware.