Glad people are finding the commit() and relaxed durability functionality! I worked on both of those in chromium (well, mostly commit()) and those were the ‘easiest’ perf increases we could find.
IDB is sometimes really straightforward but sometimes really weird due to needing to be ‘smart’ about interacting with javascript.
If anyone wants to increase the perf further in chromium and has a LOT of time to kill…. I have some big refactoring projects to pitch you lol.
Thank you for your work on this, and sorry if I came off a bit harsh in the post. I’m sure relaxed durability and commit() are a big improvement for cases where the page is using multiple transactions. In this Chrome bug I noticed that the benchmark naïvely uses idb-keyval‘s set() once per item insertion, probably without realizing each one gets its own transaction. I’m sure there is a lot of this pattern out there on the web, so it makes sense to optimize for it.
In the spirit of trying to be more helpful and not just complain-y, I did write down some ideas for a bulk insert. I’m not sure how much of a perf win it would be though.
Relaxed durability also lets us use less fsyncs I think? I forget. It shouldn’t only be multi-transactional improvements, but maybe that is how it mostly shows up? I wasn’t the primary dev on this one so I don’t remember the details much.
More info about how commit speeds things up:
https://docs.google.com/presentation/d/e/2PACX-1vRFevPoyBj-ntqpFmD_lRVFcBi7f1Q4fBHWR04UvLniOu_98gfgDPFo7FOv8S27ZBrqgSKexVbz5YE4/pub?start=false&loop=false&delayms=3000
(copied from my tpac presentation from a while ago)
Basically it lets us skip a whole cycle of the event loop. The biggest perf hit that I found for IDB in chromium is that our current blink-side implementation double-schedules (and sometimes triple-schedules) all “responses” / “events” on the event loop. So if the event loop is super full a lot (like on page startup), then this can really delay the IDB responses reaching JS. IDBTxn.commit() helps with this (and helps on every browser, not just chromium, because all of them have to wait an extra round trip to make sure that no more IDB operations are scheduled). This double-schedule situation could definitely be refactored away to not do this, but it would probably be a quarter-long (maybe 1.5 quarter long?) eng project for someone. This is where I personally expect the biggest perf improvement potential in chromium, especially if code is waiting on idb, multiple txns are going on, etc.
We actually had an intern project for putAll - it ended up not being faster at all with our naive implementation, and ‘better’ implementations would require pretty big refactors as well?
Caveat - it’s been years since poking at this code & my memory isn’t great here lol.
Glad people are finding the commit() and relaxed durability functionality! I worked on both of those in chromium (well, mostly commit()) and those were the ‘easiest’ perf increases we could find.
IDB is sometimes really straightforward but sometimes really weird due to needing to be ‘smart’ about interacting with javascript.
If anyone wants to increase the perf further in chromium and has a LOT of time to kill…. I have some big refactoring projects to pitch you lol.
Thank you for your work on this, and sorry if I came off a bit harsh in the post. I’m sure relaxed durability and
commit()are a big improvement for cases where the page is using multiple transactions. In this Chrome bug I noticed that the benchmark naïvely usesidb-keyval‘sset()once per item insertion, probably without realizing each one gets its own transaction. I’m sure there is a lot of this pattern out there on the web, so it makes sense to optimize for it.In the spirit of trying to be more helpful and not just complain-y, I did write down some ideas for a bulk insert. I’m not sure how much of a perf win it would be though.
Relaxed durability also lets us use less fsyncs I think? I forget. It shouldn’t only be multi-transactional improvements, but maybe that is how it mostly shows up? I wasn’t the primary dev on this one so I don’t remember the details much.
More info about how commit speeds things up: https://docs.google.com/presentation/d/e/2PACX-1vRFevPoyBj-ntqpFmD_lRVFcBi7f1Q4fBHWR04UvLniOu_98gfgDPFo7FOv8S27ZBrqgSKexVbz5YE4/pub?start=false&loop=false&delayms=3000 (copied from my tpac presentation from a while ago) Basically it lets us skip a whole cycle of the event loop. The biggest perf hit that I found for IDB in chromium is that our current blink-side implementation double-schedules (and sometimes triple-schedules) all “responses” / “events” on the event loop. So if the event loop is super full a lot (like on page startup), then this can really delay the IDB responses reaching JS. IDBTxn.commit() helps with this (and helps on every browser, not just chromium, because all of them have to wait an extra round trip to make sure that no more IDB operations are scheduled). This double-schedule situation could definitely be refactored away to not do this, but it would probably be a quarter-long (maybe 1.5 quarter long?) eng project for someone. This is where I personally expect the biggest perf improvement potential in chromium, especially if code is waiting on idb, multiple txns are going on, etc.
We actually had an intern project for putAll - it ended up not being faster at all with our naive implementation, and ‘better’ implementations would require pretty big refactors as well?
Caveat - it’s been years since poking at this code & my memory isn’t great here lol.
Thanks, this is a great explanation! I added a link to your Lobsters comment in my post. :)