1. 10
  1. 2

    On page 86 it says:

    • Enable primarycache=all where working set exceeds RAM
    • Enable primarycache=metadata where working set fits in RAM

    And zfs’s manual page states:

    primarycache=all |	none | metadata

    Controls what is cached in the primary cache (ARC). If this property is set to all, then both user data and metadata is cached. If this property is set to none, then neither user data nor metadata is cached. If this property is set to metadata, then only metadata is cached. The default value is all.

    I was confused about this at first. On the surface it looked like if you have enough RAM you want to cache less. That didn’t seem right. I know neither PostgreSQL nor ZFS that well, so is my following reasoning about the recommendation above correct?

    If your data fits into RAM you don’t want to cache it with ZFS, but rely on PostgreSQL’s own caching. And if PostgreSQL can’t keep it all in RAM you should leave it to ZFS to do its best at caching the blocks, which PostgreSQL requests.

    1. 2

      While I don’t know, my guess is that in the case of working set not fitting in RAM they are trying to minimize the cost of paging to disk by caching stuff. Also, they say before that, I think, to used Compressed ARC which lets you cache more in RAM than it seems Postgresql does (until it supports compressing its cache).

      1. 1

        Yes, I would guess that’s the reason. You can see on the following slides that in the primarycache=metadata case he’s also recommending allocating the bulk of system RAM to Pg shared_buffers which is where it keeps its own cache, and says (well, implies) that it’s to avoid having the same data uselessly in RAM twice. But if the dataset is significantly bigger, you might as well let ARC (and maybe even L2ARC) do its thing. And in that case, the caches will differentiate to some extent anyway; the truly hot data will stay in Pg’s cache, which means it won’t issue disk reads for it often, which means it will probably thrash right out of ZFS’s cache.

        1. 1

          Thanks for confirming, that I wasn’t on the wrong track with my thinking.

      2. 1

        I found the video for this talk, which was given at the Southern California Linux Expo 15x on 2017-03-02.