1. 16

  2. 3

    I wish the author went more in-depth into how changing hardware will affect kernel space. A lot of our OS concepts are based on optimizing for slow,inexpensive drives. What about now with NVM drives and in the future where servers could be just a bunch of NVDIMMS?

    1. 1

      No clue if NVDIMMS. I did submit a SSD-optimized filesystem here.

    2. 2

      This piece inspired a simplistic key-value store, we wrote to replace memcachedb. During development, the pcstat tool came in handy to answer the question:

      “is Linux caching my data or not?”

      pcstat gets that information for you using the mincore(2) syscall.

      1. 2

        Is this still accurate in 2019? I guess kernels’ internals algorithms have not changed too much since 2006. However, CPUs today have much more cache (especially server-grade CPUs), and servers can have plenty of RAM, thus avoiding excessive swapping.

        1. 7

          PHK knows enough about the filesystem to use it right. Most people don’t understand the correctness and performance issues at play when using files in performance sensitive environments, and should use a database instead. And if you’re building a database, there’s a pretty high chance you’ll want to bypass the pagecache by using O_DIRECT so you can have very fine-grained control over the order that IO is performed in, letting you reason about your own guarantees in your crash recovery logic and generally making it more clear how to scale up writes to utilize hardware.

          Kernels have generally not provided users with the interfaces required to keep up with the dramatic improvements to storage performance that have been happening since the PHK article. Linux is getting io_uring very soon which will be a really major step to unlock higher performance in a decent API and not having to take the hard paths of AIO / SPDK / lightnvm etc… But I’m not sure where the line will be between people using io_uring and people using a database that does it for them.

        Stories with similar links:

        1. So what's wrong with 1975 programming? (2006) via kyle 7 years ago | 6 points | no comments