1. 17
  1.  

  2. 14

    I’ve found the article from which this is inspired by to be meatier:

    https://lyngvaer.no/log/writing-pseudo-device-driver

    1. 4

      Yep, it’s a really great article. I didn’t want to just pull paragraphs from it though so thought it best to link off and keep mine briefer.

    2. 3

      If you want to get into kernel mode programming on other systems, then please take note that OS manufacturers make it harder and harder to load thirdparty kernel code for each OS update.

      For example, in recent macOS versions Apple has deprecated kernel extensions. Instead, you should write ‘system extensions’ that don’t have ring-0 permissions. So if you want to distribute some privileged code with your software, then if system extensions won’t cover it you can just forget about this functionality.

      Linux kernel modules require distributing the source code along with them, not only because of the license, but also because the ABI between kernel versions is completely unstable by design. So it’s hard to distribute kernel modules with your software. Not impossible, but you’re much better off if you’ll figure out how to introduce your kernel module in the main kernel tree somehow.

      Some BSD distributions have completely disabled the ability to load kernel code, so if you’re not a kernel hacker then it’s pointless to even consider writing it.

      I think the only OS that allows loading kernel drivers from thirdparty vendors without much hassle is Windows. You still need to sign the driver with MS certificates (I think), and that will probably cost some money, but I don’t think MS tried to discourage developers from writing kernel code.

      1. 4

        For example, in recent macOS versions Apple has deprecated kernel extensions. Instead, you should write ‘system extensions’ that don’t have ring-0 permissions. So if you want to distribute some privileged code with your software, then if system extensions won’t cover it you can just forget about this functionality.

        This leads somewhat to the question of what your code actually needs to do. Modern operating systems are moving towards a more microkernel-like model. A lot of the old preformance-related arguments against microkernels no longer apply because thing the kernel does tend to fall into two categories:

        • Sufficiently far off the critical path that an extra context switch is fine.
        • Sufficiently performance-critical that even a monolithic kernel’s system call is too slow.

        The first means that things like CUSE, FUSE, and so on can happily run things in userspace without users seeing any performance problems. The second means that there’s a trend for high-performance devices to do kernel bypass and map a PCI VF directly into userspace: The kernel sets up some IOMMU mappings and then gets out of your way. With S-IOV (Intel) and Revere (Arm), hardware is moving towards making this even more scalable so that future devices will support hundreds of contexts instead of the handful that’s possible with SR-IOV.

        Linux kernel modules require distributing the source code along with them, not only because of the license, but also because the ABI between kernel versions is completely unstable by design. So it’s hard to distribute kernel modules with your software. Not impossible, but you’re much better off if you’ll figure out how to introduce your kernel module in the main kernel tree somehow.

        nVidia gets around this with a shim. The shim is distributed in source form and is recompiled against the current kernel. There’s some infrastructure now for Linux that automatically rebuilds kernel modules when you install a new kernel, which helps avoid KBI changes breaking things, but doesn’t help with KPI changes breaking things. The nVidia model provides a permissively licensed kernel module that talks to the kernel interfaces and treats the majority of the driver as a black box. The rest of the driver is a blob that expects to talk to a kernel abstraction layer and is the same code for Linux, Windows, FreeBSD, and macOS.

        Some BSD distributions have completely disabled the ability to load kernel code, so if you’re not a kernel hacker then it’s pointless to even consider writing it.

        BSDs have had a securelevel mechanism for decades. At the highest securelevel, you can’t load kernel modules and you can’t open /dev/kmem. I believe OpenBSD defaults to (or at least encourages running at) the higher levels, other BSDs don’t. I’m not sure what the ABI guarantees are in Open- and NetBSD, but FreeBSD provides strong KBI guarantees across an entire major release series. Before an X.0 version is branched, they typically add a few padding fields to structures to allow them to be extended. If you’ve written a kernel module against one major release it should work with any later ones in that release series.

        I think the only OS that allows loading kernel drivers from thirdparty vendors without much hassle is Windows. You still need to sign the driver with MS certificates (I think), and that will probably cost some money, but I don’t think MS tried to discourage developers from writing kernel code.

        Windows will not load unsigned drivers by default. If you enable this, then some features that depend on the secure boot attestation will not work. Like other kernels, NT is encouraging developers to write extensions in userspace where possible, with userspace frameworks for writing USB devices, filesystems, and a few other things.

        The Windows situation isn’t that different from a Linux distro that enables kernel code signing, except that the Windows KBI guarantees are a lot stronger than Linux and weaker than FreeBSD. Windows exposes a set of symbols to loadable drivers and these are the only guaranteed stable ones. Linux similarly exposes a subset of kernel symbols to loadable modules, but does not guarantee that they’re stable. In FreeBSD, any kernel structure and any symbol that is not static is part of the KBI.

        1. 1

          BSDs have had a securelevel mechanism for decades. At the highest securelevel, you can’t load kernel modules and you can’t open /dev/kmem. I believe OpenBSD defaults to (or at least encourages running at) the higher levels, other BSDs don’t. I’m not sure what the ABI guarantees are in Open- and NetBSD, but FreeBSD provides strong KBI guarantees across an entire major release series. Before an X.0 version is branched, they typically add a few padding fields to structures to allow them to be extended. If you’ve written a kernel module against one major release it should work with any later ones in that release series.

          I believe OpenBSD completely removed loadable kernel modules entirely.

          1. 1

            Yes, that is correct. Meaning as I tested this I would need to rebuild the kernel each time (and hence why I decided to do it on Linux)

            1. 1

              Out of interest, how long does an incremental build of the OpenBSD kernel take? For Linux kernel work, I use [eudyptula-boot][https://github.com/vincentbernat/eudyptula-boot], which spins up a VM with the new kernel and a minimal init, with my host FS mounted read-only there and drops me into a shell. I do this even when testing kernel modules, because a bug in my kernel module can corrupt kernel state and I don’t want to do that on my host system. It takes around 10 seconds to boot the VM, so if an incremental rebuild of the kernel takes a comparable amount of time, my compile-test-debug cycle on OpenBSD would be similar. If I have to do a clean rebuild, it would be more painful. FreeBSD can do make buildkernel -DNO_CLEAN very quickly (not as quickly as I’d like), I’ve never managed to successfully do an incremental recompile of Linux.

              1. 1

                I do know OpenBSD relinks the kernel on boot, so I assume it’s short, or it’s long and they’re OK with the punishment in the name of security.