FreeBSD is my daily driver. But I’m still baffled, that C-States deeper than C1 are disabled by default.
hw.acpi.cpu.cx_lowest is set to C1 after install and the first thing I do is set it to CMAX. ( The OS itself can only set C3 and the hardware goes lower, enabling features like Coreparking and so on, which is invisible to the FreeBSD Kernel without kernel modules to detect lower C-states. )
The reason for only C1 by default is historically C6 / CoreParking being possibly buggy I guess?
Either way, on my dual 16 core Opteron server simply allowing them to go into AMD’s “CC6” and park their cores cuts system powerdraw by 20 something Watts on idle. (Holy smokes that architecture is inefficient) On a laptop is of course not as dramatic, but I’m still confused why a more sensible default isn’t … default.
Great write-up! Didn’t know even ZFS had a performance related tweak. Tuning filesystems for power efficency is kind of insane as a concept. There should be an official laptop.sh enabling all the sane defaults to make FreeBSD less daunting outside of the server use-case.
In May I want to do one of these again and make a proper powerdraw measurement with the battery removed in idle + watching a YouTube video compared to Windows, Linux vs FreeBSD in it’s default state vs going through a guide like yours. Especially interested in Powerdraw with Intel’s Turbo-Bias setting for the YouTube video use-case. For work I received a ZBook Fury G7 (i7-10750H, NVIDIA Quadro T2000) and it should be a good benchmark of a modern system without jumping into modern unknowns like Big/Little chips with their performance cores and an NVIDIA GPU is present to test compare how true wezm’s observation is on other systems.
It would be nice to periodically poll the sysctl that exposes whether you’re on battery or mains power and toggle between the different configuration options. I’m quite happy for my laptop to consume power when it’s connected to a wire.
The reason for only C1 by default is historically C6 / CoreParking being possibly buggy I guess?
As I recall, all of this requires poking ACPI tables and a lot of these were spectacularly buggy. I had a ThinkPad that, if you took it into lower sleep states, would then leave the CPU at a crazily low CPU speed (something like 10% of the default) when you came back, some of the time. Even Windows didn’t come with work arounds for this out of the box, so you needed to reboot if this happened. I believe Lenovo later released a driver that worked around this in some way. Additionally, I think some of the deeper states required that all drivers also be aware of the sleep state because they would affect other things on the SoC or even the rest of the motherboard.
I have no idea whether more recent hardware has these problems or whether FreeBSD now has better work-arounds. I think OpenBSD has done a lot of work in this area (it was a running joke that OpenBSD had great support for suspend, they just needed to get resume working, and that motivated some of their hackers to put in a lot of effort to make things work smoothly).
The sleep functionality of most brands of 86 motherboards over the years hasn’t been great. Even Windows doesn’t really do it all that well on most non-Surface devices. The only reliable combinations for sleep/resume has been all-Apple or all-Microsoft stacks. So if I were doing OSS stuff I’d turn most of it off, too.
Even Windows doesn’t really do it all that well on most non-Surface devices
Even there it’s not perfect. Windows 11 on the Surface Book 2 appears to have a bug where if you didn’t gracefully shut down the machine (i.e. on the first boot after a crash) it can’t take the CPU speed over about 1 GHz. Windows 11 on a 400 MHz CPU is very painful. Fortunately, an additional reboot seems to fix it for me, though booting and then shutting down with the CPU sitting at 400-700 MHz takes quite a long time.
FreeBSD runs great on my Huawei MateBook X Pro however there appears to be no way to disable the NVIDIA GPU like there is with bumblebee on Linux. As a result it runs much hotter and the battery life is much shorter than Linux. I’ve done some searching and found things like this script but it didn’t work for me. Just as I did with my desktop, when I next buy a laptop I’ll be avoiding anything with an NVIDIA GPU.
FreeBSD is my daily driver. But I’m still baffled, that C-States deeper than C1 are disabled by default. hw.acpi.cpu.cx_lowest is set to C1 after install and the first thing I do is set it to CMAX. ( The OS itself can only set C3 and the hardware goes lower, enabling features like Coreparking and so on, which is invisible to the FreeBSD Kernel without kernel modules to detect lower C-states. ) The reason for only C1 by default is historically C6 / CoreParking being possibly buggy I guess?
Either way, on my dual 16 core Opteron server simply allowing them to go into AMD’s “CC6” and park their cores cuts system powerdraw by 20 something Watts on idle. (Holy smokes that architecture is inefficient) On a laptop is of course not as dramatic, but I’m still confused why a more sensible default isn’t … default.
Thia can help you:
https://vermaden.wordpress.com/2018/11/28/the-power-to-serve-freebsd-power-management/
Great write-up! Didn’t know even ZFS had a performance related tweak. Tuning filesystems for power efficency is kind of insane as a concept. There should be an official laptop.sh enabling all the sane defaults to make FreeBSD less daunting outside of the server use-case.
In May I want to do one of these again and make a proper powerdraw measurement with the battery removed in idle + watching a YouTube video compared to Windows, Linux vs FreeBSD in it’s default state vs going through a guide like yours. Especially interested in Powerdraw with Intel’s Turbo-Bias setting for the YouTube video use-case. For work I received a ZBook Fury G7 (i7-10750H, NVIDIA Quadro T2000) and it should be a good benchmark of a modern system without jumping into modern unknowns like Big/Little chips with their performance cores and an NVIDIA GPU is present to test compare how true wezm’s observation is on other systems.
Thanks.
Good idea about laptop.sh script :)
It would be nice to periodically poll the sysctl that exposes whether you’re on battery or mains power and toggle between the different configuration options. I’m quite happy for my laptop to consume power when it’s connected to a wire.
As I recall, all of this requires poking ACPI tables and a lot of these were spectacularly buggy. I had a ThinkPad that, if you took it into lower sleep states, would then leave the CPU at a crazily low CPU speed (something like 10% of the default) when you came back, some of the time. Even Windows didn’t come with work arounds for this out of the box, so you needed to reboot if this happened. I believe Lenovo later released a driver that worked around this in some way. Additionally, I think some of the deeper states required that all drivers also be aware of the sleep state because they would affect other things on the SoC or even the rest of the motherboard.
I have no idea whether more recent hardware has these problems or whether FreeBSD now has better work-arounds. I think OpenBSD has done a lot of work in this area (it was a running joke that OpenBSD had great support for suspend, they just needed to get resume working, and that motivated some of their hackers to put in a lot of effort to make things work smoothly).
The sleep functionality of most brands of 86 motherboards over the years hasn’t been great. Even Windows doesn’t really do it all that well on most non-Surface devices. The only reliable combinations for sleep/resume has been all-Apple or all-Microsoft stacks. So if I were doing OSS stuff I’d turn most of it off, too.
Even there it’s not perfect. Windows 11 on the Surface Book 2 appears to have a bug where if you didn’t gracefully shut down the machine (i.e. on the first boot after a crash) it can’t take the CPU speed over about 1 GHz. Windows 11 on a 400 MHz CPU is very painful. Fortunately, an additional reboot seems to fix it for me, though booting and then shutting down with the CPU sitting at 400-700 MHz takes quite a long time.
FreeBSD runs great on my Huawei MateBook X Pro however there appears to be no way to disable the NVIDIA GPU like there is with bumblebee on Linux. As a result it runs much hotter and the battery life is much shorter than Linux. I’ve done some searching and found things like this script but it didn’t work for me. Just as I did with my desktop, when I next buy a laptop I’ll be avoiding anything with an NVIDIA GPU.