I would have rather seen the HardenedBSD code just get merged back into FreeBSD, I’m sure there are loads of reasons, but I’ve never managed to see them, their website doesn’t make that clear. I imagine it’s because of mostly non-technical reasons.
That said, It’s great that HardenedBSD is now setup to live longer, and I hope it has a great future, as it sits in a niche that only OpenBSD really sits in, and it’s great to see some competition/diversity in this space!
Originally, that’s what HardenedBSD was meant for: simply a place for Oliver and me to collaborate on our clean-room reimplementation of grsecurity to FreeBSD. All features were to be upstreamed. However, it took us two years in our attempt to upstream ASLR. That attempt failed and resulted in a lot of burnout with the upstreaming process.
HardenedBSD still does attempt the upstreaming of a few things here and there, but usually more simplistic things: We contributed a lot to the new bectl jail command. We’ve hardened a couple aspects of bhyve, even giving it the ability to work in a jailed environment.
The picture looks a bit different today. HardenedBSD now aims to give the FreeBSD community more choices. Given grsecurity’s/PaX’s inspiring history of pissing off exploit authors, HardenedBSD will continue to align itself with grsecurity where possible. We hope to perform a clean-room reimplementation of all publicly documented grsecurity features. And that’s only the start. :)
I view any OS as simply a tool. HardenedBSD’s goal isn’t to “win users over.” Rather, it’s to perform a clean-room reimplementation of grsecurity. By using HardenedBSD, you get all the amazing features of FreeBSD (ZFS, DTrace, Jails, bhyve, Capsicum, etc.) with state-of-the-art and robust exploit mitigations. We’re the only operating system that applies non-Cross-DSO CFI across the entire base operating system. We’re actively working on Cross-DSO CFI support.
I think OpenBSD is doing interesting things with regards to security research, but OpenBSD has fundamental paradigms may not be compatible with grsecurity’s. For example: by default, it’s not allowed to create an RWX memory mapping with mmap(2) on both HardenedBSD and OpenBSD. However, HardenedBSD takes this one step further: if a mapping has ever been writable, it can never be marked executable (and vice-versa).
On HardenedBSD:
void *mapping = mmap(NULL, getpagesize(), PROT_READ | PROT_WRITE | PROT_EXEC, ...); /* The mapping is created, but RW, not RWX. */
mprotect(mapping, getpagesize(), PROT_READ | PROT_EXEC); /* <- this will explicitly fail */
munmap(mapping, getpagesize());
mapping = mmap(NULL, getpagesize(), PROT_READ | PROT_EXEC, ...); /* <- Totally cool */
mprotect(mapping, getpagesize(), PROT_READ | PROT_WRITE); /* <- this will explicitly fail */
It’s the protection around mprotect(2) that OpenBSD lacks. Theo’s disinclined to implement such a protection, because users will need to toggle a flag on a per-binary basis for those applications that violate the above example (web browsers like Firefox and Chromium being the most notable examples). OpenBSD implemented WX_NEEDED relatively recently, so my thought is that users could use the WX_NEEDED toggle to disable the extra mprotect restriction. But, not many OpenBSD folk like that idea. For more information on exactly how our implementation works, please look at the section in the HardenedBSD Handbook on our PaX NOEXEC implementation.
I cannot stress strongly enough that the above example wasn’t given to be argumentative. Rather, I wanted to give an example of diverging core beliefs. I have a lot of respect for the OpenBSD community.
Even though I’m the co-founder of HardenedBSD, I’m not going to say “everyone should use HardenedBSD exclusively!” Instead, use the right tool for the job. HardenedBSD fits 99% of the work I do. I have Win10 and Linux VMs for those few things not possible in HardenedBSD (or any of the BSDs).
By default, migrating a memory mapping from writable to executable is disallowed (and vice-versa).
HardenedBSD provides a utility that users can use to tell the OS “I’d like to disable exploit mitigation just for this particular application.” Take a look at the section I linked to in the comment above.
Just to expound on the different philosophies approach, OpenBSD would never bring ZFS, Bluetooth, etc into the OS, something HardenedBSD does.
OpenBSD has a focus on minimalism, which is great from a maintainability and security perspective. Sometimes that means you miss out on things that could make your life easier. That said OpenBSD still has a lot going for it. I run both, depending on need.
If I remember right, just the ZFS sources by themselves are larger than the entire OpenBSD kernel sources, which gives ZFS a LOT of attack surface. That’s not to say ZFS isn’t awesome, it totally is, but if you don’t need ZFS for a particular compute job, not including it gives you a lot smaller surface for bad people to attack.
There have been quite a number of changes to the kernel to accommodate ZFS. It’d be interesting to see if the kernel can be made to be more simple when ZFS is fully removed.
Besides being large, I think what makes me slightly wary of ZFS is that it also has a large interface with the rest of the system, and was originally developed in tandem with Solaris/Illumos design and data structures. So any OS that diverges from Solaris in big or small ways requires some porting or abstraction layer, which can result in bugs even when the original code was correct. Here’s a good writeup of such an issue from ZFS-On-Linux.
Congratulations on getting setup for funding your project. Hope that works out!
I have a quick question, though. So, OpenBSD and HardenedBSD both do mitigations against code attacks. The other set of problems I had to harden Windows/Linux for were configurations allowing too much access or leaks, turn off dangerous services, and so on. I know OpenBSD addresses them to be secure by default. Is FreeBSD secure by default out of the box or do you recommend a hardening guide(s)?
If it is, then that means there’s two, extra BSD’s competing for most secure: CheriBSD with capability-secure instructions and custom hardware; Criswell et al’s SVA-OS that combines safe interface for low-level stuff with compile-time safety for C programs. Then, the most accessible and supported would be HardenedBSD and OpenBSD. Further, it’s probably wise to move low-cost mitigations into other two as extra layer.
Thanks! So, my perspective is a bit different. I don’t view HardenedBSD as a competitor with the other BSDs. I’m more of a collaborative person and I believe that the best innovations come with collaboration.
I’m also not a fan of the phrase “secure by default”. Secure against what? OpenBSD does a lot to harden an out-of-the-box fresh installation. HardenedBSD focuses mostly on exploit mitigations, but does include some level of service hardening. FreeBSD needs a lot of work.
There’s some interesting research coming out of the CHERI project. The rather large hurdle they’ll have to jump over, though, is getting their hardware-based work landing in consumer hardware. Not only will their hardware improvements need to land in silicon, they will need to be used by the general public. So that means they’re at least three to four decades out from seeing any hardware-based security mechanisms effectively used.
Of course, all these projects could do still more to harden a system.
Appreciate the heads up on FreeBSD needing hardening. I expected as much since it’s a full-featured OS. I like your collaborative attitude, too. I used to be about slamming the competition. I switched a few years ago to live-and-let-live with folks helping each other out. We’ll all be better off that way since strong security will never have massive talent or financial support. On top of having more fun together. :)
Far as CHERI, you actually need a FPGA or ASIC depending on use case. There’s lots of embedded systems whose processors aren’t fast. A FPGA could reach that performance with CHERI’s level of security at high unit cost and watts. Terasic board already runs it, too. There’s also security-focused and legacy apps that don’t have to be fast that might benefit.
If they do an ASIC, a hardware team has to convert it to ASIC-ready form, buy some IP for things like I/O, stitch them together, and do verification. I think that can happen in less than a year if it’s funded. Draper/Dover already converted SAFE in a few years to support three ISA’s, too. They had little to nothing to work with vs CHERI’s open hardware.
Well, what I meant to convey was that CHERI’s hardware-based work still needs to be used by the average consumer for it to be of any real use. That means Intel needs to adopt their work. Which then means it’ll be another decade before it’s available on the shelves. And then two more decades before it’s broadly used. So, we’re talking a minimum of three decades before CHERI’s work is useful outside of academia.
Or, perhaps, I’m being way too pessimistic. I like a lot of CHERI’s hardware-based work, so I really hope I’m wrong.
Well, what I meant to convey was that CHERI’s hardware-based work still needs to be used by the average consumer for it to be of any real use.
You are being too pessimistic but you’re still right to a large degree. Obviously, most people will want the full-custom performance their x86 boxes are getting them. The ARM processors that were targeted at embedded, which CHERI could enter, weren’t good enough for that. Smartphone companies and later tablet companies still convinced them to use stuff like that for much of their day to day experience. They included hardware accelerators for stuff like MPEG 2 the CPU couldn’t handle. Now, they’re selling so much they’re eating into desktop and laptop markets. They’re also Standard Cell (cheaper/easier) rather than Full-Custom (ultra-expensive). So, those products might use an ARM or RISC-V SoC with CHERI extensions the addition of which would be dirt-cheap, fasts work compared to rest of SoC.
For Intel/AMD/IBM doing full-custom, I don’t know where you’re getting decade. They can crank out whole CPU’s with billions of transistors done full-custom in a year or two. These extensions should be smaller than the processors college students build in their coursework with standard tooling. If companies wanted, they could throw out a CHERI-RISC processor probably in months. Cavium especially since they’re MIPS experts and CHERI is MIPS. On high end, easier to add to POWER than x86 since it’s already RISC. IBM had INFOSEC pioneer Paul Karger build a MLS-enforcing processor that one report said was in small-scale, commercial use. More realistically, I told companies interested to get Intel or AMD to do a modification trimming the fat plus adding some security through their semi-custom business. Nobody from Intel/AMD/IBM to their purchasers are interested in doing this probably because people like us can’t sell the executives on the idea directly. It’s tech people talking to business people or no need is seen. That is why it isn’t happening. If they wanted it, it could be done in a few months to a few years with FreeBSD support already there or a doable amount of work for them.
If not desktops, there’s also the embedded market which sells everything from sub-100Mhz 8-bitters to fixed-purpose setups with high-end chips. I wanted SiFive’s RISC-V team to consider targeting secure, fast, cheap networking before anything else. Switches, routers, VPN’s, firewalls, caches, SSL/TLS offloading, and so on. That would get the networking IP in better shape for the attacks that will be coming later. Inspectable, too, except for NDA parts. If it’s general purpose, it could become a more expensive and more open version of Raspberry Pi, too, with the volume sales of the commercial products keeping it cheap. Throw it in some servers they use in ARM-based clouds, too, differentiating on security and privacy.
For funding, it’s worth noting that hardware startups happen regularly funded by VC’s and taxpayers (esp university spinouts). My best idea was getting paid pro’s to work with good academics with cheap tooling to just build an open, secure CPU that they published in a form industry could use. Although not security-focused, the Leon3 GPL core was done that way. Then, the pro’s would stitch together using sponsored or crowdfunded money the third-party I.P. necessary to make it useful and buy the masks. Then, they begin selling it with sales making it self-sustaining maybe. Alternatively, it’s a loss-leader for one or more sellers of hardware appliances or software. An example of that was Zylin doing the ZPU.
I would have rather seen the HardenedBSD code just get merged back into FreeBSD, I’m sure there are loads of reasons, but I’ve never managed to see them, their website doesn’t make that clear. I imagine it’s because of mostly non-technical reasons.
That said, It’s great that HardenedBSD is now setup to live longer, and I hope it has a great future, as it sits in a niche that only OpenBSD really sits in, and it’s great to see some competition/diversity in this space!
Originally, that’s what HardenedBSD was meant for: simply a place for Oliver and me to collaborate on our clean-room reimplementation of grsecurity to FreeBSD. All features were to be upstreamed. However, it took us two years in our attempt to upstream ASLR. That attempt failed and resulted in a lot of burnout with the upstreaming process.
HardenedBSD still does attempt the upstreaming of a few things here and there, but usually more simplistic things: We contributed a lot to the new
bectl jailcommand. We’ve hardened a couple aspects of bhyve, even giving it the ability to work in a jailed environment.The picture looks a bit different today. HardenedBSD now aims to give the FreeBSD community more choices. Given grsecurity’s/PaX’s inspiring history of pissing off exploit authors, HardenedBSD will continue to align itself with grsecurity where possible. We hope to perform a clean-room reimplementation of all publicly documented grsecurity features. And that’s only the start. :)
edit[0]: grammar
I’m sorry if this is a bad place to ask, but would you mind giving the pitch for using HardenedBSD over OpenBSD?
I view any OS as simply a tool. HardenedBSD’s goal isn’t to “win users over.” Rather, it’s to perform a clean-room reimplementation of grsecurity. By using HardenedBSD, you get all the amazing features of FreeBSD (ZFS, DTrace, Jails, bhyve, Capsicum, etc.) with state-of-the-art and robust exploit mitigations. We’re the only operating system that applies non-Cross-DSO CFI across the entire base operating system. We’re actively working on Cross-DSO CFI support.
I think OpenBSD is doing interesting things with regards to security research, but OpenBSD has fundamental paradigms may not be compatible with grsecurity’s. For example: by default, it’s not allowed to create an RWX memory mapping with
mmap(2)on both HardenedBSD and OpenBSD. However, HardenedBSD takes this one step further: if a mapping has ever been writable, it can never be marked executable (and vice-versa).On HardenedBSD:
It’s the protection around
mprotect(2)that OpenBSD lacks. Theo’s disinclined to implement such a protection, because users will need to toggle a flag on a per-binary basis for those applications that violate the above example (web browsers like Firefox and Chromium being the most notable examples). OpenBSD implemented WX_NEEDED relatively recently, so my thought is that users could use the WX_NEEDED toggle to disable the extra mprotect restriction. But, not many OpenBSD folk like that idea. For more information on exactly how our implementation works, please look at the section in the HardenedBSD Handbook on our PaX NOEXEC implementation.I cannot stress strongly enough that the above example wasn’t given to be argumentative. Rather, I wanted to give an example of diverging core beliefs. I have a lot of respect for the OpenBSD community.
Even though I’m the co-founder of HardenedBSD, I’m not going to say “everyone should use HardenedBSD exclusively!” Instead, use the right tool for the job. HardenedBSD fits 99% of the work I do. I have Win10 and Linux VMs for those few things not possible in HardenedBSD (or any of the BSDs).
So how will JITs work on HardenedBSD? is the sequence:
allowed?
By default, migrating a memory mapping from writable to executable is disallowed (and vice-versa).
HardenedBSD provides a utility that users can use to tell the OS “I’d like to disable exploit mitigation just for this particular application.” Take a look at the section I linked to in the comment above.
Just to expound on the different philosophies approach, OpenBSD would never bring ZFS, Bluetooth, etc into the OS, something HardenedBSD does.
OpenBSD has a focus on minimalism, which is great from a maintainability and security perspective. Sometimes that means you miss out on things that could make your life easier. That said OpenBSD still has a lot going for it. I run both, depending on need.
If I remember right, just the ZFS sources by themselves are larger than the entire OpenBSD kernel sources, which gives ZFS a LOT of attack surface. That’s not to say ZFS isn’t awesome, it totally is, but if you don’t need ZFS for a particular compute job, not including it gives you a lot smaller surface for bad people to attack.
I would find a fork of HardenedBSD without ZFS (and perhaps DTrace) very interesting. :)
Why fork? Just don’t load the kernel modules…
There have been quite a number of changes to the kernel to accommodate ZFS. It’d be interesting to see if the kernel can be made to be more simple when ZFS is fully removed.
You may want to take a look at dragonflybsd then.
Besides being large, I think what makes me slightly wary of ZFS is that it also has a large interface with the rest of the system, and was originally developed in tandem with Solaris/Illumos design and data structures. So any OS that diverges from Solaris in big or small ways requires some porting or abstraction layer, which can result in bugs even when the original code was correct. Here’s a good writeup of such an issue from ZFS-On-Linux.
[Comment removed by author]
Congratulations on getting setup for funding your project. Hope that works out!
I have a quick question, though. So, OpenBSD and HardenedBSD both do mitigations against code attacks. The other set of problems I had to harden Windows/Linux for were configurations allowing too much access or leaks, turn off dangerous services, and so on. I know OpenBSD addresses them to be secure by default. Is FreeBSD secure by default out of the box or do you recommend a hardening guide(s)?
If it is, then that means there’s two, extra BSD’s competing for most secure: CheriBSD with capability-secure instructions and custom hardware; Criswell et al’s SVA-OS that combines safe interface for low-level stuff with compile-time safety for C programs. Then, the most accessible and supported would be HardenedBSD and OpenBSD. Further, it’s probably wise to move low-cost mitigations into other two as extra layer.
Thanks! So, my perspective is a bit different. I don’t view HardenedBSD as a competitor with the other BSDs. I’m more of a collaborative person and I believe that the best innovations come with collaboration.
I’m also not a fan of the phrase “secure by default”. Secure against what? OpenBSD does a lot to harden an out-of-the-box fresh installation. HardenedBSD focuses mostly on exploit mitigations, but does include some level of service hardening. FreeBSD needs a lot of work.
There’s some interesting research coming out of the CHERI project. The rather large hurdle they’ll have to jump over, though, is getting their hardware-based work landing in consumer hardware. Not only will their hardware improvements need to land in silicon, they will need to be used by the general public. So that means they’re at least three to four decades out from seeing any hardware-based security mechanisms effectively used.
Of course, all these projects could do still more to harden a system.
Appreciate the heads up on FreeBSD needing hardening. I expected as much since it’s a full-featured OS. I like your collaborative attitude, too. I used to be about slamming the competition. I switched a few years ago to live-and-let-live with folks helping each other out. We’ll all be better off that way since strong security will never have massive talent or financial support. On top of having more fun together. :)
Far as CHERI, you actually need a FPGA or ASIC depending on use case. There’s lots of embedded systems whose processors aren’t fast. A FPGA could reach that performance with CHERI’s level of security at high unit cost and watts. Terasic board already runs it, too. There’s also security-focused and legacy apps that don’t have to be fast that might benefit.
If they do an ASIC, a hardware team has to convert it to ASIC-ready form, buy some IP for things like I/O, stitch them together, and do verification. I think that can happen in less than a year if it’s funded. Draper/Dover already converted SAFE in a few years to support three ISA’s, too. They had little to nothing to work with vs CHERI’s open hardware.
Well, what I meant to convey was that CHERI’s hardware-based work still needs to be used by the average consumer for it to be of any real use. That means Intel needs to adopt their work. Which then means it’ll be another decade before it’s available on the shelves. And then two more decades before it’s broadly used. So, we’re talking a minimum of three decades before CHERI’s work is useful outside of academia.
Or, perhaps, I’m being way too pessimistic. I like a lot of CHERI’s hardware-based work, so I really hope I’m wrong.
You are being too pessimistic but you’re still right to a large degree. Obviously, most people will want the full-custom performance their x86 boxes are getting them. The ARM processors that were targeted at embedded, which CHERI could enter, weren’t good enough for that. Smartphone companies and later tablet companies still convinced them to use stuff like that for much of their day to day experience. They included hardware accelerators for stuff like MPEG 2 the CPU couldn’t handle. Now, they’re selling so much they’re eating into desktop and laptop markets. They’re also Standard Cell (cheaper/easier) rather than Full-Custom (ultra-expensive). So, those products might use an ARM or RISC-V SoC with CHERI extensions the addition of which would be dirt-cheap, fasts work compared to rest of SoC.
For Intel/AMD/IBM doing full-custom, I don’t know where you’re getting decade. They can crank out whole CPU’s with billions of transistors done full-custom in a year or two. These extensions should be smaller than the processors college students build in their coursework with standard tooling. If companies wanted, they could throw out a CHERI-RISC processor probably in months. Cavium especially since they’re MIPS experts and CHERI is MIPS. On high end, easier to add to POWER than x86 since it’s already RISC. IBM had INFOSEC pioneer Paul Karger build a MLS-enforcing processor that one report said was in small-scale, commercial use. More realistically, I told companies interested to get Intel or AMD to do a modification trimming the fat plus adding some security through their semi-custom business. Nobody from Intel/AMD/IBM to their purchasers are interested in doing this probably because people like us can’t sell the executives on the idea directly. It’s tech people talking to business people or no need is seen. That is why it isn’t happening. If they wanted it, it could be done in a few months to a few years with FreeBSD support already there or a doable amount of work for them.
If not desktops, there’s also the embedded market which sells everything from sub-100Mhz 8-bitters to fixed-purpose setups with high-end chips. I wanted SiFive’s RISC-V team to consider targeting secure, fast, cheap networking before anything else. Switches, routers, VPN’s, firewalls, caches, SSL/TLS offloading, and so on. That would get the networking IP in better shape for the attacks that will be coming later. Inspectable, too, except for NDA parts. If it’s general purpose, it could become a more expensive and more open version of Raspberry Pi, too, with the volume sales of the commercial products keeping it cheap. Throw it in some servers they use in ARM-based clouds, too, differentiating on security and privacy.
For funding, it’s worth noting that hardware startups happen regularly funded by VC’s and taxpayers (esp university spinouts). My best idea was getting paid pro’s to work with good academics with cheap tooling to just build an open, secure CPU that they published in a form industry could use. Although not security-focused, the Leon3 GPL core was done that way. Then, the pro’s would stitch together using sponsored or crowdfunded money the third-party I.P. necessary to make it useful and buy the masks. Then, they begin selling it with sales making it self-sustaining maybe. Alternatively, it’s a loss-leader for one or more sellers of hardware appliances or software. An example of that was Zylin doing the ZPU.