1. 32

  2. 25

    FreeBSD also needs to work on security. It’s rather unacceptable to be two decades behind in terms of userland exploit mitigations. That an attacker can reuse exploits with hardcoded addresses with 100% reliability should bring tears to the eyes of security-conscious users.

    Those who claim “ASLR is dead!” either do not know what ASLR was meant to protect against or have an inferior agenda to sell. I’m glad FreeBSD is finally working on an implementation of Address Space Randomization (ASR), but the extremely high perf hits of ASR will cause most people to disable ASR.

    There is no need to complete rewrite mmap to implement W^X, as is told by the University of Cambridge team (Brooks Davis et al). There are certainly advantages to rewriting mmap’s backend and providing an mmap shim, but doing so is not required for W^X. I really like a lot of the work the CHERIBSD project is doing, but for the hardware aspects, it’ll likely be multiple decades (hint: a minimum of three) to be of any real-world use.

    There’s a lot of work to be done on the front of CFI and SafeStack. Both ASLR and PaX NOEXEC (a much stronger form of W^X) are prerequisites to Cross-DSO CFI and SafeStack. So FreeBSD picking up Cross-DSO CFI and SafeStack are meaningless without a PaX NOEXEC implementation, which HardenedBSD has had for years now.

    I may sound harsh here, but it is indeed reality. I don’t mean to come off as combative, though I realize I most likely do. Information security is something of a passion for me, given my background. I welcome polite and constructive discussion, though ad hominem attacks will result in an end to the discussion.

    1. 4

      I really like a lot of the work the CHERIBSD project is doing, but for the hardware aspects, it’ll likely be multiple decades (hint: a minimum of three) to be of any real-world use.

      It could be put into production in a year or two with funding given it’s already a usable design. eASIC could make a S-ASIC out of it for about $50,000 in a month. All depends on the performance and power usage the market cares about. I see potential in whatever embedded markets are using Linux without high performance requirements. They use CheriBSD (or other RTOS on CHERI) for improved security. Maybe flexibility, too.

      1. 3

        Perhaps I’m wrong about CHERI’s hardware enhancements taking so long: https://www.theregister.co.uk/2019/01/28/ukgov_secure_by_design_70m_arm_cambridge/

        This is one case in which I’m very glad to be wrong. :) Even if ARM OEMs put a chip on the market with CHERI’s hardware enhancements this very day, I still think it would take at least a decade to be considered in general use. There’s a lot of ARM devices out there. The Cavium ThunderX2 is brand spankin’ new, and it’s ARMv8.1 even though ARMv8.5 is out.

        1. 3

          Nice that someone with money is noticing but I tend to ignore most of those stories. Been hyped up and let down too much in the past. The RISC-V chips were already turned into ASIC’s. There’s a ton of field-proven silicon from ARM and MIPS. Essentially, you’re looking at paying for a license of something that already works (six digits or 7 if ARM), spending another six digits on a relatively-tiny addition, buying some 3rd-party IP on same node for RAM/PCI/whatever, stiching it together (that part is cheap), and then fabbing it. Smaller firms do larger projects than this regularly. That’s why I’m confident it could be done in a year to a few years if it just got a pile of money.

          “I still think it would take at least a decade to be considered in general use. “

          Whoa, whoa, that’s a different goalpost. There’s whether we can have it out there. Then, there’s whether everyone else will be using it. High-security tech is something that we rarely have everyone using. Just ask OpenBSD folks about their massive userbase dwarfing insecure solutions. Worse if hardware-based security: only Burroughs B5000 and System/38 did that in a way that’s still around… without the hardware security. People just didn’t pay for that shit. So, it’s literally going to have to be embedded in products people would otherwise buy without hardware security.

          I see them initially in defense, safety-critical, banking with crypto processors added, security-focused kiosks/appliances, etc. They will initially not be profitable because (a) it’s always a hard upsell and (b) they’ll have to figure out the marketing plan. They can be available, though, selling at low-volume, higher-per-unit rate like Raptor on POWER9 and Freedom on RISC-V were. It might get cheaper over time as usage gradually goes up. There might form a secondary market for older processors at a discount rate since even 10 year old hardware is still performant enough for tons of use-cases for tons of people.

          Note: An exception is the SAFE architecture which is already commercially available for a market where you can crank out new chips easily and cheaply. Supports FreeRTOS: one of most popular, which even Amazon was offering recently.

          1. 2

            Whoa, whoa, that’s a different goalpost.

            Sorry, that’s what I meant all along. Sometimes I suck at proper communication. Can I blame the lack of caffeine? Or perhaps I should just take responsibility and own up to my poorly-worded statement. ;)

            I really, really want CHERI’s work to be widely available and widely used. I think, though, that both of those cases will take a minimum of one decade, but likely longer. Intel released the details on CET at least two years ago, yet there’s not a single Intel chip on the market with CET (though, there should be one by Q2 2020, I think). Even once it launches, CET won’t be widely used since operating system vendors and application developers will need to make use of it. Then, people will need to upgrade their hardware. There’s sooooooo much hardware in production that hasn’t seen any updates in well over a decade.

            Perhaps I’m being too pessimistic. I’ll fully admit that. And I’d absolutely love to be wrong.

      2. [Comment from banned user removed]

        1. 6

          This is the kind of pointless and incorrect argument I’ve grown extremely tired of. Shall we try again, this time for a productive discussion? ;)

          1. 1

            it’s not a pointless argument. the responses for freebsd security-officer emails are very lacking.

      3. 18

        I’m feeling a bit disillusioned with FreeBSD as a user running it on their home server (mainly for ZFS). For all its promises of being a coherent system it hasn’t felt that way to me (unlike OpenBSD). Here are a few things that rub me the wrong way:

        Many projects start and stall development, but it’s rarely clear when this happens and they tend to linger around in the system or still listed on support sites as solutions (e.g. docker port, relayd port).

        In the mount_unionfs(8) manpage:


        On FreeBSD 12. How long has that message been there? If it doesn’t work, then remove it! It’s things like this that make me distrust the system. If things like this are still hanging around, then what else has been forgotten about?

        I don’t trust the FreeBSD wiki at all, there is too much outdated information. I don’t think it’s really viable in the same way the ArchWiki is, and it shouldn’t be on the official domain, it’s an inconsistent mess and not a good look. Unlike some others (who rave about the handbook) I find FreeBSD documentation very hit and miss.

        There are three firewalls. Pick one and run with it (and it it’s PF, then rename it because it causes confusion with the OpenBSD version). The justification in the handbook is this:

        FreeBSD provides multiple firewalls in order to meet the different requirements and preferences for a wide variety of users.

        If one firewall doesn’t meet the different requirements then it needs to be improved. Why don’t others have this problem? It’s another reason FreeBSD does not feel coherent or straightforward to me.

        Why is rubygems split up from the ruby package? I have never had a problem with them being packaged together on any other system, and it has caused me no end of pain on FreeBSD and just feels arbitrary. There are other strange packaging decisions I’ve run into, though I can’t recall them right now. In general, I prefer package maintainers make as few modifications to the software as possible and keep it simple and predictable.

        If bhyve and jails had simple, consistent interfaces designed for humans (see OpenBSD’s vmm), there wouldn’t be such a need for the multitude of wrapper tools.

        A few minor things…

        I still find /usr/local/etc and /usr/home catch me out on occasion.

        I find freebsd-update slow and clunky (how many reboots do I need?).

        In my experience FreeBSD is a bloated system, without a good sense of direction and needs to cut a lot of fat. There’s plenty I like about it, and I’m going to continue using it for now (for ZFS), but to be honest I don’t see much of a future for it unless things change. I don’t see that happening though.

        Edit: And I completely disagree with the assertion that it ‘just works’. I’ve had to fiddle with workarounds for strange quirks numerous times.

        1. 12

          Unlike some others (who rave about the handbook) I find FreeBSD documentation very hit and miss.

          I was drawn to FreeBSD by its reputation to have a superb documentation. And for the most part, I find the documentation (i.e., the handbook) is actually really good. However, it is constantly in danger of becoming outdated, and it is obviously a lot of work to maintain quality and up-to-dateness. To make matters worse, the freebsd-doc mailing list seems to be filled with spam and it is not really obvious how to contribute. As an (maybe extreme) example, this patch for the OpenLDAP server page that was first submitted in May 2017 was only merged in December 2018: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219142

          When setting up various services (LDAP, mail, etc.) on my home server, the handbook was not sufficient to avoid the classic googling of random blog posts. But it still was the most helpful source for me.

          1. 4

            Yeah I do think the handbook is a very good resource, but as you say you do run into the odd issue which does affect my trust in it somewhat. I often just feel like FreeBSD is trying to do too much, spreading itself too thin for the size of its community to reasonably support, so quality suffers.

            I also think the handbook should be very careful about covering supporting software which is outside the base system.

            None of this is unfixable!

          2. 4

            If one firewall doesn’t meet the different requirements then it needs to be improved. Why don’t others have this problem? It’s another reason FreeBSD does not feel coherent or straightforward to me.

            I stongly agree. FreeBSD does need to pick a firewall (Dragonfly’s ipfw3 maybe?) and drop the other ones.

            There seems to be lots of cruft like this in FreeBSD. Yet it seems like I am always reading some flameup on the mailing list whenever anything old and crusty is even considered for removal. Here is a recent example. This is from someone running current! I guess timed was in 4.3 BSD, but it should have probably been moved to ports ages ago..

            Another example are all the “r”-commands in base — currently deprecated, but still present (in 12.x so far).

            Time for housecleaning[1]!

            [1]: Heck, maybe rip out nasty old ipsec and implement wireguard instead.

          3. 13

            An article reflecting the future of FreeBSD without a single mention of OpenBSD, NetBSD or DragonFlyBSD seems incomplete to me.

            It’s not only about competing with Linux, but especially distinction compared with other BSD’s. OpenBSD is known for its security approach, NetBSD is known for a wide range of hardware support (for better or worse) and DragonFlyBSD offers HAMMER and very interesting kernel features.

            What does FreeBSD offer? Come to think of it, it is the most “boring” of them all, and I say that as a FreeNAS user who likes boring for many applications. The sad truth is though, that if OpenBSD caught on with ZFS support and other things, FreeBSD would be abandoned for lots of server applications. FreeBSD is caught in a state of “stability”, where the true performance comes from reliability and continuity. It’s very difficult to spark anything from this state that might be exciting to newcomers, as this state is usually known from big stagnant companies.

            1. 9

              FreeBSD may be boring because it just works and it has most of the things that are needed. Besides ZFS (with Boot Environments which means bulletproof updates/changes - https://is.gd/BECTL) it comes with GEOM framework for storage, the devd(8) is what hald/udev always wanted to be. The sound architecture with 256 OSS channels is just great and works out-of-the-box. VirtualBox and WINE works, lots of FUSE filesystems work, automounting works with plenty of solutions like mine sysutils/automount for example. You have up to date packages regularly built with great pkg(8) manager. WiFi and 3G work well for me also.

              FreeBSD should also import lots (if not all) HardenedBSD features for security.

              Sure FreeBSD is not perfect but after trying all major BSDs I always come back to FreeBSD because as boring as it is it just delivers.

              For example OpenBSD packages are built only on RELEASE which means 6 months of security holes in packages week or more after the release. This is big concern for me because most of the security problems are from the applications you use.

              There is no virtualization on OpenBSD (Qemu is just slow emulation and vmm(8) is for OpenBSD and limited Linux only).

              Wine does not work on OpenBSD.

              OpenBSD has no modern filesystem with bit-rot protection, no compression, no volume manager, you can only use mirror and partition on that … not to mention that OpenBSD ports/packages are only about 8000 while there are more then 36000 ports/packages on FreeBSD.

              NetBSD’s Xen is too out of date and also no ZFS.

              HAMMER2 is still not ready, and no VirtualBox or Bhyve on DragonFly BSD so that is also ‘no go’ for me … but it will be interesting to see where this development will go as HAMMER2 aims to be CLUSTER filesystem which is very interesting for me.

              1. 8

                OpenBSD has no modern filesystem with bit-rot protection, no compression, no volume manager, you can only use mirror and partition on that

                This is actually one of the few pain points I really have with my use of OpenBSD. I really would like checksummed data (even if Apple decided it’s not necessary for APFS), but I doubt that can be done in a backwards-compatible manner. I don’t mind compression being absent. I think it’s mildly dangerous for the filesystem to also manage volumes as it violates a fundamental separation of concerns, but it’s frustrating that softraid layers can’t be combined yet.

                I know, bring your own patches for the softraid stuff, but that’s way above my skillset and not enough of a pain point to actually go deep into an extremely critical subsystem as an outsider.

                1. 7

                  There is no virtualization on OpenBSD (Qemu is just slow emulation and vmm(8) is for OpenBSD and limited Linux only).

                  Nitpick, but the part in parentheses proves the part before that wrong. Maybe “There is very limited virtualization”?

                2. 4

                  FreeBSD is somehow both boring and exciting. Solid fundamentals and cool features.

                  • biggest community of the non-Linux free unixes
                  • tickless kernel (Not sure about NetBSD but Open and DFly are still always ticking)
                  • hardware support — 10GbE, NVMe, AMD GPU, Bluetooth, any USB peripherals (webcams, tablets, joysticks etc.) Linux supports (via webcamd)…
                  • the well known features — jails, ZFS, DTrace
                  • sandboxing innovation with Capsicum and CloudABI
                  • evdev in input drivers, which brings
                  • Wayland support!
                3. 8

                  I’m glad they mentioned container tech. I think that would make a huge difference. I was using Docker on FreeBSD a while back and it worked fairly well. But that Docker build is super old and wasn’t updated when Docker did their big modular refactor. Newer clients can’t even connect to it (get API mismatch bugs):


                  I think a cool project at this point would just be re-implementing the Docker engine API, but recreate the layers with ZFS and launch containers in ZFS jails. For like 80% - 90% of services/apps, I think the Linux binary layer in FreeBSD should run them fine. Can jails provide most/all the same security features of cgroups?

                  I’m not sure how networking works in jails or if they can be translated over from how Docker does things, but if there was a stable, usable way to just pull and launch Docker containers in FreeBSD, I think we’d see a lot more people using them in hosting providers and data centers.

                  1. 3

                    This is the big one for me, because a large part of my work is with Linux containers and virtualisation. The “Linux” part of that means by definition that I’m not immediately likely to adopt a BSD as my daily driver, but I’d love to see a modern BSD take on those technologies. I suspect there’s a lot that the BSDs could bring which would improve the state of the art.

                  2. 5

                    From my perspective the BSDs are just “unfriendly”. I set up a VM at some point to learn how to use FreeBSD but the tooling seemed to be a decade behind everything else. With Ubuntu everything just works as I expect out of the box and package management is mostly a solved problem with lxd providing isolation as necessary. I remember getting pkg stuck in some weird loop and couldn’t get unstuck without blowing up the VM and starting over. After about a month or so of these weird unexpected issues I just went back to Ubuntu.

                    If FreeBSD wants to remain relevant I don’t think they need to fix the issues this post outlines. I think they need to fix the user experience issue because most people these days learn on some flavor of Linux (usually Ubuntu) and the BSD conventions are completely foreign to them (like they were to me when I tried to use it). If you’re an application developer then what compelling reason is there to use FreeBSD when the application runtime/toolchain works just fine on Ubuntu?

                    1. 7

                      From documentation through to implementation, OpenBSD is very user friendly. Everything is well-documented and the irc/mailing lists are active and actively helpful. Try it out, read the manpages, it’s great. And the package management just werks.

                      1. 10

                        I remember getting pkg stuck in some weird loop and couldn’t get unstuck without blowing up the VM and starting over

                        The “new” pkg tools are an abomination. The old tools definitely had their issues, but the replacement is even worse. It’s the systemd of FreeBSD.

                        I used FreeBSD extensively in the past, and when the old package tools were deprecated I couldn’t update a single of my four FreeBSD machines. All of them suffered from different bugs in pkg (segfault or logic errors).

                        I reported the bugs, and either never got a response or a cavalier brush-off. I then decided that the old “quality over anything else” BSD attitude was no longer present in the FreeBSD project, and just reinstalled the machines with Linux or OpenBSD.

                        Some of these bugs are still unfixed to this day, and this comment seems to conform it’s still a mess four years later.

                        1. 2

                          huh, odd. pkg is one of my favorite parts of FreeBSD, and being the maintainer of several ports I’ve done a lot of weird things with pkg — mixing local port builds and official packages, locking, unlocking, aliasing… It handled everything very well.

                        2. 4

                          From my experience using Gentoo, ports-style systems are just really easy to get into a mess. That being said, I wouldn’t give them up for the world.

                          1. 2

                            Agreed on both of these. Perhaps a little off topic but I can’t give enough credit to Gentoo for eselect news read, other ports systems that don’t have this as a means of communicating possible issues are missing out.

                        3. 5

                          Quite a few of the links in the post have URL encoded byte order marks in the URLs, breaking them, e.g. https://openconnect.netflix.com/en/software/%EF%BB%BF.

                          1. 5

                            This post doesn’t address the massive human side of this problem, as I have worked at several of the companies listed. Software usage is a social phenomenon, mostly.

                            I don’t know of any of the companies OP listed that have used BSD for their newest platforms, and in some cases (NetApp, Cisco) the BSD usage was not a technical choice that the company made, but through acquisitions. Curiously not mentioned is Isilon, they were big BSD committers, but again, most of them have moved on to companies / products that use Linux.

                            The big companies that contributed significantly to BSD aren’t relevant anymore - this is your biggest hurdle to making BSD relevant. If you had all the features listed implemented magically today, would companies pick BSD? If not, what would your actual strategy be?

                            1. 1

                              One of the traps I’ve found building products is trying to compete by achieving feature parity with competitors rather than focusing energy on what makes what you’re building unique. Otherwise you’re always just playing catch-up.

                              This post largely focused on how FreeBSD can be an okay way to run Linux software. Why? We have a perfectly good way to run Linux software called Linux. I want to know what’s different about FreeBSD’s world-view. I want features that make me not interested in running docker.

                              1. 4

                                features that make me not interested in running docker


                                Granted, it’s not a cool easy solution, it’s a huge paradigm shift for application development. Instead of the docker paradigm of “your app is an entire packaged Linux distro now”, it’s the complete opposite: “your app is a process with zero rights, except descriptors to whatever was explicitly injected/passed to it”.

                                1. 1

                                  That already works on Linux so why would I choose a different kernel?

                                  1. 2

                                    CloudABI doesn’t work securely on Linux/macOS/etc, that’s a userspace implementation intended only for development. The real implementation is in FreeBSD.