Reading this at the same time as being on Day 2 of installing something from /usr/ports on FreeBSD. Day 1 was wasted because the one small thing I wanted to install tried to install and build 16 GB of dependencies, including building GCC and LLVM from source.
I had to start over with a bigger disk. Now on hour 5 of compiling LLVM. jwz used to say about Linux “Linux is only free if your time is worth nothing” FreeBSD is only better if your time is worth nothing.
Seriously though, FreeBSD just seems like UNIX from a long time ago, when servers were lovingly crafted by BOFHs and could only be truly understood by a bearded elite. I miss those days, and I can still administer a FreeBSD box just fine, even though I have shaved my beard. But now we’re in the “cattle, not pets” era of operations, and FreeBSD’s days seem numbered.
doesn’t having the base dependencies for a port installed like via pkg satisfy the requirements for building a port? I’ve built packages on DragonFly and I’ve not had to build llvm or gcc from scratch - only pkg install them once.
Yes, that is assuming you know what the dependencies are. Is there a way to tell ports to tell you what software it depends on, other than going down the entire tree of dependencies?
If memory serves, I believe there is also a way to install dependencies with pkg; I’m not sure if it was a make target or an external tool (been a while).
oh, I’d just hit make, then cancel out of it when it started grabbing something absurd. When I’ve pkg installed devel packages for something like python, usually that’s enough to grab a fairly large dependency tree that satisfies most cases. Given the nature of dependencies, there’s usually a few that cascade into many and by getting those, you end up shortening your build time by a lot.
I wonder if someone has a command to list out dependencies such that you can do something like pkg install $(list my dependencies). That would probably solve a lot of cases right there. 🙂
This is my biggest problem with FreeBSD. FreeBSD people advise not mixing ports and packages.
Arch Linux for example has the ABS, which is very similar and to and inspired by the FreeBSD ports tree. Where I believe the ABS is superior is the aim of the ABS is to build a package which is then installed by the package manager.
Then put a front end over make eg $pkg make foo/bar with some flag to install dependencies where possible from binaries. I can only dream
Reading this at the same time as being on Day 2 of installing something from /usr/ports on FreeBSD
I wish someone would update the docs. The advice for new users should be never build anything from /usr/ports. If you even have a /usr/ports on a modern FreeBSD system, then the odds are that you’re doing something wrong. If you want to install a port with options different from the standard package build, then your best bet is to use poudriere. If you’re in a hurry, you can pre-populate the cache with packages that you grab from the package site (LLVM can take a few hours to build on a slow machine).
But now we’re in the “cattle, not pets” era of operations, and FreeBSD’s days seem numbered.
I can’t disagree there. I’d love to see FreeBSD fully adopt UCL, grow an init system that can manage dynamic resource allocation properly, and a decent set of jail management tools so that you can configure everything with a UCL plugin to your favourite orchestration service.
So another frustration that I didn’t mention before was the state of the documentation. What the OP says is a good thing is actually quite terrible. The docs are outdated and contradictory.
There are three different documents for hooking up FreeBSD with SSSD, none of them are correct, and they all use out of date information. It’s a frigging mess.
The one good thing is that it finally kicked me in the ass enough to put in a PR for the Terminator INSTALL file
I do recommend you do explore the other BSDs too, if you haven’t yet.
Also, please consider running it on real hardware. An old workstation or laptop, or even a raspberry pi will do. The experience just isn’t the same with virtualization or an emulator.
If you want cheap, realiable and be sure to have working laptop for FreeBSD then use one of these:
ThinkPad X220
ThinkPad T420
ThinkPad T420s
ThinkPad T520
ThinkPad W520
… and they come with great real 7-row keyboard too.
Personally I use 9 years old W520 with 4C/8T, 16GB RAM and large SSD along with that great keyboard and bright 15” FullHD screen it has everything that is needed.
A lot of the points in this article aren’t really technical advantages IMHO, but rather “I prefer the way FreeBSD does things”. This is fair enough, but … not a unique technical advantage FreeBSD has over Linux.
Most most purposes, Linux and FreeBSD are roughly equivalent technically, just as Debian and Arch Linux are equivalent. The implementations differ, but at the end of the day, both implementations work well, with minimal user-visible differences. Much comes down to what fits best with your goals. For many purposes, both fit equally well.
Some specific notes (which got a bit longer than expected):
I think the “clean separation” in /usr/local is not really all that useful in practice, and the choice of /usr/local to install packages means there is no longer a logical place to install self-compiled software (NetBSD’s /usr/pkg makes more sense, IMO).
Debian issues are, well, Debian issues, not Linux ones. As the article points out, many Linux distros don’t frob with packages like Debian. Comparing FreeBSD to dozens of Linux distros is easy, because you can always find a fault in some distro.
Both Linux and FreeBSD are stable systems; FreeBSD had a leg up here in 2000 or so when Linux was a bit less mature, but those days are long gone (and as I’ve written about here before, the introduction of pkg-ng was a mess of segfaults and other errors; not very stable).
The documentation in FreeBSD is indeed pretty good, and I learned a lot from the handbook back in the day. Many man pages also tend to be a bit better (let’s not even start about GNU info), but the Linux docs aren’t all that terrible these days; certainly a lot better than 15 years ago, and the Arch wiki is more or less the de-facto unofficial handbook these days (most stuff can be applied to other Linuxes as well).
FreeBSD ports is essentially just a package build system; pretty much all Linux distros come with something similar, and it’s not really that unique to BSD. The key difference is that in FreeBSD ports are much more user-visible, but compiling everything from ports strikes me as a massive waste of CPU cycles to be honest. The technical implementation of ports is … well … just look at bsd.port.mk. I’ve done some hacking on this back in the day, and it wasn’t a lot of fun.
ZFS works just as well on Linux as FreeBSD, and has for many years now. The GPL/CDDL incompatibility make shipping it by default and running it on root a bit harder, but these are fairly minor details, IMHO. There was a time when ZFS ran well on FreeBSD and Linux support wasn’t quite mature yet, but that was a decade ago, or longer. Dtrace runs on Linux as well.
I much prefer FreeBSD’s /etc/rc.conf which gives a good single overview about what a system is running in a single file. I wish the defaults would be a bit better though (e.g. maybe not start sendmail by default, requiring not just one but a bunch of variables to disable). I also wish it would be a proper service manager instead of this PID-file hackery.
FreeBSD jails are neat and were a bit ahead of the curve on the whole containers thing. I think the part that’s really missing is the management: you’re expected to extract the base.tar.gz in your container, mount /dev, and do all sorts of other setup. Yeah, you can automate it with scripts and whatnot, but compared to docker pull it’s a lot of work. I think it’s the easy distribution and UX part that really made Docker popular. Bastille looks neat, but as far as I can see it doesn’t do distribution of images. There was another management tool as well (I forgot the name), which has/had the same shortcoming.
bhyve is cool; Linux has kvm/qemu; is there really any difference?
The three different firewalls in FreeBSD reminds me of the Hollywood “zomg he just hacked through 3 different firewalls!” What’s the reason for having 3? I don’t really see the point, and find it rather clashes with the “clean base OS”-doctrine.
Linux can be tuned as well, this is hardly unique to FreeBSD.
GEOM is really neat; this is probably the only clear technical advantage listed here.
I think the “clean separation” in /usr/local is not really all that useful in practice
Its VERY useful. No matter how fscked up your packages are, just do these and you are CLEAN and still have fully working system:
# rm -rf /usr/local /boot/modules /var/cache/pkg
Both Linux and FreeBSD are stable systems.
After systemd became the king of init system in Linux land Linux started to behave very unstable and unpredictable. Maybe you have different observations. I made them using RHEL/CentOS systems.
let’s not even start about GNU info
Little off topic but this is my way to cope with GNU info pages:
# info ls | less
FreeBSD ports is essentially just a package build system; pretty much all Linux distros come with something similar
Gentoo like systems, yes. RHEL like systems and its clones, nope.
ZFS works just as well on Linux as FreeBSD, and has for many years now.
Nope. Linux still does not have ZFS Boot Environments out of the box. There are some guides, some third party tools but nothing consistent. Even latest Ubuntu 20.04 does not support them while having Root on ZFS.
maybe not start sendmail by default
100% agreed. The sendmail is PITA in FreeBSD. I hope it will die in FreeBSD land soon with 13 version.
FreeBSD jails are neat and were a bit ahead of the curve on the whole containers thing. I think the part that’s really missing is the management: you’re expected to extract the base.tar.gz in your container, mount /dev, and do all sorts of other setup. Yeah, you can automate it with scripts and whatnot, but compared to docker pull it’s a lot of work.
Yes. This is the reason why Docker sky rocketed and Jails are still just Jails. Tooling.
bhyve is cool; Linux has kvm/qemu; is there really any difference?
Bhyve is cool, fast and nice but its still pretty young so while KVM provides mature live migration in oVirt like environments for example, the live migration from what I remember is just a proof of concept or very early implementation.
The three different firewalls in FreeBSD reminds me of the Hollywood “zomg he just hacked through 3 different firewalls!” What’s the reason for having 3?
Agreed. This is pointless really. IPFilter should be gone by now and FreeBSD should focus on either PF or IPFW. IMHO PF should be cross BSD project to have one simple firewall across BSDs but I am dreaming here. Even FreeBSD PF is different from OpenBSD PF.
GEOM is really neat; this is probably the only clear technical advantage listed here.
Yes its really nice, once you get used to it its pretty strange that its not available anywhere else …
Not sure where to start … You can create 3 mirrors of two disks or partitions … then make 3-way stripe of these … (you get RAID10 from 6 disks) … and if you would create 4 of these, then you could create RAID5 on top of that making it RAID105 (maybe stupid idea) … then create gjournal on that device (for journaling) and then FAT32 filesystem on it and then expose that ‘storage’ to another FreeBSD server (or several) using GEOM GATE mechanism … without any iSCSI daemons or other third party tools.
That is just one example (probably not the best one).
GEOM is like LEGO for various storage possibilities/features on FreeBSD.
Its all about layers, you just put one layer on top of another …
Its all about layers, you just put one layer on top of another …
That’s what you do on linux anyway. mdadm creates a new disk whcih you can use for luks, which you can use for lvm.
and then expose that ‘storage’ to another FreeBSD server (or several) using GEOM GATE mechanism
Hm, now that is interesting. In my experience, setting up networked file systems on Linux a bit finicky. How does it handle the multiple writers case? E.g. when two writes need to modify the same block on disk.
Hm, now that is interesting. In my experience, setting up networked file systems on Linux a bit finicky. How does it handle the multiple writers case? E.g. when two writes need to modify the same block on disk.
Depends how you implement it.
If you have simple 1 x SERVER <-> 1 x CLIENT then this problem does not exists.
If you have 1 x SERVER <-> N x CLIENT then you can either use some HA cluster on these CLIENTS with non-clustered filesystem (UFS/ZFS/FAT32/…) and only single CLIENT will write or put some clustered filesystem there (like GFS2) and then the cluster logic will decide which CLIENT writes first. I do not know any clustered filesystem such as GFS2 for FreeBSD thou. There are distributed Minio, LeoFS, LizardFS, CEPH, GlusterFS but from what I know each of them require single DEV single CLIENT attachment and then data is presented on network, not directly to block devices.
Yeah, rm -rf /usr/local is a small advantage I suppose, but it’s a fairly minor detail, IMO, just like the ZFS boot thing. I mean, do you really want to base your entire OS choice on just that? There are many far more important considerations, I think.
Both Linux and FreeBSD are stable systems.
After systemd became the king of init system in Linux land Linux started to behave very unstable and unpredictable. Maybe you have different observations. I made them using RHEL/CentOS systems.
I don’t really want to start a YASA (“Yet Another Systemd Argument”), but suffice to say that systemd is also not my most favourite piece of software in the world, and that I, too, am not always impressed by the attitude of its developers, but at the end of the day … it does work. I’ve certainly had weird issues, but in all the years I’ve used it (including on quite a few servers) I’ve only really had two issues, and none were that critical. I’ve had far more issues with pkg-ng, for example.
I currently use a combination of Void and Alpine Linux, partly because it comes without systemd. But that’s really just a personal preference thing. and I think that “systemd made Linux very unstable and unpredictable” seems rather exaggerated to me.
Yeah, rm -rf /usr/local is a small advantage I suppose, but it’s a fairly minor detail, IMO, just like the ZFS boot thing. I mean, do you really want to base your entire OS choice on just that? There are many far more important considerations, I think.
There are many things that ‘got me’ to use FreeBSD instead of Linux in most cases. I can not state all of them here of course (I probably will on a blog post some day). Its one of many. Things I need work on FreeBSD so moving to Linux would not give me anything …
… it does work. I’ve certainly had weird issues, but in all the years I’ve used it (including on quite a few servers) I’ve only really had two issues, and none were that critical.
For most of the time it works, but its weird crashes and problems are very problematic to debug. For example after yum update on CentOS 8.0 from VMware here:
I was not able to determine what was wrong. I reverted this VM to earlier snapshot made yum update again and everything worked like a charm. Also recently my buddy needed to setup puppetserver on CentOS 8.1. The systemd startup script was not able to start the service. He then tried on RHEL 8.1. Worked. CentOS 8.x is basically the same as RHEL 8.1 (red hat logos package is different and /etc/redhat-release file is different) but for some reason it does not work on CentOS … and he was not able to determine the cause of the problem. He finally went with Ubuntu (as RHEL is paid and Ubuntu is for free). Yes Ubuntu also uses systemd but at least it worked.
With rc.d(8) on FreeBSD or with old Linux init.d system I was able to debug it very easily with sh -x /path/to/startup/script.sh start. Now while systemd offers its own debug options its nowhere near that simple and obvious to debug problems.
I’ve had far more issues with pkg-ng, for example.
In its early life PKGng had some issues but currently I do not recall any problem from more then 2 years. What problems did you faced?
I currently use a combination of Void and Alpine Linux, partly because it comes without systemd. But that’s really just a personal preference thing. and I think that “systemd made Linux very unstable and unpredictable” seems rather exaggerated to me.
I also ‘follow’ the development of these two Linux systems. Their design philosophy is very close to BSDs. Personally if I would HAVE TO move to Linux it will either be Ubuntu MATE (to not interest about internals at all) or one of Void or Alpine.
There are many things that ‘got me’ to use FreeBSD instead of Linux in most cases. I can not state all of them here of course (I probably will on a blog post some day). Its one of many. Things I need work on FreeBSD so moving to Linux would not give me anything …
Yeah, that’s fair enough. my point was mostly that many of the items in the article are just a personal preference, rather than a real advantage or something. As you know, I was a FreeBSD user for many years (“Carpetsmoker”, in case you’ve forgotten who I am with the username change 😉), but have used mostly Linux for the last few years, and as a user … I find that I don’t really notice the differences most of the time.
And yeah, when things go wrong with systemd it can be really hard to actually debug and fix, which is one of the better criticisms of it IMHO. Like I said, I’m not a fan either, but for the overwhelming majority of the cases, it does work.
In its early life PKGng had some issues but currently I do not recall any problem from more then 2 years. What problems did you faced?
I had a bunch of segfaults and “solver errors” (if memory serves, it was a long time ago). Being unable to upgrade several systems was the direct cause for installing Linux, and haven’t looked back since.
I’d be surprised if these issues still exist, as this was like 5 years ago; it’s just an example of FreeBSD unfortunately not always being as stable as is sometimes claimed (another example might be the FreeBSD 5 release, which was also a long time ago, but was not great release, to put it mildly; I don’t know of any more recent examples as I haven’t used FreeBSD in a few years, but there have certainly been problems over the years).
As you know, I was a FreeBSD user for many years (“Carpetsmoker”, in case you’ve forgotten who I am with the username change 😉
I have forgotten many things but I do remember You sir :)
Will have to remember arp242 now.
Yeah, that’s fair enough. my point was mostly that many of the items in the article are just a personal preference, rather than a real advantage or something.
For example sound. It works on both FreeBSD with OSS and on Linux with Alsa/Pulseaudio mix. I have lots of problems with Pulseaudio when I used that on Ubuntu several years ago. The only problem I had with FreeBSD sound is that I needed to add some really strange two lines into /boot/device.hints file so I can hear sound on mini jack port with earphones … and that was only on one laptop - ThinkPad T420s. T420/T520/W520/X220 does not have that problem. With Pulseaudio I have to reboot the machine to get the sound back.
I not so long ago wanted to migrate my wife laptop from Windows 8.1 to Linux (because desktop distributions are a lot more ready for out of the box experience). I have tried Ubuntu MATE and ElementaryOS. Both freezed after one or two days of suspend/resume sessions. It was Dell XPS 12 9Q23. GhostBSD was not that polished by then and I really did not want to make a custom desktop for ‘casual’ user on FreeBSD by then. Of course she got back to Windows.
I had a bunch of segfaults and “solver errors” (if memory serves, it was a long time ago). Being unable to upgrade several systems was the direct cause for installing Linux, and haven’t looked back since.
I see your point, yes I remember some of those several years ago.
it’s just an example of FreeBSD unfortunately not always being as stable as is sometimes claimed.
I think statistically it is more stable then Linux, but as any OS with active development sometimes bad things will happen.
another example might be the FreeBSD 5 release, which was also a long time ago, but was not great release, to put it mildly
This is a big mystery for me. I started my FreeBSD journey with FreeBSD 5 … the worst FreeBSD release ever … and that sill made me use it instead of any Linux. Linux must have been really bad when it comes to FreeBSD 5 times :p
Pulseaudio is a bit better than it was a few years ago; I used to have to pulseaudio -k and restart it quite regularly, but it’s been perhaps 2-3 years since I last had to do that. But yeah, I agree sound on Linux is a mess, and has been for a long time.
In general I have good experiences with Ubuntu; and I’ve found that suspend/resume can be a bit fickle on any non-Windows system; I had a laptop where it worked great on Linux but not FreeBSD too; so yeah, in some cases FreeBSD works better here.
I think statistically it is more stable then Linux
Maybe, I don’t know. I suspect when averaged out over all hardware the difference will be negligible. Same for Windows, actually. But no way to be sure until someone runs those kind of tests (which would be rather interesting, actually!)
This is a big mystery for me. I started my FreeBSD journey with FreeBSD 5 … the worst FreeBSD release ever … and that sill made me use it instead of any Linux. Linux must have been really bad when it comes to FreeBSD 5 times :p
At that time, Linux was more fragmented and a bit more “gung-ho” than it is now, and FreeBSD was rather more mature in some respects, even with FreeBSD 5 (the later releases were better). It also offered some things that were not really available at all on Linux at the time (jails, GEOM, kqueue, probably some more). But since then, the differences have evened out quite a bit.
When FreeBSD talks about stability, the project doesn’t mean in the not-crashing sense: that’s expected from a modern OS. They mean that ABIs and KBIs don’t change within a major release (and ABIs are supported across multiple major releases with compat layers). That’s also where the separation is important. If something depends only on components in the base system, you can be sure it will keep working across an upgrade.
Linux (the kernel) has very good ABI stability: statically linked programs from Linux 1.0 typically work today. It has terrible KBI compatibility: kernel modules built last week may not work on this week’s kernel. Glibc has superb ABI compatibility. It makes extremely good use of symbol versioning to avoid breaking programs or libraries linked against earlier versions. libstdc++ tries hard. Everything else in a typical Linux distro is very hit and miss. Because many of the core components of any given distro are developed elsewhere, this can cause problems: upstream breaks an ABI in something that the distro wants to make strong ABI guarantees about, so the distro has to make a choice of whether to stick with an old version, maintain a fork for the duration of a distro release’s lifecycle, or break their ABI policy.
On FreeBSD, this layering means that there is a clean separation between things that have long-term documented ABI stability guarantees (provided by the core OS project) and things that don’t (anything from third-party ports / packages). For a lot of things, that doesn’t matter, but for core bits of your infrastructure it can be valuable knowing that you have both long-term stable ABIs and the ability to run the latest versions of other packages on top. The separation in the filesystem means that you know that things in /usr are safe stable dependencies, things in /usr/local are not (and, if you want things installed yourself from source without packaging, that’s what /opt is for: I generally use a naming scheme of /opt/{package-name}-{version} so that I can uninstall things I’ve built from source with rm -rf). Again, this is where the ABI compatibility is valuable: I know everything in /opt that depends only on things in /usr and things that I’ve installed in /opt will keep working across an upgrade, even if all of the packages from the ports collection are upgraded and some of the core ones have ABI breakages.
Yeah, that’s fair enough, and I agree there is some value in that. But that’s not really the kind of stability the article is talking about though, as I read it anyway.
I hope to run BSD at home, but I wish for Nix to be available first. This isn’t an idle desire; fortunately, there is an active effort to make FreeBSD a first-class target for Nix. But, just like how the author has gotten comfortable with what they perceive as the technical superiority of FreeBSD over GNU/Linux, I have gotten comfortable with Nix and NixOS.
An excellent list. While it is true that many of these have Linux counterparts, some
alternatives (like ZFS on Linux) are only well-supported on a small number of Linux
distros.
Jails are really interesting. While Linux has good support for cgroups/namespaces to
create “containers”, they’re not the same as jails. Tools like
bubblewrap (minimal sandboxing tool that
Flatpak uses), Firejail, chroot, Systemd-nspawn, LXC, and Podman
(unprivileged/rootless and daemonless Docker alternative that supports cgroup v2) all
replicate enough functionality for most use-cases, but jails are still quite
compelling: they offer the low-overhead VM-like experience of containers while
providing stronger sandboxing and less complexity.
I also agree that FreeBSD jails is an attractive feature.
However, when doing more detailed analysis, I could not find ‘cluster’ management software available for freeBSD
Something that automatically brings jails up and down, migrates network traffic to healthy jails, etc.
In other words, something that is done in Linux world by Kubernetis, or by a combination of Nomad-Consul-Vault (by Hashi Corp)
Thank you. Now I can see why leaning towards Nomad makes sense .. it has FreeBSD support.
(I was educating myself on Nomad+Consul vs k8 and I tended to lean against the ‘conventional wisdom’)
It’s the CDDL, yes. It’s a separate cddl directory. I’m not sure what’s so scary about it though? It’s incompatible with the GPL, which can be an inconvience for sure, but that’s not really “scary”?
They could sue with the GPL as well; it’s pretty much the same license. As far as I know, there is nothing inherently in the CDDL that makes it any more or less scary than the GPL.
Is Oracle a great company? Clearly not. But that’s an entirely different thing than whether or not the CDDL is “scary” or not (never mind that FreeBSD imported it from Sun Solaris, before the Oracle acquisition; I’m actually not sure if either FreeBSD or OpenZFS integrated any code from after the Oracle acquisition since they were pretty quick to kill OpenSolaris).
Look I’m the first to argue in favor of choosing a Copyleft license but doesn’t incorporating a Copyleft component into the core of your system go against the licensing principles of FreeBSD?
As for the scariness of CDDL, it’s a weird incompatible license with a litigious author. That’s scary. Wouldn’t want to base a business on code that’s licensed like that. At least Oracle license the JVM under the GPL.
doesn’t incorporating a Copyleft component into the core of your system go against the licensing principles of FreeBSD?
There’s some GPL code as well; it contained gcc in base for a very long time (old GPL2 version), and there’s still stuff like GNU diff, grep, etc. There are also some GPL2 kernel modules I believe (although not compiled in by default, but rather a loadable module). While in general FreeBSD tend to prefer simple permissive licenses, I’m not sure if you can really talk about “license principles”; from what I’ve seen, the considerations tend to be quite pragmatical in this regard.
As for the scariness of CDDL, it’s a weird incompatible license with a litigious author.
Oracle didn’t write the CDDL, Sun did. I’m not sure why it’s “weird”? The reason they wrote it is because they wanted a file-based copyleft that could be more easily mixed with other licenses and doesn’t apply to the binary. Seems to make sense to me; it’s just a different take on copyleft; I don’t see what’s so weird about that.
Reading this at the same time as being on Day 2 of installing something from /usr/ports on FreeBSD. Day 1 was wasted because the one small thing I wanted to install tried to install and build 16 GB of dependencies, including building GCC and LLVM from source.
I had to start over with a bigger disk. Now on hour 5 of compiling LLVM. jwz used to say about Linux “Linux is only free if your time is worth nothing” FreeBSD is only better if your time is worth nothing.
Seriously though, FreeBSD just seems like UNIX from a long time ago, when servers were lovingly crafted by BOFHs and could only be truly understood by a bearded elite. I miss those days, and I can still administer a FreeBSD box just fine, even though I have shaved my beard. But now we’re in the “cattle, not pets” era of operations, and FreeBSD’s days seem numbered.
Why not
pkg install <whatever>? - is there a reason not to install a package this way?I wish. This is the virtualbox drivers and are only available as a port, AFAICT.
doesn’t having the base dependencies for a port installed like via pkg satisfy the requirements for building a port? I’ve built packages on DragonFly and I’ve not had to build llvm or gcc from scratch - only
pkg installthem once.Yes, that is assuming you know what the dependencies are. Is there a way to tell ports to tell you what software it depends on, other than going down the entire tree of dependencies?
make all-depends-listfor the full list of build and run-time dependencies.make missingfor what’s needed but isn’t currently installed.If memory serves, I believe there is also a way to install dependencies with
pkg; I’m not sure if it was a make target or an external tool (been a while).There’s a ports Makefile target:
Oh this might come in handy! Thanks!
oh, I’d just hit make, then cancel out of it when it started grabbing something absurd. When I’ve
pkg installed devel packages for something like python, usually that’s enough to grab a fairly large dependency tree that satisfies most cases. Given the nature of dependencies, there’s usually a few that cascade into many and by getting those, you end up shortening your build time by a lot.I wonder if someone has a command to list out dependencies such that you can do something like
pkg install $(list my dependencies). That would probably solve a lot of cases right there. 🙂This is my biggest problem with FreeBSD. FreeBSD people advise not mixing ports and packages.
Arch Linux for example has the ABS, which is very similar and to and inspired by the FreeBSD ports tree. Where I believe the ABS is superior is the aim of the ABS is to build a package which is then installed by the package manager.
Then put a front end over make eg
$pkg make foo/barwith some flag to install dependencies where possible from binaries. I can only dreamI wish someone would update the docs. The advice for new users should be never build anything from
/usr/ports. If you even have a/usr/portson a modern FreeBSD system, then the odds are that you’re doing something wrong. If you want to install a port with options different from the standard package build, then your best bet is to use poudriere. If you’re in a hurry, you can pre-populate the cache with packages that you grab from the package site (LLVM can take a few hours to build on a slow machine).I can’t disagree there. I’d love to see FreeBSD fully adopt UCL, grow an init system that can manage dynamic resource allocation properly, and a decent set of jail management tools so that you can configure everything with a UCL plugin to your favourite orchestration service.
So another frustration that I didn’t mention before was the state of the documentation. What the OP says is a good thing is actually quite terrible. The docs are outdated and contradictory.
There are three different documents for hooking up FreeBSD with SSSD, none of them are correct, and they all use out of date information. It’s a frigging mess.
The one good thing is that it finally kicked me in the ass enough to put in a PR for the Terminator INSTALL file
I do recommend you do explore the other BSDs too, if you haven’t yet.
Also, please consider running it on real hardware. An old workstation or laptop, or even a raspberry pi will do. The experience just isn’t the same with virtualization or an emulator.
I was planning on trying out FreeBSD on my new RPi 4 but sadly it doesn’t seem to be ported to that platform yet.
If you want cheap, realiable and be sure to have working laptop for FreeBSD then use one of these: ThinkPad X220 ThinkPad T420 ThinkPad T420s ThinkPad T520 ThinkPad W520
… and they come with great real 7-row keyboard too.
Personally I use 9 years old W520 with 4C/8T, 16GB RAM and large SSD along with that great keyboard and bright 15” FullHD screen it has everything that is needed.
I’m using openbsd on a ThinkPad X395. Works fine, except the wireless.
Do not use /usr/ports of you do not compile.
Just use pkg(8) for packages.
A lot of the points in this article aren’t really technical advantages IMHO, but rather “I prefer the way FreeBSD does things”. This is fair enough, but … not a unique technical advantage FreeBSD has over Linux.
Most most purposes, Linux and FreeBSD are roughly equivalent technically, just as Debian and Arch Linux are equivalent. The implementations differ, but at the end of the day, both implementations work well, with minimal user-visible differences. Much comes down to what fits best with your goals. For many purposes, both fit equally well.
Some specific notes (which got a bit longer than expected):
I think the “clean separation” in /usr/local is not really all that useful in practice, and the choice of /usr/local to install packages means there is no longer a logical place to install self-compiled software (NetBSD’s /usr/pkg makes more sense, IMO).
Debian issues are, well, Debian issues, not Linux ones. As the article points out, many Linux distros don’t frob with packages like Debian. Comparing FreeBSD to dozens of Linux distros is easy, because you can always find a fault in some distro.
Both Linux and FreeBSD are stable systems; FreeBSD had a leg up here in 2000 or so when Linux was a bit less mature, but those days are long gone (and as I’ve written about here before, the introduction of pkg-ng was a mess of segfaults and other errors; not very stable).
The documentation in FreeBSD is indeed pretty good, and I learned a lot from the handbook back in the day. Many man pages also tend to be a bit better (let’s not even start about GNU info), but the Linux docs aren’t all that terrible these days; certainly a lot better than 15 years ago, and the Arch wiki is more or less the de-facto unofficial handbook these days (most stuff can be applied to other Linuxes as well).
FreeBSD ports is essentially just a package build system; pretty much all Linux distros come with something similar, and it’s not really that unique to BSD. The key difference is that in FreeBSD ports are much more user-visible, but compiling everything from ports strikes me as a massive waste of CPU cycles to be honest. The technical implementation of ports is … well … just look at bsd.port.mk. I’ve done some hacking on this back in the day, and it wasn’t a lot of fun.
ZFS works just as well on Linux as FreeBSD, and has for many years now. The GPL/CDDL incompatibility make shipping it by default and running it on root a bit harder, but these are fairly minor details, IMHO. There was a time when ZFS ran well on FreeBSD and Linux support wasn’t quite mature yet, but that was a decade ago, or longer. Dtrace runs on Linux as well.
I much prefer FreeBSD’s /etc/rc.conf which gives a good single overview about what a system is running in a single file. I wish the defaults would be a bit better though (e.g. maybe not start sendmail by default, requiring not just one but a bunch of variables to disable). I also wish it would be a proper service manager instead of this PID-file hackery.
FreeBSD jails are neat and were a bit ahead of the curve on the whole containers thing. I think the part that’s really missing is the management: you’re expected to extract the base.tar.gz in your container, mount /dev, and do all sorts of other setup. Yeah, you can automate it with scripts and whatnot, but compared to
docker pullit’s a lot of work. I think it’s the easy distribution and UX part that really made Docker popular. Bastille looks neat, but as far as I can see it doesn’t do distribution of images. There was another management tool as well (I forgot the name), which has/had the same shortcoming.bhyve is cool; Linux has kvm/qemu; is there really any difference?
The three different firewalls in FreeBSD reminds me of the Hollywood “zomg he just hacked through 3 different firewalls!” What’s the reason for having 3? I don’t really see the point, and find it rather clashes with the “clean base OS”-doctrine.
Linux can be tuned as well, this is hardly unique to FreeBSD.
GEOM is really neat; this is probably the only clear technical advantage listed here.
Linux can do event auditing.
Its VERY useful. No matter how fscked up your packages are, just do these and you are CLEAN and still have fully working system:
After systemd became the king of init system in Linux land Linux started to behave very unstable and unpredictable. Maybe you have different observations. I made them using RHEL/CentOS systems.
Little off topic but this is my way to cope with GNU info pages:
Gentoo like systems, yes. RHEL like systems and its clones, nope.
Nope. Linux still does not have ZFS Boot Environments out of the box. There are some guides, some third party tools but nothing consistent. Even latest Ubuntu 20.04 does not support them while having Root on ZFS.
100% agreed. The sendmail is PITA in FreeBSD. I hope it will die in FreeBSD land soon with 13 version.
Yes. This is the reason why Docker sky rocketed and Jails are still just Jails. Tooling.
Bhyve is cool, fast and nice but its still pretty young so while KVM provides mature live migration in oVirt like environments for example, the live migration from what I remember is just a proof of concept or very early implementation.
Agreed. This is pointless really. IPFilter should be gone by now and FreeBSD should focus on either PF or IPFW. IMHO PF should be cross BSD project to have one simple firewall across BSDs but I am dreaming here. Even FreeBSD PF is different from OpenBSD PF.
Yes its really nice, once you get used to it its pretty strange that its not available anywhere else …
Hope that helps.
Regards.
Yes, this is one of the primary reasons I switched to Arch from Fedora.
How exactly does it differ from mdadm/luks/lvm?
Check from page 30: https://is.gd/bsdstg
So instead of running (say) mdadm, you do something with /dev?
Now that is interesting.
Not sure where to start … You can create 3 mirrors of two disks or partitions … then make 3-way stripe of these … (you get RAID10 from 6 disks) … and if you would create 4 of these, then you could create RAID5 on top of that making it RAID105 (maybe stupid idea) … then create gjournal on that device (for journaling) and then FAT32 filesystem on it and then expose that ‘storage’ to another FreeBSD server (or several) using GEOM GATE mechanism … without any iSCSI daemons or other third party tools.
That is just one example (probably not the best one).
GEOM is like LEGO for various storage possibilities/features on FreeBSD.
Its all about layers, you just put one layer on top of another …
That’s what you do on linux anyway. mdadm creates a new disk whcih you can use for luks, which you can use for lvm.
Hm, now that is interesting. In my experience, setting up networked file systems on Linux a bit finicky. How does it handle the multiple writers case? E.g. when two writes need to modify the same block on disk.
Depends how you implement it.
If you have simple 1 x SERVER <-> 1 x CLIENT then this problem does not exists.
If you have 1 x SERVER <-> N x CLIENT then you can either use some HA cluster on these CLIENTS with non-clustered filesystem (UFS/ZFS/FAT32/…) and only single CLIENT will write or put some clustered filesystem there (like GFS2) and then the cluster logic will decide which CLIENT writes first. I do not know any clustered filesystem such as GFS2 for FreeBSD thou. There are distributed Minio, LeoFS, LizardFS, CEPH, GlusterFS but from what I know each of them require single DEV single CLIENT attachment and then data is presented on network, not directly to block devices.
Not sure if that helps.
Oh, so GEOM GATE only works for one mounter? That is, only one client can mount the underlying block device.
Its like NFS, for one, for several or for entire network.
Here is an example from ggated(8) man page:
Yeah,
rm -rf /usr/localis a small advantage I suppose, but it’s a fairly minor detail, IMO, just like the ZFS boot thing. I mean, do you really want to base your entire OS choice on just that? There are many far more important considerations, I think.I don’t really want to start a YASA (“Yet Another Systemd Argument”), but suffice to say that systemd is also not my most favourite piece of software in the world, and that I, too, am not always impressed by the attitude of its developers, but at the end of the day … it does work. I’ve certainly had weird issues, but in all the years I’ve used it (including on quite a few servers) I’ve only really had two issues, and none were that critical. I’ve had far more issues with pkg-ng, for example.
I currently use a combination of Void and Alpine Linux, partly because it comes without systemd. But that’s really just a personal preference thing. and I think that “systemd made Linux very unstable and unpredictable” seems rather exaggerated to me.
There are many things that ‘got me’ to use FreeBSD instead of Linux in most cases. I can not state all of them here of course (I probably will on a blog post some day). Its one of many. Things I need work on FreeBSD so moving to Linux would not give me anything …
For most of the time it works, but its weird crashes and problems are very problematic to debug. For example after
yum updateon CentOS 8.0 from VMware here:https://imgur.com/a/ZURPn1G
I was not able to determine what was wrong. I reverted this VM to earlier snapshot made
yum updateagain and everything worked like a charm. Also recently my buddy needed to setuppuppetserveron CentOS 8.1. The systemd startup script was not able to start the service. He then tried on RHEL 8.1. Worked. CentOS 8.x is basically the same as RHEL 8.1 (red hat logos package is different and/etc/redhat-releasefile is different) but for some reason it does not work on CentOS … and he was not able to determine the cause of the problem. He finally went with Ubuntu (as RHEL is paid and Ubuntu is for free). Yes Ubuntu also uses systemd but at least it worked.With
rc.d(8)on FreeBSD or with old Linuxinit.dsystem I was able to debug it very easily withsh -x /path/to/startup/script.sh start. Now while systemd offers its own debug options its nowhere near that simple and obvious to debug problems.In its early life PKGng had some issues but currently I do not recall any problem from more then 2 years. What problems did you faced?
I also ‘follow’ the development of these two Linux systems. Their design philosophy is very close to BSDs. Personally if I would HAVE TO move to Linux it will either be Ubuntu MATE (to not interest about internals at all) or one of Void or Alpine.
Regards.
Yeah, that’s fair enough. my point was mostly that many of the items in the article are just a personal preference, rather than a real advantage or something. As you know, I was a FreeBSD user for many years (“Carpetsmoker”, in case you’ve forgotten who I am with the username change 😉), but have used mostly Linux for the last few years, and as a user … I find that I don’t really notice the differences most of the time.
And yeah, when things go wrong with systemd it can be really hard to actually debug and fix, which is one of the better criticisms of it IMHO. Like I said, I’m not a fan either, but for the overwhelming majority of the cases, it does work.
I had a bunch of segfaults and “solver errors” (if memory serves, it was a long time ago). Being unable to upgrade several systems was the direct cause for installing Linux, and haven’t looked back since.
I’d be surprised if these issues still exist, as this was like 5 years ago; it’s just an example of FreeBSD unfortunately not always being as stable as is sometimes claimed (another example might be the FreeBSD 5 release, which was also a long time ago, but was not great release, to put it mildly; I don’t know of any more recent examples as I haven’t used FreeBSD in a few years, but there have certainly been problems over the years).
I have forgotten many things but I do remember You sir :)
Will have to remember arp242 now.
For example sound. It works on both FreeBSD with OSS and on Linux with Alsa/Pulseaudio mix. I have lots of problems with Pulseaudio when I used that on Ubuntu several years ago. The only problem I had with FreeBSD sound is that I needed to add some really strange two lines into /boot/device.hints file so I can hear sound on mini jack port with earphones … and that was only on one laptop - ThinkPad T420s. T420/T520/W520/X220 does not have that problem. With Pulseaudio I have to reboot the machine to get the sound back.
I not so long ago wanted to migrate my wife laptop from Windows 8.1 to Linux (because desktop distributions are a lot more ready for out of the box experience). I have tried Ubuntu MATE and ElementaryOS. Both freezed after one or two days of suspend/resume sessions. It was Dell XPS 12 9Q23. GhostBSD was not that polished by then and I really did not want to make a custom desktop for ‘casual’ user on FreeBSD by then. Of course she got back to Windows.
I see your point, yes I remember some of those several years ago.
I think statistically it is more stable then Linux, but as any OS with active development sometimes bad things will happen.
This is a big mystery for me. I started my FreeBSD journey with FreeBSD 5 … the worst FreeBSD release ever … and that sill made me use it instead of any Linux. Linux must have been really bad when it comes to FreeBSD 5 times :p
Wish You all the best.
Regards.
Pulseaudio is a bit better than it was a few years ago; I used to have to
pulseaudio -kand restart it quite regularly, but it’s been perhaps 2-3 years since I last had to do that. But yeah, I agree sound on Linux is a mess, and has been for a long time.In general I have good experiences with Ubuntu; and I’ve found that suspend/resume can be a bit fickle on any non-Windows system; I had a laptop where it worked great on Linux but not FreeBSD too; so yeah, in some cases FreeBSD works better here.
Maybe, I don’t know. I suspect when averaged out over all hardware the difference will be negligible. Same for Windows, actually. But no way to be sure until someone runs those kind of tests (which would be rather interesting, actually!)
At that time, Linux was more fragmented and a bit more “gung-ho” than it is now, and FreeBSD was rather more mature in some respects, even with FreeBSD 5 (the later releases were better). It also offered some things that were not really available at all on Linux at the time (jails, GEOM, kqueue, probably some more). But since then, the differences have evened out quite a bit.
When FreeBSD talks about stability, the project doesn’t mean in the not-crashing sense: that’s expected from a modern OS. They mean that ABIs and KBIs don’t change within a major release (and ABIs are supported across multiple major releases with compat layers). That’s also where the separation is important. If something depends only on components in the base system, you can be sure it will keep working across an upgrade.
Linux (the kernel) has very good ABI stability: statically linked programs from Linux 1.0 typically work today. It has terrible KBI compatibility: kernel modules built last week may not work on this week’s kernel. Glibc has superb ABI compatibility. It makes extremely good use of symbol versioning to avoid breaking programs or libraries linked against earlier versions. libstdc++ tries hard. Everything else in a typical Linux distro is very hit and miss. Because many of the core components of any given distro are developed elsewhere, this can cause problems: upstream breaks an ABI in something that the distro wants to make strong ABI guarantees about, so the distro has to make a choice of whether to stick with an old version, maintain a fork for the duration of a distro release’s lifecycle, or break their ABI policy.
On FreeBSD, this layering means that there is a clean separation between things that have long-term documented ABI stability guarantees (provided by the core OS project) and things that don’t (anything from third-party ports / packages). For a lot of things, that doesn’t matter, but for core bits of your infrastructure it can be valuable knowing that you have both long-term stable ABIs and the ability to run the latest versions of other packages on top. The separation in the filesystem means that you know that things in
/usrare safe stable dependencies, things in/usr/localare not (and, if you want things installed yourself from source without packaging, that’s what/optis for: I generally use a naming scheme of/opt/{package-name}-{version}so that I can uninstall things I’ve built from source withrm -rf). Again, this is where the ABI compatibility is valuable: I know everything in/optthat depends only on things in/usrand things that I’ve installed in/optwill keep working across an upgrade, even if all of the packages from the ports collection are upgraded and some of the core ones have ABI breakages.Yeah, that’s fair enough, and I agree there is some value in that. But that’s not really the kind of stability the article is talking about though, as I read it anyway.
I hope to run BSD at home, but I wish for Nix to be available first. This isn’t an idle desire; fortunately, there is an active effort to make FreeBSD a first-class target for Nix. But, just like how the author has gotten comfortable with what they perceive as the technical superiority of FreeBSD over GNU/Linux, I have gotten comfortable with Nix and NixOS.
Thank you!
Welcome :)
An excellent list. While it is true that many of these have Linux counterparts, some alternatives (like ZFS on Linux) are only well-supported on a small number of Linux distros.
Jails are really interesting. While Linux has good support for cgroups/namespaces to create “containers”, they’re not the same as jails. Tools like bubblewrap (minimal sandboxing tool that Flatpak uses), Firejail, chroot, Systemd-nspawn, LXC, and Podman (unprivileged/rootless and daemonless Docker alternative that supports cgroup v2) all replicate enough functionality for most use-cases, but jails are still quite compelling: they offer the low-overhead VM-like experience of containers while providing stronger sandboxing and less complexity.
I also agree that FreeBSD jails is an attractive feature. However, when doing more detailed analysis, I could not find ‘cluster’ management software available for freeBSD Something that automatically brings jails up and down, migrates network traffic to healthy jails, etc. In other words, something that is done in Linux world by Kubernetis, or by a combination of Nomad-Consul-Vault (by Hashi Corp)
I saw the article mentioned https://bastillebsd.org/templates/ , but that seems to be more of installation tool (similar to Ansible)
The FreeBSD Journal had an article an issue or two ago about Nomad on BSD w/jails. Jails work fine under Nomad.
https://www.freebsdfoundation.org/past-issues/containerization/
For Nomad drivers for FreeBSD jails exist. pot appears to be the closest one to something like Docker on Nomad.
Consul and Vault packages exist as well.
Thank you. Now I can see why leaning towards Nomad makes sense .. it has FreeBSD support. (I was educating myself on Nomad+Consul vs k8 and I tended to lean against the ‘conventional wisdom’)
Does FreeBSD have it’s own ZFS yet or is it the Solaris one with the scary Oracle license?
It’s the CDDL, yes. It’s a separate cddl directory. I’m not sure what’s so scary about it though? It’s incompatible with the GPL, which can be an inconvience for sure, but that’s not really “scary”?
I mean does Oracle ever sue anyone over the licenses on their sort of open source software? No? Never? Great, that’s a comfort.
They could sue with the GPL as well; it’s pretty much the same license. As far as I know, there is nothing inherently in the CDDL that makes it any more or less scary than the GPL.
Is Oracle a great company? Clearly not. But that’s an entirely different thing than whether or not the CDDL is “scary” or not (never mind that FreeBSD imported it from Sun Solaris, before the Oracle acquisition; I’m actually not sure if either FreeBSD or OpenZFS integrated any code from after the Oracle acquisition since they were pretty quick to kill OpenSolaris).
Look I’m the first to argue in favor of choosing a Copyleft license but doesn’t incorporating a Copyleft component into the core of your system go against the licensing principles of FreeBSD?
As for the scariness of CDDL, it’s a weird incompatible license with a litigious author. That’s scary. Wouldn’t want to base a business on code that’s licensed like that. At least Oracle license the JVM under the GPL.
There’s some GPL code as well; it contained gcc in base for a very long time (old GPL2 version), and there’s still stuff like GNU diff, grep, etc. There are also some GPL2 kernel modules I believe (although not compiled in by default, but rather a loadable module). While in general FreeBSD tend to prefer simple permissive licenses, I’m not sure if you can really talk about “license principles”; from what I’ve seen, the considerations tend to be quite pragmatical in this regard.
Oracle didn’t write the CDDL, Sun did. I’m not sure why it’s “weird”? The reason they wrote it is because they wanted a file-based copyleft that could be more easily mixed with other licenses and doesn’t apply to the binary. Seems to make sense to me; it’s just a different take on copyleft; I don’t see what’s so weird about that.
FreeBSD imported OpenSolaris/Illumos version of ZFS, not Oracle version.
The FreeBSD base system comes with this ZFS.
If you want to try latest OpenZFS then install and use these ports/packages instead:
Is the FreeBSD kernel (as distributed in the base system) licensed under a BSD license or a mix of BSD and Copyleft (like CDDL)?
The ‘default’ license that covers most of the FreeBSD system is the BSD 2-Clause license:
https://www.freebsd.org/copyright/freebsd-license.html
The ZFS was imported with its CDDL license and is still under CDDL.
For the other licenses, it depends what the license is.
You can check which Contributed Software is included in each FreeBSD release in the Release Notes:
https://www.freebsd.org/releases/12.1R/relnotes.html#userland-contrib
For example
ntpduses its own NTP license and BearSSL uses MIT license.