Sounds like another one of those common software development tradeoffs. Building a system with several very disconnected building blocks gives you a certain kind of flexibility, and building a system where everything is developed together as an integrated whole gives you a different kind of flexibility. And they each have their constraints as well.
The Linux way allows you to mix and match building blocks however you want, hence the huge number of distros built for a wide variety of people with different preferences and needs. But that means you have to be careful and slow when making changes.
The BSD way allows you to change the whole system a lot easier because the building blocks aren’t scattered all over the place. You can move faster and easier, but you make it harder to mix and match, hence the relatively few different flavors of BSD.
Both ways make a ton of sense. I am personally glad both options exist.
I do not fully understand they did not just update ifconfig(8) – random IRC rumors say there was a failed attempt due to ifconfig(8)’s convoluted code-base.
The reason is that ifconfig maintains bugs. The Linux kernel is irrelevant. If ifconfig would change its output format, many scripts around the world would break.
So the real title should be: “ifconfig maintains bugs: The real reason ifconfig on Linux is deprecated”.
(Also, there are ways of transitioning over to new APIs; you could release an ifconfig2 or whatever with new behavior, or add a flag for a new structured output that might be more pleasant to parse anyhow.)
This seems like a typical versioning API problem. You have those naturally at the boundaries of teams or maintainership. FreeBSD seems to move the boundary further up so that legacy support is clearly separated from core interfaces by layering. I like it.
I wonder how the compat layer is used. It sounds like you need to choose to use it which might be bothersome. Your stuff stops working, you debug, you use the compat layer. Much wasted effort.
If the interface is clearly versioned, things might be too conservative but automatic.
From a kernel perspective, with FreeBSD, lots of the compat is on by default for quite a while(one might argue too long). FreeBSD really is known for not breaking things willy nilly. In my experience, even when they do finally, almost reluctantly, turn off compat, they leave it optionally enabled by a recompile. FreeBSD is amazingly stable in that regard. Unless it’s a giant security hole that can only be fixed by breaking userland, the stable branch tends to be very very stable. They would only break compat during a major release (say 11.X to 12)
For instance, FreeBSD right now is in the middle of merging the linux ZFS stack into FreeBSD replacing the Illumos upstream, which comes with lots of potential incompatibility. In 12.X(read stable) it’s an optional package one can install and play with, and it’s very well labelled as experimental. It should become the default with the 13.X release. Meanwhile they keep updating the experimental version in 12, working through the bugs. So users can play with the “new hotness” now, even in the stable branch, if they want, but are warned. Plus of course you can install the 13 branch(unstable/development) to play and test as well if you want.
From a kernel perspective, with FreeBSD, lots of the compat is on by default for quite a while(one might argue too long). FreeBSD really is known for not breaking things willy nilly
That’s true to a degree, but it’s also a bit more complicated. In some ways, FreeBSD’s compat guarantees are a lot stronger than Linux’s, but in others they’re weaker. FreeBSD does not break ABIs (or KBIs) within a major release. I was amused to learn recently that Debian/kFreeBSD added FreeBSD support to DKMS, even though it’s solving a problem that simply does not exist with a FreeBSD kernel: a kernel module for FreeBSD X.Y will work with FreeBSD X.(Y+1). It will (almost?) definitely not work with FreeBSD X+1.
For the userspace APIs, FreeBSD guarantees backwards compatibility within a major release and across releases if you install the compat libraries and build your kernel with the right COMPAT_* options. The kernel options will include compatibility versions of system calls (when a system call changes in an incompatible way, it gets a new syscall number. The old one is supported only when the relevant compat options are included). Most FreeBSD core libraries use symbol versioning, so you don’t need to have multiple versions installed, but the compat packages include old versions of libraries that have had SONAME bumps or have been removed in newer FreeBSD.
One place where Linux does a lot better; however, is in admin interfaces. FreeBSD does not guarantee compat between major versions for the ioctls used for control plane things. For example, all of the things that ifconfig uses may change in incompatible ways between major versions. Linux tries much harder to keep these stable (though they may completely remove things, like OSS, and just expect the world to rewrite software to suit this week’s APIs - basically, all bets are off when it comes to device drivers and each one has a different support story), so an old ifconfig still works on a new Linux.
This is a problem for FreeBSD when you combine it with jails. You could run a FreeBSD 9 jail on a FreeBSD 10 system for testing things on older systems, but unfortunately you couldn’t run FreeBSD 9’s ifconfig because the interfaces it needed had changed. I believe that there’s been work recently to add compat layers for a bunch of those things, though I’d prefer to see a setup where the host system could configure everything for a jail, without needing all of the init / RC goo in each jail.
merging the linux ZFS stack into FreeBSD replacing the Illumos upstream
huh, is that actually happening? I’ve seen some patches from ZoL land, but in -CURRENT the stock ZFS still uses opensolaris.ko, and the ZoL-based thing is still only in the external openzfs-kmod, regardless of whether you run -STABLE or -CURRENT
Last I heard[0], but I’m not part of that team, so I can’t say with any certainty. I imagine it’s still their goal, but perhaps there are some blockers delaying the adoption? I don’t have any new information around this.
The downside for BSD is a higher coupling between user- and kernel land. Suppose a bug is found in BSDs ifconfig and it is fixed in a newer BSD release. If you cannot backport the bug fix patch (for whatever reason), then you have to update your whole BSD. On Linux, you just use the new ifconfig.
its worth noting that OpenBSD and NetBSD do not have these libraries because the kernel interface itself is highly stable anyways. FreeBSD even provides a COMPAT layer in the rare cases that an older binary fails to run on modern versions of FreeBSD.
Sounds like another one of those common software development tradeoffs. Building a system with several very disconnected building blocks gives you a certain kind of flexibility, and building a system where everything is developed together as an integrated whole gives you a different kind of flexibility. And they each have their constraints as well.
The Linux way allows you to mix and match building blocks however you want, hence the huge number of distros built for a wide variety of people with different preferences and needs. But that means you have to be careful and slow when making changes.
The BSD way allows you to change the whole system a lot easier because the building blocks aren’t scattered all over the place. You can move faster and easier, but you make it harder to mix and match, hence the relatively few different flavors of BSD.
Both ways make a ton of sense. I am personally glad both options exist.
^ That seems like the “real reason”.
The reason is that ifconfig maintains bugs. The Linux kernel is irrelevant. If ifconfig would change its output format, many scripts around the world would break.
The reasons seem connected, though. If maintaining known bugs weren’t the policy, then
ifconfigcould have been rewritten.It can still be rewritten and use the new interfaces.
So the real title should be: “ifconfig maintains bugs: The real reason ifconfig on Linux is deprecated”.
(Also, there are ways of transitioning over to new APIs; you could release an
ifconfig2or whatever with new behavior, or add a flag for a new structured output that might be more pleasant to parse anyhow.)This seems like a typical versioning API problem. You have those naturally at the boundaries of teams or maintainership. FreeBSD seems to move the boundary further up so that legacy support is clearly separated from core interfaces by layering. I like it.
I wonder how the compat layer is used. It sounds like you need to choose to use it which might be bothersome. Your stuff stops working, you debug, you use the compat layer. Much wasted effort.
If the interface is clearly versioned, things might be too conservative but automatic.
From a kernel perspective, with FreeBSD, lots of the compat is on by default for quite a while(one might argue too long). FreeBSD really is known for not breaking things willy nilly. In my experience, even when they do finally, almost reluctantly, turn off compat, they leave it optionally enabled by a recompile. FreeBSD is amazingly stable in that regard. Unless it’s a giant security hole that can only be fixed by breaking userland, the stable branch tends to be very very stable. They would only break compat during a major release (say 11.X to 12)
For instance, FreeBSD right now is in the middle of merging the linux ZFS stack into FreeBSD replacing the Illumos upstream, which comes with lots of potential incompatibility. In 12.X(read stable) it’s an optional package one can install and play with, and it’s very well labelled as experimental. It should become the default with the 13.X release. Meanwhile they keep updating the experimental version in 12, working through the bugs. So users can play with the “new hotness” now, even in the stable branch, if they want, but are warned. Plus of course you can install the 13 branch(unstable/development) to play and test as well if you want.
That’s true to a degree, but it’s also a bit more complicated. In some ways, FreeBSD’s compat guarantees are a lot stronger than Linux’s, but in others they’re weaker. FreeBSD does not break ABIs (or KBIs) within a major release. I was amused to learn recently that Debian/kFreeBSD added FreeBSD support to DKMS, even though it’s solving a problem that simply does not exist with a FreeBSD kernel: a kernel module for FreeBSD X.Y will work with FreeBSD X.(Y+1). It will (almost?) definitely not work with FreeBSD X+1.
For the userspace APIs, FreeBSD guarantees backwards compatibility within a major release and across releases if you install the compat libraries and build your kernel with the right
COMPAT_*options. The kernel options will include compatibility versions of system calls (when a system call changes in an incompatible way, it gets a new syscall number. The old one is supported only when the relevant compat options are included). Most FreeBSD core libraries use symbol versioning, so you don’t need to have multiple versions installed, but the compat packages include old versions of libraries that have had SONAME bumps or have been removed in newer FreeBSD.One place where Linux does a lot better; however, is in admin interfaces. FreeBSD does not guarantee compat between major versions for the
ioctls used for control plane things. For example, all of the things thatifconfiguses may change in incompatible ways between major versions. Linux tries much harder to keep these stable (though they may completely remove things, like OSS, and just expect the world to rewrite software to suit this week’s APIs - basically, all bets are off when it comes to device drivers and each one has a different support story), so an old ifconfig still works on a new Linux.This is a problem for FreeBSD when you combine it with jails. You could run a FreeBSD 9 jail on a FreeBSD 10 system for testing things on older systems, but unfortunately you couldn’t run FreeBSD 9’s ifconfig because the interfaces it needed had changed. I believe that there’s been work recently to add compat layers for a bunch of those things, though I’d prefer to see a setup where the host system could configure everything for a jail, without needing all of the init / RC goo in each jail.
agreed and thanks for the detailed response!
I was trying to keep my response simple, but you definitely added more detail, and it’s much appreciated!
huh, is that actually happening? I’ve seen some patches from ZoL land, but in -CURRENT the stock ZFS still uses
opensolaris.ko, and the ZoL-based thing is still only in the externalopenzfs-kmod, regardless of whether you run -STABLE or -CURRENTLast I heard[0], but I’m not part of that team, so I can’t say with any certainty. I imagine it’s still their goal, but perhaps there are some blockers delaying the adoption? I don’t have any new information around this.
0: https://papers.freebsd.org/2019/bsdcan/jude-the_future_of_openzfs_and_freebsd/
The downside for BSD is a higher coupling between user- and kernel land. Suppose a bug is found in BSDs ifconfig and it is fixed in a newer BSD release. If you cannot backport the bug fix patch (for whatever reason), then you have to update your whole BSD. On Linux, you just use the new ifconfig.
According to the article:
The updated BSD is compatible with binaries built for the older BSD though