Not very much. Most of the core utilities are incrementally improved from the 4BSD versions and so have only supported newer kernel features when they actually need them. There’s no benefit, for example, for ls to use kqueue.
This is different for newer tools. Things like pkg support jails sort-of natively, though in the case of pkg it is by running a copy of the tool inside the jail, rather than doing a jail-attach thing. You can probably just disable the -J option.
Things like ifconfig depend on kernel interfaces that are very specific to FreeBSD. I doubt you could port ifconfig to Linux without it being close to a complete rewrite. I suspect the same is true of things like systat, iostat, fstat and all of the similar things that make a FreeBSD environment actually pleasant to work with. The FreeBSD utilities generally install a SIGINFO handler because ^T on a FreeBSD terminal sends SIGINFO to the controlling process. This isn’t supported by Linux and is one of the main reasons I prefer *BSD (including macOS) to Linux for terminal work. For example, if you’re doing a big cp or dd, you can hit ^T to discover how far it’s got and get some useful statistics. If you run these commands on a Linux kernel, this won’t work.
I wonder how much of the kernel API surface for ifconfig and *stat could be polyfilled with a kernel module.
Part of the problem is that these are not long-term stable KBIs. The system call interface on FreeBSD has very strong backwards compat guarantees. You may need to load a COMPAT{version} module to enable old behaviours, but they’re maintained for ages (and generally compiled into the GENERIC kernel). A generic userland program from FreeBSD 4 will run happily on FreeBSD 12. The same guarantees are present in libc and most of the system libraries: a dynamically linked FreeBSD 4.x binary that links against the userland libraries will also work today.
In contrast, the control interfaces are stable only within a release series. ifconfg from FreeBSD 12 is not guaranteed to work on FreeBSD 13. That’s generally fine, because these programs are part of the base system and so you have the version that matches your kernel installed (note: jails for older FreeBSD versions are a bit of a problem here, so sometimes you end up with some compat stuff in the kernel) but they are a problem if you want to write a compat layer and not have to rewrite it every few years to support the newer tools. The compat code in FreeBSD itself is the wrong way around for you: it’s aimed at supporting old userland on new kernels, there’s never any expectation that new userland will work with old kernels.
That said, I’d be really happy to review any patches to allow FreeBSD libc to work on Linux. There’s stuff now to separate out the syscalls layer, so I can imagine using libkqueue and a few other things as a basis for a Linux version of libsyscalls.
It’s interesting to see how different platforms choose different ABI boundaries.
Linux has a fanatical commitment to maintaining the kernel’s userspace ABI but internally everything is up for refactoring. That allows any old software to continue to run on newer kernels while also allowing the internal kernel APIs to evolve over time.
I guess with FreeBSD the stable kernel interface is the subset needed by anything except system software? How is that distinction documented? These days Linux is littered with legacy kernel APIs that are usually, in practice disabled or only present if you load the right kernel module.
Does FreeBSD have a concept of stability for the interface of tools like ifconfig so that even as the kernel interfaces change scripts that call it will work across releases?
I guess with FreeBSD the stable kernel interface is the subset needed by anything except system software?
What do you mean ‘kernel interface’? The KBI is incredibly stable within a FreeBSD major release series, to the extent that you should always be able to compile a kernel module for X.0 and load it on X.n. The major releases all have around a 5-year support cycle. Before a release is branched, folks go and add padding fields to any internal structures that they may want to extend over the lifetime of the branch. There are a few special ones that are allowed to grow at the end (and may only be allocated in specific places or by dynamically querying their size).
The ABI is stable for system calls. If a system call is replaced with a newer version, a new system call number is allocated and the old one is preserved. libc papers over the difference. Eventually the old system call is deprecated and, once libc no longer exposes it, it is hidden behind #ifdef COMPAT{VERSION} defines. Eventually, the compat code is removed (or, at least, removed from the default builds).
The ABI is not stable for system configuration interfaces. This basically means control-plane ioctls.
Does FreeBSD have a concept of stability for the interface of tools like ifconfig so that even as the kernel interfaces change scripts that call it will work across releases?
Most of these tools on FreeBSD use libxo and so can produce structured JSON or XML output. If you want to consume them in a tool, you’re recommended to parse the JSON. In practice, the human-readable output tends to be very stable, but it’s not guaranteed.
Most of the tools these days have a library that wraps the kernel functionality in stable interfaces (e.g. libprocstat). If you link against those, rather than parsing the output of the tool, then you get stable interfaces. Because all of these things are 2-clause BSD licensed, there’s no legal problem for anyone to link against them.
I would argue that control plane ioctls form part of the kernel syscall interface, but that’s not a particularly interesting argument.
The problem is that these typically leak a lot of implementation details. For example, when FreeBSD 3 first gained support for wireless networking, the kernel implementation separated the physical interfaces from logical ones. You configured the NIC instances with ifconfig but then had to create a wlan device with wiconfig (which wrapped one or more physical interfaces) to configure the WiFi-specific parameters. Eventually (around FreeBSD 6?) these were combined and the NIC-configuration interfaces were exposed as part of the NIC devices. The user-facing wiconfig command went away, but so did /dev/wlan[0-9]+ entirely and therefore so did all of the corresponding ioctls. The devices that ifconfig used gained some extra ioctls.
It might be possible to implement a compat wlan layer that mimics the semantics by forwarding ioctls to the underlying one. One of the things I’ve been tempted to play with for a while is implementing some of these with CUSE so that legacy jails can run keep the compat code out of the kernel and running with the restricted rights of the jail.
Your comments come off as pretty dismissive and judgemental. If you wanted to ask “what is the advantage of this?” or “what can I do with this that I can’t with regular Linux?” Then sure, those are perfectly reasonable questions.
But your comments, to me at least, read more like “why would you even do this?” or “why are you wasting your time on this?”
There is an expected audience. I’m here, reading about this persons project.
I truly don’t get what you’re complaining about.
If you want some practical applicability, you can consider this a way to use Linux while avoiding any GPLv3 code. That’s certainly practically useful for a bunch of commercial projects which aren’t too fond of the anti-tivoization clause. But even if there had been no such utility to it, having side projects is great, and posting about technical side projects is appropriate for lobste.re.
Depends on what you mean by “this,” I guess. I’m not sure this is the first attempt at a hybrid like this. There is or was a Debian project that tried to implement a GNU user land on a BSD kernel IIRC.
Not sure I see a lot of value in it, but that’s one of the nice things about FOSS, people can do these things and sometimes they turn out great.
I’ve always wanted the FreeBSD userland with Linux kernel. Whether the Linux kernel is “better” or not aside, it’s clearly the more pragmatic choice, and I much prefer the FreeBSD userland.
I tried exactly this over ten years ago. It was too much work because of libc issues at the time, and also because I’m a bad C programmer. Looks like musl filled in the missing piece on that.
Although by now I’m pretty used to the GNU tools, so it makes less difference than it used to.
musl is not necessarily a factor, we have bsdutils working on glibc as well; but since i’m already doing this, might as well, since you’re not going to get compatibility with existing binaries either way, and musl is the cleaner C library (not to mention things like reasonable long doubles on powerpc64 and so on)
To be honest I don’t recall exactly; there were loads of issues even with smaller tools, more than I was prepared to fix anyway. I think there might still be a post on the FreeBSD forums or daemonforums about it somewhere.
One advantage of using GNU libc would be that you can run programs that don’t run well with musl libc, but then you’d of course lose the “No GNU” bullet point. Purely practically speaking, it’s an advantage on desktop systems especially (and also why I stopped using musl on Void and use GNU libc; musl it works great right up to the point it doesn’t).
But I mostly stopped caring what exact system I use beyond some basics (e.g. Debian’s package philosophy doesn’t really jibe for me).
If you ever feel like porting FreeBSD libc to Linux for this project, I’d be very happy to review any patches you want to send upstream. I’d love to see FreeBSD libc as a reusable component for other systems.
very nice.
A good number of use cases for this that I could think of:
I tend to use different compilers/databases/OS targets to clean up and sharpen interfaces/installation procedures/dependencies/etc. Even if the particular commercial deliverables are targeting just one os/compiler/db combination.
This is also an interesting platform for a ‘holistic’ benchmarking. Meaning that differences in benchmarks of the same program on the same hardware for Chimera vs, say, Linux may be, at least, directionally indicative were the particular bottlenecks are.
Another tangential comparasing: Android is a type of a ‘chimera’ OS (linux core + a very different ‘user land’). So just the work that goes into Leah’s effort, fundamentally enables these kinds of separations, and could, some time in the future, enable projects that create ‘device centric’ userland while keeping core rooted in particular kernel
I wonder how much of the core FreeBSD userland is dependent on FreeBSD-isms like jails and kqueue.
Not very much. Most of the core utilities are incrementally improved from the 4BSD versions and so have only supported newer kernel features when they actually need them. There’s no benefit, for example, for
lsto usekqueue.This is different for newer tools. Things like
pkgsupport jails sort-of natively, though in the case ofpkgit is by running a copy of the tool inside the jail, rather than doing a jail-attach thing. You can probably just disable the-Joption.Things like
ifconfigdepend on kernel interfaces that are very specific to FreeBSD. I doubt you could portifconfigto Linux without it being close to a complete rewrite. I suspect the same is true of things likesystat,iostat,fstatand all of the similar things that make a FreeBSD environment actually pleasant to work with. The FreeBSD utilities generally install aSIGINFOhandler because ^T on a FreeBSD terminal sends SIGINFO to the controlling process. This isn’t supported by Linux and is one of the main reasons I prefer *BSD (including macOS) to Linux for terminal work. For example, if you’re doing a bigcpordd, you can hit ^T to discover how far it’s got and get some useful statistics. If you run these commands on a Linux kernel, this won’t work.I wonder how much of the kernel API surface for
ifconfigand*statcould be polyfilled with a kernel module.As for
^TI bet that’s a simple kernel patch. The Linux tty subsystem already knows how to send signals to the controlling process on^Zand^C.Part of the problem is that these are not long-term stable KBIs. The system call interface on FreeBSD has very strong backwards compat guarantees. You may need to load a COMPAT{version} module to enable old behaviours, but they’re maintained for ages (and generally compiled into the GENERIC kernel). A generic userland program from FreeBSD 4 will run happily on FreeBSD 12. The same guarantees are present in libc and most of the system libraries: a dynamically linked FreeBSD 4.x binary that links against the userland libraries will also work today.
In contrast, the control interfaces are stable only within a release series.
ifconfgfrom FreeBSD 12 is not guaranteed to work on FreeBSD 13. That’s generally fine, because these programs are part of the base system and so you have the version that matches your kernel installed (note: jails for older FreeBSD versions are a bit of a problem here, so sometimes you end up with some compat stuff in the kernel) but they are a problem if you want to write a compat layer and not have to rewrite it every few years to support the newer tools. The compat code in FreeBSD itself is the wrong way around for you: it’s aimed at supporting old userland on new kernels, there’s never any expectation that new userland will work with old kernels.That said, I’d be really happy to review any patches to allow FreeBSD libc to work on Linux. There’s stuff now to separate out the syscalls layer, so I can imagine using libkqueue and a few other things as a basis for a Linux version of libsyscalls.
It’s interesting to see how different platforms choose different ABI boundaries.
Linux has a fanatical commitment to maintaining the kernel’s userspace ABI but internally everything is up for refactoring. That allows any old software to continue to run on newer kernels while also allowing the internal kernel APIs to evolve over time.
I guess with FreeBSD the stable kernel interface is the subset needed by anything except system software? How is that distinction documented? These days Linux is littered with legacy kernel APIs that are usually, in practice disabled or only present if you load the right kernel module.
Does FreeBSD have a concept of stability for the interface of tools like
ifconfigso that even as the kernel interfaces change scripts that call it will work across releases?What do you mean ‘kernel interface’? The KBI is incredibly stable within a FreeBSD major release series, to the extent that you should always be able to compile a kernel module for X.0 and load it on X.n. The major releases all have around a 5-year support cycle. Before a release is branched, folks go and add padding fields to any internal structures that they may want to extend over the lifetime of the branch. There are a few special ones that are allowed to grow at the end (and may only be allocated in specific places or by dynamically querying their size).
The ABI is stable for system calls. If a system call is replaced with a newer version, a new system call number is allocated and the old one is preserved. libc papers over the difference. Eventually the old system call is deprecated and, once libc no longer exposes it, it is hidden behind
#ifdef COMPAT{VERSION}defines. Eventually, the compat code is removed (or, at least, removed from the default builds).The ABI is not stable for system configuration interfaces. This basically means control-plane
ioctls.Most of these tools on FreeBSD use
libxoand so can produce structured JSON or XML output. If you want to consume them in a tool, you’re recommended to parse the JSON. In practice, the human-readable output tends to be very stable, but it’s not guaranteed.Most of the tools these days have a library that wraps the kernel functionality in stable interfaces (e.g.
libprocstat). If you link against those, rather than parsing the output of the tool, then you get stable interfaces. Because all of these things are 2-clause BSD licensed, there’s no legal problem for anyone to link against them.I would argue that control plane ioctls form part of the kernel syscall interface, but that’s not a particularly interesting argument.
It seems like a Linux port of
libprocstatwould allow a lot of system tools from FreeBSD to work just fine.The problem is that these typically leak a lot of implementation details. For example, when FreeBSD 3 first gained support for wireless networking, the kernel implementation separated the physical interfaces from logical ones. You configured the NIC instances with
ifconfigbut then had to create awlandevice withwiconfig(which wrapped one or more physical interfaces) to configure the WiFi-specific parameters. Eventually (around FreeBSD 6?) these were combined and the NIC-configuration interfaces were exposed as part of the NIC devices. The user-facingwiconfigcommand went away, but so did/dev/wlan[0-9]+entirely and therefore so did all of the correspondingioctls. The devices thatifconfigused gained some extraioctls.It might be possible to implement a compat
wlanlayer that mimics the semantics by forwardingioctls to the underlying one. One of the things I’ve been tempted to play with for a while is implementing some of these with CUSE so that legacy jails can run keep the compat code out of the kernel and running with the restricted rights of the jail.Fantastic
But why
because i wanted to
Reason enough for me.
Just because we can.
Why not?
[Comment removed by moderator pushcx: Don't insult people for making projects.]
Are we really going to judge how people choose to spend their spare time? We could be having this conversation on half the stories posted to Lobsters.
Well, I ask this question on embedded interpreter projects too. If you’re not expecting an audience, then why even post the link?
Your comments come off as pretty dismissive and judgemental. If you wanted to ask “what is the advantage of this?” or “what can I do with this that I can’t with regular Linux?” Then sure, those are perfectly reasonable questions.
But your comments, to me at least, read more like “why would you even do this?” or “why are you wasting your time on this?”
There is an expected audience. I’m here, reading about this persons project.
I truly don’t get what you’re complaining about.
If you want some practical applicability, you can consider this a way to use Linux while avoiding any GPLv3 code. That’s certainly practically useful for a bunch of commercial projects which aren’t too fond of the anti-tivoization clause. But even if there had been no such utility to it, having side projects is great, and posting about technical side projects is appropriate for lobste.re.
Weird that it tool this long before somebody try this.
Depends on what you mean by “this,” I guess. I’m not sure this is the first attempt at a hybrid like this. There is or was a Debian project that tried to implement a GNU user land on a BSD kernel IIRC.
Not sure I see a lot of value in it, but that’s one of the nice things about FOSS, people can do these things and sometimes they turn out great.
I’ve always wanted the FreeBSD userland with Linux kernel. Whether the Linux kernel is “better” or not aside, it’s clearly the more pragmatic choice, and I much prefer the FreeBSD userland.
I tried exactly this over ten years ago. It was too much work because of libc issues at the time, and also because I’m a bad C programmer. Looks like musl filled in the missing piece on that.
Although by now I’m pretty used to the GNU tools, so it makes less difference than it used to.
musl is not necessarily a factor, we have bsdutils working on glibc as well; but since i’m already doing this, might as well, since you’re not going to get compatibility with existing binaries either way, and musl is the cleaner C library (not to mention things like reasonable long doubles on powerpc64 and so on)
To be honest I don’t recall exactly; there were loads of issues even with smaller tools, more than I was prepared to fix anyway. I think there might still be a post on the FreeBSD forums or daemonforums about it somewhere.
One advantage of using GNU libc would be that you can run programs that don’t run well with musl libc, but then you’d of course lose the “No GNU” bullet point. Purely practically speaking, it’s an advantage on desktop systems especially (and also why I stopped using musl on Void and use GNU libc; musl it works great right up to the point it doesn’t).
But I mostly stopped caring what exact system I use beyond some basics (e.g. Debian’s package philosophy doesn’t really jibe for me).
If you ever feel like porting FreeBSD libc to Linux for this project, I’d be very happy to review any patches you want to send upstream. I’d love to see FreeBSD libc as a reusable component for other systems.
very nice. A good number of use cases for this that I could think of:
I tend to use different compilers/databases/OS targets to clean up and sharpen interfaces/installation procedures/dependencies/etc. Even if the particular commercial deliverables are targeting just one os/compiler/db combination.
This is also an interesting platform for a ‘holistic’ benchmarking. Meaning that differences in benchmarks of the same program on the same hardware for Chimera vs, say, Linux may be, at least, directionally indicative were the particular bottlenecks are.
Another tangential comparasing: Android is a type of a ‘chimera’ OS (linux core + a very different ‘user land’). So just the work that goes into Leah’s effort, fundamentally enables these kinds of separations, and could, some time in the future, enable projects that create ‘device centric’ userland while keeping core rooted in particular kernel