Just wanted to drop a big, heartfelt thank you to the Void crew. If you’re reading this: you folks make the modern linux experience pleasurable in the face of ever-encroaching bloat and complexity.
Not just bloat and complexity, but also general wrong-headedness.
Also, Void is interesting and innovative - involving a new package manager, using LibreSSL (rather than OpenSSL) by default and having a musl libc variant available, amongst other things.
You see, one of the biggest problems with a Linux distro or any large system is bootstrapping. At some point you get to the point where you need a previous version of your tooling to build the current version. This is not really desirable as it ultimately can become a very difficult to break build loop. Because of this, xbps-src is in fact not a binary, it is a collection of very intricate shell scripts that use standard tooling available on almost any Linux distribution to bootstrap a compiler, build chroot, and XBPS itself.
It’s cool that their build system is a bunch of shell scripts that will run on any Linux distro. I wrote about some related stuff here in January [1]:
Alpine’s abuild script is also a pure shell script – a pure busybox ash script! But it doesn’t run on anything besides Alpine.
Debootstrap is a shell script that also only runs on Debian (as far as I know). I believe building Debian packages (as opposed to the base image) requires a lot of native / non-shell tools.
Aboriginal Linux is a bunch of pure shell scripts as well.
So it sounds like I should download xbps-src and try it out with Oil :)
However, I also don’t think the bootstrapping issue is as big a deal as he thinks? Because you want/need some kind of isolation anyway. You don’t want your build platform leaking into your binaries, i.e. a Void package built under Debian or Alpine should not be different than a Void package built under Void.
The usual solution for Alpine is to download an Alpine image and build packages there; likewise for Debian you download a Debian image. And this image can generally be a chroot – i.e. you don’t need a dedicated piece of hardware (or a VM, or a Docker container). (However I wish that Alpine would make the chroot story more straightforward. It doesn’t feel “first class”.)
Without the chroot or similar to pin down the versions of your build toolchain, I think it’s hard to solve the “reproducibility” issue. I don’t know exactly how Void Linux does it, but I am a bit confused by the description. You still need two stages to build, and it’s natural to use your own distro as the first stage, rather than some other distro.
Linux From Scratch has the steps “preparing the host system”, “constructing a temporary system”, and “building LFS”.
Void’s build system uses isolation to build packages. From readme [1]:
xbps-src requires an utility to chroot and bind mount existing directories into a masterdir that is used as its main chroot directory. xbps-src supports multiple utilities to accomplish this task:
xbps-uunshare(1) - XBPS utility that uses user_namespaces(7) (part of xbps, default).
xbps-uchroot(1) - XBPS utility that uses namespaces and must be setgid (part of xbps).
Wow totally unrelated but “an utility” sounds so wrong. I didn’t realize there were words starting with vowels that don’t use “an.” Apparently it’s words starting with vowel sounds that get an “an.” Utility, as well as words like user and euphemism, phonetically start with a y sound and therefore don’t use “an.” Crazy that I knew that intuitively, but until I read “an utility” just now I would have said “an” goes before words starting with vowels. I had thought “an hour” and a few other h words were the only exceptions. Weird.
The bootstrap process is done by building a meta-package base-chroot (https://github.com/voidlinux/void-packages/blob/master/srcpkgs/base-chroot/template) and its dependencies using the tools available on the host.
After the bootstrap xbps-src uses base-chroot to create clean/void environment which can be used to build all the packages available.
OK thanks for the info. I guess my point is that this is very much like other Linux distros, including Alpine, and even Debian.
At least this is true in theory – I don’t see that Void necessarily has a unique take on bootstrapping. But in practice it might be better, because with the distros I’ve tried this area feels a little like undocumented black magic that only the core devs know about.
I’ve actually been using fpm to do a lot of my packing as of late, rather then dealing with deb scripts and RPM spec files. fpm does things amazingly well for most use cases.
When I first started using Void I’ll fully admit I found it via a website that listed distributions that don’t use SystemD for init.
That’s actually how I started using Void as well. It currently runs on my Home Theater PC. So far I’ve been pretty impressed by it. Although runit can be a little janky at times, it’s pretty elegant in its simplicity.
Base on Void’s implementation of runit (I have not used runit elsewhere) there are a few things that feel odd to me:
Using ‘sv disable servicename’ doesn’t really seem to disable a service. You have to rm it from /var/service anyway.
Service directory naming conventions that I always get the wrong way around. “/etc/sv/” and “/var/service/” are correct, “/etc/service/” and “/var/sv/” are not.
Service logging seems to clash with the elegance of the rest of the system. You have to add a second service dir inside the original service dir just for the logger. And you have to do this for every service.
On the other hand there are some things that I really appreciate about runit that keep me here:
The service files are damned simple. Many of them are only two lines (shebang + exec).
Easy to read bringup/boot/shutdown scripts in /etc/runit
Damn fast. Not as fast as my custom init (mainly due to a stupid btrfs scan being done if you have btrfs tools installed. Let’s not mention what it does when I have a CD in my optical drive…)
To elaborate on (1): the metaphor of “the service should stay in the foreground” is so much better than trying to handle forking services. Compared to the openrc implementation in Alpine, runit is a godsend. Openrc tries to track forking services (tradition?) and the service files have their own long list of keywords, syntax and quasi-hidden default scripts.
Previously I edited /etc/runit/core-services/03-filesystems.sh and commented out the feature:
if [ -x /bin/btrfs ]; then
msg "Activating btrfs devices..."
btrfs device scan || emergency_shell
fi
Unfortunately this file gets overwritten again when runit updates.
Alternatively I could uninstall btrfs-progs, but then I wouldn’t be able to work with other people’s (non-boot) btrfs filesystems without having to install it again.
Deactivating the btrfs scan may be easy, but keeping it deactivated permanently is harder.
Just wanted to drop a big, heartfelt thank you to the Void crew. If you’re reading this: you folks make the modern linux experience pleasurable in the face of ever-encroaching bloat and complexity.
Not just bloat and complexity, but also general wrong-headedness.
Also, Void is interesting and innovative - involving a new package manager, using LibreSSL (rather than OpenSSL) by default and having a musl libc variant available, amongst other things.
It’s cool that their build system is a bunch of shell scripts that will run on any Linux distro. I wrote about some related stuff here in January [1]:
So it sounds like I should download xbps-src and try it out with Oil :)
However, I also don’t think the bootstrapping issue is as big a deal as he thinks? Because you want/need some kind of isolation anyway. You don’t want your build platform leaking into your binaries, i.e. a Void package built under Debian or Alpine should not be different than a Void package built under Void.
The usual solution for Alpine is to download an Alpine image and build packages there; likewise for Debian you download a Debian image. And this image can generally be a chroot – i.e. you don’t need a dedicated piece of hardware (or a VM, or a Docker container). (However I wish that Alpine would make the chroot story more straightforward. It doesn’t feel “first class”.)
Without the chroot or similar to pin down the versions of your build toolchain, I think it’s hard to solve the “reproducibility” issue. I don’t know exactly how Void Linux does it, but I am a bit confused by the description. You still need two stages to build, and it’s natural to use your own distro as the first stage, rather than some other distro.
Linux From Scratch has the steps “preparing the host system”, “constructing a temporary system”, and “building LFS”.
http://www.linuxfromscratch.org/lfs/view/stable/
[1] Success with Aboriginal, Alpine, and Debian http://www.oilshell.org/blog/2018/01/15.html
Void’s build system uses isolation to build packages. From readme [1]:
I’m very slowly developing a Linux distribution using xbps-src. I can build a (still quite useless) squashfs image without root privileges [2].
[1] https://github.com/voidlinux/void-packages/blob/master/README.md
[2] https://github.com/hadrianw/tomatoaster
Wow totally unrelated but “an utility” sounds so wrong. I didn’t realize there were words starting with vowels that don’t use “an.” Apparently it’s words starting with vowel sounds that get an “an.” Utility, as well as words like user and euphemism, phonetically start with a y sound and therefore don’t use “an.” Crazy that I knew that intuitively, but until I read “an utility” just now I would have said “an” goes before words starting with vowels. I had thought “an hour” and a few other h words were the only exceptions. Weird.
The bootstrap process is done by building a meta-package
base-chroot(https://github.com/voidlinux/void-packages/blob/master/srcpkgs/base-chroot/template) and its dependencies using the tools available on the host. After the bootstrapxbps-srcusesbase-chrootto create clean/void environment which can be used to build all the packages available.OK thanks for the info. I guess my point is that this is very much like other Linux distros, including Alpine, and even Debian.
At least this is true in theory – I don’t see that Void necessarily has a unique take on bootstrapping. But in practice it might be better, because with the distros I’ve tried this area feels a little like undocumented black magic that only the core devs know about.
I have used: https://github.com/alpinelinux/alpine-chroot-install
But I haven’t yet rebuild a lot of packages under it.
Voids take reminds me more of netbsd here, which isn’t a huge surprise given who started it.
Also if anyone is interested in trying Void build scripts with OSH let me know :-)
https://github.com/oilshell/oil/issues/92
I’m not familiar with Void Linux, but it sounds BSD-ish and interesting.
I’ve actually been using fpm to do a lot of my packing as of late, rather then dealing with deb scripts and RPM spec files. fpm does things amazingly well for most use cases.
That’s actually how I started using Void as well. It currently runs on my Home Theater PC. So far I’ve been pretty impressed by it. Although runit can be a little janky at times, it’s pretty elegant in its simplicity.
What do you mean by janky here?
Base on Void’s implementation of runit (I have not used runit elsewhere) there are a few things that feel odd to me:
On the other hand there are some things that I really appreciate about runit that keep me here:
To elaborate on (1): the metaphor of “the service should stay in the foreground” is so much better than trying to handle forking services. Compared to the openrc implementation in Alpine, runit is a godsend. Openrc tries to track forking services (tradition?) and the service files have their own long list of keywords, syntax and quasi-hidden default scripts.
You can deactivate the btrfs scanning pretty easily.
Previously I edited /etc/runit/core-services/03-filesystems.sh and commented out the feature:
Unfortunately this file gets overwritten again when runit updates.
Alternatively I could uninstall btrfs-progs, but then I wouldn’t be able to work with other people’s (non-boot) btrfs filesystems without having to install it again.
Deactivating the btrfs scan may be easy, but keeping it deactivated permanently is harder.
Thats where something like OpenBSD’s sysmerge comes in handy.