Quite nice… until the X11 Forwarding – when you let an untrustworthy process connect to your X11 server, you are lost. Maybe XACE could help, but default settings are quite insecure (designed for trusted clients). Xephyr might also help. Another way is Xpra, VNC, SPICE… i.e. protocols that are somehow restricted and designed for different purpose than X11.
Modern www browsers are indeed terrible and overly complex software and I am afraid that sane way to run them would be VM (e.g. a qemu image) or dedicated hardware with read-only media.
Are FreeBSD jails really considered to be synonymous with containers? I thought jails were glorified chroots, while most container systems are based on Linux namespaces (plus chroot, plus other stuff).
Edit: I suppose it doesn’t really matter, “containerized” doesn’t have to mean a “Linux container”.
The term “container” is a marketing invention – there’s no such thing as a “Linux container” at the OS level. Linux provides primitives (chroot, cgroups, namespaces) which can be combined to create something like a FreeBSD jail. They can also be combined in other ways, for example to limit which threads can access the network, or to provide unprivileged filesystem virtualization to a process tree.
When discussing FreeBSD’s jails you’d need to compare them with some specific tool (such as LXC or OpenVZ). An OpenVZ container is pretty close in design and purpose to a FreeBSD jail, in a way that (for example) Docker containers are not.
Separately, there’s a difference of security culture. Security-oriented Linux users don’t consider chroot/namespaces to be a strong security boundary because the Linux kernel has a long history of local privilege escalations, so in the specific case of sandboxing a web browser they would probably use a virtual machine. Popular options for that are Firecracker (a sort of hardened, Linux-only QEMU) or Qubes (based on Xen).
I feel like there’s a practicality aspect of usually underestimated importance.
I’m not even talking about docker as that is extreme marketing at play. Using chroot always had its limitations and there was no well defined workflow on how much convenience you sprinkled at it.
When lxc came, one could essentially slice a machine in many virtual machines without resource overhead. And there were well defined APIs for networking, storage volumes, etc.
This is hugely convenient. If you want to experiment setting up a system you could spin up a container and try things without without polluting the host system.
Security boundaries only matter on the face of an hostile entity in the same system. This is not the case for like 90%+ of the usage people give it. Some. People even provision machines and run a single docker container in them because they have built their app with a dockerfile.
At an old job of mine, we took an old desktop computer and threw lxc with a web admin tool at it. All of the sudden, everyone could create virtual servers to test and experiment stuff. I don’t think many people (or any at all) would rely on such a setup if the remaining teanants of the host system are to be assumed hostile.
All of these are just OS Level Virtualization [1] solutions:
FreeBSD Jails - 2000
Solaris Containers/Zones - 2004
Docker - 2014
LXC - 2008
The Sun named its Solaris Zones as Containers a decade (10 years) before Docker even existed. You may call it Containers, you may call it OS Level Virtualization alternately.
Linux containers have many more build tools, and the “layers” concept (OverlayFS, aufs) is useful because of the way Unix software is traditionally installed (spread all over /lib, /bin, /usr, etc.)
Docker did actually add something (despite being sloppy in many ways). Otherwise you basically have shell scripts and file system images with a lot of duplication.
The two systems I’ve seen that use jails and zones are NearlyFreeSpeech and Joyent Manta, respectively. Neither of them allowed users to upload containers. The containers were for admins only.
In fact Joyent actually implemented the Docker interface for their cloud, accepting the file format and using Linux syscall emulation.
Also Linux was the first kernel to get resource isolation (cgroups), even though FreeBSD and Solaris had namespace isolation (chroot and family) long before.
Are FreeBSD jails really considered to be synonymous with containers?
A container is a massively overloaded term. It is used to describe (at least):
A self-contained application.
A distribution mechanism for a self-contained applications.
An isolation mechanism used to run self-contained applications in such a way that they don’t interfere with the rest of the system.
On Windows or macOS, containers typically run with hypervisor-based VM isolation. On Linux, they often do in the cloud but can also use a pile of ducktape, string, seccomp-bpf, and namespaces to provide shared-kernel virtualisation that approximates what FreeBSD jails and Solaris Zones provide out of the box.
I think setting the jail’s securelevel to 3 via jail.conf would be a good edition.
A few things to note: virtualization (at least, on FreeBSD (Windows is somewhat of a different story in certain cases)) does not stop exploitation. As such, virtualization tech don’t really increase security. This is especially true for jails, wherein the kernel is shared among all the jails. FreeBSD has a habit of “kernel infoleak as feature”, which makes escaping the jail much easier. When in a jailed context, as an unprivileged user, you can run sysctl -a | grep 0xf and gain a lot of information about the kernel (more than enough to make exploitation incredibly easy). And, remember, there’s a whole heck of a lot more ways to interact with the kernel than just devfs.
I use jails quite a bit. I use them for natural segregation of services and to keep things organized. FreeBSD jails are incredibly attractive for this particular use case.
It’s a shame that the Capsicum patches were never upstreamed (Google refuses to take patches to support any operating system that they don’t ship products for, which is definitely not illegally using their dominant market position in the browser market to influence the OS market, honest). They required about 100LoC to isolate the renderer process on FreeBSD. David Drysdale tried to get Capsicum support upstreamed to Linux for a while on behalf of the CromeOS team but eventually gave up.
There are a lot of details in the original Capsicum paper. More recent FreeBSD has a reworked Capsicum implementation that avoids the wrapper file descriptors described in the paper and gives a more extensible permissions mask.
Quite nice… until the X11 Forwarding – when you let an untrustworthy process connect to your X11 server, you are lost. Maybe XACE could help, but default settings are quite insecure (designed for trusted clients). Xephyr might also help. Another way is Xpra, VNC, SPICE… i.e. protocols that are somehow restricted and designed for different purpose than X11.
Modern www browsers are indeed terrible and overly complex software and I am afraid that sane way to run them would be VM (e.g. a qemu image) or dedicated hardware with read-only media.
Ready to use solution is probably QubesOS.
I have just added UPDATE 1 - Use XPRA Instead of X11 Forwarding section with XPRA method if that pleases you :)
nice :-)
Are FreeBSD jails really considered to be synonymous with containers? I thought jails were glorified chroots, while most container systems are based on Linux namespaces (plus chroot, plus other stuff).
Edit: I suppose it doesn’t really matter, “containerized” doesn’t have to mean a “Linux container”.
The term “container” is a marketing invention – there’s no such thing as a “Linux container” at the OS level. Linux provides primitives (chroot, cgroups, namespaces) which can be combined to create something like a FreeBSD jail. They can also be combined in other ways, for example to limit which threads can access the network, or to provide unprivileged filesystem virtualization to a process tree.
When discussing FreeBSD’s jails you’d need to compare them with some specific tool (such as LXC or OpenVZ). An OpenVZ container is pretty close in design and purpose to a FreeBSD jail, in a way that (for example) Docker containers are not.
Separately, there’s a difference of security culture. Security-oriented Linux users don’t consider chroot/namespaces to be a strong security boundary because the Linux kernel has a long history of local privilege escalations, so in the specific case of sandboxing a web browser they would probably use a virtual machine. Popular options for that are Firecracker (a sort of hardened, Linux-only QEMU) or Qubes (based on Xen).
I feel like there’s a practicality aspect of usually underestimated importance. I’m not even talking about docker as that is extreme marketing at play. Using chroot always had its limitations and there was no well defined workflow on how much convenience you sprinkled at it.
When lxc came, one could essentially slice a machine in many virtual machines without resource overhead. And there were well defined APIs for networking, storage volumes, etc. This is hugely convenient. If you want to experiment setting up a system you could spin up a container and try things without without polluting the host system. Security boundaries only matter on the face of an hostile entity in the same system. This is not the case for like 90%+ of the usage people give it. Some. People even provision machines and run a single docker container in them because they have built their app with a dockerfile.
At an old job of mine, we took an old desktop computer and threw lxc with a web admin tool at it. All of the sudden, everyone could create virtual servers to test and experiment stuff. I don’t think many people (or any at all) would rely on such a setup if the remaining teanants of the host system are to be assumed hostile.
Thanks for the explanation.
All of these are just OS Level Virtualization [1] solutions:
The Sun named its Solaris Zones as Containers a decade (10 years) before Docker even existed. You may call it Containers, you may call it OS Level Virtualization alternately.
[1] https://en.wikipedia.org/wiki/OS-level_virtualization
AFAIK it’s the other way. Containers just wish they were as good as jails and zones
Linux containers have many more build tools, and the “layers” concept (OverlayFS, aufs) is useful because of the way Unix software is traditionally installed (spread all over /lib, /bin, /usr, etc.)
Docker did actually add something (despite being sloppy in many ways). Otherwise you basically have shell scripts and file system images with a lot of duplication.
The two systems I’ve seen that use jails and zones are NearlyFreeSpeech and Joyent Manta, respectively. Neither of them allowed users to upload containers. The containers were for admins only.
In fact Joyent actually implemented the Docker interface for their cloud, accepting the file format and using Linux syscall emulation.
Also Linux was the first kernel to get resource isolation (cgroups), even though FreeBSD and Solaris had namespace isolation (chroot and family) long before.
A container is a massively overloaded term. It is used to describe (at least):
On Windows or macOS, containers typically run with hypervisor-based VM isolation. On Linux, they often do in the cloud but can also use a pile of ducktape, string, seccomp-bpf, and namespaces to provide shared-kernel virtualisation that approximates what FreeBSD jails and Solaris Zones provide out of the box.
I think setting the jail’s securelevel to 3 via
jail.confwould be a good edition.A few things to note: virtualization (at least, on FreeBSD (Windows is somewhat of a different story in certain cases)) does not stop exploitation. As such, virtualization tech don’t really increase security. This is especially true for jails, wherein the kernel is shared among all the jails. FreeBSD has a habit of “kernel infoleak as feature”, which makes escaping the jail much easier. When in a jailed context, as an unprivileged user, you can run
sysctl -a | grep 0xfand gain a lot of information about the kernel (more than enough to make exploitation incredibly easy). And, remember, there’s a whole heck of a lot more ways to interact with the kernel than just devfs.I use jails quite a bit. I use them for natural segregation of services and to keep things organized. FreeBSD jails are incredibly attractive for this particular use case.
It’s a shame that the Capsicum patches were never upstreamed (Google refuses to take patches to support any operating system that they don’t ship products for, which is definitely not illegally using their dominant market position in the browser market to influence the OS market, honest). They required about 100LoC to isolate the renderer process on FreeBSD. David Drysdale tried to get Capsicum support upstreamed to Linux for a while on behalf of the CromeOS team but eventually gave up.
I’m not familiar with this - can you tell us more?
There are a lot of details in the original Capsicum paper. More recent FreeBSD has a reworked Capsicum implementation that avoids the wrapper file descriptors described in the paper and gives a more extensible permissions mask.
Thank you!