The real point seems to be that linux container tech isn’t a supported product, it’s a big name for a collection of tools and patterns and kernel features.
Now jails and zones probably use tools and patterns and kernel features, but there’s an overarching cathedral-style spec. Containers are a bazaar-style buffet of different tools interacting in (sometimes surprising) ways.
This almost reads (to me) like an attempt to argue that it is better that containers provide no security, because security adds complexity, and because legos are great. Legos are great, but I don’t think I would want to put my money in a safe built out of legos.
I think people who want to promote containers should try arguing more along the lines of “containers are like static binaries for ecosystems that can’t build static binaries”. That argument at least makes more sense to me.
Linux containers as they exist today were originally written at Google for process management, in particular for resource sharing / limiting / guarantees. So whether or not it’s better that containers provide no security is immaterial, they weren’t designed to provide security, that’s not what they’re for. There is no debate about whether you should put your money in a safe built out of legos. You shouldn’t, that’s not what Linux containers are for, the end.
OpenVZ is the other big contributor to Linux container tech, but lots of their work isn’t mainline. I don’t personally know whether their containers are secure, though that does seem to be their aim.
So whether or not it’s better that containers provide no security is immaterial, they weren’t designed to provide security, that’s not what they’re for.
The usage of “immaterial” seems a bit odd to me . Certainly what level of security they provide is material if I’m planning on running internet-facing applications in a container. SMTP wasn’t built with security in mind either, a fact which has become of material concern.
I suppose another way to interpret your comment is that one should not assume they provide any security at all, which is probably what people should do. Unfortunately, in my limited experience that message hasn’t seemed to have really spread. I think securing a Linux containers deployment is probably something that could use a bit more attention.
I’m not sure who this is targeted towards. The people who like zones/jails or the people who say containers are a thing. Because from my occasional interactions with these groups, there’s a bunch of people saying “containers are awesome!!!” and then there’s some BSD yahoos saying “no, actually, jails are better.” I think this second group is wrong only to the extent that they’ve accepted the premise put forth by the first group? I guess the lesson is if I’m arguing with a linux advocate who tells me containers are great, I should just reply “nope, wrong, containers aren’t real”. That sounds kinda trollish though.
I love this. I’ve heard a ton of startup bros talk about containers as if they’re some brand new panacea that Linux crafted from the ether, without actually knowing any of the history behind them. This was a very satisfying read.
I’ve spent some time elucidating on the history to those smitten with the container “gold rush” and it’s made exactly zero impact – not positive or negative – just zero change. it has not helped those I’ve spoken with understand anything more clearly, or approach their usage (planned or existing) of containers differently…literally no impact.
the hype is very real. very strong. and very, very vague.
filesystems can be constructed as needed using zfs + nullfs, and hopefully soon 9p2000 and shared across containers
you design the network as required, you can drop in loopbacks, external interfaces as you like, and access them from the host if you want to
I’m not sure on how much of this could be changed at runtime, but most of the usual needs (introspection, sharing) have straightforward solutions.
I haven’t succeeded in getting dtrace working from the host system into a jailed process but maybe somebody will point out how to do that. Then its pretty much a full house as far as I’m concerned.
Her point seems to be somewhere between “they were explicitly designed to not be secure” and “look at the things we can do with the insecurity”. For myself, the arguments made for why the lack of security is a feature didn’t seem very compelling to me, but maybe it’s just because I haven’t used containers in anger? Why not just run wireshark in the same container as the one I’m wiresharking? Is the lack of security so valuable that I don’t want to do that?
“Containers allow for a flexibility and control that is not possible with Jails, Zones, or VMs. And THAT IS A FEATURE.”
She seems fully aware that there are things that aren’t secure, but she also seems dedicated to fixing that. I asked on twitter, and got no response, about contrasting slinging together crypto primitives vs. slinging together these primitives. Can we expect that people will shoot themselves in the foot when given powerful primitives? I think the answer is yes. Can we make it such that when you put these primitives together the safety stays on? I’m a bit pessimistic about that given the ways in which people do things incorrectly anyway… because the protection got in their way.
The “dedicated to fixing it” part is what I found confusing, though. The current state of insecurity seems to be touted as a strength (you can do all these sharing things that are not possible in jails) But if one wants to fix it, does that mean it’s not actually a strength? Or fixing it involves retaining those somehow? It’s like someone selling an older model car without ABS saying “yes, and you can do cool skids in the snow with them, it’s a feature! But the next model will have ABS….”
I agree that it’s confusing. It definitely seems to have been written a bit haphazardly!
Or fixing it involves retaining those somehow?
My read on it is that she believes a world is possible where you can have all of those things, securely. And if you did, that’d be far more powerful than Jails or Zones. I think I can’t do much but agree with that. Though, I do question the possibility of it.
She’s got a kubecon talk coming up this week. Hopefully she’ll talk more about this topic?
In theory you can put a wrapper on top, which prevents everything insecure. Just like Rust is a wrapper around assembly, which prevents a lot of insecure things.
I guess the hard part is that it was not designed with security in mind (only resource partitioning). This means security has to be tacked on, which is much harder. Some would say impossible.
So, the way I see it, you either have something like Docker that does everything for you in a default and secure (which you can still screw up!) way, and gives you the illusion of jails. Or you have a library of functions that you can call directly that gives you ultimate flexibility, but opens up the potential for subtle sec issues.
A lot of people don’t understand how to reap child processes, but we’re going to make it easy and secure for them to share a namespace with a net capability to allow for witeshark?
There are at least two projects attempting to do that. runc is a spin-off of an approach to Linux containers built by Docker as libcontainer, and LXD, from Canonical, aims to build a “secure by design” hypervisor-like system. Both are built on top of a mixture of the assorted Linux container tech and the capabilities/sandboxing tech.
As other people have noticed, this is a vague rant directed at an even more vague audience. There is no technical content in here, and in my mind there isn’t any kind of non-technical content either.
The author is correct in pointing out that FreeBSD jails and Solaris Zones have a very different architecture than Linux containers. The author fails to provide evidence on how this specifically affects Linux container security. It’s all labeled under the complexity umbrella which can mean whatever the reader wants to mean.
The reason why Linux container security is hard is because the fundamental primitives (that the author think are so great) are not orthogonal, and the reason they are not orthogonal is that the people who implemented them didn’t know better or didn’t care. I would argue that implementing containers with private namespaces is vastly simpler that doing something like FreeBSD jails, not more complex, as the author claims.
In Plan 9 (Linux’s clone(2) is a rip-off of Plan 9’s rfork(2)) this all works much better since there is only one namespace (the file system) and there is no superuser (so no suid and /etc/passwd).
Combined with bind(2) you can pick and chose what to share and what not to share just as the author wants, processes, networks, whatever, it’s just part of the file system namespace, and there are no rough edges.
And it’s all vastly more simple than FreeBSD jails and vastly more flexible.
The real point seems to be that linux container tech isn’t a supported product, it’s a big name for a collection of tools and patterns and kernel features.
Now jails and zones probably use tools and patterns and kernel features, but there’s an overarching cathedral-style spec. Containers are a bazaar-style buffet of different tools interacting in (sometimes surprising) ways.
This almost reads (to me) like an attempt to argue that it is better that containers provide no security, because security adds complexity, and because legos are great. Legos are great, but I don’t think I would want to put my money in a safe built out of legos.
I think people who want to promote containers should try arguing more along the lines of “containers are like static binaries for ecosystems that can’t build static binaries”. That argument at least makes more sense to me.
Linux containers as they exist today were originally written at Google for process management, in particular for resource sharing / limiting / guarantees. So whether or not it’s better that containers provide no security is immaterial, they weren’t designed to provide security, that’s not what they’re for. There is no debate about whether you should put your money in a safe built out of legos. You shouldn’t, that’s not what Linux containers are for, the end.
OpenVZ is the other big contributor to Linux container tech, but lots of their work isn’t mainline. I don’t personally know whether their containers are secure, though that does seem to be their aim.
The usage of “immaterial” seems a bit odd to me . Certainly what level of security they provide is material if I’m planning on running internet-facing applications in a container. SMTP wasn’t built with security in mind either, a fact which has become of material concern.
I suppose another way to interpret your comment is that one should not assume they provide any security at all, which is probably what people should do. Unfortunately, in my limited experience that message hasn’t seemed to have really spread. I think securing a Linux containers deployment is probably something that could use a bit more attention.
To me the point came across as you have more opportunities to screw up a sandbox homebrewing it from primitives not designed for that job.
I’m not sure who this is targeted towards. The people who like zones/jails or the people who say containers are a thing. Because from my occasional interactions with these groups, there’s a bunch of people saying “containers are awesome!!!” and then there’s some BSD yahoos saying “no, actually, jails are better.” I think this second group is wrong only to the extent that they’ve accepted the premise put forth by the first group? I guess the lesson is if I’m arguing with a linux advocate who tells me containers are great, I should just reply “nope, wrong, containers aren’t real”. That sounds kinda trollish though.
This post seemed like a new level of your “Features as faults”, but in this case, the fault is the feature!
I love this. I’ve heard a ton of startup bros talk about containers as if they’re some brand new panacea that Linux crafted from the ether, without actually knowing any of the history behind them. This was a very satisfying read.
I’ve spent some time elucidating on the history to those smitten with the container “gold rush” and it’s made exactly zero impact – not positive or negative – just zero change. it has not helped those I’ve spoken with understand anything more clearly, or approach their usage (planned or existing) of containers differently…literally no impact.
the hype is very real. very strong. and very, very vague.
and it’s here to stay, unfortunately.
Jess is super smart and doing amazing things with containers, that being said the claims she makes about what jails & zones can’t do are not correct.
I would also argue that while you can’t take apart the jails deathstar, in practice you have all the flexibility you need:
I’m not sure on how much of this could be changed at runtime, but most of the usual needs (introspection, sharing) have straightforward solutions.
I haven’t succeeded in getting dtrace working from the host system into a jailed process but maybe somebody will point out how to do that. Then its pretty much a full house as far as I’m concerned.
I’ll stay in my FreeBSD prison tyvm.
Her point seems to be somewhere between “they were explicitly designed to not be secure” and “look at the things we can do with the insecurity”. For myself, the arguments made for why the lack of security is a feature didn’t seem very compelling to me, but maybe it’s just because I haven’t used containers in anger? Why not just run wireshark in the same container as the one I’m wiresharking? Is the lack of security so valuable that I don’t want to do that?
“Containers allow for a flexibility and control that is not possible with Jails, Zones, or VMs. And THAT IS A FEATURE.”
She seems fully aware that there are things that aren’t secure, but she also seems dedicated to fixing that. I asked on twitter, and got no response, about contrasting slinging together crypto primitives vs. slinging together these primitives. Can we expect that people will shoot themselves in the foot when given powerful primitives? I think the answer is yes. Can we make it such that when you put these primitives together the safety stays on? I’m a bit pessimistic about that given the ways in which people do things incorrectly anyway… because the protection got in their way.
The “dedicated to fixing it” part is what I found confusing, though. The current state of insecurity seems to be touted as a strength (you can do all these sharing things that are not possible in jails) But if one wants to fix it, does that mean it’s not actually a strength? Or fixing it involves retaining those somehow? It’s like someone selling an older model car without ABS saying “yes, and you can do cool skids in the snow with them, it’s a feature! But the next model will have ABS….”
I agree that it’s confusing. It definitely seems to have been written a bit haphazardly!
My read on it is that she believes a world is possible where you can have all of those things, securely. And if you did, that’d be far more powerful than Jails or Zones. I think I can’t do much but agree with that. Though, I do question the possibility of it.
She’s got a kubecon talk coming up this week. Hopefully she’ll talk more about this topic?
What is the problem with making it secure?
In theory you can put a wrapper on top, which prevents everything insecure. Just like Rust is a wrapper around assembly, which prevents a lot of insecure things.
I guess the hard part is that it was not designed with security in mind (only resource partitioning). This means security has to be tacked on, which is much harder. Some would say impossible.
So, the way I see it, you either have something like Docker that does everything for you in a default and secure (which you can still screw up!) way, and gives you the illusion of jails. Or you have a library of functions that you can call directly that gives you ultimate flexibility, but opens up the potential for subtle sec issues.
A lot of people don’t understand how to reap child processes, but we’re going to make it easy and secure for them to share a namespace with a net capability to allow for witeshark?
There are at least two projects attempting to do that. runc is a spin-off of an approach to Linux containers built by Docker as libcontainer, and LXD, from Canonical, aims to build a “secure by design” hypervisor-like system. Both are built on top of a mixture of the assorted Linux container tech and the capabilities/sandboxing tech.
As other people have noticed, this is a vague rant directed at an even more vague audience. There is no technical content in here, and in my mind there isn’t any kind of non-technical content either.
The author is correct in pointing out that FreeBSD jails and Solaris Zones have a very different architecture than Linux containers. The author fails to provide evidence on how this specifically affects Linux container security. It’s all labeled under the complexity umbrella which can mean whatever the reader wants to mean.
The reason why Linux container security is hard is because the fundamental primitives (that the author think are so great) are not orthogonal, and the reason they are not orthogonal is that the people who implemented them didn’t know better or didn’t care. I would argue that implementing containers with private namespaces is vastly simpler that doing something like FreeBSD jails, not more complex, as the author claims.
In Plan 9 (Linux’s clone(2) is a rip-off of Plan 9’s rfork(2)) this all works much better since there is only one namespace (the file system) and there is no superuser (so no suid and /etc/passwd).
Combined with bind(2) you can pick and chose what to share and what not to share just as the author wants, processes, networks, whatever, it’s just part of the file system namespace, and there are no rough edges.
And it’s all vastly more simple than FreeBSD jails and vastly more flexible.
[Comment removed by author]