yes, containers are great, that security model bit is nothing to scoff at. But consider a better world: [ basically describes nixos ]
I don’t know if the author is around, but in this section of “dreams” for a Linux distro, you’re basically exactly what NixOS is doing today. I highly suggest anyone who thought “geez wouldn’t that be nice” when reading that section to go check it out.
I was at Twitter when we deployed the first production deployment of mesos. It did not use “containers” as they are understood in the docker sense. We shipped binaries, there was a shared filesystem you could see if you logged into one of the machines your service was running on, and mesos set up service discovery to connect the local ports you were randomly assigned at boot time in an (iirc) env variable.
For mesos, and, I believe, borg before it, containers came after to make things easier.
containers came after to make things easier.
That’s the whole point of it! Obviously you can do without, but like I said, imagine how complex it would be!
I’m not speaking about the configuration details but the whole concept. Can someone point me to a technology that correctly and easily describe the runtime convention such as port binding, volume mounting, but as the same time enable extensibility and security?
Sûre many tools exist to do things here and there, but none use them offer a coherent solution to build on top of.
Sure, containers enabled this paradigm. But it’s not as if networking, data storage/access, scaling, and security didn’t exist before containers. They were just handled differently with different technologies.
Does kubernetes make this easier/better? Probably depends who you ask. And for my take, you still have to run kubernetes on something. That something still has the same needs/requirements as always. (This is where you introduce the cloud layer, or someone else’s computer.)
Similarly to how containers apply the benefits you can easily get from the Maven model to any runtime, I’ve always thought that kubernetes was just Greenspun’s Tenth Law but about the benefits of Erlang.
Artifacts would be shipped around as e.g. .zip files downloaded from HTTP servers, instead of containers pulled from container registries. Those artifacts would be tagged with metadata to indicate which runtimes they require, and nodes in the cluster would be tagged to indicate which runtimes they provide; the scheduler would schedule jobs based on those constraints. Kubernetes, at least, already supports this. On balance, I think dropping containers for this aspect of things would actually make the overall system simpler.
Resource limits and namespaces would need to leverage underlying OS primitives directly, rather than going through the container abstraction. Probably a proto-container spec (like OCI?) would arise from these requirements naturally. This is where things would get more complex.
In terms of a framework that would allow you to programmatically deploy (virtual) machines, get applications installed then configure those and start them, it’s basically what I built at my last job. I didn’t get onto any advanced features like automatic load distribution and automatic machine setups when the environment needs more resources but it was completely possible on the back of the basic features as well as the fact that the system did set up distributed (peer to peer and mastered) services together with their own encrypted virtual networks.
I don’t know what other advanced features are possible in Kubernetes/Mesos that wouldn’t work without containers but AFAIK the security isolation is still better for VMs than containers and the networking should be easier since it’s simplified by not having a machine effectively be a router for the containers.
I think part of the author’s argument is flawed because he muddies the term artifact. I’d say when you use a Makefile, your executable absolutely is an artifact. Also you could build a static library (pretty close to a jar I guess, from the build steps). This is more in the preceding (previous: maven) article, but he sticks to this in this one as well.
I’m ignoring the “static compiled is bad because lack of security updates” angle because I’m torn on this one, but the Java/Minecraft example. ugh. I don’t know, I don’t want 5 different JVMs because everyone wants to bundle it. I think JetBrains does it as well, but at least there it’s not 300% overhead, the IDE itself is pretty big and nags you to update and the user group is usually technically versed enough to do it. I wouldn’t say the same about gamers.
Using 2 containers (1 for build, 1 for running) seems like a good idea, but using “RUN go get” for reproducabilty kind of defeats the point. Did he even use go for more than Hello World or am I missing something. He mentions vendoring as state of the art (I think it’s more like a 50/50 split between vendoring and dep/glide/etc) but if there’s one criticism about Go I have, it’s lack of reproducible builds.
In short, there’s no reason why apt and dnf or whatever couldn’t work in a similar way npm or bundle do
I’d say the reason is that it works better regarding “one set of dependencies that work together” vs the Wild West of people not specifying their dependency version ranges. Maybe I’m biased but I see zero benefit of using npm for the points he’s actually trying to make. The benefit is that it’s distributed and open, and low-barrier, and a lot quicker to ship new versions - aka the opposite of stable.
I don’t know if the author is around, but in this section of “dreams” for a Linux distro, you’re basically exactly what NixOS is doing today. I highly suggest anyone who thought “geez wouldn’t that be nice” when reading that section to go check it out.
No. It’s closer to plan 9 and its namespaces.
Just envision how complex would it be to deploy to mesos/kubernetes without the container abstraction…
Nice and complete article!
I was at Twitter when we deployed the first production deployment of mesos. It did not use “containers” as they are understood in the docker sense. We shipped binaries, there was a shared filesystem you could see if you logged into one of the machines your service was running on, and mesos set up service discovery to connect the local ports you were randomly assigned at boot time in an (iirc) env variable.
For mesos, and, I believe, borg before it, containers came after to make things easier.
Because those are built for containers. Containers came first. Maybe I am missing your point…
I’m not speaking about the configuration details but the whole concept. Can someone point me to a technology that correctly and easily describe the runtime convention such as port binding, volume mounting, but as the same time enable extensibility and security?
Sûre many tools exist to do things here and there, but none use them offer a coherent solution to build on top of.
Sure, containers enabled this paradigm. But it’s not as if networking, data storage/access, scaling, and security didn’t exist before containers. They were just handled differently with different technologies.
Does kubernetes make this easier/better? Probably depends who you ask. And for my take, you still have to run kubernetes on something. That something still has the same needs/requirements as always. (This is where you introduce the cloud layer, or someone else’s computer.)
Similarly to how containers apply the benefits you can easily get from the Maven model to any runtime, I’ve always thought that kubernetes was just Greenspun’s Tenth Law but about the benefits of Erlang.
Honestly, I think it wouldn’t be too bad.
Artifacts would be shipped around as e.g. .zip files downloaded from HTTP servers, instead of containers pulled from container registries. Those artifacts would be tagged with metadata to indicate which runtimes they require, and nodes in the cluster would be tagged to indicate which runtimes they provide; the scheduler would schedule jobs based on those constraints. Kubernetes, at least, already supports this. On balance, I think dropping containers for this aspect of things would actually make the overall system simpler.
Resource limits and namespaces would need to leverage underlying OS primitives directly, rather than going through the container abstraction. Probably a proto-container spec (like OCI?) would arise from these requirements naturally. This is where things would get more complex.
In terms of a framework that would allow you to programmatically deploy (virtual) machines, get applications installed then configure those and start them, it’s basically what I built at my last job. I didn’t get onto any advanced features like automatic load distribution and automatic machine setups when the environment needs more resources but it was completely possible on the back of the basic features as well as the fact that the system did set up distributed (peer to peer and mastered) services together with their own encrypted virtual networks.
I don’t know what other advanced features are possible in Kubernetes/Mesos that wouldn’t work without containers but AFAIK the security isolation is still better for VMs than containers and the networking should be easier since it’s simplified by not having a machine effectively be a router for the containers.
I don’t even know where to begin.
I think part of the author’s argument is flawed because he muddies the term artifact. I’d say when you use a Makefile, your executable absolutely is an artifact. Also you could build a static library (pretty close to a jar I guess, from the build steps). This is more in the preceding (previous: maven) article, but he sticks to this in this one as well.
I’m ignoring the “static compiled is bad because lack of security updates” angle because I’m torn on this one, but the Java/Minecraft example. ugh. I don’t know, I don’t want 5 different JVMs because everyone wants to bundle it. I think JetBrains does it as well, but at least there it’s not 300% overhead, the IDE itself is pretty big and nags you to update and the user group is usually technically versed enough to do it. I wouldn’t say the same about gamers.
Using 2 containers (1 for build, 1 for running) seems like a good idea, but using “RUN go get” for reproducabilty kind of defeats the point. Did he even use go for more than Hello World or am I missing something. He mentions vendoring as state of the art (I think it’s more like a 50/50 split between vendoring and dep/glide/etc) but if there’s one criticism about Go I have, it’s lack of reproducible builds.
I’d say the reason is that it works better regarding “one set of dependencies that work together” vs the Wild West of people not specifying their dependency version ranges. Maybe I’m biased but I see zero benefit of using npm for the points he’s actually trying to make. The benefit is that it’s distributed and open, and low-barrier, and a lot quicker to ship new versions - aka the opposite of stable.