As long as the /usr/sysroot/ subdirectories remain immutable and synchronized across all environments in which binaries run, we are guaranteed to always get a deterministic version of glibc for a given binary.
and then
Containers are, usually, not the best solution to a systems problem. While they might be coerced to deliver the desired results, they come with a heavy cost. Unix has been around for a long time and there exist alternate solutions to problems that don’t require full replicas of a functioning system.
So the author is saying we need a way to replicate some folders across systems and make sure they’re immutable. We could build a server that allows us to download immutable “images”, and then have a command-line tool that makes those images part of the filesystem… just need to pick a name for this concept…
To answer the question seriously (and as someone who does have store paths rather than images):
I suspect store paths can and even do waste less space than images.
I suspect that store paths can make it easier to see what all files one has installed on a system (e.g. to scan for vulnerabilities), relative to listing all files in images.
I think store paths, at least in some applications, may be a more direct, less abstracted, simpler, maybe even more sensible way than images to have multiple copies of software components in a system without their clashing with each other.
I think store paths, at least as used in NixOS, still suffer from difficulties around rapid security updates; how they compare to images in this regard I don’t know.
The way I see it, the difference lies in chroot. That’s what brings cognitive complexity to containers. Using the article, people can make an informed decision considering their own specific context. Cost/benefit, yadda yadda…
Correct. Containers are good for some things but they aren’t the only solution and they aren’t always the best solution. Furthermore, containers can be lightweight or heavyweight, and from what I observe, they tend to be much larger than they could be.
The problem I face (which I didn’t fully describe in the article) is much easier to solve in the way I described. Chroot-style solutions, like containers, could of course work… but they would make the solution much harder to manage logistically and much more inefficient.
* Execing the dynamic linker is a slightly obscure feature, which can increase the chance of hitting bugs (e.g. BZ#16381, BZ#24900).
* If my_executable tries to open("/proc/self/exe") or readlink("/proc/self/exe") or similar, it'll get ld-linux-x86-64.so.2 rather than itself.
* Having a launcher script is aesthetically displeasing.
In addition, /proc//exe and proc//cmdline would look different and there might be weird re-exec issues.
rcombs has a nice musl solution for this, which builds a special static-pie (ET_DSO, no PT_INTERP) that links in a custom loader (dcrt1.o, which replaces rcrt1.o).
The static-pie mmap’s ld-musl, adjusts auxv to point to it, and then jumps to ld-musl’s entry point.
Oh, thanks. I hadn’t thought about those implications. Good to have one more argument against the idea of prefixing all binary invocations with an explicit mention of ld-linux.so (which I didn’t like anyway).
Same here. I learned about dynamic linking on Windows first and this post helped understand the differences between how it’s done on Linux:
Windows: single linker that is a part of the system (maybe even the kernel).
Linux (or maybe the correct term here is “ELF-based systems”): user mode facility bundled together with libc. glibc has one, musl libc has its own too.
and then
So the author is saying we need a way to replicate some folders across systems and make sure they’re immutable. We could build a server that allows us to download immutable “images”, and then have a command-line tool that makes those images part of the filesystem… just need to pick a name for this concept…
What if rather than “images” we had “store paths” and instead of arbitrary version numbers we used some kind of hash?
To answer the question seriously (and as someone who does have store paths rather than images):
I suspect store paths can and even do waste less space than images.
I suspect that store paths can make it easier to see what all files one has installed on a system (e.g. to scan for vulnerabilities), relative to listing all files in images.
I think store paths, at least in some applications, may be a more direct, less abstracted, simpler, maybe even more sensible way than images to have multiple copies of software components in a system without their clashing with each other.
I think store paths, at least as used in NixOS, still suffer from difficulties around rapid security updates; how they compare to images in this regard I don’t know.
Container images are stored by hash and you can so reference them if you desire.
The way I see it, the difference lies in chroot. That’s what brings cognitive complexity to containers. Using the article, people can make an informed decision considering their own specific context. Cost/benefit, yadda yadda…
Correct. Containers are good for some things but they aren’t the only solution and they aren’t always the best solution. Furthermore, containers can be lightweight or heavyweight, and from what I observe, they tend to be much larger than they could be.
The problem I face (which I didn’t fully describe in the article) is much easier to solve in the way I described. Chroot-style solutions, like containers, could of course work… but they would make the solution much harder to manage logistically and much more inefficient.
@corsix has a nice writeup https://www.corsix.org/content/for-want-of-a-relative-path , which mentions some drawbacks if you invoke the dynamic loader directly.
In addition, /proc//exe and proc//cmdline would look different and there might be weird re-exec issues.
rcombs has a nice musl solution for this, which builds a special static-pie (ET_DSO, no PT_INTERP) that links in a custom loader (dcrt1.o, which replaces rcrt1.o). The static-pie mmap’s ld-musl, adjusts auxv to point to it, and then jumps to ld-musl’s entry point.
Here is the glibc feature request: https://sourceware.org/bugzilla/show_bug.cgi?id=31959
Oh, thanks. I hadn’t thought about those implications. Good to have one more argument against the idea of prefixing all binary invocations with an explicit mention of ld-linux.so (which I didn’t like anyway).
I never fully understood what the whole glibc and interpreter stuff was all about, this article made it click for me :) Very interesting!
Same here. I learned about dynamic linking on Windows first and this post helped understand the differences between how it’s done on Linux:
The full summary on Wikipedia: https://en.wikipedia.org/wiki/Dynamic_linker
Not at runtime. It’s “Picking glibc at startup, without any container or chroot” and I like the amount of details.