1. 41
  1. 12

    The author forgets about plugins. Basically if you want to support new media, file format, capabilities, etc. without a completely new version is the main app - you need multi-file distribution.

    Go already has lots of problems with those - they completely ignored (or deprioritised) that use case at the design stage. Now things like terraform are doing weird hacks to add in the support and it’s terrible in practice.

    From the other side we’ve got consistency. Do you want every application’s print dialog to look just a bit different? That’s what static linking and single-file distribution gives you.

    1. 4

      […] plugins […]

      I’m not sure how plugins will affect the static linking… For once, at least in C there is dlopen, that at least technically should work with static linking.

      However, many of the tools I cite as example (say Hugo), provide many of the plugins built-in (in addition to a minimal build). Alternatively if one actually needs a plugin he could just re-compile the code (provided it’s open source).

      On the other side, perhaps we should look more closely at the plugin mechanism: do we really want to load unknown code into our process? Because if something goes wrong we can’t easily point out what the problem is.

      How about switching to a sub-process model for plugins? Set out a simple plugin protocol, and just run the plugin (which is now a binary itself) as an sub-process and communicate with it via pipes or IPC. One could say this is sub-optimal due to serialization, however at least on Linux one could uses file-descriptor passing, shared memory, and other such constructs.

      […] multi-file destribution […]

      Having a “single executable” is not a mandate, instead I see it as a guideline. If your tool requires multiple executables or a few large files, it’s OK to just create a tar with them; just make sure the user can extract it anywhere and still work.

      Do you want every application’s print dialog to look just a bit different?

      First of all said that perhaps GUI heavy application can’t easily apply this model.

      However I don’t see how static linking goes against this? How does configuration tie to static linking? The widget library, regardless of how it’s linked, can just read the same configuration files than describe the visual properties (font, colors, etc.)

      Perhaps you are referring to the GUI “engines” (GTK?, Java’s Swing) where the “look-and-feel” is actually code not configuration. But then again, do we really want to go down this road?

      1. 5

        For once, at least in C there is dlopen, that at least technically should work with static linking.

        This depends on the platform but in general statically-linked UNIX programs can’t call dlopen. This function is provided by the run-time linker. When you start a statically linked program, the kernel maps it into the new process’s address space, sets up an initial stack, sets PC to the entry point, and then starts the process executing. When you start a dynamically linked program, the kernel reads an ELF header to tell it where to find the corresponding run-time linker and loads that instead. The run-time linker then loads the real program and remains resident to provide services such as library loading, late binding of symbols, and so on. In the static case, the run-time linker is not present.

        On GNU/Linux, glibc doesn’t support static linking and so the closest that you get is a dynamically linked program that doesn’t link anything except libc.so and the VDSO. I’m not sure if musl bundles the run-time linker in statically linked binaries. FreeBSD now bundles a minimal one so that ifuncs can work in statically linked binaries.

        1. 4

          Apart from trivial examples, plugins over IPC/subprocess are not great. For example: music app with sound processor plugins. You want to share memory, not serialise the audio. (But if you share with an external app, it either has to do call back to the main one to interpret the memory or serialise anyway) You want to embed configuration GUI in the main app. You do not ever want to context switch while processing real-time stream. You just can’t do VST-like plugins this way.

          Alternatively if one actually needs a plugin he could just re-compile the code

          That does not fly with normal users.

          The GUI issue is not about configurable decoration, but rather code defaults which do come from linking. You can notice them already if you have multiple GUI apps running through flatpak from more than one source.

          1. 2

            […] music app with sound processor plugins […]

            OK, granted this is a valid case where plugins should be shared code. However this kind of application is far from your normal CLI tool (the main use-case I speak about).

            GUI issue is not about configurable decoration, but rather code defaults which do come from linking […]

            I still don’t see why this wouldn’t work with static linking… Think about it this way: if defaults come from a compiled shared library, that is deployed system wide, why can’t the developer just extract those defaults into a base configuration file that is deployed system wide, and make the existence of that file mandatory (perhaps also look in a few standard places like XDG)? (Without the shared library the tool doesn’t work, so making the base configuration file mandatory doesn’t break the usability more than it already was…)

            1. 5

              your normal CLI tool (the main use-case I speak about).

              I think you would avoid a lot of issues raised here (and possibly elsewhere) if you open with this. You mention 56k packages in OpenSUSE - relatively few, but still many very used ones are not CLI. There are still issues remaining to be discussed (and are), but you probably know these. However, a very blanket statement “ditch the apt/rpm” is going to illicit a lot of knee-jerk reactions (and justifiably so, if you don’t qualify your statements like you just did).

              Note that this is not a critique of the article contents, just this one remark. When I was reading it, it was somewhat clear, after a while, that you mean “CLI tools”, but it’s not 100% there, and I don’t remember reading that explicitly.

        2. 2

          (…) plugins (…) Go (…) terraform are doing weird hacks (…)

          There’s the plugin package in Go; it’s a relatively late addition, but seems to be quietly improving support for platforms; that said, I never tried it personally yet, so I’m curious: is it not working well enough still? or just nobody remembers it exists?

          1. 2

            It’s Linux-only unfortunately. So if you’re targeting more systems it’s not usable.

        3. 5

          I still feel like there’s some synthesis out there on the static linking/dynamic linking (early binding/late binding) question. Static is better for production, no doubt about that. In basically every scenario, it’s better to have a system that is tested and known to work than an untested glued together conglomeration. But dynamic has its pull, and there must be reasons for that too. I think until we find a way to do static with the benefits of dynamic, people will keep trying to use dynamic in production and getting burnt.

          1. 10

            Static is better for production, no doubt about that

            It’s not quite so clear cut. Static linking has some big advantages: it makes updates easier, it enables whole-program optimisation, it avoids indirection for cross-library calls, and so on. It has a couple of big down sides, one technical and the other legal:

            • There is no code sharing between processes that link the same library.
            • Distributing statically linked binaries that use LGPL libraries is a headache.

            The first has three levels of impact. You need more space on disk for the duplicated code but that doesn’t matter very much with modern disk sizes. You need more space in RAM for the duplicated code. This can impact performance directly if it leads to swapping, indirectly if it causes less RAM to be available as file cache. This is typically only an issue if there’s a lot of code shared between processes. Finally, it increases instruction cache usage, causing cache misses in both programs that call different copies of the same function. This is more of an issue and affects systems with multiple processes in the runnable state, though I don’t know to what degree. It’s less of an issue on server systems where the trend is towards a single program in a single container on a single VM.

            The second is a mess because you must provide recipients of your program the ability to relink the resulting program against their own version of the LGPL’d dependency. With dynamic linking, this is trivial (just replace the .so), with static linking it depends on providing the .o files for your application and a build script.

            There’s also the auditing issue. Unless your statically linked binary comes with an SBOM, it’s difficult to tell if it links against a vulnerable dependency, whereas with dynamic linking you can see the library that it depends on and update it when there’s a security fix.

            Apple has a somewhat hybrid model where they link a load of core libraries together into a massive blob and inject that into all processes, so that they can have cross-library optimisation within their own code and can avoid the start-up costs of applying dynamic relocations into their code and get good code sharing. More than 90% of the (used) code in a typical Cocoa app is in these libraries (there’s also a load of code that is dead in any given program). They also do a neat trick of reusing the leaf page-table pages for this entire section between all processes, which means that they’re always hot in the cache (and the higher-level pages typically are for the running program) which significantly reduces the cost of a TLB miss in this region.

            This works for Apple because they have a rigid separation between system libraries (i.e. things bundled with macOS/iOS) and third-party libraries (i.e. things bundled with an app / installed via Homebrew or whatever). They don’t get much benefit from sharing in the second category because you’re unlikely to have very many apps that all ship their own copy of the same third-party library but they get a huge benefit from sharing the first category because everything links that. Generally, apps can statically link third-party dependencies if they need to and use rpath linking for LGPL (or similar) .dylibs that are stored within the app bundle (note: I believe this is probably illegal with signed binaries, because the end user can’t replace the .dylib without preventing the application from running but I’m not aware of anyone testing that in court).

            In *NIX systems, the library providing the web view is probably the largest one that multiple programs link and keep resident at the same time. For security, this is increasingly a fairly thin wrapper library that runs a child process that does all of the rendering and just provides event injection and rendered output. This model works fine with static linking (as long as you have a stable RPC ABI between the child process and the parent).

            I liked the old PC-BSD PBI model, which installed every application in a separate directory tree, with all of its libraries in ${APPROOT}/lib and so on. It then used hard links to deduplicate redundant copies of the same library. Everything used $ORIGIN rpaths for linking. It wasn’t fantastic for download speed (every KDE app required you to download 300+ MiB of KDE libraries, for example, which were then thrown away because you had a copy locally) but that would be avoidable fairly easily. This gave you the separate update ability but with the total code-size advantages of dynamic linking.

            Vikram Adve also has a project calls ALLVM that looks promising. The idea here is to distribute everything as IR and make a decision at install time which bits belong in a static binary and which in shared libraries. This (in theory) lets you identify functions that are common to multiple programs and pull those out into dynamic libraries but keep specialised implementations of any of them privately, and redo this decision at any point.

            1. 3

              There is no code sharing between processes that link the same library.

              Unfortunately I don’t think this is actually an issue currently. For example there have been quite a few writings about this subject, especially:

              […] it’s difficult to tell if it links against a vulnerable dependency, whereas with dynamic linking you can see the library that it depends on and update it when there’s a security fix […]

              Unfortunately this also isn’t usually true, especially when using pre-built binaries, because many projects often ship their own version of well known libraries. A good example is zlib that is “vendored” quite a lot; another example is openssl. Thus these vendored libraries don’t appear on a simple ldd or readelf invocation.


              All in all thanks for all the insight into what Apple is doing and PC-BSD used to do.

              About the LGPL legality issue, I wasn’t aware of that requirement, although given most tools are open-source, legally I don’t think its actually an issue as the user can just ditch the pre-built binary and use the source original code.

              1. 10

                https://drewdevault.com/dynlib – by Drew DeVault, written in June 2020;

                Looking at the list, it appears that Drew ran this on a system that had almost no modern GUI applications installed. The only GUI libraries that show up are in the low-level X ones. You’ll see something very different on a system with KDE or GNOME installed, but even then he’s measuring the wrong thing. It doesn’t matter how many programs link shared libraries, it matters how many are running at the same time and how big the shared libraries are. On a Windows / macOS system, a typical GUI app is linking tens or hundreds of MiBs of shared libraries and those same libraries are used by all graphical apps. Something similar is true if you’re running a suite of KDE of GNOME apps, though the fact that things like Firefox and OpenOffice use their own toolkits alters this somewhat.

                The claim about 4.6% of symbols is ignoring transitive dependencies. For a trivial example, if I call printf then I’m calling a libc function that is half a dozen instructions. Definitely smaller in static linking, right? Well, no, because printf is a thin wrapper around vfprintf, which calls localeconv and ends up pulling in a huge amount of object code in total. I probably hit more than 4.6% of the symbols in libc from a single printf call, which makes me deeply suspicious of the rest of this claim. Note that the ‘a good linker will remove unused symbols’ is true only in languages with no dynamic dispatch. In C, if you take the address of a function but never call it, it (and everything reachable from it) has to be in the final binary. The linker can’t do any analysis here (go can do a lot more in the front end), it doesn’t know that you’ve assigned the function pointer to a structure but you never actually call that function.

                https://lore.kernel.org/lkml/… – reply by Linus Torvalds, written last in May 2021;

                The key bit in Linus’ comment is this:

                Yes, it can save on disk use, but unless it’s some very core library used by a lot of things (ie particularly things like GUI libraries like gnome or Qt or similar), the disk savings are often not all that big - and disk is cheap. And the memory savings are often actually negative (again, unless it’s some big library that is typically used by lots of different programs at the same time).

                Note the qualification. Dynamically linking a small library, especially one where you use few symbols, is a net loss. Dynamically linking a large library that provides you with a lot of services and that a lot of things link is generally a net win.

                The specific context of this thread is dynamically linking clang. This is well documented as a bad idea. There are two situations when you should do it:

                • If you want a short compile-test-debug cycle, because building a dynamically linked clang is a lot faster than building a statically linked one. Don’t do this if you run the whole test suite because it takes longer to run the test suite and dynamically link than it takes to run the test suite and statically link.
                • If you are in an environment where disk space is very constrained and you are installing all of the LLVM tools.

                LLVM is currently exploring a crunchgen-like approach for the second case, where clang, lldb, lld, and so on are all in the same statically linked binary that dispatches based on the value of argv[0]. The follow-up post points out that dynamically linked LLVM is under half the size of statically linked LLVM and neither is particularly small. This is a lot worse for some of the binutils replacements, which are a few KiB of code per utility and which depend on a few MiBs of LLVM library code, than it is for clang which has a large chunk of clang-only code (though shared with libclang, which a load of other tools such as clangd use).

                even your arguments are heavily qualified with conditions;

                Because it’s not a clear-cut call. If static linking were universally better, we’d use it everywhere. We don’t because it’s definitely worse in some cases and evaluating which is better is a fairly non-trivial decision.

                Unfortunately this also isn’t usually true, especially when using pre-built binaries, because many projects often ship their own version of well known libraries. A good example is zlib that is “vendored” quite a lot; another example is openssl. Thus these vendored libraries don’t appear on a simple ldd or readelf invocation.

                I guess that depends a bit on your corpus as well. The packaged versions of these often use the system-provided one, for precisely the auditing reason.

                About the LGPL legality issue, I wasn’t aware of that requirement, although given most tools are open-source, legally I don’t think its actually an issue as the user can just ditch the pre-built binary and use the source original code.

                Assuming that most tools are open source is a very big assumption. It’s definitely true for things in open source package repos (by definition) but it’s far from true in the general case.

            2. 2

              Though it’s clearly a trendy perspective, I don’t think this is really an absolute truth:

              Static is better for production

              I think part of your justification is not unreasonable, i.e.,

              it’s better to have a system that is tested and known to work than an untested glued together conglomeration

              But that’s not really a question of linking, it’s a question of testing and of packaging and distribution.

              If you test a particular combination of dynamic libraries and executables and then ship exactly those files to production, then your test coverage is just as good as with a static binary. This is arguably what people have been doing for quite some time with Docker and other container image formats.

              1. 2

                At $WORK, we have a few services written in Lua. To ease installation, the program is basically a simple C program where Lua, the Lua modules we use (both written in C and Lua) and the main program written in Lua, are all statically linked into an executable. That way, ops just has to install a single executable when we update the code, instead of the platforms package format (we use Solaris in production—does Solaris even have something like RPMs? I honestly don’t know).

                1. 3

                  we use Solaris in production—does Solaris even have something like RPMs? I honestly don’t know

                  Sure! It depends which version. Solaris 10 had older packaging tools that are similar in principle to RPM. You generally had to shepherd the packages onto a system yourself or use NFS, and dependency management was less than stellar.

                  Solaris 11 systems have a newer packaging technology, IPS (aka pkg), which provides much richer tooling, strong management of dependencies between packages and version locking, network accessible repositories, etc.

                  I would just build an IPS package from the Lua files and then you can install it over and over again on new systems.

                  1. 2

                    The program started out on Solaris 5.10 and we have it working. I don’t think we’ll change, even if we’ve upgraded to Solaris 5.11 (we have a department that does the deployments and I’m not sure how they set things up).

                2. 1

                  I’m not looking at the very narrow question of “use DLLs/.so files or not?” I am trying to ask about static vs. dynamic in the very broadest meaning possible. The essential part of static vs. dynamic is not “does this use one file or multiple files?” As the words “static” and “dynamic” mean, the question is “can part of this be swapped out or not?” Swapping parts out is an operational risk. If you distribute systems as a collection of files but don’t swap them out, they’re effectively static, you’re just relying on multiple files for whatever reason.

              2. 5

                I generally try to avoid picking on the style of anyone’s technical writing, but if the writer of this piece happens to read: you are leaving too many thoughts dangling/open.

                Treat each thought like a file descriptor–let the reader close it as soon as possible.

                Consider putting some arbitrary constraints on your sentence length and commit to them for a similarly-arbitrary timeframe (maybe a year if you write often, longer if not). It will help your readability greatly.

                I don’t care if you count words, characters, lines, or commas. Just set a reasonable length. Notice when you feel like something reads nicely. Pop the text out, and break it down to discover the average number of words per terminal punctuation, number of commas/semicolons/ellipses per word, etc. Then, set some arbitrary goals based on this and force yourself to deal with them.


                To keep it quantifiable, this page has:

                • about 3000 words
                • 221 commas, or 1 per 13.574
                • 155 semicolons, or 1 per 19.35
                • Of 115 periods on this page, only 11 of them are terminal punctuation (69 of them appear as part of 23 ellipses, 11 more in etc./i.e./e.g/, 7 in ./commands, 3 in domains/emails; this leaves ~14 uncounted with most probably in code examples)
                • 40 – ~em-dashes (can stand in for colons, commas, parens, semicolons…)
                • 27 colons
                • 16 question marks, 14 of which are roughly terminal punctuation.
                • 7 exclamation marks, 2 of which are roughly terminal punctuation (rest for emphasis)

                Adding these up:

                • 1 unit of terminal punctuation (actually-terminal periods, question marks, and exclamations) per ~111.11 words
                • 1 unit of continuing punctuation (commas, colons, semicolons, dashes, and ellipses) per 6.77 words
                1. 2

                  I’m the author of this article, thus thanks for the feedback! It’s important for me to receive feedback also about the writing style, especially since English is not my main language, and I write only seldomly.

                  […] you are leaving too many thoughts dangling/open.

                  Indeed this is as issue with my writing style, I often find myself using ... quite a lot to end phrases… :)

                  Consider putting some arbitrary constraints on your sentence length and commit to them for a similarly-arbitrary timeframe […]

                  This is an interesting thought. Do you have any suggestion for a good small tool (CLI if possible) that could help here?

                  1. 1

                    There might be something more finely tuned to the task (making it easier to do some of the weirder forms I mentioned) but I think textlint can do this (whether that’s through a readily-available rule or writing one).

                    While looking around a little, I also noticed a more recent Go project named vale that can likely do this. I see a reference to a SentenceLength.yml near the top of https://docs.errata.ai/vale/styles; in the source it looks like a demo rule. It might be the case that one of the style packages, like https://github.com/testthedocs/Openly, is a better place to start.

                    I think both are pretty customizable, so I imagine you could use just a few rules if the rest are distracting.

                2. 2

                  asdf is the package manager that the author is looking for. I use it daily and I have contributed with patches and plug-ins. I also learned a few bash tricks on the way :)

                  1. 1

                    Unfortunately I don’t think yet-another-package-manager is the solution… (Nix, Guix, Homebrew, asdf, Hermit, etc.)

                    For once, users in general don’t have enough “bandwidth” (energy, time, willing, etc.) to learn yet another “thing” just to get to the actual “thing” they need to learn and use.

                    Then there is the subject of opportunity and feasibility: sometimes you need to quickly debug a production system and you need a non-mainstream tool. You don’t want / are allowed to touch that system in such a major way as installing a new package manager; you just want to rsync your tool somewhere in /tmp/debugging, do the debugging and get out of there as quickly as possible and without leaving traces.

                    1. 1

                      For once, users in general don’t have enough “bandwidth” (energy, time, willing, etc.) to learn yet another “thing” just to get to the actual “thing” they need to learn and use.

                      Sure, and that’s fine for one-off use cases, and I do agree that self-contained executables are great in general, but once you start to do anything more than one-offs you want something to manage your tools. And your own article mentions some of these cases:

                      • have multiple versions of the same tool installed and running at the same time;
                      • have it installed and running even when you don’t have sudo rights; just place it in /home/user/.bin (and have that in your ${PATH};)
                      • have it stowed in your .git repository so that everyone in your team gets the same experience without onboarding scripts;

                      These use cases are exactly what Hermit solves - it doesn’t do any packaging, library installation, dependency management, etc. - it just download binaries and makes them available. Its stubs can be checked into Git, and for users Hermit doesn’t require installation, it bootstraps itself.

                      1. 2

                        Precisely. I don’t have enough “bandwidth” to go around websites and executing weird build instructions, or fish around for downloads, even on my single system. If I had to do this each time I had a new dev on my team, I would definitely want to manage this somehow. And yes, the proposed “check it into source control” thing from the article partially solves this, but it’s not always feasible.

                    2. 2

                      That’s basically where we’re heading with Flatpak, Appimage, … Instead of statically linking all the dependencies, the app is executed in a container.

                      The author links to an article to explain why he’s dismissing Flatpak, but the article exposes the same problems you’ll have with statically linking; the image gets bigger, sandboxing is not perfect (except that it’s non-existent with static linking), GTK compatibility issues, …

                      Also statically linking C projects is no easy task. Even with Go, you can quickly hit some corner-cases where it actually makes sense to link against Glibc.

                      1. 1

                        The more I think about my writing, the more I think that the underlying message I try to convey is this:

                        • let’s take a minimalist approach to packaging; let’s trim down our dependencies, let’s trim down our deployment footprint (especially with regard to the number of resource files); let’s make deployment (if needed) as simple as a tar -xf tool and then rsync ./tool/ root@remote /opt/tool/;
                        • dynamic linking is fine, just don’t depend on non-mainstream libraries (that are most likely missing from one’s distribution), and don’t depend on the version just released last month; if you must depend on some non-mainstream library, then perhaps link it statically;
                        • try to provide pre-built binaries that work across a broad range of OS’s (Linux, OSX, BSD’s), distributions (especially in the case of Linux), and even versions; (obviously not everything in the same binary, but overall a broad spectrum should be covered;)
                        • (static linking is only the simplest approach to achieving some of the above;)

                        And the reason I think Flatpack, AppImage, Snap, and all the others – including Nix, GUIX, Homebrew, Hermit, asdf, etc. – are not the solution is because they just hide the issues and complexity, not actively work to solve it.

                        1. 1

                          I see your point. What you are saying is that without the forcing function of statically-linked restrictions, package authors will be lazy and pull the entire world in their docker/flatpack/nix/… closure. This is quite true and observable out there.

                          Traditionally, the Linux distro was also serving as a forcing function in terms of dependency compatibility. If packages A and B wanted to be shipped with Debian, they also had to agree on which version of C they would use. That was an important forcing function in order to agree on ABI. In that role, the distribution was also responsible for security updates.

                          Now with the number of dependencies exploding, traditional Linux distributions are facing a sort of reckoning. The combinatorial problem is too large to solve, as you clearly pointed out in your article. What happening instead, is that dependency management and security updates are becoming the responsibility of the application itself. Tools like Dependabot, security scanners, … start to appear that slowly massage the code tree in the right direction.

                          So instead of building all the apps statically, we could also start seeing bots appear that try to reduce the size of applications. That might be another way to do things.

                      2. 2

                        I agree that the status quo of “unbundling” can’t survive, and we need some alternative, but a single exectuable is extreme.

                        All of the examples given are command-line tools or servers. For a desktop app, you may need an icon, manual, config file, maybe a template project. Games need GBs of data files. They theoretically bundle all of data into the executable, but that’s needlessly inconveniencing everyone involved.

                        1. 1

                          Idk, while I agree in cases where your build environment is highly controlled and observable (at a company with a robust CI and artifact storage infra), I think the security issues are massive and make the proposed solution unacceptable as-is for general use.

                          Given a binary procured from some vendor, I can’t reason about what dependencies it has, let alone the versions. The best bets I have are using objdump(1) or nm(1) or at worst strings(1) and guessing. Maybe it’s on a system where symbol versioning is present, and I can use that info to make an assertion about what versions it’s not, but that’s not perfect.

                          Without that info, if a security vulnerability is announced, I can’t even audit the system to figure out what packages are affected. Hell, with just a binary, unless the author has provided some flag to get the version, I can’t even deduce what version it is.

                          However, I feel like all of this is fixable? It might be nice to see a newer language where statically-linked (ish, this isn’t true on many systems) single binaries are the default, like with Go or Rust. I could imagine a adding a new ELF section where cargo/go embeds this information at build time, and a small ecosystem of tools that would query this information from the binary. The big problem is that there wouldn’t be authenticity guarantees, unless the binary was signed by the author.

                          A solution like that, plus an addition to the proposed theoretical package manager by the author to query this information for installed packages would satisfy the security/auditability needs imo.

                          1. 1

                            The package manager he is describing is Hermit!

                            1. 2

                              These use cases are exactly what Hermit solves - it doesn’t do any packaging, library installation, dependency management, etc. - it just download binaries and makes them available. Its stubs can be checked into Git, and for users Hermit doesn’t require installation, it bootstraps itself.

                              OK, I’ve taken a quick look at Hermit, and for once it is itself a “single binary executable” (written in Go).

                              However I don’t think it’s the right solution:

                              • it works mainly with projects, although I think one could “convince” it to install globally;
                              • it supports only Linux and OSX, although I don’t see anything that technically would stop it being ported to BSD’s; (especially OpenBSD which seems to have very little “love” from many tool developers;)
                              • it supports only bash and zsh; other shells I assume won’t usually work due to the “activation” requirement (see below);
                              • it depends on a centralized repository for available “packages”; I’m sure one can change the URL, but still in order for one to be able to install something there must be a “recipe” in that repository; (at the moment there are only 141 such “packages”;)
                              • it also depends on a working internet connectivity; (at least for the initial install;)

                              But most importantly, it relies heavily on “touching” my shell environment, which personally I’m very sensitive of. For example hermit install something doesn’t even work without “activation”, which implies sourcing a bunch of bash in my environment… (I will never do that!)

                              Also, a deal breaker for me, the shims that Hermit installs in my project’s bin folder are bash scripts… Just looking at them I see one quoting issue (if my HERMIT_STATE_DIR contains spaces I’m doomed):

                              export HERMIT_EXE=${HERMIT_EXE:-${HERMIT_STATE_DIR}/pkg/hermit@${HERMIT_CHANNEL}/hermit}
                              

                              But even more worryingly, as part of the normal execution (if the tool is not installed) it just curl’s and runs a bash script:

                              curl -fsSL "${HERMIT_DIST_URL}/install.sh" | /bin/bash 1>&2
                              

                              Say GitHub has a bad day and just spits out some 404 HTML… I’ll all of a sudden start executing HTML with bash

                              So overall, although it’s a nice and pragmatic tool, I don’t think it’s the right choice, especially given the security implications with all that curl | bash-ing…

                              1. 3

                                You seem to be fixated on a specific approach you have in mind, which is fine - each to their own - but I’ll just correct a few invalid observations.

                                various limitations such as shell support, OS support

                                Shell support is only required for “activation”. Executing Hermit binaries directly works in any shell. For OS etc., it’s just a matter of a) package availability and b) someone to test and support it.

                                export HERMIT_EXE=${HERMIT_EXE:-${HERMIT_STATE_DIR}/pkg/hermit@${HERMIT_CHANNEL}/hermit}

                                This one is interesting, we run shellcheck on our scripts so it’s not clear to me why it’s not picking this up. I’ll have to take a look.

                                it depends on a centralized repository for available “packages”

                                This is incorrect - repositories can define their own manifest sources, including bundling their own in a sub-directory.

                                For example hermit install something doesn’t even work without “activation”, which implies sourcing a bunch of bash in my environment… (I will never do that!)

                                Incorrect. It is not necessary to activate an environment to use the stub scripts, you can just run them eg. ./bin/make

                                it also depends on a working internet connectivity; (at least for the initial install;)

                                Quite a bizarre complaint given that basically everything has to come from the Internet, including the repository the user presumably cloned. Of course it needs to be bootstrapped from somewhere. Once bootstrapped, packages are shared.

                                curl -fsSL “${HERMIT_DIST_URL}/install.sh” | /bin/bash 1>&2

                                Say GitHub has a bad day and just spits out some 404 HTML… I’ll all of a sudden start executing HTML with bash…

                                Also completely incorrect, for two reasons which I’ll leave as an exercise for the reader.

                                1. 3
                                  export HERMIT_EXE=${HERMIT_EXE:-${HERMIT_STATE_DIR}/pkg/hermit@${HERMIT_CHANNEL}/hermit}
                                  

                                  This one is interesting, we run shellcheck on our scripts so it’s not clear to me why it’s not picking this up. I’ll have to take a look.

                                  Though I would probably put the quotes in anyway because it can’t hurt and is easier to remember, I don’t think it’s actually unsafe – which is probably why shellcheck is not complaining. In the specific context of assigning a variable, the kind of word expansion that injects spaces between arguments does not actually occur:

                                  $ cat /tmp/trial.bash
                                  a='a b c'
                                  export b=$a
                                  export c=${C:-$b}
                                  
                                  set | grep '^[a-c]'
                                  
                                  $ bash /tmp/trial.bash
                                  a='a b c'
                                  b='a b c'
                                  c='a b c'
                                  
                                  $ C='value with spaces' bash /tmp/trial.bash
                                  a='a b c'
                                  b='a b c'
                                  c='value with spaces'
                                  
                            2. 1

                              Rust – say goodbye to cross-compiling, but if you stay away from OS provided libraries, you are kind of covered;

                              how easy is it to cross-compile the above? just prepend GOOS=darwin; how easy is to do that in Rust / C / C++? let’s just say that sometimes, especially if the code is small enough, it’s easier to just rewrite the damn thing in Go…

                              I’m not sure what experience the author has had with Rust, but I find it very easy to cross-compile things:

                              rustup target add aarch64-unknown-linux-gnu
                              apt install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu
                              cargo build --target aarch64-unknown-linux-gnu
                              

                              If you depend on libraries, you’ll want to install the native version of the library. Eg libssl-dev:arm64, or build it from source, which many Rust crates will do for you. Even for macOS or Windows targets, cross compilation is relatively easy. Magnitudes better than setting it up for C/C++, in my experience.

                              1. 4

                                It’s easy only on Linux where the distro provides all of the missing bits. From perspective of a macOS user, Rust can barely cross-compile to other Apple platforms, and everything else is half-broken. From perspective of a Windows user*, Rust can’t cross-compile anything.

                                (if you have to run a full Ubuntu in a VM (WSL) to avoid cross-compilation, that doesn’t really prove that Windows Rust can cross-compile)

                                Some bits of Rust are extremely Linux-centric and I worry that core Rust devs don’t even realize how painful things are elsewhere. “Just configure a linker” is treated as a triviality in Rust, too simple for Cargo to even bother, but outside of Linux distros it’s nearly impossible to find a working cross-linker to use.

                                1. 2

                                  Cross compiling generally has the same set of problems, irrespective of the front-end language.

                                  • Can the back end generate instructions for the target ISA? The back end is such a tiny part of a modern compiler that you may as well enable all of the back ends in every compiler build, so this is pretty easy.
                                  • Can the back end generate the correct binary format (ELF, Mach-O, PE/COFF)? Again, this is a very small part of the compiler, so it’s easier to leave it enabled than disable it.
                                  • Can the rest of the toolchain handle the correct binary format? This is where things start to get tricky. PE/COFF, ELF, and Mach-O linkage are sufficiently different that LLD now separates them into different binaries, so you won’t have all of the tools unless you’ve explicitly installed them. On Windows there are some bits of toolchain that don’t have direct analogues elsewhere.
                                  • Do you have a sysroot for the target platform? For C, this includes header files, for everything it includes at least a proxy for the shared libraries (and real code for the static libraries). Apple’s sysroots include magic text (YAML?) files that their linker knows to pretend are .dylib files for the purpose of symbol resolution, which makes them nice and small, but with both Windows and Darwin you hit legal problems: are you legally allowed to redistribute everything? I believe the Windows SDK redistributable things do permit this but I don’t think the Apple ones do. Zig tries to work around this for Apple by building a Darwin SDK from the open source bits, but that doesn’t work if you want to link to any of the proprietary libraries (including any of the GUI bits).
                                2. 2

                                  I do have quite a bit of experience with Rust, and at least for my Rust-based Scheme interpreter (https://github.com/volution/vonuvoli-scheme) I once tried to cross-compile it (only the generic Rust stuff, without external C/C++ dependencies) for OSX (while running on Linux) and the experiment isn’t something I would recommend to others; actually I’m not sure if I can even replicate it today (two years since I’ve tried it).

                                  For reference, here is the article I’ve based my experiments on, and some accompanying projects:

                                  It might be easy if your distribution happens to have the target libraries packaged, like for example when cross-compiling running on Linux x86 and targeting Linux ARM (and the same distribution), but things are quite different when you are targeting a completely different OS like OSX or OpenBSD for example.