1. 32
  1.  

  2. 26

    One small useful fact that not many people realize: for Linux specifically, if you make a fully static executable, it will be portable to all Linux distributions, no matter which libc they use, or if they use a nonstandard dynamic linker path. If the application needs to open a window and use the graphics driver, it gets a lot more complicated to be portable.

    1. 2

      Another approach to handle portability when targetting multiple Linux distributions is to compile the app on an old distribution (e.g. some old CentOS with old glibc). This can be done through Docker. This covers different use cases than static linking, so it’s worth considering (e.g. usage of LGPL libs).

      1. 1

        I might be wrong here, but what about kernel version?

        I had a thing last year where we compiled stuff with a certain Qt version which was built on a 4.x kernel and Qt used some ifdefs and it wouldn’t run on a 3.x kernel on an old RHEL. This was dynamic linking, but the underlying problem is the same, C++ code used a kernel call via libc that wasn’t there, not sure how this would’ve worked with a purely static build.

        1. 3

          I think there’s a chance that if you’d compile it on an old RHEL with 3.x kernel, it would run on newer distros with 4.x kernels.

          1. 1

            I’d even go so far and say that will be the case 99% of the time because it falls under the kernel’s “don’t break APIs” mantra as far as I know.

          2. 1

            Any binary would still need to use syscalls, so that limits the backward compatibility as linux keep adding new syscalls.

        2. 14

          This uses a specific definition of fragility related to portability and availability of of environmental requirements. Which is interesting. I would be more interested though in an examination of the fragility of a codebase broken down by language.

          • How frequently will an upgrade of the interpreter/compiler/stdlib break you.
          • How quickly will the language ecosystem change enough to break you.
          1. 11

            Some of their assessment is pretty questionable too. For example, they object to C because it depends on libc, but that’s a part of the OS install (or bare-metal environment) - you might as well object to something depending on the kernel (and then you’re left with a much smaller list of ones that can run in a freestanding environment). To give a concrete example, Go doesn’t depend on libc and so every Go program broke a couple of years back because Apple changed the arguments to the gettimeofday system call. Everything calling it via the public interface in libSystem.dylib kept working because it was updated for the new system call ABI, everything calling it via the private syscall interface broke. XNU, Windows, and Solaris all provide public C APIs but not a public syscall interface and periodically move things from the kernel to userspace libraries or vice versa.

            Similarly, they object to clang and gcc because they depend on a package manager or installer to install (because apparently untaring a tarball is something you can’t do in their future world), but not to go, which depends on being able to connect to remote git repos to grab dependencies for pretty much everything and which also ships as multiple binaries and support files to provide a working toolchain.

            1. 2

              That would be interesting. I wonder how similar the lists would be.

              1. 1

                Really, it’s a modularity problem.

              2. 8

                Can someone explain this to me? I assume by “copy to another computer” they mean another computer with the same CPU type and OS. But then how is a compiled C program not portable? I mean, most of the commercial-software industry is literally based on copying the output of cc onto a {floppy, CD-ROM, Zip archive} and selling it to people to run on their computers.

                Is this portability problem something specific to Linux? The comment in the table says “no, libc”, but libc is either built into the OS or statically linked into the executable, no?

                (Also, how can Nim rate as more portable than C, when Nim is translated and run through a C compiler?)

                1. 6

                  Yes, I assume the author’s complaints mostly revolve around Linux, where different distros (and different distro versions) may have different versions of libc which are not ABI-compatible. It is possible to build an executable which will run against most glibc versions on most distros, and proprietary software vendors do this, but it is fair to say that there’s effort involved, whereas with say Go it “Just Works.”

                  I’m given to understand that similar problems arise across major versions of some of the FOSS BSDs, but I’m less familiar there.

                  Unsurprisingly, the FOSS world is much less built around distributing object code than the proprietary software world, and the toolchains reflect this.

                  You can get around this by using static linking, since the kernel ABI is stable. In some cases this is easy, in others it can be a little annoying, e.g. because your distro doesn’t actually ship the static libraries for you to link against. But it does work.

                  1. 9

                    You can get around this by using static linking, since the kernel ABI is stable. In some cases this is easy, in others it can be a little annoying, e.g. because your distro doesn’t actually ship the static libraries for you to link against. But it does work.

                    Of course, this only does you good for basic POSIX or server applications. It kinda falls apart when you get into more complex scenarios like as Andrew pointed out, “displaying a window”. The userland stability of say, FreeBSD at least gets you libc and perhaps things like audio, albeit not really X. The advantage of Win32 isn’t just the ABI stability, but the fact it covers at least the foundation to build most scenarios on without fuckery. You might need to ship a static libpng to display an image, but you can talk to USER32.DLL to display your window with minimal fuss.

                    If I were looking into shipping a desktop Linux application in a sustainable way and couldn’t rely on distros, the options are probably down to AppImage and Flatpak. My tendencies would lean towards Flatpak because that’s seemingly where the puck is going, but you never know with desktop Linux. Or maybe just mandate Wine. Win32 is the stable userland ABI for Linux, after all. Just ask Valve.

                    1. 3

                      I’ve wondered for a while why, when asked about an OpenBSD release, the developer of Ripcord (coincidentally also the author of this post) said that it wasn’t going to happen because “The BSDs do not provide ABI stability or compatibility.”.

                      Your post helped me finally understand, thank you!

                      1. 4

                        The BSDs vary wildly here. OpenBSD has zero ABI stability and intentionally breaks it between releases. FreeBSD and NetBSD try to provide dynamic library-level (so libc plus whatever additional libraries are part of the base system) backwards compatibility across releases with optional syscall compatibility through modules with varying degrees of support. (@david_chisnall could certainly elaborate on FreeBSD’s policy here.) edit: FreeBSD’s policies are better than Linux’s here, if only because you at least get libc and supporting cast. I don’t know how ports/pkgs come into play, since you prob want i.e. X.

                        1. 9

                          @david_chisnall could certainly elaborate on FreeBSD’s policy here.

                          For the base system, FreeBSD provides complete backwards (but not forwards) ABI compat within a major release series, including kernel interfaces. A binary (program, library, kernel module) built for FreeBSD 12, for example, is guaranteed to work on any later FreeBSD 12.x release. A binary built for FreeBSD 12.1 may use interfaces not in 12.0 and so not work on older versions. Note that this applies only to things in the base system

                          Between major versions, all kernel control-plane interfaces (e.g. the things that ifconfig or fstat depend on) may change. System calls may be removed but if they are then they are hidden behind a COMPAT_ configuration option and so you can build FreeBSD 13 with COMPAT_12 all the way back to COMPAT_4 and get access to old system calls. Most system libraries use symbol versioning and provide compatibility versions of symbols for older versions. The few that don’t will have a .so version bump between major versions if they change (recently, that’s only happened to move from not-using-symbol-versions to using-symbol-versions editions of the libraries) and the old version available in a compat package.

                          The userspace guarantees are similar to glibc. The kernel guarantees are stronger than Linux within a major release (kernel modules don’t need recompiling) but weaker between major releases (control interfaces can break), on paper at least (Linux changes the behaviour of ioctls periodically, though typically not for anything Linus uses).

                          Note that this applies only to the base system. Packages are on a rolling release. Some of these have very strong ABI guarantees. For example, X applications that use Xlib directly will work even if their X-facing parts haven’t changed for 30 years. Some have much weaker guarantees, for example GTK and Qt have much weaker guarantees. LLVM is really bad in this regard, reserving the right to break all of the C++ APIs between every 6-monthly release. We typically end up shipping multiple versions of these in the packages collection and maintaining them as long as they have consumers in the packages collection (llvm 7, 8, 9, 10, 11, 12, 13, and 14 are currently maintained in packages, for example). As long as you don’t link one of them directly and a different one indirectly via a dependency then things tend to be fine.

                          If you’re shipping a proprietary application (as the linked rant seems to imply the author is doing), you’re better off shipping your own copy of the toolkit that you’re using (either statically linked or installed in a private location and dynamically linked via -rpath if you need to easily comply with the LGPL). If you do this then it’s guaranteed to work on FreeBSD for the lifetime of a major release (typically about 5 years) and will almost certainly work on the next one (which will have the previous version’s compat layer installed by default) and probably the one after that.

                      2. 2

                        Containers are the new static linking :P

                    2. 4

                      Also, how can Nim rate as more portable than C, when Nim is translated and run through a C compiler?

                      I was wondering about that too, so I asked him. His reply:

                      Nim has an extra dependency that requires an actual C compiler installed and working. It requires the Nim compiler, and also a C compiler, because the Nim compiler emits C code which the C compiler then compiles. That’s why it’s more fragile and complicated and more likely to break. (I have experienced this first-hand.)

                      1. 3

                        Isn’t that kind of repeating the objection? Sounds like he considers it no more portable than C, so his table has a typo(?) in it.

                        1. 1

                          I suppose it just means there are more moving parts that can break. If the path to the C compiler that’s baked into the Nim binary isn’t correct, for example, it won’t work. If the C compiler isn’t invoked with the correct flags, it won’t work, etc.

                      2. 3

                        how is a compiled C program not portable?

                        Unsure given that the author doesn’t provide many rationales, but maybe they’re putting C in the not-portable-by-default bucket because libc isn’t statically linked by default? 🤷‍♂️

                        1. 5

                          …on Linux, I guess. As I thought, this article seems implicitly Linux-specific. On Apple/Darwin platforms, and I think on Windows, the C library is built into the OS and everything dynamically links it.

                          1. 5

                            Windows would prefer not to have any C runtime, and C programs are supposed to have an installer and bundle their runtime: https://devblogs.microsoft.com/oldnewthing/20140411-00/?p=1273 (statically linking MSVCRT.lib still requires a DLL at run time).

                            macOS doesn’t have a stable kernel interface, so you can’t link libc (libSystem) statically. Fortunately it’s stable-enough for dynamic linking, provided that you set MACOSX_DEPLOYMENT_TARGET to an old-enough version.

                            1. 4

                              Windows doesn’t ship a C library*. What it does guarantee is everything that isn’t the C/C++ library. It ships language neutral (not linked against libc) libraries (not syscalls!) that applications link against instead. In practice, this means ABI stability that works for complex GUI applications.

                              *: Well kinda. Now there’s UCRT which came after the Chen post, but there was MSVCRT/CRTDLL before it. But shipping the libc means you can’t go wrong.

                              1. 2

                                You make it sound exotic to link against libSystem. ¯_(ツ)_/¯ Everything does. Shipping a C binary isn’t exotic, it’s the way nearly every piece of software on Apple platforms works.

                                As for Windows … I’ve learned it doesn’t pay to overestimate it.

                              2. 1

                                On most Linux-based OS it is the same.

                            2. 2

                              Is this portability problem something specific to Linux? The comment in the table says “no, libc”, but libc is either built into the OS or statically linked into the executable, no?

                              On Linux, you can pick your poison of libc: musl or glibc. Sure, most distros go for glibc, but musl is very popular, especially in container world.

                              (I personally think this list is mostly author’s personal aesthetics though.)

                              1. 4

                                Having a choice of C libs seems more like a curse than a boon, honestly.

                            3. 4

                              Not true for Perl: FatPacker can bundle pure-Perl dependencies, and PAR::Packer will even go so far as to bundle compiled XS and C architecture-specific libraries along with a full Perl interpreter.

                              1. 2

                                Nice. Worth emailing the author? By the looks of it they keep their post up-to-date.

                                1. 2

                                  Yep, just did.

                              2. 4

                                Not to be pedantic, but Haskell isn’t a compiler. It’s a language specification. They probably mean GHC.

                                1. 2

                                  Is there any other implementation that matters outside of GHC? Not like Hugs seems to be doing so hot…

                                  1. 4

                                    I mean, not really, but I think it’s still important to point out. There was Ajhc. Eta runs on the JVM. Ghcjs is a fork of ghc targeting JavaScript. There’s a wasm fork out there too, I forget the name. Intel has a research Haskell compiler. There are a bunch of projects out there. Yeah, none anywhere near as popular as GHC, but still.

                                2. 4

                                  If the internet stopped functioning, would you be left helpless?

                                  Yes, in terms of application deployment, I think all of us would.

                                  1. 3

                                    For Rust, why does this say that you need no_std to get portable binaries? I don’t feel like this is true. If you just want a portable binary within the same OS, you can use the MUSL libc target and statically compile everything. It’s crazy easy.

                                    1. 1

                                      Suppose you need to write something on a bare metal architecture, no operating system even, just an assembly bootstrap.

                                      1. 1

                                        Ok, sure, then you’d need no_std. But if we’re just talking about a binary being portable between different environments with the same kernel, then Rust + musl libc gets you there.

                                        The biggest hit to portability is dynamic linking. Once you start dynamically linking, you have to worry about what version of your dependencies are on your users’ machines. Or your users have to worry about it, at least.

                                        Portability between different operating systems is a less important problem, though seemingly still interesting given how much attention Cosmopolitan libc gets.

                                        Portability to the extent of “I don’t even need an OS” is probably not that valuable.

                                    2. 2

                                      There was a script a long time ago which let you package what is needed to run a program on one linux system to run it on another. I don’t recommend this as a common occurrence, but it’s been useful when I’ve had in-house executables for which source wasn’t available.

                                      1. [Comment removed by author]

                                        1. 1

                                          Because from their perspective C isn’t portable either. Which is a weird perspective, but I get why they have it.

                                        2. 1

                                          POSIX requires the cc and c99 command to be available.

                                          1. 1

                                            Emailed the author to hopefully get them to include https://crystal-lang.org/

                                            1. 1

                                              It’s interesting to see this as a modularity failure.