1. 30
  1. 23

    Waiting for ““Static Linking Considered Harmful” Considered Harmful” Considered Harmful

    1. 7

      it’s just harmful considerations all the way down

      1. 1

        Allthough if we’re to keep the cadence, the next harmful consideration is due in September 2036 or something.

      2. 2

        I think that would be hilarious!

        1. 2

          Rule of Threes. After ““Static Linking Considered Harmful” Considered Harmful” Considered Harmful, we’ll get tired of the joke. So, still one more to go lol

          1. 1

            We need a British person beyond that point. Because we all know that they have a good feeling for overdoing jokes to the point where they get funny again.

            1. 2

              But you have to listen very carefully, because they will say it only once.

          2. 2

            Reminds me of a notice I saw that said (translated) all posting except posting about posting forbidden forbidden.

          3. 9

            The argument against system call stability is not merely one of numbers, but of the entire shape of the system call interface. This includes things like which calls to even provide, and how to make the calling convention work with respect to register use or stack layout or even which mode switch mechanism to use. There’s a lot of detail in there that provides room for improvement, but only if you’re not committed to the specifics of what is really an internal implementation detail. The proposed solution of just deprecating calls you want to be rid of is not especially empathetic toward users, and if your OS has a strong culture of backwards compatibility for binaries it’s not really an option at all.

            1. 4

              I upvoted you because you have good points. But…

              I would argue that it is still only one of numbers.

              If an OS wants to try a new calling convention, or register use, or stack layout, they can add new syscalls that they define as using that new ABI while the existing syscalls use the old one.

              If it pans out, the old ones can be deprecated. If not, the new ones can be.

              In 32 bits, we have 4 billion numbers to use for syscalls. I doubt ABI’s are changing often enough to fill that. If it does, use 64 bits.

              In fact, I doubt syscall ABI’s really change that much anymore because there’s only so many options, and many OS’s have explored the space. We see this at a higher level with the slow convergence of OS’s on certain features and with the convergence of programming languages on certain features.

              This (in my opinion) kind of makes your argument moot, though we could certainly debate whether the convergence has happened enough yet to declare existing syscall ABI’s stable.

              The proposed solution of just deprecating calls you want to be rid of is not especially empathetic toward users, and if your OS has a strong culture of backwards compatibility for binaries it’s not really an option at all.

              Even Linux, the paragon of backwards compatibility for syscalls, is removing old syscalls.

              Also, I specifically said that removing the syscall should only happen after a deprecation period. I think this period should be 10-15 years. If software has not been updated in that time, then it really should only be run on old versions of the OS anyway, in my opinion.

              But regardless of all of that, I actually laid out some ideas that would allow OS’s to get rid of the problems with dynamic system libraries while keeping them. I personally think that using those ideas with dynamic system libraries would be better than our current system with a stable syscall ABI. That’s one reason I put out those ideas: to convince OS authors to adopt them. Implement my ideas, and I’ll lose my desire for a stable syscall ABI.

              So I don’t really agree, but I also think you are correct on some level.

              1. 9

                I would argue that it is still only one of numbers.

                I have been working for a long time on an OS that has some users, though likely not in the millions. It’s pretty rare that anything is as simple an issue as you’re making this out to be.

                I think this period should be 10-15 years. If software has not been updated in that time, then it really should only be run on old versions of the OS anyway, in my opinion.

                Maybe? Our (dynamically linked) ABI has been sufficiently backwards compatible that I can still run binaries from 2006 (15 years ago) without recompiling. I suspect I can probably run binaries from earlier than that as well, but that’s your 15 years and it’s also the oldest binary I can see in $HOME/bin right now. Pushing people to rebuild their software puts them on a treadmill of busy work at some frequency, and that doesn’t feel like it adds a tremendous amount of value. At some point clearly you can draw the line; e.g., it will likely not be possible for existing 32-bit binaries to be 2038 safe in any sense, for the obvious reasons about integer width. Because these binaries are only built against dynamic libraries with stable sonames and library versions (not the same as symbol versions), and don’t directly encode any system call specifics or other internals, we have at least a tractable, if not easy, time of making this promise.

                I have heard the noisy trumpeting of the static linking people for a pretty long time now, and it just doesn’t speak to me at all. It’s not like I hate them, I just don’t really share or even viscerally appreciate the problem they claim I must have, because I link a lot of software dynamically. There are, of course, exceptions!

                I agree that libraries like OpenSSL have, especially in the not-that-distant past, presented a capriciously moving target, both at the ABI level and even at the source compatibility level. That’s not really anything to do with dynamic (or static) linking, though, it’s a question of what the people who work on that project value – and it was, at least historically, not stable interfaces really at all.

                As we do a lot of our “linking” nowadays not merely statically or dynamically through ELF, but increasingly through IPC (e.g., sigh, D-Bus) and RPC (e.g., REST), the question of interface stability is the interesting part to me, rather than the specific calling mechanism. Once I put software together I’d really rather spend the minimum amount of time keeping it working in the future. Many, many software projects today don’t value stability, so they just don’t have it.

                But some projects do value stability – and thus, in my estimation, the time and energy of their consumers and community – and so I try my level best to depend only on those bodies of software. Dynamic linking just isn’t the problem I have.

                1. 7

                  I get that you don’t have the problem. But Docker containers exist precisely because a majority of software developers do have that problem and solved it the only way they were able to because no one was meeting that need except Containers.

                  1. 7

                    IME docker containers largely contain ruby code or python code or JavaScript code: things that have no concept of linking at all.

                    1. 5

                      They frequently contain C/C++ code as well. They get used for python/ruby/javascript frequently for the same reason though. Those languages have a form of dynamic linking that causes the exact same problem as dynamic linking causes for C code.

                  2. 1

                    I have been working for a long time on an OS that has some users, though likely not in the millions. It’s pretty rare that anything is as simple an issue as you’re making this out to be.

                    Fair, although I talked about why I thought it could be that simple.

                    In my opinion, our job as programmers is to make things as simple as we can without hindering our users.

                    Maybe? Our (dynamically linked) ABI has been sufficiently backwards compatible that I can still run binaries from 2006 (15 years ago) without recompiling. I suspect I can probably run binaries from earlier than that as well, but that’s your 15 years and it’s also the oldest binary I can see in $HOME/bin right now. Pushing people to rebuild their software puts them on a treadmill of busy work at some frequency, and that doesn’t feel like it adds a tremendous amount of value.

                    I don’t think it’s fair to say that 15 years of deprecation is a “treadmill of busy work,” even if it is at higher frequencies.

                    At some point clearly you can draw the line; e.g., it will likely not be possible for existing 32-bit binaries to be 2038 safe in any sense, for the obvious reasons about integer width.

                    This is true, and this is actually an example of an ABI break.

                    Because these binaries are only built against dynamic libraries with stable sonames and library versions (not the same as symbol versions), and don’t directly encode any system call specifics or other internals, we have at least a tractable, if not easy, time of making this promise.

                    I don’t believe you. You said that you have one or more 15-year-old binaries. Are you running on Linux or another platform? Regardless, I highly doubt there has not been an ABI break in 15 years, which means there might be some hidden bug(s) in those old binaries.

                    Sure, it might work, but if there have been any ABI breaks, you don’t know for sure.

                    I have heard the noisy trumpeting of the static linking people for a pretty long time now, and it just doesn’t speak to me at all. It’s not like I hate them, I just don’t really share or even viscerally appreciate the problem they claim I must have, because I link a lot of software dynamically. There are, of course, exceptions!

                    This, I understand. I don’t think everyone will have the same problems, and it seems that static linking won’t solve yours. That’s fine. That’s why my post from January is harmful: because it doesn’t leave room for that.

                    I agree that libraries like OpenSSL have, especially in the not-that-distant past, presented a capriciously moving target, both at the ABI level and even at the source compatibility level. That’s not really anything to do with dynamic (or static) linking, though, it’s a question of what the people who work on that project value – and it was, at least historically, not stable interfaces really at all.

                    I also agree that static linking is not going to help with software that is a moving target.

                    1. 6

                      I don’t believe you. You said that you have one or more 15-year-old binaries. Are you running on Linux or another platform? Regardless, I highly doubt there has not been an ABI break in 15 years, which means there might be some hidden bug(s) in those old binaries.

                      It’s common for large organizations using Windows to have a few ancient apps; there’s a reason that Microsoft only recently dropped support for 16bit Windows and DOS applications. API breaks aren’t an issue on Win32; Microsoft goes to heroic means to keep even poorly written applications running.

                      Here’s one example of Windows backward compatibility from Strategy Letter II: Chicken and Egg Problems

                      Jon Ross, who wrote the original version of SimCity for Windows 3.x, told me that he accidentally left a bug in SimCity where he read memory that he had just freed. Yep. It worked fine on Windows 3.x, because the memory never went anywhere. Here’s the amazing part: On beta versions of Windows 95, SimCity wasn’t working in testing. Microsoft tracked down the bug and added specific code to Windows 95 that looks for SimCity. If it finds SimCity running, it runs the memory allocator in a special mode that doesn’t free memory right away.

                      1. 1

                        That’s a thing on Windows.

                        Also, reading about how Windows managed to do that, I would argue that the complexity required makes it not worth it. At a certain point, you have to let buggy code remain buggy and don’t help it keep running.

                        1. 10

                          Disclaimer: I’m a Microsoft employee.

                          reading about how Windows managed to do that, I would argue that the complexity required makes it not worth it

                          Can I ask what you’re referring to with this? From what I’ve seen, most of the issue is just about following practices that make a maintainable ABI, and with that done, the complexity is not really that much. Stories like Simcity stand out specifically because they’re exceptional.

                          The issue in open source is that a lot of developers don’t know or don’t care to make a stable ABI, and once any common package takes that position, it forces downstream developers to work around it in some form, whether static linking, containers, whatever. Some are actively opposed to the concept of ABI stability because they believe in source code availability, so API stability is all that they’re aiming for. It’s gotten to the point where people equate “ABI” and “syscall” as equivalent concepts, but a stable ABI can exist at any layer, and it just so happens that Linus is aware of the need for it and capable of delivering it, so that layer has a stable ABI. (Frankly, it’d be very hard to build a non-ABI compatible kernel, because every kernel change would require rebuilding usermode to test the kernel; ain’t nobody got time for that.)

                          What we see with Linux distributions today is a pattern where upstream developers might make ABI incompatible changes in their development, then fix a security problem, then distributions take that patch and apply it to an older version in a way that doesn’t break the ABI, then offer 10 years of LTS support. The distributions can’t issue a security update that breaks the ABI, since they’re not Gentoo and don’t want to rebuild every program, and yet they issue updates on a regular cadence. Unfortunately this code flow means the learnings and practices of the distributions don’t necessarily flow back up to the original authors.

                          As I can tell, ABI compatibility is basically a necessity today. If you have a platform, applications run on it, and you want to be able to patch security updates every month (or similar), then it implies patches need to be able to be made without disturbing applications. And if you have a system that’s designed to do that every month, 15 years just starts to look like a large number of non-breaking changes. IMHO, the more this happens the easier it gets, because it forces patterns that allow non-breaking changes to be made repeatedly and reliably.

                          1. 4

                            Stories like Simcity stand out specifically because they’re exceptional.

                            Oh yeah! Stories like Simcity stand out, and not just because they’re exceptional, but also because they seemingly solve a trivial problem – all that work “just” to keep a “stupid game” running. It’s puzzling and challenges you to think a bit (and realize that supporting a popular game was really a significant problem in terms of adoption!), which is why it makes such a good story.

                            But it’s crazy just how much of the wheels that keep the real world spinning survive on ABI (and, to some degree, API) compatibility, and I think the FOSS community derives the wrong lessons out of this.

                            Lots of people in the Linux space think about it in terms of just how much effort goes into maintaining compatibility with legacy apps just because megacorps hate progress, lazy programmers won’t switching to the new and better versions of everything, evil companies would rather suck money out of their customer’s pockets than improve their products and so on.

                            Absurdly few people think of it in terms of just how much time that makes available to solve more important problems and add functionality that helps people do more things with their computers.

                            Yes, it looks stupid that so much effort went into keeping Sim City running – but that’s also one of the reasons why Sim City is the impressive franchise it is today, and why modern Sim City games are light years ahead of their old selves (nostalgia/personal preferences notwithstanding, I also like Sim City 2000 the most because that’s what I played when I was young): because Will Wright & co. could spend time working on new Sim City games, instead of perpetually impedance-matching Sim City to make it work on the latest DOS and Windows machines, the way we do with Linux software today.

                            1. 2

                              The other part is if someone’s favourite game didn’t work, they’d just blame it on Windows, even if it’s clearly the application’s fault. (Or if it was an actually critical business application.)

                            2. 1

                              Can I ask what you’re referring to with this? From what I’ve seen, most of the issue is just about following practices that make a maintainable ABI, and with that done, the complexity is not really that much. Stories like Simcity stand out specifically because they’re exceptional.

                              I’m referring to Tales of Application Compatibility.

                              For the record, I applaud the work that Microsoft does to maintain backwards compatibility. Of my many complaints about Microsoft, that is emphatically not one of them.

                              The issue in open source is that a lot of developers don’t know or don’t care to make a stable ABI, and once any common package takes that position, it forces downstream developers to work around it in some form, whether static linking, containers, whatever. Some are actively opposed to the concept of ABI stability because they believe in source code availability, so API stability is all that they’re aiming for.

                              I agree. Upstream developers rarely have any empathy for their users, who are downstream developers.

                              It’s gotten to the point where people equate “ABI” and “syscall” as equivalent concepts, but a stable ABI can exist at any layer, and it just so happens that Linus is aware of the need for it and capable of delivering it, so that layer has a stable ABI.

                              Agreed; I just think that the syscall layer is the best place to put the stability.

                              As I can tell, ABI compatibility is basically a necessity today. If you have a platform, applications run on it, and you want to be able to patch security updates every month (or similar), then it implies patches need to be able to be made without disturbing applications. And if you have a system that’s designed to do that every month, 15 years just starts to look like a large number of non-breaking changes. IMHO, the more this happens the easier it gets, because it forces patterns that allow non-breaking changes to be made repeatedly and reliably.

                              I agree that ABI compatibility is basically a necessity today. The ideas I put in my post are there to make it unnecessary tomorrow because even if you have great reliability about not having non-breaking changes, 15 years is long enough to assume it happened somewhere.

                              I guess, in essence, I agree with you, and you got an upvote; I just want to push us all in a direction where we don’t have to worry about such things.

                        2. 5

                          I don’t believe you. You said that you have one or more 15-year-old binaries. Are you running on Linux or another platform? Regardless, I highly doubt there has not been an ABI break in 15 years, which means there might be some hidden bug(s) in those old binaries.

                          This is on an illumos system now. Some of the oldest binaries I probably built on Solaris systems before we parted ways. In my experience, outside of perhaps RHEL which has sought traditionally to supplant systems like Solaris in enterprises with an understandably low tolerance for churn, there aren’t many Linux-based systems that have had as strong a focus on keeping old binaries working. The Linux kernel project has a strong focus on system call stability, but this ethos does not appear to extend to many distributions.

                          I can’t unequivocally state that the ABI hasn’t changed, but I can say that if we are made aware of a problem we’ll fix it – and that when we extend the system we are careful to avoid breakage. Some examples of ABI-related issues that I can recall include:

                          • The addition of open_memstream(3C) and related stdio functionality; the person who did this work wrote fairly extensively on some of the compatibility mechanisms in our stdio implementation

                          • We found that there had been a shift in certain alignment decisions made by the compiler over time, and some sigsetjmp(3C) calls were causing crashes; this was fixed to ensure that we accepted now-incorrectly aligned sigjmp_buf arguments, which will ensure that things continue to work for both old and new binaries.

                          1. 2

                            In my experience, outside of perhaps RHEL which has sought traditionally to supplant systems like Solaris in enterprises with an understandably low tolerance for churn, there aren’t many Linux-based systems that have had as strong a focus on keeping old binaries working. The Linux kernel project has a strong focus on system call stability, but this ethos does not appear to extend to many distributions.

                            I agree with this, and it makes me sad! Linux should be the best at this, but distros ruined it.

                            I can’t unequivocally state that the ABI hasn’t changed, but I can say that if we are made aware of a problem we’ll fix it

                            This is what I am afraid of. I hope there are not any unfixed issues in your systems, but there is no guarantee of that.

                            and that when we extend the system we are careful to avoid breakage.

                            This is good, but it is also the target of one section of my post: “ABI Breaks Impede Progress”. There is always going to be some place where ABI breaks cause problems. In fact, it your story about the compiler’s alignment decisions, it seems the compiler was defining your ABI and shifting it out from under you.

                            To minimize that, people reduce the progress they make, which is why the fear of ABI breaks impedes progress.

                            The point I was trying to make regarding syscalls was that I believe that they are the best place to guarantee ABI because you can always replace syscalls with bad ABI’s with different syscalls with good ABI’s, where you can’t really replace a bad dynamic library ABI without a lot of pain.

                  3. 8

                    One part of the analysis that I’m not confidently convinced by is this:

                    So on average, executables use about 6% of the symbols in shared libraries on my machine.

                    This whole section is using number of exported symbols referenced as a proxy for proportion of code reachable at runtime.

                    The functions you call can transitively call other functions and expose more code through function pointers and things that resemble eval().

                    For example you can exercise a really huge chunk of sqlite with just sqlite_open(), sqlite _exec(), sqlite_free() and sqlite_close()

                    1. 1

                      You have fair criticisms, and your example is correct. For that, you get an upvote.

                      I adopted that method because it was the method used by Drew DeVault in his post, which I cite in mine.

                    2. 6

                      Here to pick some nits:

                      Startup Times Ulrich Drepper also said:

                      With prelinking startup times for dynamically linked code is as good as that of statically linked code. Let’s see how true that is.

                      Emphasis mine. Did you actually use prelinking? Apologies if so and you just weren’t explicit about it. I’ve seen some impressive improvements with it, though in those cases I did not have the option of static vs. dynamic vs. prelink to really compare head to head, just prelinked vs non-prelinked.

                      [re ltrace] The truth is that you could do this with a proper build of a static library that tracked calls for you. In fact, that would give you more control because you could pick and choose which functions to trace at a fine-grained level.

                      For the record ltrace lets you pick and choose which functions to trace at a fine grained level.

                      1. 2

                        Emphasis mine. Did you actually use prelinking?

                        No, I did not. I was under the impression that glibc’s linker automatically did it. I will do it, update the post, and let you know.

                        For the record ltrace lets you pick and choose which functions to trace at a fine grained level.

                        Good to know. I will update the post.

                        1. 2

                          To update: I was not able to get prelinking to work, so I put a note in instead.

                          1. 2

                            I believe prelinking is now disabled because ASLR requires libraries to be linked at different addresses. I think OpenBSD has a prelinking-like thing that works with their form of address-space randomisation.

                            Note that static linking means that you are completely opting out of this kind of randomisation. Even if you build a statically linked PIE (mmm, PIE) then the displacements from any function pointer to any gadget are the same. I remain unconvinced that ASLR is anything other than security theatre but it is something you give up with most static linking designs (there are some half-way places where you compile with -ffunction-sections and -fdata-sections and randomise the binary layout in between each launch).

                      2. 5

                        Complaining about ABI stability causing language stagnation and design issues and then complaining about a lack of ABI stability in BSDs (because they don’t want to be held back) forcing the use of dynamic linking is a little interesting.

                        And default statically-linked binaries is one of the reasons the Go language became popular. I don’t think that’s a coincidence.

                        An appeal to popularity.

                        Other than that, the possible solutions section at the end FINALLY addresses actual problems I have always had with statically linking everything with very practical and simple solutions. I applaud this. I’ve read all three related articles now (this one, Dynamic Linking Needs To Die and Static Linking Considered Harmful) and this really is the best of all three.

                        I would note though that this doesn’t propose a solution for my most annoying problem with static linking in the C world: namespace pollution. My current solution is to give all functions, that are not part of the API, macros which suffix/prefix their name with something random to minimize the possibility of collisions.

                        1. 1

                          Complaining about ABI stability causing language stagnation and design issues and then complaining about a lack of ABI stability in BSDs (because they don’t want to be held back) forcing the use of dynamic linking is a little interesting.

                          They’re different types of ABI’s. But then again, I also present ideas to get around the problems with dynamic linking, allowing BSD’s to have the lack of ABI stability. At that point, I’d be okay with them not having ABI stability too.

                          An appeal to popularity.

                          I understand your sentiment, but I used it because some people want their software to be popular. I’m just saying that this might be something that helps. I want my programming language to be popular, so this appeal to popularity works on me, at least.

                          But I also had an implied argument there: that static linking is easiest for downstream developers. It’s implied because people usually gravitate to what’s easiest.

                          Other than that, the possible solutions section at the end FINALLY addresses actual problems I have always had with statically linking everything with very practical and simple solutions.

                          I apologize that it took so long to get there; that’s how it flowed in my mind. Should I have put them first?

                          I applaud this. I’ve read all three related articles now (this one, Dynamic Linking Needs To Die and Static Linking Considered Harmful) and this really is the best of all three.

                          Thank you. :) And I apologize for “Dynamic Linking Needs to Die”.

                          I would note though that this doesn’t propose a solution for my most annoying problem with static linking in the C world: namespace pollution.

                          I would like to talk to you more about this because my programming language uses C-like names and could suffer from namespace pollution. It’s helped somewhat because prefixes are automatically added for items in packages (i.e., for the function foo() in the package bar, its C name is bar_foo()), but I have not been able to ensure it avoids all of the problems.

                          My current solution is to give all functions, that are not part of the API, macros which suffix/prefix their name with something random to minimize the possibility of collisions.

                          Is this automatic? Should it be automatic?

                          I kind of think I know what you’re going at here, and I think it’s about the same thing that gcc and other C compilers do for functions with static visibility. So in my language, any functions and types private to a package would be treated as though they have static visibility.

                          I think that would be sufficient because I think the problem C has is that something does not have static visibility unless you mark it so, leading to a lot of functions that should be static polluting the namespace.

                          I’m not entirely sure, so what do you think?

                          1. 4

                            But I also had an implied argument there: that static linking is easiest for downstream developers. It’s implied because people usually gravitate to what’s easiest.

                            There was an article on here or the other website recently about people’s focus on what is easiest for developers. I personally think that we should be looking at what’s best for users not what’s easiest for developers.

                            I apologize that it took so long to get there; that’s how it flowed in my mind. Should I have put them first?

                            No, I just meant I read dozens of these “dynamic linking bad” articles which complain about dynamic linking and tout the benefits of static linking as if there were literally no benefits to dynamic linking (usually after claiming that because 90% of libraries aren’t shared as much as 10% that means we should ditch 100% of dynamic linking). This was also the vibe of your original article (and a similar vibe of the “static linking considered harmful” article except in the opposite direction).

                            The problem is that while yes, on the face of it you could say that static linking as things currently stand is less of a shit show, nobody seemed to acknowledge that it might be worth trying to solve the problem in a way which keeps some of the key benefits of dynamic linking. Your article is the first I’ve read so far which actually took this into consideration. That’s why I said I finally have read an article which covers these things.

                            Is this automatic? Should it be automatic?

                            It’s not so easy to automate these things portably. I have not looked into automating it but at the same time I almost never write libraries (my opinion is on the other extreme from npm proponents, don’t write a library unless you’ve already written similar code in a number of places and know what the API should look like from real experience).

                            I kind of think I know what you’re going at here, and I think it’s about the same thing that gcc and other C compilers do for functions with static visibility. So in my language, any functions and types private to a package would be treated as though they have static visibility.

                            I think that would be sufficient because I think the problem C has is that something does not have static visibility unless you mark it so, leading to a lot of functions that should be static polluting the namespace.

                            I’m not entirely sure, so what do you think?

                            Fundamentally the problem is that in C and C++ to get a static library on linux you compile your code into object files and then you put them into an weird crusty old archive format. At the end of the day, unlike with a shared object, you can’t mark symbols as being local to the library since when it comes to linking, it’s as if you had directly added the list of object files to the linking command.

                            Let me give you an example:

                            ==> build <==
                            #!/bin/sh -ex
                            cc -std=c11 -c lib.c
                            cc -std=c11 -c internal.c
                            ar rc lib.a lib.o internal.o
                            ranlib lib.a
                            cc -shared lib.o internal.o -o liblib.so
                            cc -std=c11 -I. -c prog.c
                            cc prog.o lib.a -o prog_static
                            cc prog.o -L. -llib -Wl,-rpath=. -o prog_dynamic
                            
                            ==> clean <==
                            #!/bin/sh
                            rm -f *.o *.a *.so prog_*
                            
                            ==> internal.c <==
                            #include "internal.h"
                            #include <stdio.h>
                            void print(const char *message)
                            {
                            	puts(message);
                            }
                            
                            ==> internal.h <==
                            #ifndef INTERNAL_H
                            #define INTERNAL_H
                            __attribute__ ((visibility ("hidden")))
                            void print(const char *message);
                            #endif
                            
                            ==> lib.c <==
                            #include "lib.h"
                            #include "internal.h"
                            __attribute__ ((visibility ("default")))
                            void api(void)
                            {
                            	print("api");
                            }
                            
                            ==> lib.h <==
                            #ifndef LIB_H
                            #define LIB_H
                            __attribute__ ((visibility ("default")))
                            void api(void);
                            #endif
                            
                            ==> prog.c <==
                            #include <lib.h>
                            int main(void)
                            {
                            	api();
                            	extern void print(const char *message);
                            	print("main");
                            }
                            

                            In the above example, lib.h exposes an API, this consists of the function api. internal.h is internal to the library and exposes a function print. The visibility has been marked but from the point of view of the linker, linking prog_static from prog.o and lib.a is equivalent to linking prog.o, lib.o and internal.o. This means the linking succeeds and print can be called from main. In the library case, the linking step happens first, the visibilities are honored and once you try to link to the library the print function is no longer exposed.

                            There is no way to hide the print function in this case. And given how internal functions usually don’t have namespaced names, they have a higher chance of colliding. The solution would be to replace internal.h with the following:

                            #ifndef INTERNAL_H
                            #define INTERNAL_H
                            #define print lib_internal_print
                            __attribute__ ((visibility ("hidden")))
                            void print(const char *message);
                            #endif
                            

                            (Or something similar like #define print print_3b48214e)

                            This would namespace the function without having the entire codebase end up having to call the function by the long name.

                            But this is all hacks, and doesn’t solve the visibility problem.

                            I hope you see what I mean now.

                            1. 1

                              There was an article on here or the other website recently about people’s focus on what is easiest for developers. I personally think that we should be looking at what’s best for users not what’s easiest for developers.

                              I agree, but I think making certain things easier on developers would reduce mistakes, which would make things easier on users. So whenever I can reduce mistakes by developers by making things easier, I do so.

                              No, I just meant I read dozens of these “dynamic linking bad” articles which complain about dynamic linking and tout the benefits of static linking as if there were literally no benefits to dynamic linking (usually after claiming that because 90% of libraries aren’t shared as much as 10% that means we should ditch 100% of dynamic linking). This was also the vibe of your original article (and a similar vibe of the “static linking considered harmful” article except in the opposite direction).

                              Oh, I see. Yeah, sorry again about the first one. It was unprofessional.

                              It’s not so easy to automate these things portably. I have not looked into automating it but at the same time I almost never write libraries (my opinion is on the other extreme from npm proponents, don’t write a library unless you’ve already written similar code in a number of places and know what the API should look like from real experience).

                              I think this is a great guideline.

                              Regarding the code, yes, I think I see what you mean.

                              As much as it pains me, I think this means that I am going to have to “break” ABI with C enough to make such private functions not visible to outside code. (Maybe I’ll add a suffix to them, and the suffix will be the compiler-internal ID of their scope?) But that means that I can make it automatic.

                              However, I can probably only do this in the compiler for my programming language, not for C itself.

                              1. 2

                                My recommendation is: make your own static linking format. But if you want static linking interop with C then there’s nothing stopping you from exposing that as a single object file (the issue comes from having multiple object files in the archive).

                        2. 5

                          This is an interesting article, but some things that stood out to me:

                          • I’d have liked to see the article engage more with the libc case. The article points out several arguments against statically linking libc, specifically - it’s the only supported interface to the OS on most *nix-ish systems that are not Linux, it’s a very widely used library (so disk/memory savings can be substantial), and as part of the system it should be quite stable. Also, Go eventually went back to (dynamically) linking against libc rather than directly making syscalls [EDIT: on many non-Linux systems]
                          • LLVM IR is platform-specific and not portable. And existing C code using e.g. #ifdef __BIG_ENDIAN__ (or using any system header using such a construction!) is not trivial to compile into platform-independent code. Yes, you can imagine compiling with a whole host of cross-compilers (and system headers!), but at some point just shipping the source code begins looking rather attractive…
                          • exploiting memory bugs is a deep topic, but the article is a bit too simplistic in its treatment of stack smashing. There’s a lot to dislike about ASLR, but ASLR is at least somewhat effective against - to think up a quick example - a dangling pointer in a heap-allocated object being use(-after-free)d to overwrite a function / vtable pointer on the stack.
                          • in general, there’s a lot of prior art that could be discussed; e.g. I believe that Windows randomizes (randomized?) a library’s address system-wide, rather than per-process.
                          1. 3

                            And existing C code using e.g. #ifdef __BIG_ENDIAN__ (or using any system header using such a construction!) is not trivial to compile into platform-independent code.

                            The article addresses this.

                            I would also note as an aside that any code utilising #ifdef __BIG_ENDIAN__ is just plain wrong. Yes, even /usr/include/linux/tcp.h. Just don’t do that. Write the code properly.

                            1. 2

                              I’ll bite. I have written a MC6809 emulator that makes the following assumptions of the host system:

                              • A machine with 8-bit chars (in that it does support uint8_t)

                              • A 2’s complement architecture

                              I also do the whole #ifdef __BIG_ENDIAN__ (effectively), check out the header file. How would you modify the code? It’s written that way to a) make it easier on me to understand the code and b) make it a bit more performant.

                              1. 3

                                I would write inline functions which would look like:

                                static inline mc6809byte__t msb(mc6809word__t word) { return (word >> 8) & 0xff; }
                                static inline mc6809byte__t lsb(mc6809word__t word) { return word & 0xff; }
                                

                                And I would use those in place of the current macro trick of replacing something->A with something->d.b[MSB] etc.

                                I don’t think this would significantly impact readability. Clang seems to produce identical code for both cases, although from a benchmark there’s still some minor (10% if your entire workload is just getting data out of the MSB and LSB) performance impact although this may be some issue with my benchmark. gcc seems to struggle to realize that they are equivalent and keeps the shr but the performance impact is only 13%.

                                If you need to support writing to the MSB and LSB you would need a couple more inline functions:

                                static inline set_msb(mc6809word__t *word, mc6809byte__t msb) { *word = lsb(*word) | ((msb & 0xff) << 8); }
                                static inline set_lsb(mc6809word__t *word, mc6809byte__t lsb) { *word = (msb(*word) << 8) | (lsb & 0xff); }
                                

                                I haven’t benchmarked these.

                                I think the point I would make is that you should benchmark your code against these and see whether there is a real noticeable performance impact moving from your version to this version. Through this simple change you can make steps to drop your assumption of CHAR_BIT == 8 and your code no longer relies on type punning which may or may not produce the results you expect depending on what machine you end up on. Even though your current code is not doing any in-place byte swapping, you still risk trap representations.

                                P.S. *_t is reserved for type names by POSIX.

                                1. 2

                                  I would definitely have to benchmark the code, but I can’t see it being better than what I have unless there exists a really magical C compiler that can see through the shifting/masking and replace them with just byte read/writes (which is what I have now). Your set_lsb() function is effectively:

                                  *word = ((*word >> 8) & 0xff) << 8 | lsb & 0xff;
                                  

                                  A 10-13% reduction in performance seems a bit steep to me.

                                  you still risk trap representations.

                                  I have to ask—do you know of any computer sold new today that isn’t byte oriented and 2’s complement? Or hell, any machine sold today that actually has a trap representation? Because I’ve been programming for over 35 years now, and I have yet to come across one machine that a) isn’t byte oriented; b) 2’s complement; c) has trap representations. Not once. So I would love to know of any actual, physical machines sold new that breaks one of these assumptions. I know they exist, but I don’t know of any that have been produced since the late 60s.

                                  1. 2

                                    a really magical C compiler that can see through the shifting/masking and replace them with just byte read/writes

                                    Yes, that’s what clang did when I tested it on godbolt. In fact I can get it to do it in all situations by swapping the order of the masking and the shifting.

                                    Here’s the result:

                                                            output[i].lower = in & 0xff;
                                      4011c1:       88 54 4d 00             mov    %dl,0x0(%rbp,%rcx,2)
                                                            output[i].upper = (in & 0xff00) >> 8;
                                      4011c5:       88 74 4d 01             mov    %dh,0x1(%rbp,%rcx,2)
                                    

                                    You underestimate the power of compilers, although I’m not sure why gcc can’t do it, it’s really a trivial optimisation all things considered.

                                    I just checked further and it seems the only reason that the clang compiled mask&shift variant performs differently is because of different amounts of loop unrolling and also because the mask&shift code uses the high and low registers instead of multiple movbs. The godbolt code didn’t use movbs, it was identical for both cases in clang.

                                    My point being that in reality you may get 10% (absolute worst case) difference in performance just because the compiler felt like it that day.

                                    I have to ask—do you know of any computer sold new today that isn’t byte oriented and 2’s complement? Or hell, any machine sold today that actually has a trap representation?

                                    I don’t personally keep track of the existence of such machines.

                                    For me it’s not about “any machine” questions, it’s about sticking to the C abstract machine until there is a genuine need to stray outside of that, a maybe 13% at absolute unrealistic best performance improvement is not worth straying outside the definitions of the C abstract machine.

                                    In general I have found this to produce code with fewer subtle bugs. To write code which conforms to the C abstract machine you just have to know exactly what is well defined. To write code which goes past the C abstract machine you have to know with absolute certainty about all the things which are not well defined.

                                    edit: It gets worse. I just did some benchmarking and I can get swings of +-30% performance by disabling loop unrolling. I can get both benchmarks to perform the same by enabling and disabling optimization options.

                                    This is a tight loop doing millions of the same operation. Your codebase a lot more variation than that. It seems more likely you’ll get a 10% performance hit/improvement by screwing around with optimisation than you will by making the code simply more correct.

                            2. 2

                              Wait, Go dynamically links to libc now? Do you have more details? I thought Go binaries have zero dependencies and only the use of something like CGO would change that.

                              1. 15

                                As the person responsible for making Go able to link to system libraries in the first places (on illumos/Solaris, others later used this technology for OpenBSD, AIX and other systems), I am baffled why people have trouble understanding this.

                                Go binaries, just like any other userspace binary depend at least on the operating system. “Zero dependency” means binaries don’t require other dependencies other than the system itself. It doesn’t mean that the dependency to the system cannot use dynamic linking.

                                On systems where “the system” is defined by libc or its equivalent shared library, like Solaris, Windows, OpenBSD, possibly others, the fact that Go binares are dynamically linked with the system libraries doesn’t make them not “zero dependency”. The system libraries are provided by the system!

                                Also note that on systems that use a shared library interface, Go doesn’t require the presence of the target shared library at build time, only compiled binaries require it at run time. Cross-compiling works without having to have access to target system libraries. In other words, all this is an implementation detail with no visible effect to Go users, but somehow many Go users think this is some kind of a problem. It’s not.

                                1. 3

                                  I (clarified) was thinking about e.g. OpenBSD, not Linux; see e.g. cks’ article.

                                  1. 2

                                    Under some circumstances Go will still link to libc. The net and os packages both use libc for some calls, but have fallbacks that are less functional when CGO is disabled.

                                    1. 6

                                      The way this is usually explained is a little bit backwards. On Linux, things like non-DNS name resolution (LDAP, etc), are under the purview of glibc (not any other libc!) with its NSCD protocol and glibc-specific NSS shared libraries.

                                      Of course that if you want to use glibc-specific NSS, you have to to link to glibc, and of course that if you elect not to link with glibc you don’t get NSS support.

                                      Most explanations of Go’s behavior are of the kind “Go is doing something weird”, while the real weirdness is that in Linux name resolution is not something under of the purview of the system, but of a 3rd party component and people accept this sad state of affairs.

                                      1. 3

                                        How is glibc a 3rd party component on, say, Debian? Or is every core component 3rd party since Debian does not develop any of them?

                                        1. 4

                                          Glibc is a 3rd party component because it is not developed by the first party, which is the Linux developers.

                                          Glibc sure likes to pretend it’s first party though. It’s not, a fact simply attested by the fact other Linux libc libraries exists, like musl.

                                          Contrast that with the BSDs, or Solaris, or Windows, where libc (or its equivalent) is a first party component developed by BSDs, Solaris, or Windows developers.

                                          I would hope that Debian Linux would be a particular instance of a Linux system, rather than an abstract system itself, and I could use “Linux software” on it but glibc’s portability quirks and ambitions of pretending to be a 1st party system prevent one from doing exactly that.

                                          Even if you think that Linux+glibc should be an abstract system in itself, distrinct from, say, pure Linux, or Linux+musl, irrespective of the pain that would instil on me, the language developer, glibc is unfeasible as an abstract interface because it is not abstract.

                                          1. 3

                                            Wait, how are the Linux developers “first party” to anything but the kernel?

                                            I would hope that Debian Linux would be a particular instance of a Linux system

                                            There’s no such thing as “a Linux system” only “a system that uses Linux as a component”. Debian is a system comparable to FreeBSD, so is RHEL. Now some OS are specifically derived from others, so you might call Ubuntu “a Debian system” and then complain that snap is an incompatible bolt on or something (just an example, not trying to start an argument about snap).

                                            1. 5

                                              Of course there is such a thing as a Linux system, you can download it from kernel.org, it comes with a stable API and ABI, and usually, in fact with the exception of NSS above, absolutely always does exactly what you want from it.

                                              Distributions might provide some kind of value for users, and because they provide value they overestimate their technical importance with silly statements like “there is no Linux system, just distributions”, no doubt this kind of statement comes from GNU itself, with its GNU+Linux stance, but from a language designer none of this matters at all. All that matter are APIs and ABIs and who provides them. On every normal system, the developer of the system dictates and provides its API and ABI, and in the case of Linux that’s not different, Linux comes with its stable API and ABI and as a user of Linux I can use it, thank you very much. The fact that on Linux this ABI comes though system calls, while on say, Solaris, comes from a shared library is an implementation detail. Glibc, a 3rd party component comes with an alternative API and ABI, and for whatever reason some people think that is more canonical than the first party API and ABI provided by the kernel itself. The audacity of glibc developers claiming authority over such a thing is unbelievable.

                                              As a language developer, and in more general as an engineer, I work with defined systems. A system is whatever has an API and an ABI, not some fuzzy notion defined by some social organization like a distribution, or a hostile organization like GNU.

                                              As an API, glibc is valuable (but so is musl), as an ABI glibc has negative value both for the language developer and for its users. The fact that in Go we can ignore glibc, means not only freedom from distributions’ and glibc’s ABI quirks, but it also means I can have systems with absolutely no libc at. Just a Linux kernel and Go binaries, a fact that plenty of embedded people make use of.

                                              1. 2

                                                Of course there is such a thing as a Linux system, you can download it from kernel.org,

                                                So is libgpiod part of the system too? You can download that from kernel.org as well. You can even download glibc there.

                                                it comes with a stable API and ABI, and usually, in fact with the exception of NSS above, absolutely always does exactly what you want from it.

                                                Unless you want to do something other than boot :)

                                  2. 2

                                    I’d have liked to see the article engage more with the libc case.

                                    Fair, though I feel like I engaged with it a lot. I even came up with ideas that would make it so OS authors can keep their dynamically-linked libc while not causing problems with ABI and API breaks.

                                    What would you have liked me to add? I’m not really sure.

                                    LLVM IR is platform-specific and not portable.

                                    Agreed. I am actually working on an LLVM IR-like alternative that is portable.

                                    And existing C code using e.g. #ifdef BIG_ENDIAN (or using any system header using such a construction!) is not trivial to compile into platform-independent code.

                                    This is a good point, but I think it can be done. I am going to do my best to get it done in my LLVM alternative.

                                    the article is a bit too simplistic in its treatment of stack smashing.

                                    Fair; it wasn’t the big point of the post.

                                    There’s a lot to dislike about ASLR, but ASLR is at least somewhat effective against - to think up a quick example - a dangling pointer in a heap-allocated object being use(-after-free)d to overwrite a function / vtable pointer on the stack.

                                    I am not sure what your example is. Could you walk me through it?

                                    1. 3

                                      Agreed. I am actually working on an LLVM IR-like alternative that is portable.

                                      This is not possible for C/C++ without either:

                                      • Significantly rearchitecting the front end, or
                                      • Effectively defining a new ABI that is distinct from the target platform’s ABI.

                                      The second of these is possible with LLVM today. This is what pNaCl did, for example. If you want to target the platform ABI then the first is required because C code is not portable after the preprocessor has run. The article mentions __BIG_ENDIAN__ but that’s actually a pretty unusual corner case. It’s far more common to see things that conditionally compile based on pointer size - UEFI bytecode tried to abstract over this and attempts to make Clang and GCC target it have been made several times and failed and that was with a set of headers written to be portable. C has a notion of an integer constant expression that, for various reasons, must be evaluated in the front end. You can make this symbolic in your IR, but existing front ends don’t.

                                      The same is true of C++ templates, where it’s easy to instantiate a template with T = long as the template parameter and then define other types based on sizeof(T) and things like if constexpr (sizeof(T) < sizeof(int)), at which point you need to preserve the entire AST. SFINAE introduces even more corner cases where you actually need to preserve the entire AST and redo template instantiation for each target (which may fail on some platforms).

                                      For languages that are designed to hide ABI details, it’s easy (see: Java or CLR bytecode).

                                      1. 2

                                        I believe you are correct, but I’ll point out that I am attempting to accomplish both of the points you said would need to happen, mostly with a C preprocessor with some tricks up its sleeve.

                                        Regarding C++ templates, I’m not even going to try.

                                        I would love to disregard C and C++ entirely while building my programming language, but I am not disregarding C at least because my compiler will generate C. This will make my language usable in the embedded space (there will be a -nostd compiler flag equivalent), and it will allow the compiler to generate its own C source code, making bootstrap easy and fast because unlike Rust or Haskell, where you have to follow the bootstrap chain from the beginning, my compiler will ship with its own C source, making bootstrap as simple as:

                                        1. Compile C source.
                                        2. Compile Yao source. (Yao is the name of the language.)
                                        3. Recompile Yao source.
                                        4. Ensure the output of 2 and 3 match.

                                        With it that easy, I hope that packagers will find it easy enough to do in their packages.

                                        1. 2

                                          If your input is a language that doesn’t expose any ABI-specific details and your output is C (or C++) code that includes platform-specific things, then this is eminently tractable.

                                          This is basically what Squeak does. The core VM is written in a subset of Smalltalk that can be statically compiled to C. The C code can then be compiled with your platform’s favourite C compiler. The rest of the code is all bytecode that is executed by the interpreter (or JIT with Pharo).

                                      2. 3

                                        Agreed. I am actually working on an LLVM IR-like alternative that is portable.

                                        If you’re looking for prior art, Tendra Distribution Format was an earlier attempt at a UNCOL for C.

                                        1. 1

                                          Thank you for the reference!

                                        2. 3

                                          With respect to libc - indeed, the article engaged quite a bit with libc! That’s why I was waiting for a clear conclusion.

                                          E.g. cosmopolitan clearly picks “all the world’s a x86_64, but may be running different OSes”, musl clearly picks “all the world’s Linux, but may be running on different architectures”. You flirt with both “all the world’s Linux” and something like Go-on-OpenBSD’s static-except-libc. Which are both fine enough.

                                          With respect to ASLR: I agree that this isn’t the main point of your article, and I don’t think I explained what I meant very well. Here’s some example code, where the data from main() is meant to represent hostile input; I mean to point out that merely segregating arrays and other data / data and code doesn’t fix e.g. lifetime issues, and that ASLR at least makes the resulting bug a bit harder to exploit (because the adversary has to guess the address of system()). A cleaned-up example can be found below, (actually-)Works-For-Me code here.

                                          static void buggy(void *user_input) {
                                              uintptr_t on_stack_for_now;
                                              /* Bug here: on_stack_for_now doesn't live long enough! */
                                              scheduler_enqueue(write_what_where, user_input, &on_stack_for_now);
                                          }
                                          
                                          static void victim(const char *user_input) {
                                              void (*function_pointer)() = print_args;
                                          
                                              if (scheduler_run() != 0)
                                                  abort();
                                          
                                              function_pointer(user_input);
                                          }
                                          
                                          int main(void) {
                                              buggy((void *)system);
                                              victim("/bin/sh");
                                          }
                                          
                                          1. 2

                                            With respect to libc - indeed, the article engaged quite a bit with libc! That’s why I was waiting for a clear conclusion.

                                            I see. Do you mean a conclusion as to whether to statically link libc or dynamically link it?

                                            I don’t think there is a conclusion, just a reality. Platforms require programmers to dynamically link libc, and there’s not much we can do to get around that, though I do like the fact that glibc and musl together give us the choice on Linux.

                                            However, I think the conclusion you are looking for might be that it does not matter because both suck with regards to libc!

                                            If you statically link on platforms without a stable syscall ABI, good luck! You can probably only make it work on machines with the same OS version.

                                            If you dynamically link, you’re probably going to face ABI breaks eventually.

                                            So to me, the conclusion is that the new ideas I gave are necessary to make working with libc easier on programmers. Right now, it sucks; with my ideas, it wouldn’t (I hope).

                                            Does that help? Sorry that I didn’t make it more clear in the post.

                                            Regarding your code, I think I get it now. You have a point. I tried to be nuanced, but I should not have been.

                                            I am actually developing a language where lifetimes are taken into account while also having separated stacks. I hope that doing both will either eliminate the possibility of such attacks or make them infeasible.

                                      3. 3

                                        Everyone trots out “DLL hell”, but I don’t think I’ve ever seen this on Windows and it’s purely rhetoric. If anything, dynamic linking is how Windows has maintained herculean backwards compatibility for so long.

                                        1. 8

                                          In 16-bit versions of Windows, a DLL is identified by its filename (for example, C:\WINDOWS\SYSTEM\MCI.DLL is identified as “MCI”), and is reference-counted. So if one application starts up and loads the MCI library, then another application starts up and loads the same library, it’s given a handle to the same library that the first application is using… even if that’s a different file on disk than the application would have loaded if it had started first.

                                          Thus, even if your application shipped the exact version of the DLL you wanted to use, and installed it beside the executable so it wouldn’t be clobbered by other applications, when your application loaded the library you could still wind up with any version of the library, with any ABI, without any bugs or bug-fixes your application relied on… and because every library in the universe had to share the 8-character identifier space, collisions with entirely unrelated libraries weren’t unheard of either.

                                          This is truly DLL Hell.

                                          Modern versions of Windows are less prone to these problems, since Windows 98 allowed different applications to load different DLLs with the same name, since the convention to ship DLLs beside the executable rather than install them to C:\WINDOWS, since modern PCs have enough RAM to occasionally load duplicate copies of a library.

                                          1. 3

                                            On 16- and 32-bit x86, position-independent code was fairly expensive: there’s no PC-relative addressing mode and so the typical way of faking it is to do a PC-relative call (which does exist) to the next instruction and then pop the return address (which is now your PC) from the stack into a register and use that as the base for loads and stores. This burns a general-purpose register (of which you have 6 on 32-bit x86).

                                            Windows avoided this entirely by statically relocating DLLs. I think on 16-bit Windows each DLL was assigned to a segment selector[1] on 286s, on real-mode or 32-bit Windows there was a constraint solver that looked at every .EXE on the system and the set of .DLLs that it linked and tried to find a base address for each DLL that didn’t conflict with other DLLs in any EXE that linked that DLL. If the solver couldn’t find a solution then you’d end up with some DLLs that would need to be resident twice with different relocations applied. I think in some versions of Windows it would just fail to load an EXE.

                                            [1] 16-bit Windows is really 24-bit Windows. The exciting 8086 addressing mode is a 16-bit segment base right shifted by 8 and added to the address, giving a 24-bit address space. As I recall, .COM files were true 16-bit programs in a 64KiB segment, Windows provided an ABI where pointers were 16-bit segment-relative things but far pointers were stored as 32-bit integers that expanded to either a 24- or 32-bit address depending on whether you were on real-mode, standard-mode (286) or 386-enhanced mode (protected mode) Windows.

                                          2. 1

                                            Even if it’s not a thing on Windows (anymore), and it probably isn’t, problems with dynamic linking are a problem on Linux.

                                            1. 3

                                              I’ve seen DLL hell on windows, never on linux. Package managers are awfully effective at avoiding it in my experience. If you go behind your distro package manager’s back, that’s on you.

                                              1. 1

                                                I agree with you, but I’ve seen it with package managers too.

                                                1. 2

                                                  “I’ve seen it” is quite annecdotal. I think the point is, like u/atk said, it’s very rare and not a problem on linux, that most people writing code for linux would have to deal with.

                                                  1. 1

                                                    That’s fair.

                                          3. 3

                                            It seems to me that proponents of dynamic linking kind of miss the forest for the trees. The system is incidental, the only thing that matters are the applications. The value of a binary that just works is enormously higher than the value of system-wide security patches or etc.

                                            1. 2

                                              I agree.

                                              I think following your comment would be a great way to design a user-friendly OS: you want to make things “just work” as you say, while having security.

                                            2. 3

                                              If you like static linking and LLVM IR then you’ll love ALLVM.

                                              1. 1

                                                That does look like the sort of thing I am talking about.

                                              2. 2

                                                Drew DeVault also measured how much of the libraries were used (by number of symbols), so let’s do that too, using his code.

                                                The number of used symbols is not a good proxy for how large the statically linked program will be. The few symbols may well be the entry point to a large number of relocatable object files.

                                                On the other hand, an executable A linking against B.a may be smaller than A + b.so because of (a) archive member extraction and section-based garbage collection (https://maskray.me/blog/2021-02-28-linker-garbage-collection), (b) the fact that using shared objects means most symbols need to be exported which defeats –gc-sections. Though I shall note that -Wl,–gc-sections is not common among distros.

                                                First, the linker can sometimes map segments containing const data into writeable memory, allowing attackers to overwrite constant data.

                                                The linked paper exaggerated things. ld.lld .bss.rel.ro fixed the problem. For compatibility, ld.lld doesn’t use .bss.rel.ro in the presence of a SECTIONS linker script command.

                                                In addition, copy relocations are almost not a problem for -fPIE. Unfortunately GCC -fPIE x86-64 is problematic and they don’t take my trivial fix https://maskray.me/blog/2021-01-09-copy-relocations-canonical-plt-entries-and-protected#summary

                                                Performance

                                                -fPIC and shared objects are not bad. We need more awareness and fixing GCC/GNU ld:) https://maskray.me/blog/2021-05-16-elf-interposition-and-bsymbolic#the-last-alliance-of-elf-and-men

                                                I actually replied to the thread with As Linus Torvalds said: but he did not listen:) Anyway, I added -fno-semantic-interposition and -Bsymbolic-functions to libLLVM-*.so and libclang-cpp.so and the performance loss due to -fPIC is mostly fixed.

                                                1. 2

                                                  “I stand corrected”

                                                  1. 2

                                                    Well, yes, I do. Not completely, however.

                                                  2. 1

                                                    The style of introducing the plots breaks the reading flow. You first share the code to generate these graphs and then shows the graphs without axes description. So I need to read the code to understand what it does and can then understand what the graph is telling me. After that I need to reread the argument to check if the graph supports the argument and to mentally get back to the actual argument. It would be nice if you help the reader understand what your graph says. I would also suggest to just link the generation code instant of embedding it, because this reduces the disruption.

                                                    1. 1

                                                      I think you have good points.

                                                      I did it the way I did to 1) make the post self-contained, and 2) to follow the style of Drew DeVault’s post.

                                                      I think I could keep them in while still helping people not break flow.

                                                      Currently, the little JavaScript I do have on the site is to expand and contract the Table of Contents. What if I added a tiny bit of JavaScript to expand the code and charts? And when not expanded, it puts a small “alt” text instead.

                                                      Would that be sufficient?