1. 22
  1. -1

    The complicators won.

    We just can’t have simple binary formats.

    1. 19

      I think this is a nice bit of history, and describes a very good reason for folks moving away from a.out format executables:

      When introducing shared libraries certain design decisions had to be made to work in the limitations of a.out. The main accepted limitation was that no reloca- tions are performed at the time of loading and afterward. The shared libraries have to exist in the form they are used at run-time on disk. This imposes a major restric- tion on the way shared libraries are built and used: every shared library must have a fixed load address; otherwise it would not be possible to generate shared libraries which do not have to be relocated.

      —Ulrich Drepper, How To Write Shared Libraries

      So someone would have to manage a registry of library load addresses, and ensure they didn’t try to load into the same bit of address space.

      In this example, it largely comes down to whether you prefer your complications automated or artisanally hand managed.

      1. 2

        Relevant if you want standardized shared libraries.

      2. 16

        I am not knowledgeable enough to contribute anything meaningful on the technical side of this discussion, but I think we should really avoid this type of discussion style. I’ve seen it over and over that people who lament the complexity of new systems and romanticize the simplicity of old ones simply don’t recognize the usecases of others, or are actively dismissive towards them. Accusing them of introducing complexity for its own sake is even worse than that.

        1. 8

          Isn’t it simpler to support the one binary format than two? Even if that one format is more complex.

          1. 1

            Sure, you can argue that supporting more than one format (ELF) is itself unnecessary complexity, but that’s reduction to absurd, once we’re considering the complexity forced by ELF itself.

          2. 3

            What’s a good reason to use a.out these days? I mean, we need the elf for things that use metadata, extra sections, fat binaries, etc. so if it’s already available, why would I want to use a.out?

            1. 3

              Simplicity. You can do everything else manually. Same sort of appeal as COM executables on Freedos.

              1. 7

                Sure, we can do a lot of things in a different way, but why simplicity? There are existing interfaces to the format so you mostly don’t need to think about that and when you do, it’s not that hard (I’ve parsed ELF for specific info and it was ok). I’m trying to understand what would we gain from simplicity apart from an esthetically purer result that’s functionally weaker.

                1. 2

                  Simplicity is valuable itself.

                  1. 12

                    No it isn’t. Simplicity is a means to an end. That end is software that can be understood, used and modified easily. Don’t lose sight of the forest for the trees.

                    1. 2

                      No, it really is valuable itself.

                      Complexity is inherently bad. Justification is needed for any complexity added: Any complexity needs to be weighted against the value of simplicity (which is even higher than what you already seem to be aware of).

                      It is indeed a high burden, but that’s reality.

                      1. 3

                        it’s simpler having 1 binary format instead of 5 though?

                        1. 1

                          Addressed here.

                    2. 9

                      Simplicity at one layer in the stack often drives complexity at another level. ELF was driven by the ‘simple formats, complex tools’ model and the complexity of an ELF run-time linker is huge but at least it’s standardised compared to a.out, where any kind of dynamic code loading or shared library usage had to be bespoke for the application(s) concerned.

                      1. 1

                        had to be bespoke for the application(s) concerned.

                        I prefer to think about it as “able to do bespoke”. I am not arguing for the removal of ELF.

                        1. 3

                          You can also do bespoke things with ELF (I have) and the container format makes it a lot easier by providing a richer mechanism for expressing metadata to drive your own mechanism.

                2. 3

                  I mean, we need the elf for things that use metadata, extra sections, fat binaries, etc.

                  Why do we need these?

                  1. 2

                    Fat binaries are convenient for distribution. Metadata or extra sections for things like binary signing, embedding icons, annotating source package, tagging versions/name understood by cve scanners, lots of other uses…

                    1. 4

                      Elf doesn’t support fat binaries. The rest can be done with symbols instead of sections.

                      It seems that in practice, few enough people are aware of the intricacies of elf, and do it with symbols regardless of what elf permits.

                      The big advantage of elf is dynamic linking, and the consensus around that is changing.

                      1. 1

                        That part is aspirational, correct. Unfortunately FatELF was not merged, but if we ever want to revisit it, we need something with dynamic sections, like ELF.

                        1. 1

                          I like AmigaOS’s HUNK executables, as they do not force complexity upon implementations.

                          HUNK is structured as a linked list. It is possible to just ignore these that aren’t essential for what you’re trying to do (e.g. running the program), thus as long as essentials happen early in the list, implementations aren’t burdened no matter how many non-essential sections are added.

                          With ELF, there’s a lot of structs to navigate through, no matter what you’re trying to do. The barrier of entry is much higher, and I argue it isn’t justified, particularly when HUNK does not need anywhere near as much work.

                          1. 2

                            Why do you think it’s easier to iterate hunks than iterate over the section list? It feels almost the same to me (you can skip unknown entries in either case)

                            1. 1

                              Because there’s only one structure to care about, a quite simple one which forms the linked list.

                              Then you can care/not care about a node. If you care, you implement that.

                              1. 3

                                Ok, but that’s basically hunk iteration:

                                for(cur=...;cur<...;cur+=cur->len)
                                  if (cur->id is interesting)
                                    do_something(cur->data, cur->len);
                                

                                Elf exec iteration (either sections or exec table)

                                for(table=elf->table,i=0;i<elf->table_size;i++)
                                  if (table[i].id is interesting)
                                    do_something(table[i].offset, table[i].len);
                                

                                Ignoring the necessary casting and precise offset calculation… The difference seems trivial.

                                1. 1

                                  Except you need a bunch of structs for even minimal access, rather than just one.

                                  I admit bias; I got traumatized a couple decades ago, when I wrote a Linux virus in x86 asm which had to touch ELFs back in high school.

                  2. 2

                    Surely you don’t need ELF for any of these?

                    1. 1

                      In theory, no, you could create a different one. In practice ELF and PE are the only supported options. Or did you mean something else?

                      1. 3

                        In practice ELF and PE are the only supported options.

                        And Mach-O.

                        1. 2

                          Yes, that’s what I meant. If you’re doing an OS, you are free to use and design what you want. That may also entail maintaining your own Binutils, but it’s still for you to decide.