1. 86
  1.  

  2. 27

    This is such a good post. By far the clearest and most compelling statement I’ve read on the topic.

    I’ve spent the past ten years working on cross platform C/C++ codebases, and I keep learning about new and weird ways that code that’s fine for one compiler/OS/architecture can break on another.

    1. 12

      There’s lots of stuff out there that just isn’t supported.

      Try running something written in “100% portable ANSI C”:

      • on a system whose encoding isn’t based on ASCII.
      • on a system that uses floats based on something other than IEEE-794.
      • on a system where pointers don’t fit in any integer type.
      • on a Harvard architecture (C99 defined some stuff to make this more doable, but…).
      • on a system where CHAR_BITS isn’t 8.
      1. 6

        Yeah, people think “oh, just target it to C” - as if C can’t be quite platform dependent already. Sometimes intentionally, sometimes assumptions. Sure, you might never need to port to i8052, PDP-10, VAX, or OS/400, but you never know…

        1. 6

          To the Committee’s credit, the Standard does generally address these situations…there’s just a disconnect between what people think the standard is and what it actually guarantees.

      2. 8

        I think there are valid arguments on both sides here, but this post doesn’t seem to be grounded in experience.

        Practically speaking, users of weird architectures do contribute patches back. Those people eventually become the maintainers. When those people go away, the project drops support for certain architectures. That happened with CPython, e.g. it doesn’t support Mac OS 9 anymore as far as I remember.

        It’s sort of a self-fulfilling prophesy – if the code is in C, you will get people who try to compile it for unusual platforms. If it’s in Rust, they won’t be able to try.

        I’m not saying which one is better, just that this post misses the point. If you want to use Rust and close off certain options, that’s fine. Those options might not be important to the project. Someone else can start a different project with the goal of portability to more architectures.

        Changing languages in the middle of the project is a slightly different case. But that’s why the right to fork exists.

        1. 25

          Author here: this post is grounded in a couple of years of experience as a packager, and a couple more years doing compiler engineering (mostly C and C++).

          Practically speaking, users of weird architectures do contribute patches back. Those people eventually become the maintainers. When those people go away, the project drops support for certain architectures. That happened with CPython, e.g. it doesn’t support Mac OS 9 anymore as far as I remember.

          This is the “hobbyist” group mentioned in the post. They do a fantastic job getting complex projects working for their purposes, and their work is critically undervalued. But the assumptions that stem from that work are also dangerous and unfounded: that C has any sort of “compiled is correct” contract, and that you can move larger, critical work to novel architectures just by patching bugs as they pop up.

          1. 6

            OK I think I see your point now. TBH the post was a little hard to read.

            Yes, the people contributing back patches often have a “it works on my machine” attitude. And if it starts “working for others”, the expectation of support can arise.

            And those low quality patches could have security problems and tarnish the reputation of the project.

            So I would say that there are some projects where having the “weird architectures” off to the side is a good thing, and some where it could be a bad thing. That is valid but I didn’t really get it from the post.


            I also take issue with the “no such thing as cross platform C”. I would say it’s very hard to write cross platform C, but it definitely exists. sqlite and Lua are pretty good examples from what I can see.

            After hacking on CPython, I was surprised at how much it diverged from that. There are a lot of #ifdefs in CPython making it largely unportable C.

            In the ideal world you would have portable C in most files and unportable C in other files. Patches for random architectures should be limited to the latter.

            In other words, separate computation from I/O. The computation is very portable; I/O tends to be very unportable. Again, sqlite and Lua are good examples – they are parameterized by I/O (and even memory allocators). They don’t hard-code dependencies, so they’re more portable. They use dependency inversion.

            1. 10

              TBH the post was a little hard to read.

              That’s very fair; I’m not particularly happy with how it came out :-)

              I also take issue with the “no such thing as cross platform C”. I would say it’s very hard to write cross platform C, but it definitely exists. sqlite and Lua are pretty good examples from what I can see.

              I’ve heard this argument before, and I think it’s true in one important sense: C has a whole bunch of mechanisms for making it easy to get your code compiling on different platforms. OTOH, to your observation about computation being generally portable: I think this is less true than C programmers generally take for granted. A lot of C is implicitly dependent on memory models that happen to be shared by the overwhelming majority of today’s commercial CPUs; a lot of primitive operations in C are under-specified in the interest of embedded domains.

              Maybe it’s possible to truly cross-platform C, but it’s my current suspicion that there’s no to verify that for any given program (even shining examples of portability like sqlite). But I admit that that’s moving the goalposts a bit :-)

              1. 11

                Maybe it’s possible to truly cross-platform C, but it’s my current suspicion that there’s no to verify that for any given program (even shining examples of portability like sqlite).

                I think the argument holds up just fine despite the existence of counterexamples like Sqlite and Lua; basically it means that every attempt to write portable and safe code in C can be interpreted as an assertion that the author (and every future contributor) is as capable and experienced as Dr. Hipp!

                1. 6

                  A lot of C is implicitly dependent on memory models that happen to be shared by the overwhelming majority of today’s commercial CPUs

                  That’s largely a result of the CPU vendors optimising for C, due to its popularity. Which leads to its popularity. Which…

                  1. 2

                    A lot of C is implicitly dependent on memory models that happen to be shared by the overwhelming majority of today’s commercial CPUs; a lot of primitive operations in C are under-specified in the interest of embedded domains.

                    As the author of a C library, I can confirm that fully portable C is possible (I target the intersection of C99 and C++). It wasn’t always easy, but I managed to root out all undefined and unspecified behaviour. All that is left is one instance of implementation defined behaviour: right shift of negative integers. Which I have decided is not a problem, because I don’t know a single platform in current use that doesn’t propagate the sign bit in this case.

                    The flip side is that I don’t do any I/O, which prevents me from directly accessing the system’s RNG.

                    Incidentally, I’m a firm believer in the separation of computation and I/O. In practice, I/O makes a relatively small portion of programs. Clearly separating it from the rest turns the majority of the program into “pure computation”, which (i) can be portable, and (ii) is much easier to test than I/O.

                  2. 5

                    I also take issue with the “no such thing as cross platform C”. I would say it’s very hard to write cross platform C, but it definitely exists. sqlite and Lua are pretty good examples from what I can see.

                    I see this as parallel to “no such thing as memory-safe C”. Sure, cross-platform C exists in theory, but it’s vanishingly rare in practice, and I’d wager even the examples you cite are likely to have niche platform incompatibilities that haven’t been discovered yet.

                    1. 1

                      I’d wager even the examples you cite are likely to have niche platform incompatibilities that haven’t been discovered yet.

                      Portability in C is hard, but it is simple: no undefined behaviour, no unspecified behaviour, no implementation defined behaviour. If you do that, and there are still are platform incompatibilities, then the platform’s compiler is at fault: it has a bug, fails to implement part of the standard, or simply conforms to the wrong standard (say, C89 where the code was C99).

                      If we’re confident a given project is free of undefined, unspecified, and implementation defined behaviour, then we can be confident we’ll never discover further niche platform incompatibilities. (Still, achieving such confidence is much harder than it has any right to be.)

                      1. 3

                        Portability in C is hard, but it is simple: no undefined behaviour, no unspecified behaviour, no implementation defined behaviour.

                        That is a very tall order, though. Probably impossibly tall for many (most?) people. I asked how to do this and the answers I would say were mixed at best. Simple isn’t good enough if it’s so hard nobody can actually do it.

                2. 3

                  If it’s in Rust, they won’t be able to try.

                  I think this is the most trenchant point here. If someone wants to maintain a project for their own “weird” architecture, then they need to maintain the toolchain and the project. I’ve been in that position and it sucks. In fact, it’s worse, because they need to maintain the toolchain before they even get to the project.

                  I’m particularly sensitive to this because I’m typing this on ppc64le. We’re lucky that IBM did a lot of the work for us, but corporate interests shift. There’s no Rust compiler for half the systems in this room.

                  1. 2

                    I’m not familiar with these systems. What are they used for? What kind of devices use them? What industries/sectors/etc.?

                    1. 3

                      Ppc is very common in aerospace and automotive industries. Of course there are also power servers running Linux and Aix, but those are comparatively a niche compared to the embedded market.

                      1. 6

                        Got it. Sounds like definitely something that would not be hobbyists working on side projects using mass-market hardware. I think the article was referring to this–these corporate users should be willing to pay up to get their platforms supported.

                        1. 3

                          So does that mean we should only care about architectures that have corporate backing? Like I say, this isn’t a situation where it’s only a project port that needs maintainers. The OP puts it well that without a toolchain, they can’t even start on it. If Rust is going to replace C, then it should fill the same niche, not the same niche for systems “we like.”

                          For the record, my projects are all officially side projects; my day job has nothing officially to do with computing.

                          1. 8

                            So does that mean we should only care about architectures that have corporate backing?

                            Yes, it does. Money talks. Open source is not sustainable without money. I can work on a pet project on the side on evenings and weekends only for a relatively short period of time. After that it’s going to go unmaintained until the next person comes along to pick it up. This is going to happen until someone gets a day job working on the project.

                            If Rust is going to replace C, then it should fill the same niche, not the same niche for systems “we like.”

                            C has a four-decade head start on Rust, if no one is allowed to use Rust until it’s caught up to those four decades of porting and standardization effort–for the sake of people’s side projects–then that argument is a non-starter.

                            1. 3

                              Yes, it does. Money talks.

                              In such a world there would be no room for hobbyists, unless they work with what other people are using. Breakage of their interests would be meaningless and unimportant. That’s a non-starter too.

                              But, as you say, you’re unfamiliar with these systems, so as far as you’re concerned they shouldn’t matter, right?

                              1. 9

                                In that (this) world, there is room for hobbyists only insofar as they support their own hobbies and don’t demand open source maintainers to keep providing free support for them.

                          2. 2

                            OpenWrt runs on TP-Link TL-WDR4900 WiFi Router. This is a PowerPC system. OpenWrt is nearly a definition of hobbyists working on side projects using mass-market hardware.

                            1. 2

                              It says on that page that this device was discontinued in 2015. Incidentally, same year Rust reached 1.0.

                              1. 2

                                I am not sure what you are trying to argue. The same page shows it to be in OpenWrt 19.07, which is the very latest release of OpenWrt.

                  2. 7

                    One thing that goes unmentioned by the people who are complaining is the effect of RMS’s and the FSF’s licensing strategy. They prioritized avoiding proprietary GCC front ends with the effect of LLVM being the nicer option for also Free out-of-tree front ends. Now we’re at the point where it’s no longer enough to have GCC for a platform but in practice you need to have LLVM as well. The people who are upset about this don’t seem to blame the strategy that made GCC not nice for out-of-tree front ends.

                    1. 2

                      Rarely submit reports to the packager (they bug the project directly instead!)

                      From a cynical but frequently correct poinf of view, reporting bugs to packagers is a waste of time. Either the packager is competent and the fault is in upstream code, or the packager is incompetent and wouldn’t be able to help. ;)

                      In practice, I believe the best suggestion for users who encounter what likely is a packager-induced bug is to do the build yourself and make sure to follow upstream’s building guidelines closely, then see if the bug still can be replicated.

                      1. 1

                        This is a misunderstanding.

                        Things can be possible and not supported. When things that are not supported are made impossible, you will raise hackles.

                        1. 1

                          Very good article, but the only point I take issue with is the part of the corporations who manufactured said non-tier-1 hardware/platform. I don’t think think this is primarily about first class support for production systems, but it’s mostly people who for one reason or other WANT to use those platforms. So I think the original manufacturer doesn’t really come into play, they are more likely to have abandoned that platform already, so why would they take money into their hand (or even be pressured to do that) if they don’t even provide support for first-class citizens (OS, etc). So this mostly hurts the hobbyists and not the corporations…