1.  

    Hmm. His goal and aim seems…. twisted. Short sighted.

    I look at the average industrial code base….. it’s been around a long time, and probably be around in some barely recognizable form even longer.

    CEO’s come and go like may flies compared to code.

    Extensibility is required because the code will be extended. For decades to come.

    Way way beyond the original design.

    Usually by programmers who are cursing short sighted quick fix shippit types with every breath.

    Re-usability is required, because if you copy-pasta and roll your own of every thing in the system…. you massively bloat the maintenance burden, inflate the bug count and decrease the functionality. Yup. Decrease the functionality. If I increase the functionality of a component in a well factored system I increase the functionality everywhere it is used.

    But beyond the obvious stated utility of extensibility and re-usability, they are pointers towards good code quality.

    Code that is easy to re-use is Good Code. (Note, I do not say code that is re-used is Good Code, I say code that is easy to re-use is Good Code.) Admittedly ease of re-use will encourage re-use, and re-use will also wear off the rough edges of bad code.

    Code that is easy to extend, is Good Code.

    Hint: You are always extending, and re-using code. The code base in the repo yesterday is not what it is today. It’s an entire new copy-pasta and tweak of yesterdays code. And yes, your old function has been re-used in todays.

    If you are never re-used and extending, Go Home, your Job is Done. Rest in Peace. Sign up for a Mac Job.

    The rest of us will keep shepherding our code bases, delivering more and new value every cadence.

    I guess I say back to him, “Yup kiddo, I’m in it for the money. Today, tomorrow, and next year. This isn’t a plastic bag to use once and throw away. Tend it well and you will deliver more and more value for decades.”

    1. 4

      In progress….

      • https://personalmba.com/ - If you’re like me and think most trilogies should be singles, and most novels should be short stories….. and just want the meat of an MBA without all the waffle.

      • Skin in the Game, Hidden Asymmetries in Daily Life Nassim Nicholas Taleb - Has some good points, also a bunch of foaming at the mouth. A book to read when one feels patient enough to sort through which is which.

      Given up on… https://www.amazon.com/Patterns-Network-Architecture-Fundamentals-paperback/dp/0137063385 Way way way too much foaming at the mouth about the hideous bureaucratic abortion that was OSI. The poor guy is scarred for life. I think he has some good ideas down there somewhere.

      1. 5

        It’s a long weekend and it’s sunny, so I’m going to try to deploy an OpenBSD router I’ve been building and then I’m going to braai.

        If it’s not too hot, I might have a play with a Bluetooth CTF I saw a while back.

        1. 1

          South African spotted!

          I mean, who else would try deploy an OpenBSD router on the weekend. ;-)

          1. 1

            Sorry, I’m British ;) I Just have a very wonderful South African partner. She’s in charge of OpenBSD routing, I’m just manning the Braai!

            She’s said that she doesn’t understand fair queuing, apparently this is not a concept in South Africa ;)

            1. 1

              Could be worse, she might be braai’ing the man.

              Understand fair queuing, apparently this is not a concept in South Africa

              Like NZ, the available queueing protocols are ruck, maul and scrum.

        1. 1

          Hmm. Not really “EverySpec”, Every MilSpec maybe….

          Ps: What do you call a MilSpec mouse?

          Elephant.

          PPS:

          While we’re on the subject of “Every*”….

          Just stumbled on https://everyrfc.org/ via the How to read an RFC story. It has a very impressively fast front end.

          1. -1

            creator of Tcl

            Not sure that was a good choice of recommendation.

            1. 1

              To each their own. I’m a big fan of (parts of) Tcl, so that detail is what piqued my interest.

            1. 1

              Yes, UB is not completely without any merit. But it is the wrong default.

              bzip2 can compile with -fundefined-overflow or whatever to take advantage of optimization, exactly like -ffast-math which is not enabled by default. Everyone else should benefit from getting rid of overflow UB.

              1. 3

                Ah, but as soon as you say there is a default… you suddenly have defined the behaviour and we having a different conversation.

                As I keep hammering at people, everything we do has undefined behaviour and it’s a good thing.

                If you rip the cover off your PC and yank a random chip off the motherboard, you’re deep in the realm of Undefined Behaviour, and it would be foolish for the designers to sit there saying, “Hmm, what should the behaviour of this system be if the user yanks that chip out?”

                There are vast swathes of life where the answer is and should be “Don’t do that, and if you do, it’s your problem not mine”.

                1. 1

                  Well, the actual default (in the absence of command-line flags explicitly requesting otherwise) is -O0 (for both gcc & clang as far as I know), so…everything is peachy?

                  1. 2

                    Yes, that is a better description…

                    Except for the perception problem.

                    It goes something like this… “I’m an important person, if what you were saying was important, I would understand it, I haven’t clue about the intricate details of this problem you’re describing, so it can’t be important.”

                    ie. Humans discount whatever they don’t understand. Ever heard the phrase, “Don’t overload me with details, just give me the Big Picture”?

                    So called “Technical Debt” is all thousands of fine fine tiny nitty gritty messy dirty concrete details, as soon as you start describing them…. you have lost.

                    Eyes glaze over and refocus on nice Big Juicy Money Making Features.

                  1. 5

                    Larger question: why are we proud of this? Do we want programming to just be wiring up components written by 25 year-olds from Facebook with an excess of free time?

                    1. 4

                      I know for certain I can knock up vast chunks of functionality by gluing together prewritten chunks. I know this, I do this, I know I deliver orders of magnitude more than I could ever write myself.

                      I also know we haven’t learnt how to do it well, how to do it in a rock solid, reliable, testable, repeatable way. That video to hexagonal architecture is an example of a way of doing the “glue” part in a rock solid, testable way.

                      We haven’t really learnt the best ways. Yet.

                      The entire industry is learning on the job. And some of those lessons are going to be really really painful…. Especially for those who don’t recognize that they still need to be learning…

                      1. 2

                        I know that selfishly I don’t want to wire things up (it’s boring, it’s frustrating etc.), but what is the justification for starting every project by hand rolling your own language and compiler? Surely that wouldn’t benefit anyone but the developer. Even though wiring up components produces software that’s suboptimal in many respects, it’s nevertheless efficient on two very important metrics: cost and development time.

                        I’m sure there are exceptions to this (enterprise software comes to mind) but in general, I struggle to make a case against reusing components.

                        Looking at it from another angle, we generally want to solve ever more complex problems with software. Managing complexity requires abstraction. Components seem to be the only widely accepted mechanism for creating large scale abstractions. I know this isn’t necessarily the best way, but what is a practical alternative and how can all the existing components be replaced in a way that doesn’t create astronomical costs?

                        1. 2

                          I’m not arguing for bootstrapping the universe just to make an apple pie. I’m actually a big fan of components. But the view that we’re “just” component wiring plumbers irks me to my core.

                          Somebody has to envision the data flow from the user to the app. Someone has to design the interface. Someone has to empathize with the user to discern an optimal workflow. Someone has to also be able to make the machine perform this task in an acceptable amount of time. And someone has to have the skill to design it in a way that doesn’t prevent future improvements.

                          I’d argue the utopian vision of software components is already here, where you can drop in various modules and glue them together. Add in an appropriately flexible language, such as JS, and there is very little friction involved overall.

                          Also note that the software problem hasn’t been solved, design skills are still needed, and people merely ship faster apps in a more buggy state.

                          So, I speak against “just component wiring” in the de-skilling sense if only to say the actual programming part is only a small part of what a programmer does.

                          1. 2

                            Just playing devils advocate, but how many of us actually have design skills in an engineering sense? To be more specific, how many of us actually design in terms of a definable process? Design is definitely what differentiates a senior and junior but is it something concrete or something more like aesthetics? Another language you learn to talk to other developers.

                            It’s interesting because the whole Requirements-Design-Architecture Formalized Design process is not really used anywhere, and places that it is used, design is done by an architecture team and locked away never to be seen by another human.

                      1. 16

                        I don’t disagree with the heading…. but I would strongly assert we a VERY BAD at doing “Just” plumbing.

                        We have this terrible habit of concreting ourselves into the design suggested by whatever the prebuilt components presented to us.

                        We then pat ourselves on the back saying we don’t need to unit test, the prebuilt component is well tested, and we don’t need to think about design, the prebuilt component has thought about that for us.

                        And when we decide that component doesn’t fit we rip up everything we have done or insert layers upon layers of kludges to shim in something else.

                        And when we want a different UI or backend…. we find we have welded these components into our very architecture.

                        At severe risk of enraging DHH further…. I point at this talk as an example of someone who has thought about the issue and has an excellent proposal for resolving (some of it).

                        We also forget that retrieving, versioning, licencing, patching, bug fixing and configuring and deploying these pre-built components is hard. This is a large part of what bitbake and OpenEmbedded do for us… and partly what docker should be doing for us.

                        Bitbake makes it nice and clear, the “recipe” for grabbing the correct version of the source, verifying it, backing up the original in case the source goes down, verifying the licence, patching, configuring, building, installing, deploying, post installation configuration, ….., for this package AND all it’s dependencies…..

                        …that recipe is our source code.

                        That is the “crown jewels” of the “just plumbing” era. That is what goes into version control.

                        And we do NOT have Good best practices for developing and designing and testing this stuff.

                        I really really don’t think this is a solved problem yet.

                        Yes, “tinker toy” development as I call it, where we plumb together very large chunks, allowing very rapid development of a huge amount of functionality is the future.

                        But we aren’t there yet.

                        Not even close.

                        Stepping stones and signposts on the journey there are….

                        • guix and nixos
                        • The reproducible builds project.
                        • bitbake and openembedded.
                        1. 2

                          I think your perspective is correct in a software for developers kind of way. But in production I would say that plumbing and rapid development through external modules is already happening. Any company right now will use npm and many language specific dependency/package managers. In fact, it’s heavily incentized. Your perspective of best practices would typically apply to companies and organizations where time to market is not a significant factor. Which usually means large companies and OSS, but in those cases they would rather do it in-house anyways for other advantages.

                          The only portion of software I see this being true is perhaps very specific embedded applications. Such as drivers, industrial controls, FPGAs, etc.

                          1. 1

                            Oh, I know we all doing it now. Gluing together hundreds of packages and calling it a product.

                            I’m just saying we’re all doing it very badly.

                            The entire industry is at that painful stage the “handcraft it out of C” part of the industry was at in the 1970’s…

                            ie. Groping for what best practice actually looks like, feeling bits of pain here and there, knowing that ten years down the line the pain is going to become excruciating…. but not having and really, umm, solid, principles and tools and practices.

                            We’re like the C++ community was prior to STL, and way way prior to the isocpp Core Guidelines.

                            We’re at the stage where major players like IBM and Rational defined what they do as best practice and everybody copies that…. only to work out years later that just because you’re big doesn’t mean you’re good at it, just means you can throw hundreds of bodies at it.

                            The “IBM” of today is facebook.

                            Software in the Large was and is really hard to do Right.

                            The era of Software in the Gargantuan that we’re in now will be harder.

                        1. 1

                          At this point, with my researcher hat firmly in place (I really do need to get such a hat)…

                          Can we pause for a moment to research what such a hat may look like?

                          I feel sure it would be related to a Jägermonster hat.

                          1. 1

                            Merlin’s Wizard Hat

                            1. 2

                              Merlin’s Wizard Hat

                              Hmm. No.

                              A Wizard’s Hat suggests a person that makes magic happen.

                              A researcher should be the person that finds out why and how magic happens and whether it does any good.

                              A researcher should be the person (dog?) looking behind curtains and explicitly paying attention to the man behind the curtain.

                              1. 1

                                Damn, you got me there. Good points.

                          1. 1

                            I think the core problem is this assumes that the compiler writers can anticipate and predict and control what the behaviour of the code they generate will be for all “undefined behaviours”.

                            Even worse, the very definitions of …

                            “ignores” the situation completely

                            and

                            “behaves in a manner characteristic of the environment”

                            …are vague beyond usefulness.

                            Those are not definitions that you could implement and test for compliance and rely on in portable programs.

                            Those are a handful of hand waving examples.

                            If you could reframe that into rules that you could test for compliance on…. maybe.

                            I think by the time you have something that a portable program could rely on…. it will no longer be undefined behaviour.

                            Take for example that simplest of functions.

                            Calculate the square root of float x.

                            There are probably thousands of implementations of this around the world.

                            Let’s break it into two parts.

                            An outer function that checks for a negative argument and returns something defined. eg. NaN or an error code or throws an exception or something.

                            And an inner function inner_sqrt( float) that assumes that check has been made and has a non-negative float to work with.

                            Now assuming….

                            • I am a wondrous smart ass,
                            • and assuming I believe my code to be perfect so I don’t need the outer check
                            • since I’m perfect and never create negative parameters
                            • and since I’m invoking this in an inner loop I elect to avoid the unneeded check

                            … I can elect to invoke the inner_sqrt directly.

                            Now assuming I’m less smart than I believe myself, and do indeed invoke “inner_sqrt( -1.0)”

                            So what behaviour can I rely on for my stupidity?

                            Ignore it? Well some implementations will crash, some will go into an infinity loop, some will produce NaN’s, some will produce random garbage.

                            What exactly do you mean by “ignore it”? Do the check again? So you have an inner and an outer function that both do the same check?

                            Document it? So you expect the implementers of inner_sqrt() to have a regression test suite that checks the behaviour in response to all classes of possible stupid, is as documented for that particular class of stupid? (The sqrt example is simple, a more complex example may have hundreds of possible classes of stupid!)

                            Terminate with error message? What!? Do the check again? That’s what the outer function was for! You elected in your infinity wisdom to optimise by invoking the inner function without the check because you’re perfect and don’t write bugs!

                            The core here is “undefined” means just that. It is not defined, nor can it be.

                            If you, in your God Given power as Lord High Absolute Programmer elect to override the protections given you by the cpu or compiler or library implementer and do stupid… there is nothing a compiler or anyone can do to save you from your stupidity.

                            Nor, I would argue, should it.

                            In the case of signed int overflow, I would argue rather if there is a flaw in the standard, it is because the standard elected to give us no protections that we could opt out of.

                            Yes I can see that checking every signed int overflow has a high cost, and I can foresee in certain cases I personally would indeed opt out of such protections.

                            Alas, the state of the C/C++ is that I cannot even opt in to such protections. (Although I can pull in BigNum libraries if I choose…. but if I choose to abuse those, they will also bite me.)

                            1. 1

                              Anyone have an idea what the prices are for the event though?

                              1. 1

                                Ticket prices haven’t been determined yet, but should be in line with previous years.

                                Usually there are varying levels from hobbyist to professional, the details will be announced when registration opens.

                                I look forward to welcoming you to Christchurch!

                              1. 1

                                On the EINTR/retry issue, it’s not clear to me why a simple ‘return to userspace and retry the instruction’ is harder to implement.

                                There is kernel-side state associated with the fd (seek position, network buffers etc) but these need to be maintained anyway for an EINTR return.

                                Put another way - what work can the kernel avoid in the ‘return EINTR and let the userspace application call the system call again’ scenario which is required in the ‘return to userspace at PC-1’?

                                1. 5

                                  There are two big issues: changing entry conditions, and blocking after interrupts.

                                  Blocking after interrupts is fairly obvious when you think about it. If you are blocking on some syscall in your main thread, receive a signal, and your syscall is restarted automatically, you cannot respond to that signal in your main thread at all. You’d just keep blocking. If the signal was SIGINT and you would want to cleanly shut down, you can’t. You’d be stuck until your syscall unblocks, which could never happen.

                                  Changing entry conditions are much more tricky. For example, some syscalls have timeouts—the amount of time they should block before returning regardless of their success or failure. If you start a syscall with a 10 second timeout, what happens if that syscall is cancelled and restarted 5 seconds in? If the same arguments are used, it would be as if the timeout had been 15 seconds. If you receive 1 signal per second indefinitely, the syscall will block forever. Unlikely but possible.

                                  When you call something with a 10 second timeout, you actually mean 10 seconds from now. To restart these syscalls, the kernel would need to preserve that start time entry condition. That’s fairly doable, but there are other entry conditions that aren’t doable at all. If you’re writing size-formatted output to the console, and you receive SIGWINCH, you don’t want to proceed with the write. Instead you need to reformat the output to match the new console dimensions. The kernel certainly can’t do that for you.

                                  There are so many reasons you might want to change syscall parameters after an interrupt, or do something else entirely. The kernel can’t know all of them. And designing an interface to conveniently accommodate all of them is a lot harder. Thus the worse-is-better solution: never assume what a program wants to do after an interrupt.

                                  1. 2

                                    Signals, I think, is now fairly well accepted as one of the “ugly” parts of posix.

                                    signalfd was an attempt at fixing it…. Here is more discussion of that..

                                    https://ldpreload.com/blog/signalfd-is-useless

                                    Having spend the last week battling the fine fine fine corner cases of signal handling….

                                    Sigh.

                                    I wish linux had something better.

                                    1. 1

                                      Thanks for this.

                                      If you are blocking on some syscall in your main thread, receive a signal, and your syscall is restarted automatically, you cannot respond to that signal in your main thread at all. You’d just keep blocking.

                                      I’m still not getting this one. I’d envisage:

                                      • application calls blocking read()
                                      • application receives SIGHUP
                                      • kernel sees incoming signal and stops doing read()
                                      • kernel calls up into user spaces to run signal handler in user context for SIGHUP. As far as application goes, it’s still doing read(), but signals can happen any time anyway, so no problem here?
                                      • kernel restarts read()

                                      If you’re writing size-formatted output to the console, and you receive SIGWINCH, you don’t want to proceed with the write.

                                      I’m not sure I agree. Given that arriving signals are inherently racy, I think it could be considered to also be correct to re-run the system call without the application making a new choice based on the new information. (The system call could easily have completed before the signal arrived - and the application should be prepared for that eventuality).

                                      When you call something with a 10 second timeout, you actually mean 10 seconds from now. To restart these syscalls, the kernel would need to preserve that start time entry condition.

                                      This is a good point. However, the optimist in me would like to think this is always solvable with API design. (In the timeout case, this would involve absolute timeout rather than relative).

                                      1. 2

                                        Your scenario doesn’t work, because signal handlers are extremely restricted. Signal handlers must be reentrant with respect to all other signal handlers, meaning they can’t allocate memory, can’t use most syscalls, can’t use any libc functions that set errno, and can’t non-atomically modify non-local variables.

                                        For this reason, signal handlers usually set a global volatile flag and return immediately, allowing the main thread to catch EINTR, check the flag, and handle the signal without restriction.

                                        In the SIGWINCH example, doing what you suggest causes significant visual tearing, even though it’s technically correct. But that assumption about racy signals only works if all syscalls are guaranteed to complete.

                                        However, the optimist in me would like to think this is always solvable with API design.

                                        Perhaps. Until such an API actually exists, we must write interruptible code somehow.

                                        (In the timeout case, this would involve absolute timeout rather than relative).

                                        What absolute time? The system clock can change. The clock that can’t is the monotonic raw clock, which simply ticks upwards from an unspecified time. I don’t think an API that takes a deadline with respect to a meaningless clock beats an API that requires restarting on interrupt.

                                        1. 1

                                          Your scenario doesn’t work, because signal handlers are extremely restricted.

                                          Yes they are, but I don’t think that stops the kernel from invoking them and then restarting the system call afterwards.

                                          In the SIGWINCH example, doing what you suggest causes significant visual tearing

                                          Which could already occur. We’re just (slightly) increasing the window in which delivery of the signal will cause it.

                                          What absolute time?

                                          The one with the same semantics as a relative timeout - i.e. monotonic. That is the behaviour you would get if you specified “10 seconds from now”.

                                          1. 2

                                            Your scenario doesn’t work, because signal handlers are extremely restricted.

                                            Yes they are, but I don’t think that stops the kernel from invoking them and then restarting the system call afterwards.

                                            What I think you’re missing here is that since you can’t do much in a signal handler besides setting a flag for the main thread to check, you must have a way to interrupt/cancel/unblock whatever system call is blocking the main thread so that the main thread can start doing whatever the signal called for. If the kernel automatically restarts the system call, the main thread obviously can’t go check that flag and react accordingly.

                                            1. 2

                                              Yes, you’re right, thank you. But in that case, the interrupted return is the feature, not the bug?

                                              The whole ‘worse is better’ thing is cast as “return EINTR” being an undesirable property.

                                              1. 2

                                                Yes indeed, it is a feature. But it is a feature that complicates life for every call that doesn’t care and doesn’t want to be interrupted.

                                                1. 2

                                                  OK, but it isn’t anything to do with Worse is Better, right? The article characterises it as a deficiency of implementation.

                                                  In fact, it’s a feature you need if you want to abort a blocking operation.

                                                  afaics, the only problem is the default. You probably want the SA_RESTART behaviour by default (and perhaps that should be per-fd, rather than per-signal).

                                                  But I don’t think the characterisation in the article is fair, unless I’ve missed something.

                                                  1. 1

                                                    Worse is better means choosing a design that’s worse in some cases because at least it works for all cases. EINTR absolutely embodies that description. It’s a dead simple way to make sure you can always handle interrupts, but extremely tedious to work with in the common case.

                                                    1. 2

                                                      That was my understanding of the pt made in the article and the received wisdom regarding this article.

                                                      However, I thought the conclusion of the discussion above was that it wasn’t any easier than not handling the interrupt. In fact, the interrupted behaviour is required (it is a feature) in many cases.

                                                      I still fail to see how it is easier for the kernel to return EINTR rather than restart the syscall. (Apart from the API issue mentioned regarding entry conditions, e.g. relative/absolute times).

                                                      It might help if someone can outline the desired behaviour. It isn’t “restart syscall”, since that has been disposed of as undesirable above.

                                                      I think my point is: “this article says the Unix EINTR approach is a short cut in kernel implementation which has imposed a cost on userspace since then”. However:

                                                      a) I can’t see the shortcut being taken (no one has pointed out how it is easier to restart a read() than return EINTR) b) arguments above are for EINTR being a good thing

                                                      Is there a 3rd way (other than ‘abort read() early and return EINTR’ or ‘restart read()’) which I’m missing here?

                                                      1. 2

                                                        I’m totally out of my element here, but did not EINTR evolve over years? Like, it started really cheap and not useful, but it became better (and more complex) over time? Is this what the author referred to when they said that unix/c improved from 50% to 90%?

                                      2. 0

                                        Thus the worse-is-better solution: never assume what a program wants to do after an interrupt.

                                        One might argue that interrupts are a broken IPC system. Errno definitely is broken.

                                        However syscalls can fail (as any other computation).

                                    1. 2

                                      Yup… And bug trackers the world over are still littered with things like… https://bugs.ruby-lang.org/issues/8770

                                      1. 1

                                        Can you clarify what you mean here? I love this piece and refer to it frequently myself. I think it brilliantly illustrates why many things in tech are the way they are. I feel like you’re suggesting a point I’m missing.

                                        1. 8

                                          Yes, it does illustrate brilliantly why things in tech are the way they are.

                                          In fact, why many things in our economy are the way they are.

                                          Their solution to the PC losering problem is to transfer the complexity from the OS to the user.

                                          ie. Every author of a program, that invokes almost any system call, must on every invocation, remember to correctly handle, the possibility of it returning EINTR.

                                          Conversely, setting up a good test that proves that your code handles EINTR correctly, every time, is hard and you receive no help from the OS in doing so.

                                          Now if the effort that has now been deployed on literally tens of thousands of packages on fixing obscure sporadic EINTR related bugs….. had been expended on solving the PC losering problem correctly…

                                          We would all be much much better off.

                                          ie. The cost of solving the PC losering problem has been externalised to all users of syscalls. This enabled the “Worse is Better” solution to win in the market place by winning the “time to market” race.

                                          Alas, as is the case with very very many parts of our economic system, we reward via perverse incentives the “cheats” that externalize their costs…. but in the long run our entire civilization pays and pays and pays.

                                          1. 2

                                            Alas, as is the case with very very many parts of our economic system, we reward via perverse incentives the “cheats” that externalize their costs…. but in the long run our entire civilization pays and pays and pays.

                                            Is there any known solution to this problem that would not also sacrifice a lot of good things in the process?

                                            1. 2

                                              I think you will find any and every proposed solution will be condemned out of hand, and indeed fought tooth and nail, by those benefiting most from externalizing their costs.

                                              Thus caution is advised when listening to anyone saying “It won’t work”.

                                              The world seems to be (deliberately) stuck in this foolish black xor white thinking about economic systems, instead of more thoughtful and nuance debate.

                                              ie. I think the world needs it’s systems refactored, not rewritten.

                                              ie. We should be focusing on sinks of productivity and value, and tweaking the rules to reduce them.

                                      1. 1

                                        Hmm. Not really happy about this.

                                        Let’s take a step back.

                                        I often liken testing to building a ship in a bottle.

                                        You have to explore and manipulate the entire contents of the bottle, only through the interface, the mouth of the bottle.

                                        The larger and more complex the inside of the bottle, the harder your task is.

                                        The more constrained your interface (narrower the mouth), the harder your task is.

                                        Yet to verify your code will work for all valid uses, you have to explore the entire inside of the bottle.

                                        Now if your function calls another function….. it’s like building a ship in a bottle, that is inside another bottle.

                                        Orders of magnitude harder problem.

                                        If you have a really simple bottle with a wide mouth…. it’s all easy.

                                        But for anything even slightly more complex, the only way to achieve the goal of verifying the correctness of the code, is to go for the smallest possible unit.

                                        What is the smallest possible unit?

                                        The smallest thing you can throw at your toolchain and produce a runnable executable test.

                                        Why are you running a whole web server?”

                                        No. I won’t chide you for…

                                        not doing real unit testing, for failing to use the correct magic formula

                                        I will do worse.

                                        I will merely ask you how did you explore the whole state space of your code? Are you sure you have verified your code will work when it is invoked with any parameters and any internal state that it is contracted to work for?

                                        If you answer Yes, and you can explain how… I’m happy.

                                        If you mumble and look at your feet and mutter about the number of atoms in the known universe, I’m less happy.

                                        If your tests take longer than the average rate of commits to the repo so you cannot cleanly decide whose commit broke the tests….. I’m getting quite disgruntled.

                                        If your tests break sporadically so it seems as if my commit broke the build…. I’m actually getting quite pissed off.

                                        1. 1

                                          Now if your function calls another function….. it’s like building a ship in a bottle, that is inside another bottle.

                                          Orders of magnitude harder problem.

                                          I disagree: calling other functions make our life easier, since we can delegate responsibilities and compartmentalise the problem. The ‘interface’ of our functions (parameters, calling convention, side-effects, error conditions, etc.) should be all we need to use them effectively. Checking that the interface is adhered to isn’t the caller’s responsibility, that’s the responsibility of that function’s own tests (although calling a function in a test does check some of its implementation, but that’s just a bonus).

                                          In other words, don’t build ships-in-bottles-in-bottles: build ships-in-bottles, then push one inside another (and take it back out for changes/fixes).

                                          1. 2

                                            Certainly the path to building defect free systems is to assemble them out of defect free components.

                                            However, to verify a function you need to be able to explore it’s state space. ie. Global, static and instance variables only via the parameters of the methods that manipulate that state space.

                                            This is why functional programming folks hate hidden mutable state.

                                            Your program needs to work for values of internal state the user has access to from the initial state.

                                            But verifying that is hard.

                                            As an extreme example, I have a gadget on my desk with a couple of buttons and an RF interface.

                                            It’s internal state is in the multimegabytes. ie. No matter how much or how long you tested via those couple of buttons… you still have only explored an absolutely negligible fraction of the states that the user may legitimately access.

                                            Or to go back to the bottle analogy….

                                            It all depends on the shape of the bottle.

                                            If you have a wide mouth, shallow bottle (ie. The interface is as broad or broader than the state space inside the function), yes you are right.

                                            If you have a narrow mouthed deep bottle, and your inner bottle is narrow mouthed and deep… a substantial part of the state of your outer bottle resides inside the inner.

                                            Thus to verify your outer bottle, you have to also explore all parts of the inner bottle that may be reached via the mouth of the outer.

                                            And that is very hard.

                                            An alternate approach is use contract tests to verify that the outer and inner functions agree on their interface. I typically use this approach for services like I/O, timers, threading primitives etc.

                                            Of course, the best approach is to minimize the amount of mutable hidden state….

                                        1. 1

                                          I had the pleasure of listening to Samuel Williams present his ideas at a Ruby meetup a few months ago….

                                          …I think he is on to something.

                                          If you have ever tried to weld libraries together, you will know that getting the concurrency behaviours to “play nice” with other libraries is one of the hard parts. ie. His drive to make async behaviour composable is spot on.

                                          We need it.

                                          While that post is certainly interesting, I believe what he is saying here… https://www.codeotaku.com/journal/2018-06/improving-ruby-concurrency/index and doing https://github.com/socketry/async/tree/master/lib/async are more compelling.

                                          I’ve tried threads and promise like systems and reactor like systems and…. I’m not happy.

                                          I need to take a bit more time to play with his Async library. It looks really promising.

                                          1. 1

                                            What I don’t get about UB in these discussions: isn’t it possible to get the tooling to bail on optimizations that cross a UB barrier?

                                            Like OK you might dereference a null pointer here, this is UB, but don’t take that as a “well let’s just infer that anything’s doable here”. Instead take a “the result is unknowable, so don’t bother optimizing”.

                                            Is there some sort of reason UB-ness can’t be taken into account in these optimization paths? Is there some… fundamental thing about C optimization that requires this tradeoff?

                                            1. 2

                                              I think the problem with this idea is that many optimisations rely on something being undefined behaviour.

                                              In any case, the result is not just “unknowable” - there is no result. If I have two signed integer values for which the sum exceeds the largest possible signed integer value, and I add them together, there is not some unknown value that is the result. There is in fact a known value which is the result but which cannot be represented by the type which the result of the operation needs to be expressed in. What does it mean to “not bother optimising” here? Yes, the compiler could decide that the result of such an operation should be an indeterminate value, rather than undefined behaviour, but that will inhibit certain optimisations that could be possible even in cases where the compiler cannot be sure that the indeterminate value will actually ever result.

                                              1. 1

                                                People keep saying this, but I still have not seen a well worked out example of how UB produces real optimizations let alone a respectable study.

                                                But do “optimizations” actually produce speedups for benchmarks? Despite frequent claims by compiler maintainers that they do, they rarely present numbers to support these claims. E.g., Chris Lattner (from Clang) wrote a three-part blog posting8 about undefined behaviour, with most of the first part devoted to “optimizations”, yet does not provide any speedup numbers. On the GCC side, when asked for numbers, one developer presented numbers he had from some unnamed source from IBM’s XLC compiler, not GCC; these numbers show a speedup factor 1.24 for SPECint from assuming that signed overflow does not happen (i.e., corresponding to the difference between -fwrapv and the default on GCC and Clang). Fortunately, Wang et al. [WCC+12] performed their own experiments compiling SPECint 2006 for AMD64 with both gcc-4.7 and clang-3.1 with default “optimizations” and with those “optimizations” disabled that they could identify, and running the results on a on a Core i7-980. They found speed differences on two out of the twelve SPECint benchmarks: 456.hmmer exhibits a speedup by 1.072 (GCC) or 1.09 (Clang) from assuming that signed overflow does not happen. For 462.libquantum there is a speedup by 1.063 (GCC) or 1.118 (Clang) from assuming that pointers to different types don’t alias. If the other benchmarks don’t show a speed difference, this is an overall SPECint improvement by a factor 1.011 (GCC) or 1.017 (Clang) from “optimizations”. http://www.complang.tuwien.ac.at/kps2015/proceedings/KPS_2015_submission_29.pdf

                                                1. 3

                                                  I still have not seen a well worked out example of how UB produces real optimizations

                                                  Some examples here (found by a search just now): https://kristerw.blogspot.com/2016/02/how-undefined-signed-overflow-enables.html

                                                  SPECint isn’t necessarily a good way to assess the affect of these particular optimisations.

                                                  1. -1

                                                    OMG! You find those compelling?

                                                    1. 3

                                                      OMG! You find those compelling?

                                                      Compelling for what? They are examples of how UB produces real optimisations, something that you said you hadn’t seen.

                                                      1. 0

                                                        They don’t show any real optimization at all. They are micro-optimizations that produce incorrectly operating code.
                                                        When the compiler assumes multiplication is commutative and at the same time produces code that has non-commutative multiplication, it’s just terrible engineering. There’s no excuse for that.

                                                        As steveklabnik says above “UB has a cost, in that it’s a footgun. If you don’t get much optimization benefit, then you’re introducing a footgun for no benefit.”

                                                        1. 3

                                                          They don’t show any real optimization at all. They are micro-optimizations

                                                          This is self-contradictory, unless by “real optimization” you mean “not-micro-optimization”, in which case you are just moving the goal posts.

                                                          that produce incorrectly operating code.

                                                          This is plainly false, except if you you mean that the code doesn’t behave as you personally think it should behave in certain cases, even though the language standard clearly says that the behaviour is undefined in those cases. In which case, sure, though I’m not sure why you think your own opinion of what the language semantics should be somehow trumps that of the committee responsible for actually deciding them.

                                                          1. 2

                                                            Real optimization means substantive. Micro-optimizations like “ we take this arithmetic calculation and replace it with a faster one that produces the wrong answer” are ridiculous.

                                                            I’m totally uninterested in this legalist chopping or this absurd deference to a committee which is constantly debating and revisiting its own decisions. It is just absurd for people to argue that the C Standard wording can’t be criticized.

                                                            1. 5

                                                              “ we take this arithmetic calculation and replace it with a faster one that produces the wrong answer”

                                                              And there you have the core of the problem.

                                                              You say “wrong answer” implying there is A Right One, as defined by the current C standard sometimes there are no Right Answers to certain operations, only “undefined”.

                                                              So Define your Right One, proclaim your New Standard, implement a Conforming compiler and everybody is happy.

                                                              I’m entirely on board with you saying the C committee lacks guts to abandon ancient marginal CPU’s and just define things…..

                                                              So I look forward to programming in VyoDaiken C.

                                                              Alas… I warn you though… you will either end up with something that just isn’t C as we know, or you will still have large areas “undefined”.

                                                              1. -1

                                                                There is a right answer. That is the nice thing about arithmetic.

                                                                1. 5

                                                                  Machine integers are not a field. Anything we call “arithmetic” on them is defined arbitrarily (usually to be kind of like arithmetic on the mathematical integers, if you ignore the fact that they’re finite), so in fact there’s not a right answer—rather, there are several reasonable ones.

                                                                  You could define them to behave however the machine behaves; but this is obviously not consistent from machine to machine. You could define them in a particular way (two’s-complement with signaling overflow, perhaps), but if this definition doesn’t match up to what the target machine actually does, you have potentially expensive checks and special-casing to shore up the mismatch, and you picked it arbitrarily anyway. (Case in point: did you agree with my suggestion? Do you think you could convince all of a room full of C programmers to?)

                                                                  There’s a reasonable argument to be made that C should have said integer overflow is implementation-defined rather than undefined, but it’s hard to claim there’s a single obviously correct implementation-independent definition it should have adopted.

                                                                  1. 0

                                                                    My suggestion is that when you feel like you should tell me something I obviously already know, you should think about what point you are trying to make.

                                                                    C has a great deal of room for machine and implementation dependent behavior - necessarily. Implementation defined would have prevented surprises.

                                                                  2. 1

                                                                    arithmetic.

                                                                    Sounds like you’ll love Scheme and Ruby then ;-)

                                                                    They have this Numeric Tower concept that does The Right Thing by arithmetic.

                                                                    ps: Have a look at this, the ultimate Good News, Bad News story and cry…

                                                                    https://lobste.rs/s/azj8ka/proposal_signed_integers_are_two_s#c_xo9ebu

                                                                    …if we ever meet in person, we can drown our sorrows together and weep.

                                                                2. 3

                                                                  It is just absurd for people to argue that the C Standard wording can’t be criticized.

                                                                  Just to be clear, I’m not arguing that (and I don’t think anybody here is doing so). However if you continue to dismiss logical arguments as “legalist chopping”, and suggest that we all defer to you instead of the language committee, I think the discussion will have to end here.

                                                                  1. 1

                                                                    This is plainly false, except if you you mean that the code doesn’t behave as you personally think it should behave in certain cases, even though the language standard clearly says that the behaviour is undefined in those cases

                                                                    I have no idea how to interpret that except as an argument that one is not permitted to question the wisdom of either the standard or the interpretation chosen by compiler writers. As several people on the standards bodies have pointed out, there is certainly no requirement in the standard that compiler developers pick some atrocious meaning. “Optimizing” code that produces a correct result to produce code that does not, produces incorrectly operating code. You can claim that the unoptimized code was in violation of the standard, but it worked.

                                                                    Specifically, the example you pointed to starts off by “optimizing” C code that calculates according the mathematical rules and replaces it with code that computes the wrong answer. 2s complement fixed length multiplication is not commutative. Pretending that it is commutative is wrong.

                                                                    1. 4

                                                                      I have no idea how to interpret that except as an argument that one is not permitted to question the wisdom of either the standard or the interpretation chosen by compiler writers.

                                                                      I do not see how pointing out what the language standard does say is the same as saying that you are not permitted to question the wording of the standard. Good day.

                                                  2. 2

                                                    No. There is nothing about C that requires compilers behave irrationally. The problem is that (1) the standard as written provides a loophole that can be interpreted as permitting irrational compilation and (2) the dominant free software compilers are badly managed ( as someone pointed out, it’s not as if they have paying customers who will select another product). There’s a great example: a GCC UB “optimization” was introduced that broke a lot of code by assuming an out of bound access to an array could not happen. It also broke a benchmark - creating an infinite loop. The GCC developers specifically disabled the optimization for the benchmark but not for other programs. The standard does not require this kind of horrible engineering, but it doesn’t forbid it.

                                                    1. 4

                                                      There’s a great example: a GCC UB “optimization” was introduced that broke a lot of code by assuming an out of bound access to an array could not happen

                                                      You know, I’m quite happy with that.

                                                      Every new gcc release comes out with a lot of new optimizations and a lot of new warnings.

                                                      Every time I go through our code base cleaning up the warnings, often fixing bugs as I go, I’m ok with that.

                                                      Sometimes we only find the borken code on the unit tests or test racks. I’m OK with that. That code was fragile and was going to bite us in the bum sooner or later.

                                                      Old working code maybe working, but as soon as you’re into undefined behaviour it’s fragile, and changes in optimization are only one of many ways which can make it break.

                                                      In my view, the sooner I find it and fix it the better.

                                                      I deliberately wiggle the optimization settings and run tests. If working code breaks… I fix.

                                                      I run stuff on CPU’s with different number of cores… that’s a really good one for knocking loose undefined behaviour bugs.

                                                      Sure my code “Works for Me” on my desk.

                                                      But I don’t control the systems where other people run my code. Thus I want the behaviour of my code to always be defined.

                                                      1. 1

                                                        I run stuff on CPU’s with different number of cores… that’s a really good one for knocking loose undefined behaviour bugs.

                                                        What do you mean by that? Just something like 2 vs 4 or as different as vs a 3 or 6 core that’s not a multiple of 2? Now that you mention it, I wonder if other interaction bugs could show up in SoC’s with mix of high-power and low-energy cores running things interacting. Maybe running on them could expose more bugs, too.

                                                        1. 3

                                                          The C standard as a bunch of fine grained wording around volatile and atomic and sequence points with undefined behaviour if you understand them wrong.

                                                          Threading is dead easy…. any fool can (and many fools do) knock up a multithreaded program in a few minutes.

                                                          Getting threading perfectly right seems to be extra-ordinarily hard and I haven’t met anybody who can, nor seen anybodies code that is perfect.

                                                          So any fools code will “Work For Them” on their desk… now deploy it on a different CPU with a different number of cores, with a different load ….

                                                          ….and the bugs come crawling out in all directions.

                                                        2. 1

                                                          But of course, there is often no warning.

                                                          1. 1

                                                            Actually gcc has been remarkably good about this… The last few releases I have dealt with there has a been a pairing of new warnings with new optimization passes.

                                                            Which makes sense, because a new optimization pass tends to mean the compiler has learn more about the structure of your code.

                                                            Where they have been gravely deficient is with function attribute…. there be dragons and unlikely to get a warning if you screw up.

                                                            Curiously enough they will suggest attributes if you haven’t got one… but won;t warn you if you have the wrong one. :-(

                                                            Bottom line. Beware of gcc function attributes. They are tricksy and easy to screw up.

                                                            1. 0

                                                              Are you reading any of the critiques of UB or even the defenses? The core issue is silent transformations of code - e.g. a silent deletion of a null pointer check because UB “can’t happen”.

                                                      2. 1

                                                        There is no fundamental reason. It’s just a lot of work and no optimizing compiler is implemented that way right now.

                                                        1. 0

                                                          It looks like part of the problem is an engineering flaw in the compilers where there are multiple optimization passes that can produce errors in combination.

                                                      1. 4

                                                        This whole area has been exercising my brain recently.

                                                        As much as I hate the C standard committee’s lack of courage in defining behaviour, as often a simple decision, even if controversial will resolve it.

                                                        However, here is one that is sort of unresolvable.

                                                        What behaviour should a program have that indexes beyond the bounds of an array?

                                                        There is no way a standard can prescribe what the result will be.

                                                        It must be undefined.

                                                        So the obvious thing to do, as Pascal did, is do bounds checking and have a defined behaviour.

                                                        That imposes substantial runtime costs in CPU and memory, so users do switch it off…..

                                                        So what should the behaviour be?

                                                        One reasonable assumption a compiler writer can make is that there is no way the programmer can intend to index out of bounds, so I can assume that the index is less than the bound and generate machine code accordingly.

                                                        You might say, these newfangled optimizations are the problem… no they aren’t.

                                                        Compilers have been relaying out data in memory according what they think best for decades.

                                                        Where this whole thing is driving me nuts is around asserts. See this thread I started here… https://readlist.com/lists/gcc.gnu.org/gcc-help/7/39051.html

                                                        Asserts, if they are compiled in, tell the compiler (without getting involved in UB optimizations) that if expression is false, then everything down stream is not reachable…. so it analyzes under the assumption the expression is true.

                                                        However it doesn’t attempt to warn you if it finds a code path where the expression is false, and completely redoes it’s optimization without that assumption if you compile the assert out.

                                                        1. 4

                                                          What behaviour should a program have that indexes beyond the bounds of an array?

                                                          There is no way a standard can prescribe what the result will be.

                                                          It must be undefined.

                                                          This gets to the heart of the matter, I think. Part of the issue is people confuse “the language standard” with “what compilers do”. The language says it is undefined behaviour for an out-of-bounds array access to occur, or for signed integers to have their value range exceeded, but there’s no reason why compilers can’t generate code which will explicitly detect these situations and throw out an error message (and terminate).

                                                          So why don’t compilers generally do that by default? Because C is used in performance critical code where these checks have a cost which is considered significant. And, despite the claims in this article, there are cases where trivial optimisations such as assuming that signed integer arithmetic operations won’t overflow can lead to significant speedups (it’s just that these cases are not something as trivial as a single isolated loop).

                                                          If you do value deterministic behaviour on program error and are willing to sacrifice some performance to get it, the obvious solution is to use a language which provides that, i.e. don’t use C. But that’s not a justification to criticise the whole concept of undefined behaviour in C.

                                                          1. 4

                                                            There is a false choice between inefficient code with run time bounds checking and compiler “optimizations” that break working code. I love the example in http://www.complang.tuwien.ac.at/kps2015/proceedings/KPS_2015_submission_29.pdf where the GCC developers introduce a really stupid UB based “optimization” that broke working code and then found, to their horror, that it broke a benchmark. So they disabled it for the benchmark.

                                                            And, despite the claims in this article, there are cases where trivial optimisations such as assuming that signed integer arithmetic operations won’t overflow can lead to significant speedups (it’s just that these cases are not something as trivial as a single isolated loop).

                                                            Great. Let’s see an example.

                                                            But that’s not a justification to criticise the whole concept of undefined behaviour in C.

                                                            I think this attitude comes from a fundamental antipathy to the design of C or a basic misunderstanding of how it is used. C is not Java or Swift - and not because its designers were stupid or mired in archaic technology.

                                                            1. 4

                                                              There is a false choice between inefficient code with run time bounds checking and compiler “optimizations” that break working code

                                                              Optimisations don’t break working code. They cause broken code to have different observable behaviour.

                                                              And, despite the claims in this article, there are cases where trivial optimisations such as assuming that signed integer arithmetic operations won’t overflow can lead to significant speedups (it’s just that these cases are not something as trivial as a single isolated loop).

                                                              Great. Let’s see an example.

                                                              I don’t have a code example to hand, and as I said they’re not trivial, but that doesn’t mean it’s not true. Since it can eliminate whole code paths, it can affect the efficacy of for example value range propagation, affect inlining decisions, and have other flow-on effects.

                                                              I think this attitude comes from a fundamental antipathy to the design of C or a basic misunderstanding of how it is used

                                                              I disagree.

                                                              1. -1

                                                                Optimisations don’t break working code. They cause broken code to have different observable behaviour.

                                                                That’s a legalistic answer. The code worked as expected and produced the correct result. The “optimization” caused it to misfunction.

                                                                I don’t have a code example to hand

                                                                Apparently nobody does. So the claimed benefit is just hand waving.

                                                                I disagree.

                                                                The thinking of Lattner is indicative. He agrees that compiler behavior using the UB loophole makes C a minefield. His solution is to advocate Swift. People who are hostile to the use of C should not be making these decisions.

                                                                1. 5

                                                                  That’s a legalistic answer.

                                                                  Alas, in absence of “legalistic answers”, the only definition of C is either…

                                                                  • An implementation of C is a valid implementation iff every program in a Blessed Set of programs compile and runs successfully and outputs exactly the same values.

                                                                    or

                                                                  • An implementation of C is a valid implementation iff, every program that compiles and runs successfully on The One True Blessed C Compiler, compiles and runs and outputs exactly the same values AND every program that fails to compile on The One True Blessed Compiler, fails to compile on the candidate compiler.

                                                                  What sort of C are you envisioning?

                                                                  Those may be appropriate ways to progress, but that is a different language and probably should be called something other than C.

                                                                  1. 3

                                                                    Apparently nobody does. So the claimed benefit is just hand waving

                                                                    Again, I don’t agree.

                                                                    1. 1

                                                                      You can disagree all you want, but you also seem to be unable to produce any evidence.

                                                                      1. 3

                                                                        You can disagree all you want, but you also seem to be unable to produce any evidence.

                                                                        I have high confidence that I could produce, given some time, an example of code which compiled to say 20 instructions if integer overflow were defined and just 1 or 2 otherwise, and probably more by abusing the same technique repeatedly, but you might then claim it wasn’t representative of “real code”. And then if I really wanted to satisfy you I would have to find some way to trawl through repositories to identify some piece of code that exhibited similar properties. It’s more work than I care to undertake to prove my point here, and so I suppose you have a right to remain skeptical.

                                                                        On the other hand, I have at least explained (even if only very briefly) how small optimisations such as assuming that integer arithmetic operations won’t overflow could lead to significant differences in code generation, beyond simple exchanging of instructions. You’ve given no argument as to why this couldn’t be the case. So, I don’t think there’s any clearly stronger argument on either side.

                                                                        1. 0

                                                                          I have high confidence that I could produce, given some time, an example of code which compiled to say 20 instructions if integer overflow were defined and just 1 or 2 otherwise

                                                                          I have no confidence of this, and it would be a completely uninteresting optimization in any case.

                                                                          On the other hand, I have at least explained (even if only very briefly) how small optimisations such as assuming that integer arithmetic operations won’t overflow could lead to significant differences in code generation, beyond simple exchanging of instructions.

                                                                          Not really. You are omitting a single instruction that almost certainly costs no cycles at all in a modern pipelined processor. Balance that against putting minefields into the code - and note there is no way in C to check for this condition. The tradeoff is super unappealing.

                                                                          1. 2

                                                                            Not really. You are omitting a single instruction

                                                                            No, I was not talking about omitting a single instruction.

                                                            2. 2

                                                              With assert(), you are telling the compiler that at this point, this is true. The compiler is trusting your assertion of the truth at that point.

                                                              Also, if you compile with -DNDEBUG -O3 you will get the warning:

                                                              [spc]saltmine:/tmp>gcc -std=c99 -Wall -Wextra -pedantic -DNDEBUG -O3 c.c
                                                              c.c: In function ‘main’:
                                                              c.c:7:20: warning: ‘*((void *)&a+10)’ is used uninitialized in this function [-Wuninitialized]
                                                              c.c:13:8: note: ‘a’ was declared here
                                                              [spc]saltmine:/tmp>gcc -std=c99 -Wall -Wextra -pedantic -O3 c.c
                                                              [spc]saltmine:/tmp>
                                                              
                                                              1. 2

                                                                No, that is a meaningless statement.

                                                                The compiler doesn’t even see an assert statement, let alone “trust it”.

                                                                It is a macro that gets expanded to “plain old code” at preprocessor time, so depending on NDEBUG settings it expands either to something like if(!(exp))abort() or nothing.

                                                                What the compiler does trust is the __attribute_((__noreturn__)) on the abort() function.

                                                                1. 1

                                                                  My file:

                                                                  #include <assert.h>
                                                                  
                                                                  int foo(int x)
                                                                  {
                                                                    assert(x >= 0);
                                                                    return x + 5;
                                                                  }
                                                                  

                                                                  My file after running it through the C preprocessor:

                                                                  # 1 "x.c"
                                                                  # 1 "/usr/include/assert.h"
                                                                   
                                                                   
                                                                  
                                                                   
                                                                   
                                                                   
                                                                  
                                                                   
                                                                  # 12
                                                                  
                                                                  # 15
                                                                  
                                                                  #ident      "@(#)assert.h   1.10    04/05/18 SMI"
                                                                  
                                                                  # 21
                                                                  
                                                                  # 26
                                                                  extern void __assert(const char *, const char *, int);
                                                                  # 31
                                                                  
                                                                  # 35
                                                                  
                                                                  # 37
                                                                  
                                                                   
                                                                  # 44
                                                                  
                                                                  # 46
                                                                  
                                                                  # 52
                                                                  
                                                                  # 63
                                                                  
                                                                  # 2 "x.c"
                                                                  
                                                                  int foo(int x)
                                                                  {
                                                                     ( void ) ( ( x >= 0 ) || ( __assert ( "x >= 0" , "x.c" , 5 ) , 0 ) );
                                                                    return x + 5;
                                                                  }
                                                                  #ident "acomp: Sun C 5.12 SunOS_sparc 2011/11/16"
                                                                  

                                                                  Not an __atttribute__ to be found. This C compiler can now generate code as if x is never a negative value.

                                                                  1. 1

                                                                    #include <assert.h> int foo(int x) { assert(x >= 0); return x + 5; }

                                                                    Can you copy paste the assembly output? (I read sparc asm as part of my day job….)

                                                                    I’d be interested to see is it is treating __assert() as anything other than a common or garden function.

                                                                    1. 1

                                                                      I’m not sure what it’ll prove, but okay:

                                                                          cmp     %i0,0
                                                                          bge     .L18
                                                                          nop
                                                                      
                                                                      .L19:
                                                                          sethi   %hi(.L20),%o0
                                                                          or      %o0,%lo(.L20),%o0
                                                                          add     %o0,8,%o1
                                                                          call    __assert
                                                                          mov     6,%o2
                                                                          ba      .L17
                                                                          nop
                                                                      
                                                                          ! block 3
                                                                      .L18:
                                                                          ba      .L22
                                                                          mov     1,%i5
                                                                      
                                                                          ! block 4
                                                                      .L17:
                                                                          mov     %g0,%i5
                                                                      
                                                                          ! block 5
                                                                      .L22:
                                                                      
                                                                      !    7    return x + 5;
                                                                      
                                                                          add     %i0,5,%l0
                                                                          st      %l0,[%fp-4]
                                                                          mov     %l0,%i0
                                                                          jmp     %i7+8
                                                                          restore
                                                                      
                                                                          ! block 6
                                                                      .L12:
                                                                          mov     %l0,%i0
                                                                          jmp     %i7+8
                                                                          restore
                                                                      

                                                                      I did not specify any optimizations, and from what I can tell, it calls a function called __assert().

                                                                      1. 1

                                                                        TL;DR; The optimiser for this compiler is crap. And it isn’t treating __assert() as special / noreturn.

                                                                        int foo(int x)
                                                                        {
                                                                           ( void ) ( ( x >= 0 ) || ( __assert ( "x >= 0" , "x.c" , 5 ) , 0 ) );
                                                                          return x + 5;
                                                                        }
                                                                        
                                                                            ;; x is register %i0
                                                                            cmp     %i0,0               ; Compare x with 0
                                                                            bge     .L18                ; If it is large branch to .L18
                                                                            nop                         ; Delay slot. Sigh sparc pipelining is makes debugging hard.
                                                                        
                                                                        ;;; This is the "call assert" branch. gcc has function __attribute__((cold)) or
                                                                        ;;; __builtin_expect() to mark this as the unlikely path.
                                                                        .L19:
                                                                            sethi   %hi(.L20),%o0       
                                                                            or      %o0,%lo(.L20),%o0
                                                                            add     %o0,8,%o1
                                                                            call    __assert
                                                                            mov     6,%o2               ;Delay slot again
                                                                            ba      .L17                ; Branch absolute to .L17
                                                                            nop                         ;Delay slot
                                                                        
                                                                            ;; Really? Is this optimized at all?
                                                                            ! block 3
                                                                        .L18:
                                                                            ba      .L22                ; Branch absolute to .L22!
                                                                            mov     1,%i5               ; put 1 in %i5
                                                                        
                                                                        ;;; Seriously? Is this thing trying to do it the hard way?
                                                                        ;;; The assert branch sets %i5 to zero.
                                                                            ! block 4
                                                                            .L17:
                                                                            ;; Fun fact. %g0 is the sparc "bit bucket" reads as zero, ignores anything written to it.
                                                                            mov     %g0,%i5             
                                                                        
                                                                            ! block 5
                                                                            ;; Falls through. ie. Expected to come 
                                                                            ;; out of __assert() *hasn't treated __assert as noreturn!*
                                                                        
                                                                            ;; Joins with the x>=0 branch
                                                                        .L22:
                                                                        
                                                                        !    7    return x + 5;
                                                                            ;; Local register %l0 is x + 5
                                                                            add     %i0,5,%l0
                                                                            st      %l0,[%fp-4]         ;WTF? Has this been inlined into a larger block of code?
                                                                            mov     %l0,%i0             ;WTF? as above?
                                                                            jmp     %i7+8               ;Return to calling addres.
                                                                            restore                     ;Unwind sparc register windowing.
                                                                        
                                                                            ;; WTF? No reference to label .L12
                                                                            ! block 6
                                                                        .L12:
                                                                            mov     %l0,%i0
                                                                            jmp     %i7+8
                                                                            restore
                                                                        
                                                                2. 1

                                                                  Actually, that isn’t quite what happens….

                                                                  Actually, it’s “Just Another Macro” which, very approximately, expands to …

                                                                   if( !(exp)) abort();
                                                                  

                                                                  …where abort() is marked __attribute__((noreturn));

                                                                  Which is almost, but not quite what one would want….

                                                                  As the compiler uses the noreturn attribute to infer that if !exp, then rest of code is unreachable, therefore for rest of code exp is true.

                                                                  Alas, I have found that it doesn’t, if it finds a path for which exp is false, warn you that you will abort!

                                                                  I certainly feel there is room for compiler and optimizer writers work with design by contract style programmers to have a “mutually beneficial” two way conversation with the programmers when they write asserts.

                                                                  1. 0

                                                                    Which is almost, but not quite what one would want….

                                                                    I’m not sure I understand you. assert() will abort if the expression given is false. That’s what it does. It also prints where the expression was (it’s part of the standard). If you don’t want to abort, don’t call assert(). If you expect that assert() is a compile-time check, well, it’s not.

                                                                    I certainly feel there is room for compiler and optimizer writers work with design by contract style programmers to have a “mutually beneficial” two way conversation with the programmers when they write asserts.

                                                                    There’s only so far that can go though. Put your foo() function in another file, and no C compiler can warn you.

                                                                    assert() is also a standard C function, which means the compiler can have built-in knowledge of its semantics (much like a C compiler can replace a call to memmove() with inline assembly). The fact that GCC uses its __attribute__ extension for this doesn’t apply to all other compilers.

                                                                    1. 2

                                                                      That’s the other bit of Joy about C.

                                                                      There are two entirely separate things….

                                                                      The compiler.

                                                                      And the standard library.

                                                                      gcc works quite happily with several entirely different libc’s.

                                                                      assert.h is part of libc, not the compiler.

                                                                      How assert() is implemented is the domain of the libc implementer not the compiler writer.

                                                                      I have poked at quite a few different compilers and even more libc’s…. as I have summarised is how all I have looked at are doing things. (Although some don’t have a concept of “noreturn” so can’t optimize based on that)

                                                                      Which compiler / libc are you looking at?

                                                                      1. 3

                                                                        The compiler that comes with Solaris right now.

                                                                        You can’t have a standard C compiler without the standard C library. I can get a compiler that understands, say, C99 syntax, but unless it comes with the standard C library, it can’t be called a compliant C99 compiler. The standard covers both the language and the library. I’m reading the C99 standard right now, and here’s an interesting bit:

                                                                        Each library function is declared, with a type that includes a prototype, in a header, (182) whose contents are made available by the #include preprocessing directive.

                                                                        And footnote 182 states:

                                                                        (182) A header is not necessarily a source file, nor are the < and > delimited sequences in header names necessarily valid source file names.

                                                                        To me, that says the compiler can have knowledge of the standard functions. Furthermore:

                                                                        Any function declared in a header may be additionally implemented as a function-like macro defined in the header … Likewise, those function-like macros described in the following subclauses may be invoked in an expression anywhere a function with a compatible return type could be called.(187)

                                                                        (187) Because external identifiers and some macro names beginning with an underscore are reserved, implementations may provide special semantics for such names. For example, the identifier _BUILTIN_abs could be used to indicate generation of in-line code for the abs function. Thus, the appropriate header could specify #define abs(x) _BUILTIN_abs(x) for a compiler whose code generator will accept it. In this manner, a user desiring to guarantee that a given library function such as abs will be a genuine function may write #undef abs whether the implementation’s header provides a macro implementation of abs or a built-in implementation. The prototype for the function, which precedes and is hidden by any macro definition, is thereby revealed also.

                                                                        So the compiler can absolutely understand the semantics of standard C calls and treat them specially. Whether a C compiler does so is implementation defined. And good luck writing offsetof() of setjmp()/longjmp() portably (spoiler: you can’t—they’re tied to both the compiler and architecture).

                                                                        So, getting back to assert() and your issues with it. Like I said, the compilers knows (whether it’s via GCC’s __attribute__(__noreturn__) or because the compiler has built-in knowledge of the semantics of assert()) that the expression used must be true and can thus optimize based on that information, much like it can remove the if statement and related code:

                                                                        const int debug = 0;
                                                                        
                                                                        {
                                                                          int x = debug;
                                                                        
                                                                          if (x)
                                                                          {
                                                                            fprintf(stderr,"here we are!\n");
                                                                            exit(33);
                                                                          }
                                                                          // ...
                                                                        }
                                                                        

                                                                        even through x, because debug is constant, x is loaded with a constant, and not modified prior to the if statement. Your wanting a warning about an invalid index to an array whose index is used in assert() is laudable, but to the compiler, you are telling it “yes, this is fine. No complaints please.” Compile the same code with NDEBUG defined, the assert() goes away (from the point of view of the compiler phase) and the diagnostic can be issued.

                                                                        Yes, it sucks. But that’s the rational.

                                                                        The intent is you run the code, you get the assert, you fix the code (otherwise, why use assert() in the first place?) or remove the assert() because the assumption made is no longer valid (this has happened to me but not often and usually after code has changed, which is something you want, no?).

                                                                3. 2

                                                                  You can do #define assert(p) if (!(p)) __builtin_unreachable() to keep the optimisation benefits! And MSVC has __assume which behaves similarly.

                                                                  1. 1

                                                                    Hmm… Interesting….

                                                                    Does it then elide the expression !(p)?

                                                                    Or does it impose the run time cost of evaluating !(p) and not the benefit of invoking abort()?

                                                                    1. 1

                                                                      Since __builtin_unreachable only exists to guide the compiler, p has to be an expression with no side effects, and then the compiler can optimise it out because you don’t use its result.

                                                                  2. 1

                                                                    I think this is an incorrectly framed question. C says that it’s not the compiler’s problem. You index past an array bound, perhaps you know what you are doing or perhaps not. The compiler is just supposed to do what you said. If you have indexed into another data structure by mistake or past the bound of allocated memory - that’s on the programmer ( BTW: I think opt-in bounds checked arrays would be great). It is unreasonable for the compiler to assume things that may be false. For example, if the programmer cautiously adds a check for overflow, I don’t want the compiler to assume that the index must be in bounds so the check can be discarded.

                                                                    1. 6

                                                                      C says that it’s not the compiler’s problem

                                                                      Actually the C standard says it’s not the compilers problem, it’s undefined behaviour and completely your problem.

                                                                      If you want it to have some weird arsed, but well defined behaviour, you need a different language standard.

                                                                      In C standardese, things that are “the compilers problem” are labelled clearly as “implementation defined”, things that are your problem are labelled “undefined behaviour”.

                                                                      perhaps you know what you are doing or perhaps not.

                                                                      Well, actually, you provably don’t know what you’re doing…. as the compiler and linker lays out the data structures in ram pretty much as they damn well feel like.

                                                                      Part of that for, example, like the struct padding and alignment is part of the ABI for that particular system, which is not part of the C standard, and most of that will change as you add or remove other data items and/or change their types. If you need to rely on such things, there are other (some non-standard) mechanisms, eg. unions types and packing pragmas.

                                                                      BTW: I think opt-in bounds checked arrays would be great

                                                                      gcc and clang now does have sanitizers to check that.

                                                                      However the C standard is sufficiently wishy-washy on a number of fronts, there are several corner cases that are uncheckable, and valgrind is then your best hope. Valgrind won’t help you, for example, if you index into another valid memory region or alignment padding.

                                                                      For example, if the programmer cautiously adds a check for overflow, I don’t want the compiler to assume that the index must be in bounds so the check can be discarded.

                                                                      How ever, if the compiler can prove that the check always succeeds, then the check is useless and the programmer has written useless code and the compiler rightly elides it.

                                                                      Modern versions of gcc will (if you have the warnings dialled up high enough, and annotated function attributes correctly) will warn you about tautologies and unreachable code.

                                                                      1. 1

                                                                        The C standard is not the C language. It is a committee report attempting to codify the language. It is not made up of laws of physics - it can be wrong and can change. My argument is that the standard is wrong. Feel free to disagree, but please don’t treat the current instance of the standard as if they were beyond discussion.

                                                                        In fact, I do want a different standard: one that is closer to my idea what the rules of the language should be in order to make the language useful, beautiful, and closer to the spirit of the design.

                                                                        The compiler and linker don’t have total freedom to change layouts even in the current standard - otherwise, for example, memcpy would not work. Note: “Except for bit-fields, objects are composed of contiguous sequences of one or more bytes, the number, order, and encoding of which are either explicitly specified or implementation-defined.”

                                                                        struct a{ int x[100];}; char *b = malloc(sizeof(int)*101; struct a *y = (struct a *)b; if(sizeof(struct a)) != sizeof(int)*100 ) panic(“this implementation of C won’t work for us\n”); …. do stuff … y->x[100] = checksum(y);

                                                                        But worse,

                                                                        message = readsocket(); for(i = 0; i < message->numberbytes; i++) if( i > MAX)use(m->payload[i]))

                                                                        if the compiler can assume the index is never greater than the array size and MAX is greater than array size, according to you it should be able to “optimize” away the check.

                                                                        How ever, if the compiler can prove that the check always succeeds, then the check is useless and the programmer has written useless code and the compiler rightly elides it.

                                                                        This is one of the key problems with UB. The compiler can assume there is no UB. Therefore the check is assumed unnecessary. Compilers don’t do this right now, but that’s the interpretation that is claimed to be correct. In fact, in many cases the compiler assumes that the code will not behave the way that the generated code does behave. This is nutty.

                                                                        1. 8

                                                                          The C standard is not the C language.

                                                                          Hmm. So what is The C Language?

                                                                          In the absence of the standard, there is no “C Language”, merely a collection of competing implementations of different languages, confusingly all named “C”.

                                                                          I don’t think calling a standard “Wrong” isn’t very helpful, as that would imply there exists some definition of Right.

                                                                          I rather call it “differing from all known implementations” or “unimplementable” or “undesirable” or “just plain bloody silly”.

                                                                          There is no One True Pure Abstract Universal C out there like an ancient Greek concept of Numbers.

                                                                          There are only the standard(s) and the implementations.

                                                                          In fact, I do want a different standard: one that is closer to my idea what the rules of the language should be in order to make the language useful, beautiful, and closer to the spirit of the design.

                                                                          Ah, the Joys of Standards! They are so useful, everybody wants their own one! ;-)

                                                                          Except for bit-fields, objects are composed of contiguous sequences of one or more bytes, the number, order, and encoding of which are either explicitly specified or implementation-defined.”

                                                                          Umm. So, yes, an array is a contiguous sequence, but we’re talking about indexing out of bounds of an array. So what is contiguous beyond that array?

                                                                          Answer 1 : Possibly empty padding to align the next object at the appropriate alignment boundary.

                                                                          Answer 2: Which is the next object? That is determined by the field order within a struct…. (with the alignment padding determined by the ABI), but if the array is not in a struct…. it’s all bets off as to which object the compiler linker chooses to place next.

                                                                          Hmm. Your example didn’t format nicely (nor was it valid syntax (missing parenthesis) so let me see if I can unravel that to see what you mean…

                                                                          struct a { 
                                                                             int x[100];
                                                                          }; 
                                                                          char *b = malloc(sizeof(int)*101); 
                                                                          struct a *y = (struct a *)b; 
                                                                          
                                                                          if(sizeof(struct a)) != sizeof(int)*100 ) 
                                                                              panic(“this implementation of C won’t work for us\n”); 
                                                                          
                                                                          …. do stuff … 
                                                                          
                                                                           y->x[100] = checksum(y);
                                                                          

                                                                          Hmm. Not sure what you’re trying to say, but try this one….

                                                                          #include <stdio.h>
                                                                          #include <stdint.h>
                                                                          
                                                                          struct {
                                                                              char c[5];
                                                                              uint32_t i;
                                                                          } s;
                                                                          
                                                                          uint64_t l;
                                                                          
                                                                          int main(void)
                                                                          {
                                                                             printf( "sizeof(s)=%lu\n sizeof(c)=%lu\n sizeof(i)=%lu\n", sizeof(s),sizeof(s.c),sizeof(s.i));
                                                                             printf( "address of s=%08lx\n address of l=%08lx\n diff = %ld\n", (uintptr_t)&s, (uintptr_t)&l, ((intptr_t)&s-(intptr_t)&l));
                                                                             return 0;
                                                                          }
                                                                          

                                                                          Outputs…

                                                                          sizeof(s)=12                                                                                                                                                                                 
                                                                          sizeof(c)=5                                                                                                                                                                                 
                                                                          sizeof(i)=4                                                                                                                                                                                 
                                                                          address of s=00601050                                                                                                                                                                        
                                                                          address of l=00601048                                                                                                                                                                       
                                                                          diff = 8                                                                                                                                                                                    
                                                                          

                                                                          https://onlinegdb.com/B1Ut3031m

                                                                          1. 3

                                                                            In the absence of the standard, there is no “C Language”, merely a collection of competing implementations of different languages, confusingly all named “C”.

                                                                            What a weird idea. So much of the core code of the internet and infrastructure was written in an impossible language prior to the first ANSI standard. And it even worked!

                                                                            Ah, the Joys of Standards! They are so useful, everybody wants their own one! ;-)

                                                                            There is a standards process. It involves people presenting proposals for modifications and discussing their merits. There are changes ! That’s totally normal and expected.

                                                                            The moment C compilers began padding, every compiler added a “packed” attribute. The reason is that many C applications require that capability. Imagine ethernet packets with artisanal compiler innovative ordering. And those attributes are not in the standard - yet they exist all the same.

                                                                            Your example is not my example.

                                                                            1. 2

                                                                              So much of the core code of the internet and infrastructure was written in an impossible language prior to the first ANSI standard. And it even worked!

                                                                              Were actually written in whatever dialect was available on the day and worked for that machine, that compiler on that day.

                                                                              And porting to a different machine, different compiler, different version of the compiler, was a huge pain in the ass.

                                                                              I know.

                                                                              I’ve done a hell of a lot of that over the decades.

                                                                              Role on tighter standards please.

                                                                              Yup, and most of them added a subtly different packed attribute, and since it was not terribly well documented and defined, I’ve had a fair amount of pain from libraries written (LWIP comes to mind), where at various points in their history got the packed attribute wrong, so it wasn’t portable from 32 to 64 bit.

                                                                              Your example is not my example.

                                                                              You example wasn’t formatted, and I didn’t quite understand the point of it. Can you format it properly and expand a bit on what you were sayign with that example?

                                                                  1. 2

                                                                    Others have made far more eloquent comments on this than I would, so I’ll leave most of it to them. With respect to this part though:

                                                                    We have an ethical responsibility to refuse to work on software that will negatively impact the well-being of other people.

                                                                    I spotted that at least one of the signatories is a former Google employee, and many of the people on the list work at large SV companies that hoover up personal data, intruding into peoples’ private lives. When we talk about ethical responsibilities, part of the problem is we all have different definitions of ethics and well-being.

                                                                    1. 2

                                                                      I would settle for…

                                                                      “I was commanded to do it” is never a valid ethical justification for doing something.

                                                                      Not in the military, not in business, not in software.

                                                                    1. 6

                                                                      It might be informative to float a strawman countermanifesto here, just to stoke discussion a bit and provide a polite but opposed viewpoint–so, in the spirit of open source, I’ll put up a fork.

                                                                      The Contra-Post-Meritocracy Manifesto

                                                                      Meritocracy is a founding principle of the open source movement, and the ideal of meritocracy is perpetuated throughout our field in the way people are recruited, hired, retained, promoted, and valued.

                                                                      But, meritocracy has consistently shown itself to mainly benefit those who exploit those with merit, to the exclusion of those who actually create artifacts of value. The idea of merit is based in the popular gestalt as recognition of the creation of value, an acknowledgement that “this person is capable of producing things which benefit me”.

                                                                      (If you are not familiar with criticisms of meritocracy, please refer to the resources on this page.)

                                                                      It is time that we as an industry abandon the notion that merit is something that can be exploited, that can be produced on equal terms by every individual, and that can never be compensated fairly.

                                                                      What does a contra-post-meritocracy world look like? It is founded on a core set of values and principles, an affirmation of progress and renumeration that applies to everyone who engages in the practice of software development.

                                                                      Our Values

                                                                      These core values and principles are:

                                                                      • We believe that our value as human beings is intrinsically tied to what we leave for others to use. While we are all individuals with diverse and special circumstances and histories, our utility to our peers is measurable in what artifacts we produce and how helpful to others those artifacts are.
                                                                      • We believe that interpersonal skills are often less important than technical skills in the pursuit of artifacts. It is a good deal more difficult to boot a machine off of proper pronouns than proper bytecode.
                                                                      • We can add the most value as professionals by delivering robust and accessible artifacts on time and under budget. The utility of our artifacts should not be shackled or limited by accidents of birth of the creator or user.
                                                                      • We understand that the creation of artifacts is a deliberate sacrifice of our time in this world and an acceptance of a certain value system. We do not enter into such sacrifices or compacts lightly, and we do not accept appropriation of those energies nor the dismissal of that value system.
                                                                      • We have the obligation to get the best price for our artifacts, and to use the proceeds to enrich and secure ourselves and whatever groups we identify with. Sacrificing our energies and artifacts for the sake of others is an act of deep charity and should not cheapened by making it an obligation.
                                                                      • We must make every effort to educate people, like us or not like us, in the creation of artifacts and our value system–beyond that, they must fend for themselves. Our value system requires the competition of competency to work properly, and cannot abide by confounding acts of charity.
                                                                      • We have an ethical responsibility to refuse to mislead others about the capabilities and obvious implications of the software we work on. It is not our place to enforce a system of morals by limiting access to artifacts and knowledge unless we are not being compensated properly.
                                                                      • We acknowledge that non-technical contributors create value that often complements and enhances but rarely exceeds the value of technical contributors. Good press for a dud still means a dud.
                                                                      • We understand that succeeding in our field is a calculation, not an assurance. Being toxic in the community or larger community will backfire in proportion to the quality of the artifact–it’s safer and easier to say things politely than to produce them correctly.
                                                                      • We are devoted to practicing correctness and not complacency. We refuse to accept ongoing technical blunders in spite of their proponents, because we know that while emotions are temporary legacy systems are forever.
                                                                      • The field of software development embraces technical change, but is made worse by distractions about social change. It’s hard enough to deal with changes in chips.
                                                                      • We often reflect our values in everything that we do. We recognize that in our value system compromises must often be made to deliver value to those who would use our artifacts.
                                                                      1. 8

                                                                        Meritocracy is a founding principle of the open source movement

                                                                        Umm. Sources for that one?

                                                                        Personally I always thought it was “Freedom to Tinker”.

                                                                        As source material I refer to the Gnu manifesto https://www.gnu.org/gnu/manifesto.html

                                                                        I think the whole Meritocracy meme came in with the Eric Raymond’s Cathedral and Bazaar essay which was a post fact observation of the movement, rather than a founding document.

                                                                        1. 4

                                                                          So, ESR et al. were instrumental in the “open-source” movement, as opposed to the free software movement.

                                                                          The bit about meritocracy is probably most closely traced to the IETF “rough consensus and running code”, parts of “Conscience of a Hacker”, and frankly the general meme predating all of it (one of which would be, say, Seymour Cray’s general reaction to people suggesting architecture ideas) that an idea is only better if it can be implemented and quantified as running more correctly.

                                                                          Blaming the whole thing on ESR misses the fact that it had been culture for quite some time, and also misses that the movement itself (in either open-source or the free-software variants) depends on the ability of people to write software (moreso than any other factor).

                                                                          1. 3

                                                                            As I said, ESR was just documenting what he observed in the free software movement…. and no doubt with a fair bit of slant towards his personal biases.

                                                                            If you read his web site, he is unashamedly a very very strongly opinionated sort. Not dissing him… I’m pretty sure he own up to that personal characteristic quite proudly.

                                                                            Personally the whole meritocracy thing has been a side show for me…

                                                                            Given the choice between freedom to tinker and meritocracy…. I’d be completely on the side of freedom to tinker.

                                                                            Why?

                                                                            Because humans are absolutely atrocious at identifying what truly worthy of merit.

                                                                            We are really still very close to being monkeys on this front.

                                                                            We’re more rationalizing beings than rational beings.

                                                                            Nor do I believe in one absolute measure of merit.

                                                                            And where there is true merit, freedom to tinker and share will uncover it in the long term.

                                                                        2. 4

                                                                          Eh, I think you used the wrong sock to write that one!

                                                                          1. 2

                                                                            The hell of it is, I agree and identify with values from both Coraline’s original document and my strawman here. :)

                                                                            1. 1

                                                                              I like the effort you both put in these texts, but I do not agree with neither of two (except for some details and pretty general concepts)

                                                                              This usually means that both options miss something important.

                                                                              OTOH when people feel they agree with opposite texts, it means that from certain point of views, the texts agree.