1. 77

  2. 28

    I always say, “Well, like they say, the only silver bullet is for Dracula.”

    Excellent post. “He demands we run blind, and then blames us for tripping” is my new favorite thing.

    1. 15

      I understand the author’s annoyance–and I really like the writeup!–but I disagree on one bit in particular here:

      We have to use everything we have to even hope of writing correct code, because there’s only one way a program is right and infinite ways a program can be wrong, and we can’t assume that any tool we use will prevent more than a narrow slice of all those wrong ways.

      For a typechecker, yeah, there’s only One True Algorithm. For a user, though, there are actually a lot of ways a program can be wrong and still delivering value! More value than the perfectly-tested and proofed thing that is shipping next quarter!

      The biggest Silver Bullet we have is coordinating with business and optimizing for what they actually need. It doesn’t make sense to spend a developer year at 120K solving a bug in a man-year that contracted support can do at $10/hr…and honestly, a lot of the time, customers will oddly be more satisfied by that high-touch treatment than if they’d never had the bug happen in the first place.

      And sure, you’ve got cases like the Therac-25 and pacemakers where we really, really can’t do that–but like 99.9% of software can be steaming garbage and still please users and deliver revenue, especially if it has obvious limitations (say, Craigslist) and workarounds (say, ctrl-alt-delete) when it is collicky.

      It sucks, but it’s been what I’ve observed in my career. A hacky barely-tested solution that solves the exact business problem a customer needs today is worth infinitely more than a perfect solution delivered once the customer has left.

      1. 23

        For a user, though, there are actually a lot of ways a program can be wrong and still delivering value! More value than the perfectly-tested and proofed thing that is shipping next quarter!

        Yeah, I have confess that I dropped a of the nuance in my beliefs in favor of more polemic. Here’s how I generally think about correctness:

        • Every product as a range of tolerances for bugs, performance, time to market, etc. Nothing will be perfect, and not everything can be ideal. Your correctness budget is finite.
        • Different correctness techniques address different classes of bugs and require different amounts of effort to use. The correctness/effort ratio is not fixed, and often has diminishing returns.
        • Different products have different classes of bugs. The correctness/effort ratio per technique is not constant across projects.
        • Therefore, consider all possible techniques in your product and aim to maximize the impact of your correctness budget.

        Uncle Bob gets this seriously wrong in two ways:

        1. All programming errors are attributable to a lack in discipline. This is absolutely insane and not how any successful project works.
        2. All programming errors are preventable with more unit tests. Completely neglects the concept of a correctness budget and diminishing returns.

        As software engineers we have to be pragmatic, not dogmatic. Uncle Bob wants us to be dogmatic, which is why his advice is bad.

        1. 7

          Very, well put. I might quote or paraphrase some of that in the future. I’ll add:

          Completely neglects the concept of a correctness budget and diminishing returns.

          This also neglects the problem of state space of input values and combinatorial explosion of combinations. Tools such as SPARK can prove absence of failures such as integer overflow without runtime penalties. Amazon’s TLA+ work was catching corner cases that took thirty-something steps of protocol execution. I’d imagine we wouldn’t have those in well-tested codebases if it was feasible to find/eliminate them with unit tests with acceptable time and money.

          On high end of it, CompCert was the first to build a C compiler with impressively low amount of bugs. They did it in C language with pervasive testing using both Check and Theft tools. That let them get it done in about three to six months with no code injections or crashes during fuzz testing. And then… just kidding. They did in Coq, used formal proofs, and extracted to Ocaml. They knew they weren’t getting intended results with testing and/or unsafe languages. ;)

          1. 3

            Thank you for the elaboration and nuance!

          2. 10

            So, how do you classify software used in Equifax? If that is classified as “can’t do that”, are you still sure of 99.9% number?

            1. 7

              I classify it as a resounding success if we’re going by all the profits and executive compensation that happened from the start of them using networked technology up to the leak. The people left holding the bag for the damages might find it a failure. From there, it can become anything from the kind of failure they’ll spend big money to avoid or just a cost of doing business that’s acceptable. The latter model is the most common with Yahoo in the lead for web companies.

              1. 7

                Frankly, I view their existence as an economic panopticon to be odious.

                Software in support of goals like that shouldn’t exist, so I don’t really care if it is implemented well–in fact, I’d rather see it fail and cause people to question the desirability of allowing a single company to gather that much information in the first place.

              2. 3

                That argument may always be true for a certain type of application, but software is eating the world—the types of applications, and devices in which software is heavily used ranges from defending democracy, to ensuring privacy and security (with life threatening possibilities—smoke detectors, locks, cars), to monitoring our loved ones with affordable medical devices, to managing our money, and investments, all the way down to our todo list.

                We can’t afford to slow down (or lose customers as you suggest), but we can’t go any faster, either, unless we all practice safe software.

              3. 7

                While this is a good read, I do think that the author is misinterpreting Uncle Bob. The original Tools are not the Answer post specifically says that he supports using these tools:

                I think that good software tools make it easier to write good software. However, tools are not the answer to the “Apocalypse”.

                His point, as I understand it, is that more tools cannot solve a lack of discipline because it takes discipline to actually use those tools. And right now most developers don’t even use the simplest tools available to help ensure correctness. If a developer isn’t disciplined enough to write a unit test, why do we think the same developer will be disciplined enough formally prove the correctness of their program?

                I stood before a sea of programmers a few days ago. I asked them the question I always ask: “How many of you write unit tests on a regular basis?” Not one in twenty raised their hands.

                Uncle Bob blames this problem on a lack of discipline. I think that’s partially correct, but also ignores that developers are embedded into dysfunctional contexts that often punish discipline or reward fast completion more than correctness.

                1. 5

                  I agreed with him on that point. Discipline is the most important factor. It doesn’t matter what it’s motivation (eg money, principles). The best returns come from a mentality that constantly tries to improve the process of writing solid code or the code itself. Such a mentality naturally seeks out methods such as unit testing, code review, or even formal methods. Without that mentality, they’ll half-ass their QA or just leave it off altogether. This is what we see a lot in both proprietary and FOSS software.

                  1. 3

                    I think Uncle Bob also ignores that the vast majority of software is not written using best available practices. Cloudbleed could have been prevented by not using C to write a parser.

                    1. 1

                      I’m not sure how he’s ignoring that? Seems to be the very thing he’s complaining about.

                      1. 10

                        He pretty explicitly says that in one of his recent posts:

                        defects are never the fault of our languages. Defects are the fault of programmers. It is programmers who create defects – not languages.

                        1. 3

                          Shit, there’s a bug in my Brainfuck code! Oh well, time to sit in the corner and think about what I’ve done…

                  2. 7

                    I really appreciate your writing about correctness! As someone who writes stuff that has devastating consequences for having bugs, I am always trying to come up with more tools to help me. Even this little tool has teased out SO MANY bugs in concurrent programs:

                    # usage: ./shufnice.sh name_of_process_to_shuffle
                    while true; do
                      PID=`pgrep $1`
                      TIDS=`ls /proc/$PID/task`
                      TID=`echo $TIDS |  tr " " "\n" | shuf -n1`
                      NICE=$((`shuf -i 0-39 -n 1` - 20))
                      echo "renicing $TID to $NICE"
                      renice -n $NICE -p $TID

                    Keep speaking the good word :)

                    1. 3

                      Before forming an opinion, check out all the code at https://github.com/unclebob?tab=repositories – which is your favorite masterpiece of software there? I think that’s a fair question.

                      1. 3



                        Just because it would be epic if @hwayne used a combo of TLA+ or other assurance activities to find a glaring bug in it that unit testing missed. Then, used it as a counterpoint in an update to this blog post. It wouldn’t mean anything going the other way, though, since it’s such a simple system. Testing and assertions might catch most or all problems in it. Might.

                        1. 3

                          If it’s hot out, the controller will overheat, turn on the cooler, go under the tooHot threshold, turn off the cooled, and then overheat again. But since the cooler turned off, it can’t turn back on for another three minutes, so you can’t guarantee it will stay in an acceptable temperature range.

                      2. 3

                        From one of the linked articles in this post.

                        And what is it that programmers are supposed to do to prevent defects? I’ll give you one guess. Here are some hints. It’s a verb. It starts with a “T”. Yeah. You got it. TEST!

                        No. It begins with “P”.

                        All these constraints, that these languages are imposing, presume that the programmer has perfect knowledge of the system; before the system is written. (…) And because of all this presumption, they punish you when you are wrong.

                        It is in the very nature of computer systems to be unforviging. They do exactly what they are programmed to do, whether it is right, wrong or not even wrong.

                        And how do you avoid being punished? (…) override all the safeties. So don’t depend on safeties to prevent catastrophes. Instead, you’d better get used to writing lots and lots of tests, no matter what language you are using!

                        Tests are one of those very safeties he decries.

                        1. 7

                          No. It begins with “P”.

                          I’m curious what word or phrase you’re thinking of here. I can come up with plausible answers, but none are obviously correct.

                          I do actually know of a silver bullet to avoid writing defects, which is 100% effective and easy to apply. It works like this: don’t write any code. Unfortunately, customers and employers typically find this approach dissatisfying, so we’re pretty much stuck with the largely ineffective and varyingly impractical regular bullets we’ve got.

                          1. 7

                            I’m curious what word or phrase you’re thinking of here. I can come up with plausible answers, but none are obviously correct.

                            Prove it correct.

                            I do actually know of a silver bullet to avoid writing defects, which is 100% effective and easy to apply. It works like this: don’t write any code.

                            It doesn’t work if we define “defect” as “failure to meet the specification”. No matter what your specification is, the non-program doesn’t meet it.

                            1. 5

                              Alas the industry is in this curious state…..

                              Programming is Hard, and requires someone with a very unusual degree of skill and competence.

                              Program proving is much harder than programming and requires someone with much greater skill and competence.

                              You might be right, but you’ll go be outcoded by a less skillful swarm of monkeys testing their stuff into the shape of a product.

                              1. 6

                                This can be solved by making bugs more expensive. For example, software companies could be made liable for damages caused by bugs in their softwares.

                                1. 2

                                  What would that gain us, exactly?

                                  Expensive bugs result in expensive process, which tends to compound on its own until the downstream prices to consumers spiral out of control.

                                  I’m not sure that that’s to the benefit of anyone.

                                  1. 14

                                    Right now we’re in a situation where damages are externalized. Look at the Equifax leak; that already cost me 30 minutes putting locks on all my credit. When I need my credit next, I’ll have to go and un-lock, which will take at least the same amount of time. There were 143 million people affected. Think about all their time lost, just to deal with it–assuming that they don’t get their identity stolen, which will result in far more time and money lost.

                                    Even smaller breaches, like the Target and Home Depot ones, end up costing time and money to the victims. The corporations mostly shrug and go on about their business. Toyota’s software cost people their lives, and yet here they are, doing great. (Note that this news article was posted on a site which also had a huge data breach.)

                                    These bugs should cost the corporations. Will it make programming more expensive? Yes, undoubtedly. Is that desirable over the current state of things? I think so.

                                    1. 9

                                      Expensive bugs result in expensive process which provides market for improvements to make such process less expensive. Alternatively, bugs are negative externality which leads to market failure.

                                      1. 1

                                        Alternatively, bugs are negative externality which leads to market failure.

                                        This is a very bold claim. Do you remember how outright buggy Windows and MacOS releases were are?

                                        Their extensive bug catalogs hardly led to market failure.

                                        1. 4

                                          Their extensive bugs never caused either Apple or Microsoft any economic harm because they were inured from the financial consequences of their failures.

                                          1. 2

                                            Correct, but again, look at the market overall.

                                            How much more software and service was delivered to OEMs and everyday users on top of those bug-riddled operating systems?

                                            The assertion is that “buggy software leads to market failure”, my contention (via historical counterexample) is that cheap software (at a price enabled by tolerance of shipping buggy code) actually causes markets to flourish.

                                            1. 9

                                              “Market failure” here doesn’t mean that “no software was sold”, but rather, that strictly better software was not able to compete, because of negative externalities – the true cost of the badness was hidden and thus pricing information was inefficient.

                                          2. 2

                                            That’s a good example because the vulnerabilities did start adding up at some point. Microsoft had an image, possibly a revenue, problem where they were forced to take action. So, they hit it from two angles: Steve Lipner from high-assurance security was brought in to implement the Secure Development Process (lightweight version of Orange Book activities); Microsoft Research invested lots of brainpower into tools to make software more robust or secure. The SDL drove defects down a lot where we started seeing more 0-days in FOSS apps than Windows kernel. A major result of MSR’s work was their driver verifier that they forced hardware vendors to use. Blue screens pretty much never happen now. They also invested in safer languages (esp C#) to, among other things, reduce security-critical defects in Windows apps.

                                            So, Microsoft immediately invested in highly-effective methods of improving software quality right after they perceived vulnerabilities as a liability for themselves. Same thing happened internally here and there with IBM where the failures led to them inventing things like Fagan Inspections and Cleanroom. It happened in regulated markets where re-certification costs led suppliers to apply as many QA techniques and tools as possible to systems submitted for TCSEC and later DO-178B (now DO-178C). There’s a whole ecosystem for the latter that gives benefits at reduced costs, esp for tooling or pre-evaluated components.

                                            So, you’re totally right that lots of vulnerabilities currently don’t lead to market failure. You also identified an example where they led to market improvement because the liability was hitting the vendor. Microsoft and other examples I identified here are strong evidence we should shift liability toward them and/or regulate commercial software to use proven QA activities. Or at least show theirs work with defect rate and severities during evaluations. Don’t want to be too prescriptive lest innovative assurance techniques don’t get used easily.

                                        2. 4

                                          It creates an incentive to not cause such expensive bugs. It takes what is a negative externality and makes the actual actor responsible for it.

                                          Here in the real world, it would do some of that, and an enormous amount of rent-seeking.

                                          1. 1

                                            Expensive bugs result in expensive process, which tends to compound on its own until the downstream prices to consumers spiral out of control.

                                            I’ve already covered that low-defect methodologies in industrial use usually saved money on less debugging. The extremely-low-defect methods like Praxis Correct by Construction cost 50% extra. Just using safe languages with human-checked configurations knocks out vast majority of attacks with little to no extra effort. The “expensive” part is greatly overstated. Especially given even things like high-assurance guards or security-enhanced CPU’s would come down to the price of regular firewalls and niche CPU’s if they were high volume. The stuff on medium side would be cost of ordinary software with the FOSS side making some of it free.

                                          2. 1

                                            It would also kill Free Software, no?