1. 21

  2. 3

    Unit testing and continuous integration are now mainstream. In due time, I think property-based testing and fuzz testing will be mainstream.

    1. 4

      I don’t think that’s likely, because fuzz testing has a much narrower scope of applicability, and widening it takes nontrivial effort. Property and fuzz testing are powerful, but niche, tools.

      1. 3

        I think that technically it’s very doable: for example, GitHub Actions could run your fuzz tests for a certain number of minutes for free (or more if you pay), as in “go test -fuzz -fuzztime 10min ./...”. However, whether that’s actually a good idea is another question. Rob Pike asks some good questions on that issue that get to the heart of this: “the cost/benefit ratio [of fuzz testing] remains unclear” and “I will say that some of the bugs it finds are not worth fixing, and certainly not the cost of finding, although it’s very hard to make that case.”

        As much as I like fuzz testing and think it’s good that Go’s making it easier, all testing does have a cost, and fixing the (sometimes very obscure) bugs it finds takes time. At my previous company I had a hard time convincing folks that a real Bobby Tables SQL injection bug on our production website needed to be prioritized.

        1. 1

          Unit testing also took nontrivial effort and required infrastructure for mocking and fixture. There will be infrastructure for property-based testing and fuzz testing too.

          1. 2

            Yeah, but I think that for most line of business applications, domain-oriented fuzzing represents an ~order of magnitude more work, for an ~order of magnitude less benefit, than unit testing.