1. 7
  1.  

  2. 6

    So should you use quickcheck? Yes you should, but you should also keep an eye on things like code coverage and branch coverage because it is possible that you have that one very strange corner case that will never be found this way.

    Or you should analyse your code for complicated data structures (e.g. I would call “196913” a “complicated” Int) and either:

    1. Write a generator for those complicated things
    2. Abstract complicated data out, so you test your functions independently

    If we see a complicated data structure, we should generate something which looks like the complicated data structure. Not just Int!

    λ let f v = if v > 196913 then True else if v < 196913 then False else error "Whoops"
    
    λ quickCheck (forAll ((+ 196913) <$> arbitrary) (\a -> f a `elem` [True, False]))
    *** Failed! (after 1 test):                             
    Exception:
      Whoops
    196913
    

    Or we could say “no more complicated data structures hardcoded” and we’d get something a lot more simple:

    λ let f n v = if v > n then True else if v < n then False else error "Whoops"
    
    λ quickCheck (\a b -> f a (b :: Int) `elem` [True, False])
    *** Failed! (after 1 test):                             
    Exception:
      Whoops
    0
    0
    
    1. 3

      This seems more like a Limit of Your Language. This function returns int or something else. Maybe TypeScript would help here? But this does show a legitimate issue in any language with exceptions.

      1. 1

        I agree. I would argue this function should have been declared with the type int -> bool, and it’s a bug that it lacks a result for specific integers.

      2. 2

        Has anyone been able to use QuickCheck for larger properties on more complicated scenarios? I see how it can replace unit tests, but I have a hard time imagining the “arbitrary crawling” successfully finding bugs in places like billing code in a larger functional test.

        I suppose the answer is to break things up until they are testable though….

        1. 1

          Depends on the language you’re in. Quviq, which sells their own version of QuickCheck for Erlang, has support for state machines, which many integrations can be boiled down to. The way it works is you define, in your test, a model for what the state machine does and commands to perform state transitions, and then checks invariants after each command. It then does shrinking on the commands to give the minimal number of commands that break the invariant.

        2. 1

          This is not unlike an example Kostis Sagonas gives to justify concolic testing as an alternative to property testing: https://www.youtube.com/watch?v=XVOV0KQAf-8