1. 15
  1.  

  2. 6

    I don’t like these “you must add comments” policies as more often than not it leads to a lot of noise and duplicate or obsolete information. The test should be well written enough that what it does is obvious and, just like for regular code, comments should be added only when something may be unclear.

    Also I don’t know about Python but, with most JavaScript test unit libraries, the test will be described directly in code:

    it('should find a character in a string', () => {
        expect('abcd'.indexOf('b')).toBe(1);
    }) 
    

    This is useful because that string, unlike comments, would show up when a test doesn’t pass.

    But the intent can’t always be summed up in a few words

    It implies that it can often be summed up in a few words, which in turns implies that most of the time comments would be useless.

    1. 9

      it('should find a character in a string'

      This is pretty much exactly what the article is arguing for. The rest is just a matter of syntax and tooling features, so why the contrarianism? There’s an entire paragraph talking about avoiding the kind of bad comments you mention.

      This is useful because that string, unlike comments, would show up when a test doesn’t pass.

      Ironically, Python’s unittest module (the one in the standard library; nowadays most people use pytest fortunately) does that with doctests which made CPython Core ban the usage of doctests because they found that confusing.

      It implies that it can often be summed up in a few words, which in turns implies that most of the time comments would be useless.

      This wildly depends on the project and type of code. It’s also really difficult to judge right now – when the code is fresh on your mind – what will confuse you in a year. At least that was my experience.

    2. 4

      I’d generally this a bit more and say “you should treat tests like any other code”. This includes making sure it’s sure it’s understandable (using e.g. documentation), but also includes things like running your linters, making sure it can be understood easily, and generally just making sure it doesn’t become this kludge I’ve seen far too often.

      1. 5

        So, I generally agree with this, but there are some important differences.

        For instance: Test code is not usually itself tested. If you fail to hook up a feature correctly, users will notice it isn’t there; but if you fail to hook up the test code correctly, it’ll never fail.

        As a result, it’s valuable to keep test code very simple, even if doing so generates verbosity. Multiple times I’ve discovered tests that are either not running at all or not executing any checks, because some clever abstraction was introduced to make them easy to write.

        1. 2

          I’ve actually joked that some tests need tests themselves.

          Yeah, I agree tests should be very simple; overcomplicated tests and testing frameworks are one of the few things I have very strong negative opinions on. Not just because what you’re saying, but also because when writing tests you need to think of both the testing code and the code being tested. Reducing the cognitive load here really helps in my experience.

          1. 2

            when writing tests you need to think of both the testing code and the code being tested

            Conversely, thinking about both the code itself and how you will test it when you write the code can help you to end up with more testable code which avoids the need for overcomplicated tests.

            1. 2

              A test by itself doesn’t have to be over complicated to need an explanation. The connection between “what am I testing” and “how do I verify I have achieved it” can and often is though.

              1. 1

                A test by itself doesn’t have to be over complicated to need an explanation

                Sorry. I didn’t mean my comment as “If you write testable code, your tests will be simple enough to be self documenting”, although given the context, I now see that it could be read as such. While it might be true that testable code leading to simpler tests might let you get away with poorer documentation, I wouldn’t encourage it. I agree with the idea that you should document tests.

                As I see it, the two issues (how simple your tests are vs. how well documented your tests are) are largely orthogonal.

              2. 1

                Perhaps; but it’s still a lot more to keep in your head.

        2. 2

          After trying to maintain a unit test I myself wrote…. and getting really pissed off with myself, I have reluctantly forced myself to adopt a given/when/then approach.

          What angered me most about me were those checks for stuff I didn’t know why I would expect such a thing anyway.

          So, speaking C just for fun…

          // To ensure requirement blah...
          static void Given_this_context_When_that_happens( void)     // Then this results...
          {
               context_t context;
               setup_this( &context);
               make_that_happen( &context, that);
               assert_binary_op( context.this(), ==, expected_result); // macro that on failure prints lhs and rhs.
               teardown( &context);
          }
          

          Note: This example is just for making the “context” explicit. I mostly write code that avoids side effects and state and hence the tests are even simpler.

          Things to note about this…

          • Each test case stands by itself. You can run only that test case, you can run all test cases in any order.
          • Each test case enforces one and only one required behaviour of the class under test.
          • No test case “rambles on and on”,
          • I know exactly what I’m testing and in what context.
          • The closer I make it to a human readable statement of requirement the better.
          • I aim for defect localization. Tell me which test failed, I’ll tell you on which line the bug is.

          Tests are Executable documentation that never goes stale. Try make them readable documentation.