I agree: the biggest advantage from my experience is that unit testing gives you much more confidence in refactoring the code, that is, changing the underlying implementation while maintaining the relevant characteristics constant.
Furthermore, making code amenable to unit testing often simplifies it and improves its design.
I can also see the appeal of well-crafted tests as documentation on the usage of the module, but my tests haven’t really attained that proud to have my name attached to it status — usually they’re a grab bag of boilerplate and ad-hoc, totally contrived, examples.
What I can’t get behind is test-driven development. I’ll echo John Ousterhout (whose book “A Philosophy of Software Design” OP might find interesting, if they haven’t already read it):
Although I am a strong advocate of unit testing, I am not a fan of test-driven development. The problem with test-driven development is that it focuses attention on getting specific features working, rather than the best design.
Executable documentation on how to use this api. Always up to date with worked examples. Conversely, if you find them hard to read and understand and use as such…. that’s telling you something important about your unit test design.
Compelling cases. They’re a list of use cases / required behaviours that compel the design and implementation. As a trivial example, testing !even(1) and even(2) are compelling. Anymore test cases are superfluous to compel the design.
Compelling API design. If an API is hard to test, it is hard to use and reuse. Fix your API design.
Excluding test cases. eg. There are an infinite supply if integer sequences http://oeis.org/ which do not have 1 and do have 2. Adding test case that prove our implementation excludes large swathes of likely wrong implementations. eg. !even(1999) even(4000)
Corner cases. !even(0), !even( UINT_MAX)
Property tests. for 100 random i {even(i) xor even(i+1)}
Defect localization. With a superbly designed api and unit test suite, tell me which test failed, and I will immediately tell you the file and line number of the bug.
Capturing and locking required behaviour to prevent regressions.
testing exceptional conditions, which is usually very hard to do with integration tests
speed of execution, to speed up the feedback loop. Integration tests may require a server, which needs to start up, they need an environment. Unit tests give you the opportunity to test your code more quickly.
I’d add, perhaps at the top: to force you to make all your dependencies explicit. This is ofc related to the bullet you called “Compelling API Design.”
Another important point not mentioned in the article: To have a fast test harness. As you’re refactoring, or just developing, you can have near instant feedback when you’ve broken something with a good UT suite. This isn’t feasible with integration or e2e tests. So I’d also add “enables developer productivity.”
Unit tests can be used as a sort of scaffolding when building things. You get to climb to places you couldn’t without, and the scaffolding helps you stay within certain limits when exploring unknown spaces. That can be a tremendous help in certain situations, especially if your programming language doesn’t natively support such scaffolding via advanced types, contracts, etc.
In contrast to actual world scaffolding, once the work is done, the scaffolding can be left in place to make it less likely that whatever was built doesn’t collapse in the future.
Seconding. Bob Martin has a fictional example of TDD where he avoids all the traditional OO trappings (objects, classes, interfaces, etc.) by using unit tests to focus on the simplest procedural code necessary to score a bowling game.
I agree: the biggest advantage from my experience is that unit testing gives you much more confidence in refactoring the code, that is, changing the underlying implementation while maintaining the relevant characteristics constant.
Furthermore, making code amenable to unit testing often simplifies it and improves its design.
I can also see the appeal of well-crafted tests as documentation on the usage of the module, but my tests haven’t really attained that proud to have my name attached to it status — usually they’re a grab bag of boilerplate and ad-hoc, totally contrived, examples.
What I can’t get behind is test-driven development. I’ll echo John Ousterhout (whose book “A Philosophy of Software Design” OP might find interesting, if they haven’t already read it):
Thanks for the book recommendation, and I’ve indeed read it. It’s really good, although I forgot that it had that take on test-driven development.
If you haven’t seen https://8thlight.com/blog/thomas-countz/2019/02/19/essential-and-relevant-unit-tests.html, it shaped my thoughts around writing tests that I’m proud of.
Yeah, in the chapter devoted to Software trends he goes underrated/overrated on a bunch of things, which I love. Thank you for the link!
That reminds me of Ron Jefferies’ attempt to solve sudoku:
The punchline? It’s not satire.
In order of importantance…
Executable documentation on how to use this api. Always up to date with worked examples. Conversely, if you find them hard to read and understand and use as such…. that’s telling you something important about your unit test design.
Compelling cases. They’re a list of use cases / required behaviours that compel the design and implementation. As a trivial example, testing !even(1) and even(2) are compelling. Anymore test cases are superfluous to compel the design.
Compelling API design. If an API is hard to test, it is hard to use and reuse. Fix your API design.
Excluding test cases. eg. There are an infinite supply if integer sequences http://oeis.org/ which do not have 1 and do have 2. Adding test case that prove our implementation excludes large swathes of likely wrong implementations. eg. !even(1999) even(4000)
Corner cases. !even(0), !even( UINT_MAX)
Property tests. for 100 random i {even(i) xor even(i+1)}
Defect localization. With a superbly designed api and unit test suite, tell me which test failed, and I will immediately tell you the file and line number of the bug.
Capturing and locking required behaviour to prevent regressions.
This creates some cognitive dissonance with the V-model. Are unit tests part of testing or part of the documentation?
I guess the logicians answer is fitting: Yes.
I would add:
So much this. The human brain can stay so much more engaged and learn so much faster when the feedback loop is tight.
Unfortunately lobste.rs no longer let’s me edit my post, otherwise I would add yours and /u/thirdtruck and /u/jonahx ’s points as well.
Good list.
I’d add, perhaps at the top: to force you to make all your dependencies explicit. This is ofc related to the bullet you called “Compelling API Design.”
Another important point not mentioned in the article: To have a fast test harness. As you’re refactoring, or just developing, you can have near instant feedback when you’ve broken something with a good UT suite. This isn’t feasible with integration or e2e tests. So I’d also add “enables developer productivity.”
Unit tests can be used as a sort of scaffolding when building things. You get to climb to places you couldn’t without, and the scaffolding helps you stay within certain limits when exploring unknown spaces. That can be a tremendous help in certain situations, especially if your programming language doesn’t natively support such scaffolding via advanced types, contracts, etc.
In contrast to actual world scaffolding, once the work is done, the scaffolding can be left in place to make it less likely that whatever was built doesn’t collapse in the future.
Seconding. Bob Martin has a fictional example of TDD where he avoids all the traditional OO trappings (objects, classes, interfaces, etc.) by using unit tests to focus on the simplest procedural code necessary to score a bowling game.
URL for this has changed to https://ahuth.github.io/articles/what-are-unit-tests-for.html
Sorry about that!