1. 11

  2. 5

    Very thoughtful review. The book is 15 years old now and it does show its age a bit, but I am proud of it.

    I don’t know whether responding to the author here works, but the two big issues he mentions: (unit testing > integration testing) and database isolation are two things I still emphasize.

    The thing about integration testing is that you lucky if you have seams that allow it. If you do, then it’s doable. The thing is, though, that writing tests at a high level to get coverage at a low level is like dropping a pebble down a hole in the the earth and trying to get it to land on on a particular ledge. Even with coverage tools it’s tough. Once you’ve done that, you have a test that works, covers a lot, and likely takes longer to run than a focused test. Since it takes longer to run, you bunch up your changes and don’t run it as often. When that test fails, you’re left wondering which particular change caused the failure and you have to spelunk to figure it out. That’s why I lean toward smaller focused (unit) tests.

    Re databases, it’s the slow down and the distance of used values from expected values that bothers me. Rails in particular is rough this way. I once worked with a team that had tests that took three hours to run on 24 cores because of ActiveRecord. If they’d only dealt with computation, I suspect the tests would’ve run in minutes.

    1. 1

      One thing I have encountered from the article:

      One of the purposes of testing is to make refactoring and subsequent behavior changes easier and safer, but if every jot and tittle of the internal code structure is encoded in the test suite via all the mocks and fakes, a simple half hour of work refactoring the code as part of adding new functionality turns into hours of tedious work restructuring the tests to match. The result is to paradoxically discourage refactoring because of the painful changes then required to the tests, defeating one of the purposes of having tests.

      Now, I know testing is something of an art and perhaps this is solved by a more careful balance between integration tests and unit tests, but have you encountered this? And how do you avoid it?

      1. 3

        Often I’ve found that overspecification in tests comes from using the wrong mechanism for faking out parts. http://blog.ploeh.dk/2013/10/23/mocks-for-commands-stubs-for-queries/ is pretty helpful for getting your head straightened out. Read it today. Read it again sometime in the future. Share it with your juniors. Hide it from the seniors that you wish to overtake ;)

    2. 3

      Tangential, but IIRC Michael Feathers is actually on Lobste.rs. I can’t recall his handle, though.

      1. 5

        Yes. I’m here.