1. 14

  2. 13

    I agree with the idea here, however usually when I’m changing the code, I am changing its behaviour. If I just refactor of course the tests should not change, but if the behaviour changes the tests must change because they are wrong now.

    1. 2

      Eh I don’t follow. For 90% of the changes I make, I am adding functionality or fixing a bug. I add new tests, and they fail at first.

      After writing the code, they will pass, and thousands of other tests should still pass. If they don’t, then I introduced a regression.

      Occasionally you will need to refactor your tests, but doing that too much is indeed a smell — a sign the tests are coupled to implementation details.

      1. 11

        It is rare that I am writing purely “new functionality” very often I must also change how some existing part of the system works, not at the implementation detail level but at a user-noticible level. In this case any tests covering the changing functionality may be rendered incorrect because they correctly specify the old behaviour which has now been intentionally changed. Yes of course many tests cover unrelated behaviour which should remain unchanged.

        1. 3

          I think it depends a lot on your testing granularity is. A good integration test probably doesn’t change very often because otherwise your users would be hit with POLA violations on a regular basis. Unit tests, on the other hand, may change a lot more often as implementation details change. Unit tests make the best regression tests because they provide easily actionable output. If an integration test fails then you know you’ve got a lot of root-cause analysis ahead of you, if a unit test fails then you already have a mostly reduced test case to work from.

          1. 2

            I would say that if your integration tests don’t change you either have not changed anything meaningful (user won’t notice) or you’ve managed to build something “purely new” which can happen, but is not the bulk of my work. Or you’ve found a place that the tests weren’t covering butbshild have been.

            POLA/POLS to me means make your changer in a way that don’t confuse your users, not to avoid changes altogether. Of course, stable UX is very valuable, but if it’s too stable then you might not have much to do :)