1. 14
  1.  

  2. 31

    I dislike TDD. It often seems to me that the correct shape of an API emerges as I write the code; I might make a first pass and then scrap it because the API doesn’t feel right, and reimplement it with a completely different “shape.” In those cases, TDD becomes a total liability: there’s no value in the tests for an API that didn’t end up working, so they simply get replaced.

    In other cases, I start with a problem that cannot really be broken down into “progressive” parts. The classic example of this is the Sudoku solver issue: Ron Jeffries started XP and knows TDD, but he didn’t know the algorithm he needed, and doesn’t seem to have stepped back to do research it–in part, I suspect, because TDD gives you an illusion of progress.

    If TDD is a “software design” practice, it is a poor one.

    1. 7

      Exactly. Additionally, TDD is test driven in the sense that you are forced to adapt your design to allow for the tests (exposing functions and data for testing, breaking things down in a different way), which leads you away from optimal readability and usability.

      If you use a practice that’s forcing emerged design, then sure, something will emerge. But I can’t understand why anyone would expect that something to be optimal - there’s no basis to expect that.

        1. 3

          “I did some up-front thinking in the shower and on the subway, then attacked the problem with TDD.”

          “When you’re using TDD to solve a tricky algorithm, you have to think about both the algorithm and the test approach.”

          “Solving a problem with a known algorithm using TDD gives more readable code than I otherwise would expect.”

          Is that a success of TDD or that of a combo of design/algorithmic focused development combined with a “known algorithm” developed without TDD? It looks much more like the think-and-review style of development I pushed in my comment as counter to TDD. I think that supporting point for TDD actually counters it on Sudoku case in favor of design- and research-driven approach with TDD a QA process on implementation of mostly-done design. The algorithm or design itself wasn’t created with test-first approach.

          That was a hypothesis from reading the intro. I just started reading the article itself to confirm or reject my claim. So, the first thing is author uses a solver with specific approach to search that should work. Writes the test after making that decision. The board interface emerges from that then author simultaneously writes tests & develops board. That’s either TDD or close enough to it. Then the design is done with author claiming a bunch of benefits for TDD. It wasn’t TDD on most important component, though. That was upfront design based on prior work with TDD just trying to catch bugs in it. The “solving Sudoku” part isn’t a success of TDD. That work was done by non-practitioners and possible opponents of TDD in CompSci.

        2. 3

          It often seems to me that the correct shape of an API emerges as I write the code; I might make a first pass and then scrap it because the API doesn’t feel right, and reimplement it with a completely different “shape.” In those cases, TDD becomes a total liability: there’s no value in the tests for an API that didn’t end up working, so they simply get replaced.

          On the contrary, those cases are where TDD adds a lot of value, because it gets you using your API immediately which helps you to get more of a sense of what it feels like, and scrap a bad design much more quickly - or even avoid writing the bad design in the first place.

          1. 3

            I disagree with that. The uses of an API while you’re testing don’t usually reflect the same priorities that you have in real code. You have to set up data, mock external interfaces, manage your assertions… the actual calls are secondary to the boilerplate of the tests, and when you actually make them it’s usually not in the same “flow” as you would in real code. After all, good test suites test very little per test – meaning that your ideal test just calls one method or function. Usually an API in genuine use is a lot less “modular:” you need to do several things connected to a single API.

            Obviously there are exceptions, where you have a few very well-isolated tasks that can be tested beforehand: parsers, serializers, mathematical functions, and the like. I usually find myself preferring to test those afterward, too.

            The one place where I really like writing tests up front is if I am working on a bug in one of those well-isolated tasks. Write a test that demonstrates the bug, then fix it. Ironically, this obvious use of up-front testing is almost entirely separate from dogmatic TDD.

            1. 3

              You have to set up data, mock external interfaces, manage your assertions… the actual calls are secondary to the boilerplate of the tests

              That’s a sign that you’re testing badly (or working with a bad API). Implement lightweight literals for your datatypes (maybe including unfold-like operations). Make your interfaces easy to stub, or use commands to decouple the declaration from the implementation that needs access to an external service and then you can test that you’re getting the correct command values.

              Usually an API in genuine use is a lot less “modular:” you need to do several things connected to a single API.

              Very much disagree; indeed I’d say that’s a sign of a bad API.

        3. 6

          I struggled with TDD for a long time because I learned it while using some nicely typed langs. It made little sense until I realized it was about things in this article and that 75% of the time types were faster and better. Now I use it a lot but typically as a layer which comes after doing significant type-driven design.

          1. 1

            Definitely! I find a lot of the same points in this article go for ‘type-first-development’ or whatever we end up calling it. Would love to see folks elaborate on the process of types->tests->implementation some more. Perhaps it could help us convert some folks over to this way of thinking - to show how types help to make the feedback loops even tighter, especially for critical design questions, that can be a pain to unpick once the tests are in place.

          2. 4

            TDD does not preclude thinking, communicating, experimenting, documenting, stepping back, sleeping on it, etc. That said, I’ve found TDD to be helpful for me in enhancing each of those and in demonstrating even if to myself, I am making incremental progress with executable results.i am not a TDD-is-all type.

            1. 3

              Such a worthwhile article! I hope he’s preaching to the choir, because doing anything without tests up-front has started to feel like Russian roulette, but harder to do.

              1. 14

                Dijkstra, Mills (Cleanroom), and Praxis did it without tests up-front. Their results were better than TDD from what I’ve seen.

                https://en.wikipedia.org/wiki/THE_multiprogramming_system

                http://infohost.nmt.edu/~al/cseet-paper.html

                http://www.anthonyhall.org/c_by_c_secure_system.pdf

                Praxis approaches space shuttle in defect rate with a 50% premium on development. Their data indicates most defects were found before testing. Maybe something to these methods. ;)

                1. 2

                  Caveat: I didn’t go very deep in these, but I have read about these back in the day.

                  To me these are not mutually exclusive. You can do a lot of planning beforehand and still do all the testing you need.

                  @alexkorban mentioned laying out the code to be tested. I believe this is also something that makes sense for testing regressions.

                  Or am I missing something?

                  1. 2

                    “doing anything without tests up-front has started to feel like Russian roulette, but harder to do”

                    That quote inspired my comment. It sounded like drinking the Kool-AID of TDD crowd. I got plenty of software working right first time without testing by using similarly-careful methods to what I described. Testing is definitely useful and caught errors in other stuff I built. It’s an extra method in the verification and validation toolbox. Using things other than TDD isn’t Russian roulette, though, if they got better results. Using no verification methods or only occasional verification would fit your metaphor.

                    1. 2

                      In my experience tests are most useful after a refactor or change in platform. Before that, they are about equal to careful review and testing your software by hand as you write it.

                      1. 2

                        I can understand that, but as it has been said, the tests are there to guard against regressions as well.

                        Someone else can change a little thing in a large code base and there are two ways to find problems: complaints or tests.

                        When you can hold the context in your head or your team is absolutely committed to making no changes without careful study of all the code, sure. But this is cumbersome and more prone to human errors.

                        Not to say testing and TDD are fool-proof silver bullets, either.

                        1. 3

                          But now you are making an argument for having tests, not TDD.

                          I write tests, but stopped practising TDD because it took comparatively forever and for reasons alexkorban stated.

                          1. 3

                            Exactly. Regressions are catching screwups after the design is done or in some usable, intermediate form. They’re not about whether tests drove that design in the first place. Like both ac and mjtorn, I’m all for tests to catch regression errors. They can be written after the fact, though. They can also be generated from specifications with some toolkits.

                    2. 1

                      My reading of the cseet-paper link made me think it was just careful code review.

                      1. 2

                        It uses box structures, a limited amount of formal specification, functional style of decomposition in hierarchical way, a simplified subset of programming constructs, code review to verify their proper use, tracking of data flows as he described it, and usage-centered testing to knock bugs out of intended use-cases. Quite a bit more than code review. Most important is the stuff before code review that makes the review more likely to catch problems.

                  2. 3

                    Good article. Interestingly, these same claims can be made for development processes centered on formal specs and/or human review. I’d love to see more empirical experiments on similar teams and projects doing one of each to see what happens for productivity and defects. Then, mixes of them aiming to find 80/20 rule of what mix gets most bang for buck.

                    Also note that there are tools that take functional-level, formal specs to generate docs, unit tests, and/or input for theorem provers. Anything that can’t be expressed easily as a formal spec becomes handwritten test + English description. This way, you can get more verification over time plus maintain less tests by hand. Now, support for this varies a ton by what language or tools you’re using. TDD is more portable across ecosystems.