1. 29

  2. 16

    I’ve never fully embraced TDD, but I’m also old. Tests are important, and automated tests doubly so. But TDD- writing tests before the code- means that I’m in the ultimate “black box testing” situation- I may know what the code’s supposed to do, but I don’t know its implementation yet.

    When writing a module, I also tend to refactor as I write- as I gain a better understanding of the problem I’m solving, I reorganize the code, even before the specific function is even complete and working. The contract changes as I write (and if I’m designing to a contract, the last step will be adapting to the contract).

    1. 4

      Yeah. That’s what annoys me with the constant commit philosophy. I try my best to design things up front to hit all the big points that are hard to fix but little things change constantly and I hate merging before I’m ready.

      1. 1

        If it is a more complex feature that I’m probably going to get wrong a few times I’ll make a temporary branch, work on that til it is perfect,then create my real feature branch, and either cherry pick the good stuff or literally delete the code in the real branch and copy the final version of the temporary branch over it then create sensible commits from there. By the end the final revisions in each should be identical, but the real branch can do it in 3 or 4 commits instead of 20 with lots of messy debug stuff.

        1. 2

          I used to do this but found it messy. Nowadays I use squash/amend more heavily, running my tests, and do a new branch (or tag) in case I feel I’ll need a backup.

          If time allows, I’ll do a pre-review and see if the commits can be reordered and/or edited further to make life easier for the person who will review it.

          In that case I bear in mind it might be future me.

      2. 1

        I think a huge part of alleviating that pain is to not do TDD with unit tests, but strictly functional tests.

        I think most people try to modularize their code, so there’s a top layer that’s publicly facing that you probably commit to pretty early in development. So you write some functional tests around that defining what’s supposed to happen in pretty broad strokes.

        I end up adding unit tests for tricky bits of internal logic, but the functional tests are almost always more important in confirming the design and well-functioning of the system as a whole.

      3. 12

        I hope with the backlash from TDD, people will stop trying to shoehorn all their testing as unit tests. Most sprawling software projects would, overall, get far more useful test coverage from an integration test suite. Unit tests should be used for actual units: test your parser, test an especially complicated pure function, etc.

        1. 3

          My takeaway from the article (maybe I’m reading too much into it) is that we can do a better job of creating units, and that we should focus on getting better of making independent, loosely coupled components. If that happens, unit tests won’t get in the way. But, in the meantime, unit tests are definitely getting in the way so maybe we should write larger tests (component, integration, etc) and spend the saved time focusing on learning better design. It still seems very theoretical to me, but I’d love for someone to demonstrate what this looks like in practice.

        2. 6

          I used to think that writing tests the TDD way would help me (and others) create better code, but I no longer think that. (Hey, we’re all naive about some things.)

          Frankly, with TDD, I got tired of writing tests that I would just keep getting rid of. Instead, I stopped and thought about how the code will likely grow out of its current state. I’d always think about how to test what I had, but I didn’t write the test until I felt the design was reasonably good and testable. That wasn’t always the case and some rewriting/refactoring was required, but the changes usually weren’t severe.

          When I and others just wrote tests, we wrote a lot crap we didn’t need. TDD didn’t make better designs simply by following the tenet of writing tests first. It tended to create a world where you had tests, which gave misplaced confidence since you could run them. Instead, it turned out to be better to always think about tests in a design and write them a little later.

          1. 6

            Curious: Did you do TDD in the “write tests first” style? Or in the “Red Green Refactor” style like Kent Beck outlines in his book.

            Because I have might a bunch of good programmers in my life who were railing against TDD, but as it turned out, they were more or less just changing the order in which they wrote the tests.

            For me personally, TDD clicked when I followed the stricter practice of writing a failing test first, then implementing just enough to get the test to pass, as soon as two or more failing tests pass, refactor. For my unit tests, this improved two things: My unit tests become more coarse, before, I would test a lot of private functions, assert on private attributes, etc. TDD meant that at the time of writing the test I didn’t know about these internals, so I started writing tests that centered around a “public” (as higher-level than internal) interface. For example, nowadays I test a CRUD REST API usually by writing HTTP requests in the unit test against the API, hardly directly accessing the database during the tests. This brings me (1) into the perspective of the API user (2) it makes me decoupled from the database schema and I have a lot of implementation freedom in the refactoring steps (3) by the “test must fail first” requirement, I don’t write redundant tests.

            Of course, conventional unit testing wisdom says that “if it talks to a a data base it is not a unit test”, but there are other ways one can make sure that the test suite runs reasonably fast (curate the test suite, delete tests occasionally, etc.).

            A few aspects that make me avoid TDD are: Algorithmic complexity (wouldn’t develop a Sudoku solver using TDD, it just doesn’t make sense, I’d start developing that solver by doign some literature research and finding a suitable description for the problem, it isn’t this TDD-suitable problem where you incrementally add functionality).

            1. 2

              Instead, I stopped and thought about how the code will likely grow out of its current state.

              Is there any reason you couldn’t do that as the first step of a TDD process? I’m not usually a TDDer, but I do find it helpful on occasion, and always make sure that I think through the overall design on a piece of paper before writing my first test in a TDD process.

              1. 2

                I find the TDD tests and/or code interfaces are a better place to think about the design than a piece of paper.

                1. 3

                  That’s my experience exactly. I’m primarily API focused, my only real options after thinking about it are either TDD, or writing some code for how the end user would be expecting to use it, these basically amount to the same thing, but in TDD I get some unit tests “for free”.

                2. 2

                  Is there any reason you couldn’t do that as the first step of a TDD process?

                  No. It’s just that it tends not to happen. I think that the desire to write tests wins out over the need to work through the design.

              2. 6

                TDD doesn’t mesh well with the work – I think of writing code as exposing the underlying primitives in a useful way. A useful API emerges from that, and locking in to an API before the shape of the underlying system materializes leads to crummy, overly complex code. TDD encourages this kind of premature crystallization.

                1. 4

                  This is a “wet pavement causes rain” confusion over causality.

                  My hypothesis for this is very simple. When you look at the TDD evangelists, all of them share something: they are all very good – probably even great – at design and refactoring.

                  TDD is how they got those skills. It’s certainly how I got the skills at design and refactoring that I have.

                  1. 3

                    I think you’re extrapolating way too far from your own experience. There may not be a confusion over causality at all - consider that those people may well have started designing software before using TDD.

                    I don’t see how TDD teaches design when it eschews design… well, by design (other than design for testability perhaps). Refactoring can be learned just as successfully without ever touching TDD, so that doesn’t stack up either.

                    1. 3

                      TDD (for me) isn’t about eschewing design; it’s a method for design, which happens to incidentally produce tests.

                      The key change for me is writing the interface to the code first (by using it in the tests). This drives me to think about what constitutes a good interface.

                      If the tests start looking complex it tells me I’ve chosen a poor interface. If I have to write mocks or setup/teardown code, it prompts me to realize I’m creating a hidden coupling.

                      That said, tooling and defaults tend to be an overriding factor; I do TDD 99% of the time in ruby, sometimes in go, but almost never in javascript or scala.

                      For reasons I don’t understand well, some toolchains/languages/whatever support this practice better than others.

                      1. 2

                        But there’s a lot more to design than the external interface of individual classes/objects/modules! What about the algorithms? What about the internal organisation of the code? What about the interaction between components at a higher level that’s not going to show up in unit tests?

                        1. 2

                          What about the algorithms? What about the internal organisation of the code?

                          Choosing external interface isn’t the end of the design work, it’s the beginning.

                          How can you make decisions about ‘how’ without understanding ‘what’?

                          What about the interaction between components at a higher level that’s not going to show up in unit tests?

                          Who said anything about unit tests? If anything, the higher-level interfaces are the most important to get right (you can fix the middle layers later).

                          My “new feature” workflow is something like:

                          • Write an integration test that fails (but defines the feature interface)
                          • Create a scaffold of the implementation in terms of lower-level interfaces (some of which will not yet exist). Choosing which lower-level primitives to use/create is probably the least rigorous stage of the workflow.
                          • Write failing tests for any new primitives (defining their interface).
                          • Sketch a quick & dirty implementation to flush out any ‘reality failures’ (eg oh, I’m gonna need a bunch of mocks because the interface is wrong)
                          1. 2

                            This is my approach as well, what some people call BDD->TDD, or outside-in->inside-out, but whatever it is, the “feature” defines my externally-visible goal, and my small tests (via TDD) ensure my implementation is working. For me, Test Doubles (aka mocks) at the outside are useful, at the TDD (inside) level, they get me in trouble.

                        2. 1

                          If the tests start looking complex it tells me I’ve chosen a poor interface.

                          This presumes that the way you test X is the same as the way you use it for real in your actual application code.

                          If it’s not, then your tests don’t actually reveal a poor interface. It’s the code that actually uses X to accomplish things that matters in that regard.

                          1. 1

                            As pointed out elsewhere in the same comment, I’m starting with the interface needed by the implementation.

                            If exercising that interface from a test is hard, it usually means there is a hidden coupling to external state.

                    2. 3

                      (copying my Hacker News comment here)

                      The tests get in the way. Because my design does not have low coupling, I end up with tests that also do not have low coupling.

                      Not to be smug, but I feel like this is a rookie mistake I learned 10 years ago immediately after starting TDD.

                      The slogan I use in my head is that testing calcifies interfaces. Once you have a test against an interface, it’s hard to change it. If you find yourself changing tests and code AT THE SAME TIME, e.g. while refactoring, then your tests become less useful, and are slowing you down.

                      Instead, you want to test against stable interfaces – ones you did NOT create. That could be HTTP/WSGI/Rack for web services, or stdin/stdout/argv for command line tools.

                      Unit test frameworks and in particular mocking frameworks can lead you into this trap. I’ve never used a mocking library – they are the worst.

                      There are pretty straightforward solutions to this problem. If I want to be fancy then I will say I write “bespoke test frameworks”, but all this means is: write some simple Python or shell scripts to test your code from a coarse-grained level. Your tests can often be in a different language than your code.

                      The last two posts on my blog are about this:

                      “How I Use Tests”: http://www.oilshell.org/blog/2017/06/22.html

                      “How I Plan to Use Tests: Transforming OSH”: http://www.oilshell.org/blog/2017/06/24.html – I want to change the LANGUAGE my code is written in, but preserve the tests, and use them as a guide.

                      And definitely these kinds of tests work better for data manipulation rather than heavily stateful code. But the point is that testing teaches you good design, and good design is to separate your data manipulation from your program state as much as possible. State is hard, and logic is easy (if you have isolated it and tested it.)

                      Summary: I use TDD, it absolutely works. But I use more coarse-grained tests against STABLE INTERFACES I didn’t create.

                      1. 2

                        I really like TDD, but honestly I haven’t found any good evidence on how effective it is, one way or another. Most of the discussion seems to stem from anecdotes or abstract principles, and all of the studies are inconclusive. That said, I think there’s one advantage of TDD that most people don’t talk about: often the alternative to TDD isn’t test-after, it’s no tests at all! With most of the developers I know (including me), if you don’t force yourself to write tests first, it’s way too easy to write the code and go “meh, I don’t feel like writing tests”. Whereas nobody writes the tests and says “meh, I don’t feel like writing code.”

                        Then again, I don’t think this applies to just unit tests. It’s a lot harder to write a TLA+ spec or end-to-end tests or Hypothesis tests if you’ve already written the code. I’d like to see DDD expanded much more into “Design Driven Development”, which unit tests are only a narrow part of.

                        What I’m saying is we should all go back to waterfall :P

                        1. 2

                          I’m also going to quote

                          The tests get in the way. Because my design does not have low coupling, I end up with tests that also do not have low coupling. This means that if I change the behavior of how class works, I often have to fix tests for other classes.

                          When I end up in that situation, it’s a huge warning that I’ve got too much coupling and a refactor is necessary until that goes away.

                          Because I don’t have low coupling, I need to use mocks or other tests doubles often

                          Which is why I see the thought “maybe I should pull in Mockito here” as a wake-up call that I need to refactor. I’ve paired with folks who throw mocks at everything and I always stop them and say that for at least the code we own, we can just, y’know, make the code easily testable directly instead of mocking things out. If things are loosely coupled and somewhat isolated, I rarely find the need to reach for a mocking framework.

                          it is possible that the design you reach through TDD is going to be worse than if you spent 15 minutes doing up-front design.

                          Yes, if you don’t know what good and/or bad design looks like, the signals that TDD sends are going to get ignored (like the use of test doubles).

                          1. 1

                            I’m completely the opposite - I find it very uncomfortable to ship code that hasn’t been tested. This article seems to be saying “tests are hard because: you have to write tests, and .. after all .. what value do they truly provide” .. without really giving any reason for why tests are a hassle to write, beyond ‘tests are a hassle to write’.

                            Okay, so you don’t want to be a software engineer, and actually test and validate the results of your hard work at programming code - but this doesn’t mean that the TDD ethos is broken. It means your ethos is broken - and if you think that is happening just because ‘writing tests is hard, and re-writing tests after changing the code is also hard’ its because .. you’re not as good at development as you think you are.

                            tl;dr 100% Code Coverage or GTFO!

                            1. 1

                              It seems like the author is railing against heavily mocked unit tests rather than TDD itself. I can certainly see that side of the argument.

                              In my experience, TDD — writing tests first — is a matter of preference. A good programmer will have a good idea of what interface their code will have before they start writing anything at all. Whether they start with tests or implementation shouldn’t influence the end result.