1. 4
  1.  

  2. 6

    What size teams do you work on? I’m asking as the larger the team, the more quickly obvious the benefits of unit testing is.

    I needed a few “aha” moments to realise that writing unit tests are not only not a waste of time, but they are a huge benefit for any project that is not throwaway.

    Early in my career, I built a long-running project mostly by myself where I wrote unit tests, “by the book”. A year later, these tests saved me as I got back to the project I’d long forgotten about and could refactor it, while being confident that it worked.

    Later, working on teams, I realized unit tests are the best way to “guard” the correctness of my code - better than any documentation or convention. I had my colleagues, who did not write tests, break my tests multiple times - and all I had to do was tell them to fix my tests. Later, a pattern showed on those not writing tests having visibly higher bug rate than those who did, and the “no unit test” hangt slowly converted, after big after big we pointed out how 90% of the time, a test would have prevented it.

    And then there’s the implication on architecture. Unit testing forces you to design testable architecture and become hands-on with things like dependent injection, abstractions via interfaces and so on.

    Everyone has a different journey, but I’d love to hear what “aha” moments you’d had (or not yet had) with unit testing.

    1. 4

      Everyone has a different journey, but I’d love to hear what “aha” moments you’d had (or not yet had) with unit testing.

      At first I found unit testing tedious: why bother writing a whole unit test when you can manually test a few relevant inputs and be done forever? More importantly, why bother spending a bunch of time setting up some unit testing framework?

      Then I found manual testing tedious: sometimes iterating on some code requires testing many times. To avoid typing the same inputs over and over again, I saved my test input as text files and used shell redirection.

      Then I found manually examining the output tedious: larger programs could have many execution paths requiring many inputs to exercise. To avoid missing differences, I wrote shell scripts to diff output with expected output.

      At some point I realized I was just unit testing indirectly with text output instead of actual function inputs and outputs. Once I finally bothered spending 2 hours getting standalone JUnit working, I felt like a huge moron. For C/C++ I used this simple #ifdef TEST idiom I found in the FreeBSD rand(3) implementation. I still use it for simple stuff. Some time later I started using googletest once I got more comfortable building and linking external projects to my own.

      I definitely thought of unit testing as 100% test coverage, even for trivial crap, rather than automating manual tests. Likewise I thought of mocks as enterprise OO hype rather than an easier way to set up and test obscure edge cases.

      1. 2

        I’ve nearly always worked on teams of 5 or less. The last two software jobs, those teams are part of a larger team as well, but code was mostly segregated. Ish.

        As far as “aha” moments have gone, I had a nice experience with working on PISC, which only had me working on it, but getting tests in place helped me keep from changing things I didn’t mean to change. Reading about how testing worked in Moon Pig has felt like another aha moment, as was working through half of 99 Bottles of OOP, which showed me how quickly unit tests should be running. I’ve yet to have unit tests protect my code as such, but I’m convinced enough of the value of testing to give it a good run.

        As an added factor, the NUnit Visual Studio extension is not a snappy experience. Comparing that to the testing framework used in 99 Bottles of OOP, I realized why the Red -> Green -> Refactor cycle is a thing.

      2. 3

        Unit testing is great, but it’s only part of the picture and not the panacea that they’re described to be. You should probably do some sort of testing, but decide what’s right for your project.

        I’d rather have full end-to-end testing of the whole product working together, and smaller runnable test programs which verify subsets of systems rather than just testing any particular unit. Much behavior of an OOP system results from the interaction of many objects, and often people construct C++ programs in ways (due to sources, local and third party library dependencies, dependencies on services) which make it incredibly difficult to create smaller test programs. Being able to mock services only helps somewhat because often you’re dealing with undocumented behavior of services written by other people when the code is unavailable.

        Most of the issues I deal with require being able to boot up and run an entire version of the involved systems and services locally to reproduce issues or develop safely for, and no one knows how to strap the entire system into a test environment to run it and it often involves many layers of hoop jumping because no one was ever required to do it.

        1. 2

          Much behavior of an OOP system results from the interaction of many objects, and often people construct C++ programs in ways (due to sources, local and third party library dependencies, dependencies on services) which make it incredibly difficult to create smaller test programs.

          This is definitely a true observation about C++ code, but it’s also part of the problem. If code isn’t written such that it can be tested in smaller units, then it’s probably too tightly integrated and isn’t very easily maintained or iterated on. I’m not a unit test zealot, but I do think code should be written in a way that it’s relatively easy to add unit tests if you wanted to (including mocking out the right things). Of course, nothing in C++ is “easy”, so even well-written C++ code won’t be ‘easy’ to mock out in any absolute sense (I used GMock back in my C++ days), but it shouldn’t feel pretty manageable and mechanical. In other words, you shouldn’t need a significant refactor or otherwise do hacky things in order to add unit tests to your code if you decided you wanted to–if you do have to resort to refactor/hacks to add unit tests, your code is probably way too hard to extend or maintain (which is perhaps okay if your project is just in maintenance mode and new features are rare).

          1. 1

            The problem I’m describing is where I see a lot of systems with unit tests, but with functional (as in for a known purpose, not as in functional programming) systems with ill-defined boundaries that you can’t stand up on their own or even together outside of production. You can mock system A for unit tests, but you can’t actually stand it up by itself because it’s relying on systems B, C, D, E, and F. In analogy: it’s the issue of being able to only either check the air in your car tires or drive your car on the highway, but not be able to run your car on a test track. Running the car engine in the garage is one thing, but there are issues you can’t address until you see it in action on a track at 200 km/h.

            1. 1

              Yeah, (some) unit tests are necessary (IMO) but not sufficient. You still need integration tests (i.e., functional tests?) to check the integration of your units and even end-to-end tests to test the complete assembly. But just because unit tests aren’t sufficient doesn’t mean they aren’t necessary or otherwise very useful.

              1. 1

                One thing I like is combining property-based tests with contracts. That gives you a lot of integration tests for free, as the inputs flow through your whole system and keep hitting your pre/postconditions.

            2. 1

              I understand that it’s only part of the picture, but, until recently, it was a part that I saw little value on for my day job.

              Condolences on your code problems.

            3. 1

              Unit testing as a practice was one of the few ones that I got instantly. However, there’s a bit of an art to it. Not everything is easily unit testable, and unit testing is not so important that everything should be fitted into that mold if they don’t naturally fit into it. For instance, processes with strict rules like “100% code coverage must be maintained at all times” are probably silly and harmful. Also, I think Dependency Injection as a way to facilitate unit testing is probably a code smell.

              1. 4

                Why would dependency injection to facilitate unit testing be a code smell? What would I be trying to avoid?

                At least for me right now, I understand using constructor DI and interfaces to separate core business logic from code that does CRUD to a database. (I also know enough about databases to be able to design a repository that will interact with the database at least semi-effiiently if large amounts of data are involved).

                I imagine that using something similar for at least the filesystem and/or time sources, and/or sockets, could all be handy.

                EDIT: I’m not saying it isn’t a code smell, I’m just trying to understand what a smell should be warning me of.

                1. 2

                  I don’t know what this exact smell is, but a couple of issues I’ve found:

                  • It implies that there’s more than one sensible production implementation for a given dependency, and you can’t tell whether or not a component is DI just for the purposes of testing. Why bother when there’s only ever going to be two implementations?
                  • It makes dependencies implicit, whereas the things that are injected (e.g. network or current time) should probably be explicitly given to the functions that require them. As such, it basically turns object classes into miniature global scopes - any function can access any dependency any other function can, and there’s no language distinction between fields that are there because that’s what the program needs and fields that are just injected to enable unit testing.

                  At $work it used to be that DI was used a lot, often just for configuration. However, since everything is in F#, new code explicitly passes everything in as function parameters. Now you can tell whether a function will access the network or current time just by looking at its signature, and there’s no ceremony of setting up dependencies in test cases. Both ways are a poor substitute for an effect system, but it’s more Functional now.

                  1. 3

                    What you’ve described in bullet point 2 is dependency injection, and your explanation captures an important reason to do it.

                    Why bother when there’s only ever going to be two implementations?

                    • To make your dependencies explicit
                    • To make the the logic testable

                    It’s hard to tell what you’re arguing since your first point seems to be an argument against DI, whereas the rest of your post is an argument for it.

                    1. 1

                      It’s about how it’s implemented, really. Giving code the external stuff it requires, rather than making it choose them itself, is just a normal part of programming, but it has the blessing of being an invoke-able Design Pattern in OOP circles.

                      DI as in “insert these components into a class instance” is a similar but more opaque version of “write functions that use no inputs other than their arguments” that functional programming has known and used since before the term “dependency injection” existed (citation needed). DI where dependencies are injected per-class is more implicit and less testable than per-function.

                      There’s also the part of DI where you find implementations of an interface at run-time based on a configuration file or whatever, which is an obvious footgun. (Edit: I may be getting this confused with IOC, I don’t know any more)

                      Edit: I suppose it’s not really useful for me to argue against DI as a whole, when there are so many ways of doing it.

                    2. 2

                      It sounds like your job was using both dependency injection and inversion of control. You can use DI without using IOC. When I am talking about DI, it’s just taking the sorts of things I depend on as either constructor arguments, or function arguments, rather than relying on them to exist in global memory space. And, for now, anyway, anything that interacts directly with the outside world (database, system time, filesystem, sockets) will be behind an interface so I can switch out the implementation of what’s going on for testing purposes. For now, things that don’t directly touch the outside world, I’m trying to keep concrete.

                      1. 1

                        I see - I may have to read up to fix my confusion.

                        1. 2

                          I’ve found https://martinfowler.com/articles/injection.html to be a semi-helpful article on the difference between the two.

                    3. 1

                      Why would dependency injection to facilitate unit testing be a code smell? What would I be trying to avoid?

                      Usually DI sacrifices clarity on the happy path (execution) for a special case (making testing easier), so it adds a bit of complexity. Usually this is worth it, but at its most extreme you’re injecting a dozen inputs that should be inline. I’m thinking of angular 1 as the most extreme case here, where half the test suite was just setting up mock injections. So it’s useful but taken too far sometimes.

                      1. 0

                        I’m just trying to understand what a smell should be warning me of.

                        Like most smells, it’s about complicating code for a purpose that has nothing to do with solving the problem.

                      2. 2

                        Also, I think Dependency Injection as a way to facilitate unit testing is probably a code smell.

                        Not only is it not a code smell, if you aren’t using DI what you’re doing is most likely not unit testing. Unless everything you’re unit testing happens not to have any external dependencies.