1. 4

  2. 11

    This happens at the unit level, hence “unit” tests. We mock out any dependencies in order to keep it fast and focused on the unit.

    Despite its plethora of existing terminology, I think these sorts of phrases (“dependency”, “mock”, “unit”, etc.) still need to be more precise about what they’re talking about, to prevent being misunderstood and having people talk past each other. The big problem, IMHO, is that some people use these phrases to mean some abstract, domain-dependent concept, whilst others use them to mean programming language features.

    From the sound of it, this person is using the word “unit” to mean “chunk of code” (e.g. a method, a function, a class, a module, etc., like updateCustomerAddress), and “dependency” to mean “separate code which that chunk of code needs” (like postCodeLookup). There’s an equally valid alternative for these same terms: “unit” can mean “unit of functionalityregardless of how the instructions happen to be arranged (e.g. “updating a customer’s address”); dependencies are those things which just-so-happen to be outside our control (e.g. a third-party service; but not necessarily a third-party library).

    Another example is phrases like “public interface”: some people take this to mean “the fields and methods of a class which are annotated with the public keyword”, some take it to mean “the ways that information can get into and out of the system”. It’s even tempting to rephrase the latter as “the API made available to the ‘consumer’ or ‘client’ of this system”, yet even that suffers the same trap, since it assumes a “consumer”/“client” is a person using this project, whilst others take it to mean “classes which reference this one”.

    I wonder if there’s a de facto standard terminology for this difference? Personally I prefer to think in the abstract, domain-dependent terms, where “unit tests” are testing that some functionality works, and “dependencies” are things like database servers and Web APIs; but I’ve worked in places where “unit” means “method” and “dependency” means “other classes”, and it’s remarkable how much confusion these seemingly “standard” terms can cause.

    I know there’s things like BDD, but that also seems imprecise in the same way. In fact, the workplace that followed the “unit = method” approach referred to what we were doing as “BDD”, presumably because we used PHPSpec instead of PHPUnit.

    BTW a nice argument in favour of the “abstract domain-dependent” perspective is https://www.infoq.com/presentations/tdd-original

    1. 3

      I don’t know why people find this so hard. Checking that my tire was made to tolerances and can handle X miles under Y conditions doesn’t mean my car will work, but it’s still pretty important.

      Integration tests are simultaneously the most important and the least useful. If I turn my key and the car won’t start, it’s broken no matter what the unit tests say. This is what the integration test comes close to telling you. Also, if I turn my key and the car won’t start that tells me nothing about what’s wrong.

      These layers of tests are about reducing cost and turnaround time via slowly adding dependencies. The unit tests on my packet handling protocol can actually run as I edit code, making the cost of finding and fixing problems small.

      I just realized that my ideal set of tests are kind of like a merkle tree. You can drill down to one piece or look at a failure at the top level and quickly find what node failed.

      1. 1

        This. People are easily confused about the definitions so there is plenty of discussions. The main source of confusion might be that you easily use a “unit” testing framework to run integration tests. Just because you use junit you do not necessarily do unit testing.

      2. 1

        re emergence

        You’re correct in that this is a basic reason for testing getting difficult. You’re using the Complexity Science viewpoint often brought up in thing such as Artificial Life:



        The Computer Science one is more actionable: state machines that interact lead to combinatorial explosion of what states the program might be in which might all need to be checked to know total correctness. In this case, you’re addressing it in terms of unit tests versus what’s traditionally called integration tests. Integration tests were often done as acceptance tests which were quite limited or black box compared to what was truly possible. Still, given a unit involves interactions of components it calls, there’s nothing stopping integration tests of things inside the component from being part of the unit test suite. It is after all a part of the unit’s function. One can keep doing this all the way up to the highest-level function or component for the application.

        Only thing it leaves out is interactions with other applications. That’s where the tests for distributed systems or modeling tools such as TLA+ come in.

        re “You can match the logic against a fault tree.”

        Ten second attempt at finding some work like that got some interesting results:



        1. 1

          We can stop fighting over whether developers or testers write unit tests.

          If you are fighting over stuff like that you have deeper problems than the meaning of “unit test”. No methodology will fix hostility between teams.