Hm, I guess if one defines testability as “modularity, plus you can test modules in isolation”, then one follows from the other. But this definition doesn’t seem maximally useful to me.
I personally would define testability differently, as a combinations of following nice-to-have:
it’s easy to cover new functionality with tests
it’s easy to adapt test suite to changing requirements (that’s generalization of the previous one)
fast testing feedback loop
the work of adding test is proportional to the complexity of the functionality being tested (as opposed to all existing functionality)
testing has low accidental complexity (generalization of the previous two)
testing provides high degree of assurance that the code is correct
For this flavor of testability, modularity is not necessary. For example, if the system under test is a pure function whose input and output is easy to describe as data, and which is roughly linear, than you don’t really care if the system inside is modular. A prime example here are compilers for high level languages, like rustc. They are clearly testable, but are not necessary modular.
Curious how you would propose to create a meaningful test for rustc at the outside – with no optimizations etc at all I could maybe see a path, but as it is any change in one part of the compiler can affect the whole output substantially.
I’d say it’s more “testability implies modularity”. You can still modularity without testability, for example if don’t have testing oracles. That can happen when you’re doing scientific research or simulation.
Then again, @matklad makes a good case for “testability without modularity”, so maybe it’s more like “testability ≈ ≈ modularity”?
To me, it’s difficult to devise a modular system when the modules are only consumed by each other. Tests add a second consumer—albeit artificial—to each module, illuminating parts of the design that are over-coupled.
The slogan I’ve heard for this is Testing is a Software Design Activity. That is, it’s not just about correctness – the more important goal could be figuring out the natural structure of your software.
Though I don’t know where I heard it from first. I tend to agree, especially with the bits about mock objects. I wrote a few years ago that testing calcifies interfaces, which means you should be careful about what interfaces you test against, because those will last:
If you find yourself changing lots of tests whenever you refactor, that slows you down and indicates that you tested against unstable interfaces, which makes the tests less compelling as an indicator of correctness.
I tend to think of example code for the API as the better test. That is, I try not to use mock objects because those are sort of a “cheat” around the fact that your code isn’t modular. If you test without mock objects it often leads you to a Unix-style, loosely coupled, modular design. So in that sense I agree that testing and modularity are closely related. They are about interface and protocols .
Hm, I guess if one defines testability as “modularity, plus you can test modules in isolation”, then one follows from the other. But this definition doesn’t seem maximally useful to me.
I personally would define testability differently, as a combinations of following nice-to-have:
For this flavor of testability, modularity is not necessary. For example, if the system under test is a pure function whose input and output is easy to describe as data, and which is roughly linear, than you don’t really care if the system inside is modular. A prime example here are compilers for high level languages, like rustc. They are clearly testable, but are not necessary modular.
Curious how you would propose to create a meaningful test for rustc at the outside – with no optimizations etc at all I could maybe see a path, but as it is any change in one part of the compiler can affect the whole output substantially.
You don’t need to compare the actual binary, you can check that the compiled binary, when executed, produces the right results.
I’d say it’s more “testability implies modularity”. You can still modularity without testability, for example if don’t have testing oracles. That can happen when you’re doing scientific research or simulation.
Then again, @matklad makes a good case for “testability without modularity”, so maybe it’s more like “testability ≈ ≈ modularity”?
To me, it’s difficult to devise a modular system when the modules are only consumed by each other. Tests add a second consumer—albeit artificial—to each module, illuminating parts of the design that are over-coupled.
The slogan I’ve heard for this is Testing is a Software Design Activity. That is, it’s not just about correctness – the more important goal could be figuring out the natural structure of your software.
Googling gives this blog post:
https://toranbillups.com/blog/archive/2011/06/14/Test-Driven-Development-Is-A-Design-Activity/
Though I don’t know where I heard it from first. I tend to agree, especially with the bits about mock objects. I wrote a few years ago that testing calcifies interfaces, which means you should be careful about what interfaces you test against, because those will last:
http://www.oilshell.org/blog/2017/06/24.html#free-tests
If you find yourself changing lots of tests whenever you refactor, that slows you down and indicates that you tested against unstable interfaces, which makes the tests less compelling as an indicator of correctness.
I tend to think of example code for the API as the better test. That is, I try not to use mock objects because those are sort of a “cheat” around the fact that your code isn’t modular. If you test without mock objects it often leads you to a Unix-style, loosely coupled, modular design. So in that sense I agree that testing and modularity are closely related. They are about interface and protocols .