It’s ok to take a dual approach. In fact, it’s often inevitable. Tests are a way of asking questions and documenting the answers. Some questions are high-level and others are low-level. In legacy code this is particularly true. Write unit tests for a monolithic controller and they become integration tests once you extract classes from it.
I read “Working Effectively With Legacy Systems” a few years ago after my company moved me to a team suddenly responsible for maintaing a codebase written by a completely different team in another company. Thanks for the tips therein, even though I struggled to get any of my new teammates to read it.
A few of us that fought the good fight did eventually manage to bring order into the chaos by writing a lot of tests though.
I really like the collaboration test approach (partially) described by JB Rainsberger….
It took me awhile to realize it implied a separate, standalone module of code, that get’s reused in multiple contexts.
ie. Atila shouldn’t be testing that libclang works, that’s the job of the unit tests inside libclang.
He should be writing collaboration tests that prove that his code and libclang agree on the interface specification.
He needs to prove that his code meets the preconditions for calling the libclang code, and that his code can handle every possible return code and result permitted by the interface specification.
ie. He needs to create a collaboration test shim that is shared between his unit tests and mocks and integration test and production code that checks the interface specification.
ie. The collaboration test shim checks whether the unit tests, the mocks, the integration tests, the production code, and libclang itself all conform to (what he believes is) the documented interface, and the test shim is common standalone code that is byte exact shared (as appropriate) between all the above.
ie. The single responsibility of a collaboration test shim is to document, understand, check and enforce the interface specification for a particular concrete 3rd party interface..
I often start from an integration/system test and work my way in. My reason is that I want to build some software because it solves a problem someone has, and the system test shows that I am solving that problem. I write a unit test when I think of some software I want to write, and the unit test shows that I have written that software. That’s fine, but I also want to know that the software I write is solving the problem :).
I might do another blog post covering how I ended up porting a codebase with pretty much only integration tests to the unit variety.
Alternative suggestion: do a blog post on switching your project to something other than libclang (what would that even be?!). That would help determine whether these concerns you’re trying to separate actually matter in practice. Porting tests is a certain amount of effort. It makes a lot more sense if it’s in aid of a specific purpose.
There’s always some dimension you can generalize in software. But if it’s never going to be needed it’s just useless work. (Or a kata, which would be a totally valid reason.)
We’ll see. Since writing the blog post I’ve had some insights that I’ll share in what will now become a series. I think I can have my cake and eat it too.
I describe how I failed to do what I preach and ended up writing integration tests instead of unit ones.
It’s ok to take a dual approach. In fact, it’s often inevitable. Tests are a way of asking questions and documenting the answers. Some questions are high-level and others are low-level. In legacy code this is particularly true. Write unit tests for a monolithic controller and they become integration tests once you extract classes from it.
In legacy code, maybe, but this is code I wrote from scratch this year.
You’re not… the Michael Feathers are you?
Yup. I am.
I read “Working Effectively With Legacy Systems” a few years ago after my company moved me to a team suddenly responsible for maintaing a codebase written by a completely different team in another company. Thanks for the tips therein, even though I struggled to get any of my new teammates to read it.
A few of us that fought the good fight did eventually manage to bring order into the chaos by writing a lot of tests though.
Great to hear. Glad I was able to help.
You’re joking?
Can confirm, we met in person. He’s super cool in real life, too!
This is one of my favorite things about the internet =)
I really like the collaboration test approach (partially) described by JB Rainsberger….
It took me awhile to realize it implied a separate, standalone module of code, that get’s reused in multiple contexts.
ie. Atila shouldn’t be testing that libclang works, that’s the job of the unit tests inside libclang.
He should be writing collaboration tests that prove that his code and libclang agree on the interface specification.
He needs to prove that his code meets the preconditions for calling the libclang code, and that his code can handle every possible return code and result permitted by the interface specification.
ie. He needs to create a collaboration test shim that is shared between his unit tests and mocks and integration test and production code that checks the interface specification.
ie. The collaboration test shim checks whether the unit tests, the mocks, the integration tests, the production code, and libclang itself all conform to (what he believes is) the documented interface, and the test shim is common standalone code that is byte exact shared (as appropriate) between all the above.
ie. The single responsibility of a collaboration test shim is to document, understand, check and enforce the interface specification for a particular concrete 3rd party interface..
I often start from an integration/system test and work my way in. My reason is that I want to build some software because it solves a problem someone has, and the system test shows that I am solving that problem. I write a unit test when I think of some software I want to write, and the unit test shows that I have written that software. That’s fine, but I also want to know that the software I write is solving the problem :).
Me too, I learned this by doing BDD. The thing is then writing and relying way more on unit tests.
Alternative suggestion: do a blog post on switching your project to something other than libclang (what would that even be?!). That would help determine whether these concerns you’re trying to separate actually matter in practice. Porting tests is a certain amount of effort. It makes a lot more sense if it’s in aid of a specific purpose.
There’s always some dimension you can generalize in software. But if it’s never going to be needed it’s just useless work. (Or a kata, which would be a totally valid reason.)
Somebody suggested gcc plugins. I don’t know how that works. But yes, using a different parser frontend would help point out where I’ve gone wrong.
This is something you will end up almost on every SOA project. Very difficult to test something without jumping to integrations tests.
We’ll see. Since writing the blog post I’ve had some insights that I’ll share in what will now become a series. I think I can have my cake and eat it too.