I cannot disagree hard enough with this article. I am surfing to relax from the stress of not five minutes ago finishing a 90m talk on this very topic to my company.
It sounds like bad unit tests. Looking at the tags on the blog I’d guess the author is talking about testing Rails, and I think the Active Record pattern (to say nothing of the AR lib) is part of this: it encourages your objects to all know about each other, maintain state, and use global variables (the db). When you’ve been doing this, you end up mocking inappropriately (so you don’t know when you’ve broken protocols) to deal with not being able to test things in isolation - or even guess if a message you send will hit the db. You end up scared of your tests because it’s unpredictable if they’ll tell you all your failures, or you’ll change something small and half of them will go red, or you’ll make a change and spend an unreasonable amount of time updating tests to match the new behavior. This is really, really common.
It doesn’t mean unit tests should not be trusted. It means your unit tests can be better, maybe because the environment you’re interacting with encouraged poor design.
I’m wrapping up a rewrite of a personal project away from the AR pattern and into tests as described by Metz and Rainsberger and immutable values + entities with identity + repositories as described by Eric Evans in Domain Driven Design and I’ve been surprised by how much easier my tests are to work with now: failures generally break only a few tests, and they’ll break a test that is very low-level so I don’t spend time digging down to find errors. Significant refactorings have not caused me to curse my tests.
Hey, I’m the author. While this blog post was mostly inspired by Rails, and I agree with most of your points, I’d say that this really depends on your definition of refactoring.
I was talking about a larger scale changes that span multiple classes, where your unit tests are inherently going to break, because you change the way the objects interact. At this point you have nothing left but your higher level tests to rely on, because all of the unit tests are failing.
Sandi Metz talks about things at a small scale. I’ve actually finished re-reading her book last week, and I have to say that it left me feeling very sad. If you have 5000 lines of messy tangled code that you need to refactor, there’s no way you can rely on the messy unit tests, because they will give you zero information about the whole system.
Yes integration tests are a scam, and they can’t test every corner of the app, but that’s not what I’m saying here. I’m talking about having something that verifies the happy path of your code, to the point where you know the core logic is working. They’re not supposed to test every single edge case, but rather keep the big green button saying this shit didn’t break.
Significant refactorings have not caused me to curse my tests.
Then either all your applications are rainbows and unicorns, or we have a different definition of significant.
No, we’re on the same page of refactoring that span multiple classes. This app is far from rainbows and unicorns (it was one of the first OO codebases I wrote). But it’s certainly young: I’ve only been using this strategy a few months and it’s just one fairly small app (~5kloc, half tests). Maybe it will really bite me in a year or when the app doubles in size, I don’t know yet. But one stressor is that the app archives emails, so it is constantly dealing with really terrible, invalid input and having to make useful things out of it, and it’s done really well on that measure.
I’ve written up some of my recent experiences on my blog and I already follow your feed (I really liked your duplication in tests post), so let’s just keep coding and blogging and maybe we’ll figure it out. :)
Actually… and I say I’m following your blog, but now that I checked, the feed your page pointed me at http://blog.jakubarnold.cz/rss looks like it’s just serving your homepage, so I’m not getting updates.
I have that on my bookshelf, as well as Refactoring, Implementation Patterns, Growing OOS Guided by Tests, and other books on this topic, though they all seem to just a bit too idealistic.
Huh, I found WELC to be a really practical guide. I read it at a job whose first bullet point on my resume was “Wrote -110,000 lines of code”. :)
I’m not a big GOOS fan, but I got a lot out of Refactoring, Patterns of Enterprise Application Architectures, and Practical Object Oriented Design in Ruby.
Maybe part of it was being solo/on small teams. If I wanted to experiment with a design style or use new refactoring techniques, I could just do it.
I cannot disagree hard enough with this article. I am surfing to relax from the stress of not five minutes ago finishing a 90m talk on this very topic to my company.
It sounds like bad unit tests. Looking at the tags on the blog I’d guess the author is talking about testing Rails, and I think the Active Record pattern (to say nothing of the AR lib) is part of this: it encourages your objects to all know about each other, maintain state, and use global variables (the db). When you’ve been doing this, you end up mocking inappropriately (so you don’t know when you’ve broken protocols) to deal with not being able to test things in isolation - or even guess if a message you send will hit the db. You end up scared of your tests because it’s unpredictable if they’ll tell you all your failures, or you’ll change something small and half of them will go red, or you’ll make a change and spend an unreasonable amount of time updating tests to match the new behavior. This is really, really common.
It doesn’t mean unit tests should not be trusted. It means your unit tests can be better, maybe because the environment you’re interacting with encouraged poor design.
Sandi Metz’s talk The Magic Tricks of Testing tells you what to test, and Rainsberger’s Integration Tests are a Scam gives you the protocol for writing tests.
I’m wrapping up a rewrite of a personal project away from the AR pattern and into tests as described by Metz and Rainsberger and immutable values + entities with identity + repositories as described by Eric Evans in Domain Driven Design and I’ve been surprised by how much easier my tests are to work with now: failures generally break only a few tests, and they’ll break a test that is very low-level so I don’t spend time digging down to find errors. Significant refactorings have not caused me to curse my tests.
Hey, I’m the author. While this blog post was mostly inspired by Rails, and I agree with most of your points, I’d say that this really depends on your definition of refactoring.
I was talking about a larger scale changes that span multiple classes, where your unit tests are inherently going to break, because you change the way the objects interact. At this point you have nothing left but your higher level tests to rely on, because all of the unit tests are failing.
Sandi Metz talks about things at a small scale. I’ve actually finished re-reading her book last week, and I have to say that it left me feeling very sad. If you have 5000 lines of messy tangled code that you need to refactor, there’s no way you can rely on the messy unit tests, because they will give you zero information about the whole system.
Yes integration tests are a scam, and they can’t test every corner of the app, but that’s not what I’m saying here. I’m talking about having something that verifies the happy path of your code, to the point where you know the core logic is working. They’re not supposed to test every single edge case, but rather keep the big green button saying this shit didn’t break.
Then either all your applications are rainbows and unicorns, or we have a different definition of significant.
No, we’re on the same page of refactoring that span multiple classes. This app is far from rainbows and unicorns (it was one of the first OO codebases I wrote). But it’s certainly young: I’ve only been using this strategy a few months and it’s just one fairly small app (~5kloc, half tests). Maybe it will really bite me in a year or when the app doubles in size, I don’t know yet. But one stressor is that the app archives emails, so it is constantly dealing with really terrible, invalid input and having to make useful things out of it, and it’s done really well on that measure.
I’ve written up some of my recent experiences on my blog and I already follow your feed (I really liked your duplication in tests post), so let’s just keep coding and blogging and maybe we’ll figure it out. :)
Actually… and I say I’m following your blog, but now that I checked, the feed your page pointed me at http://blog.jakubarnold.cz/rss looks like it’s just serving your homepage, so I’m not getting updates.
I’ve fixed the url, it’s supposed to be http://blog.jakubarnold.cz/feed.xml
I’ll check out your blog :)
… you should be reading “Working effectively with Legacy Code” by Feathers rather than POODR, as much as I love it.
I have that on my bookshelf, as well as Refactoring, Implementation Patterns, Growing OOS Guided by Tests, and other books on this topic, though they all seem to just a bit too idealistic.
Different strokes for different folks, I guess. I found “Legacy” to be one of the most practical books I’ve read.
Huh, I found WELC to be a really practical guide. I read it at a job whose first bullet point on my resume was “Wrote -110,000 lines of code”. :)
I’m not a big GOOS fan, but I got a lot out of Refactoring, Patterns of Enterprise Application Architectures, and Practical Object Oriented Design in Ruby.
Maybe part of it was being solo/on small teams. If I wanted to experiment with a design style or use new refactoring techniques, I could just do it.
Sounds to me like a problem with mockist unit tests, not unit tests in general.
Refactoring Tips: Use a Statically Typed Language. ;)