Yes, sure, there are limitations to this study. But this is also a pattern! Almost all studies on TDD either find a weakly positive signal or no signal at all. We don’t have solid evidence it’s better than any other testing methodology. It could work- I believe it works. But we need to be epistemically humble, and admit we’re basing our opinions off our beliefs and not facts.
And with regard to TDD, Martin is anything but humble. From his post Professionalism and TDD:
If I am right… If TDD is as significant to software as hand-washing was to medicine and is instrumental in pulling us back from the brink of that looming catastrophe, then Kent Beck will be hailed a hero, and TDD will carry the full weight of professionalism. After that, those who refuse to practice TDD will be excused from the ranks of professional programmers. It would not surprise me if, one day, TDD had the force of law behind it.
He wants us to reach a point where not using TDD is breaking the law. How do you possibly argue with that? How do you convince someone like that to accept his beliefs are not facts, to believe without polemicizing, to keep exploring with an open mind?
Someone once told me that I was hypocrite for using TDD and TLA+. Another person told me that if a bug slipped past TDD, it was because you weren’t doing TDD right. It’s one thing to believe a technique works, another thing to reject everything else.
It’s a strong statement, but preceded by disclaimers: “If I am right” and “It would not surprise me if, one day.” So, technically, it’s not as arrogant as it might look if you skip past the disclaimers. It’s vividly speculating about a possible future while signposting that it’s just speculation and nobody knows the future.
But it’s true that this writing style, while common, can sometimes be annoying to read, because it comes across as making a provocative claim and then almost entirely taking it back.
In the article he compares TDD to doctors washing their hands, so I’m not inclined to give him the benefit of the doubt here. He makes the strong statement because he genuinely believes it.
Also surrounded by disclaimers. He’s saying what he believes without insisting anyone else believe it, because it’s speculation. It seems like that is admitting to “basing our opinions off our beliefs and not facts” which you were saying is epistemically humble? Can you be humble about being a true believer?
Ah, I think I see the disconnect here: I’m only presenting one of his articles, when I’ve formed my stance on him in the wider context of his work. In The Dark Path he says language safety features are “the wrong path” because we should be testing more. In The Programmer’s Oath his third oath is “I will produce, with each release, a quick, sure, and repeatable proof that every element of the code works as it should.”, which in The Obligation of the Programmer he associates with TDD. In Professionalism and Test Driven Development (the IEEE article, not the blog post) he concludes with
My green band reminds me that TDD’s disciplines are a huge help in meeting professional-ism’s requirements and that it would therefore be unprofessional of me not to follow them.
In the linked article he also says that “I plead guilty to claiming that the association exists” wrt the association between professionalism and TDD. It’s also a recurrent theme in his Twitter, where he regularly compares it to double-entry bookkeeping.
He hedges on occasion, and does say it’s “not a silver bullet”, but just because he’s adding disclaimers doesn’t mean he’s not dogmatic.
TDD doesn’t work strikes me as the same kind of statement as “agile doesn’t work”.
In both cases there was a form of wild success from the practice - it was much better than the status quo ante for a significant number of people. Then lots of people wanted in on it, and once scaled out to everyone, we found that the industry average level of productivity wasn’t wildly better than it was before anyone heard of these concepts.
This doesn’t mean that no form of it works for no-one. It means that in each case the term has lost a lot of its meaning; and that neither was a silver bullet for everyone in every context. Indeed silver bullets themselves only help if you’re committed to violence against werewolves.
Probably a better way to think about these - and all software engineering practices - is to identify when they work, what it means for them to work, and what defines those situations where the practices work. This would allow us a profession to refine the practices to work across more contexts and know when not to bother with them.
Of course, as an industry we haven’t figured how to pay people to work on our most important software (that is all open source projects with millions of installations and fewer than 10 people paid to work on them). So, I don’t know how we get to that plateau of empirical adequacy, but I don’t know that these hot takes help.
Yes, sure, there are limitations to this study. But this is also a pattern! Almost all studies on TDD either find a weakly positive signal or no signal at all. We don’t have solid evidence it’s better than any other testing methodology. It could work- I believe it works. But we need to be epistemically humble, and admit we’re basing our opinions off our beliefs and not facts.
And with regard to TDD, Martin is anything but humble. From his post Professionalism and TDD:
He wants us to reach a point where not using TDD is breaking the law. How do you possibly argue with that? How do you convince someone like that to accept his beliefs are not facts, to believe without polemicizing, to keep exploring with an open mind?
Someone once told me that I was hypocrite for using TDD and TLA+. Another person told me that if a bug slipped past TDD, it was because you weren’t doing TDD right. It’s one thing to believe a technique works, another thing to reject everything else.
It’s a strong statement, but preceded by disclaimers: “If I am right” and “It would not surprise me if, one day.” So, technically, it’s not as arrogant as it might look if you skip past the disclaimers. It’s vividly speculating about a possible future while signposting that it’s just speculation and nobody knows the future.
But it’s true that this writing style, while common, can sometimes be annoying to read, because it comes across as making a provocative claim and then almost entirely taking it back.
In the article he compares TDD to doctors washing their hands, so I’m not inclined to give him the benefit of the doubt here. He makes the strong statement because he genuinely believes it.
Also surrounded by disclaimers. He’s saying what he believes without insisting anyone else believe it, because it’s speculation. It seems like that is admitting to “basing our opinions off our beliefs and not facts” which you were saying is epistemically humble? Can you be humble about being a true believer?
Ah, I think I see the disconnect here: I’m only presenting one of his articles, when I’ve formed my stance on him in the wider context of his work. In The Dark Path he says language safety features are “the wrong path” because we should be testing more. In The Programmer’s Oath his third oath is “I will produce, with each release, a quick, sure, and repeatable proof that every element of the code works as it should.”, which in The Obligation of the Programmer he associates with TDD. In Professionalism and Test Driven Development (the IEEE article, not the blog post) he concludes with
In the linked article he also says that “I plead guilty to claiming that the association exists” wrt the association between professionalism and TDD. It’s also a recurrent theme in his Twitter, where he regularly compares it to double-entry bookkeeping.
He hedges on occasion, and does say it’s “not a silver bullet”, but just because he’s adding disclaimers doesn’t mean he’s not dogmatic.
TDD doesn’t work strikes me as the same kind of statement as “agile doesn’t work”.
In both cases there was a form of wild success from the practice - it was much better than the status quo ante for a significant number of people. Then lots of people wanted in on it, and once scaled out to everyone, we found that the industry average level of productivity wasn’t wildly better than it was before anyone heard of these concepts.
This doesn’t mean that no form of it works for no-one. It means that in each case the term has lost a lot of its meaning; and that neither was a silver bullet for everyone in every context. Indeed silver bullets themselves only help if you’re committed to violence against werewolves.
Probably a better way to think about these - and all software engineering practices - is to identify when they work, what it means for them to work, and what defines those situations where the practices work. This would allow us a profession to refine the practices to work across more contexts and know when not to bother with them.
Of course, as an industry we haven’t figured how to pay people to work on our most important software (that is all open source projects with millions of installations and fewer than 10 people paid to work on them). So, I don’t know how we get to that plateau of empirical adequacy, but I don’t know that these hot takes help.
My preferred TDD is “Thanks, didn’t debug”. It’s what I say to my boss when he notices something working.