I really like the idea they had for a “micro experiment”….
ie. You don’t need to do a controlled experiment… you just have to
rely on humans being inconsistent in their application of any
methodology… and compare the case where they did X vs the case where
they did Y.
That web site http://jot.fm you are linking to is quite an interesting
one to explore, and will do so when I have time.
Yeah, just attempting a method to see if it improves things over the status quo can by itself be enlightening. The indicators might be certain things easier to express than before, bugs easier to track down than before, faster compile times, faster run times, easier extension to something involving many parts, and so on. Design-by-Contract almost always boosts at least one of these over development with no interface checks. So, a micro-experiment can help prove things.
There’s at least two things to watch out for. One is redoing a known system with a new method. Any improvements might just come from developers’ better understanding of the problem or solution space. This happened sometimes with formal specifications in the past, esp an IBM rewrite project. The other is ensuring the method is tested against a realistic workload, preferably many different ones. Something that’s great on one might fall short on others. Lots of “silver bullets” were like that.