Agile Zone is brought to you in partnership with:

Allan Kelly has held just about every job in the software world, from sys admin to development manager. Today he provides training and coaching to teams in the use of Agile and Lean techniques. He is the author of "Changing Software Development: Learning to become Agile" (2008) and "Business Patterns for Software Developers" (2012) and a frequent conference speaker. Allan is a DZone MVB and is not an employee of DZone and has posted 77 posts at DZone. You can read more from them at their website. View Full User Profile

Study on benefits of TDD

09.01.2010
| 2065 views |
  • submit to reddit

OK, this isn’t news, this study came out a couple of years ago and was covered by many people then. But, I find myself regularly referring to it trying to find the link. So I’m going to blog about it then I’ll always be able to find the link.

The study is by Nagappan, Maximilien, Bhat and Williams and is entitled: Realizing quality improvement through test driven

development: results and experiences of four industrial

teams and is freely downloadable from Microsoft.

This is the key findings are summarised in this table:

Team /

Metric

IBM drivers

Microsoft Windows

MS MSN

MS Visual Studio

Defect density of comparable team not using TDD

W

X

Y

Z

Defect density of team using TDD

0.61W

0.38X

0.24Y

0.09Z

Increase in time taken because of TDD

15-20%

25-25%

15%

25-20%


The way to read this is: the researches looked at two Microsoft MSN teams, one team did not use TDD and had a defect density of Y. The second MSN team had a defect density less than a quarter of Y but took 15% longer.

To my mind that proves that which was to be proven, i.e. TDD reduced bugs. But, I’m also aware that other writers have disputed this and I’ve heard of studies which disprove it. (Anyone got a link? Thanks)

Most people who I’ve met, and who have practices, or understand TDD agree it is effective. However there are those who don’t believe it. It reminds me of the episode of Black Adder where Rowan Atkinson’s Black Adder hires a ship Captained by Tom Baker. When there is a problem it plays out like this:

Black Adder: “Someone in the crew will know... you do have a crew don’t you?”

Captain: “Arh, opinion is divided on the subject... all the other captains say you need a crew, and I say You Don’t”

At the end of the day Confirmation Bias will probably decide which set of results you choose to believe.

References
Published at DZone with permission of Allan Kelly, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Emma Watson replied on Sun, 2012/01/15 - 8:40am

Hi , this is a very interesing results. Always wanted to see some research work on TDD in big organisations.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.