Hyperproductivity in Scrum

Last year sometime, I had the pleasure of hearing Jeff Sutherland speak at the Agile Atlanta group here in town. One of the things that Jeff always brings up in his talks, is that Scrum creates hyper-productive teams. I asked him how he defined hyper-productivity, and how he measured it. He told me, but rather than explain my recollection of what he said, I want to reference how he explains it in print.

According to his paper Shock Therapy: A Bootstrap for Hyper-Productive Scrum:

"We define Hyper-Productivity here at 400% higher velocity than average waterfall team velocity with correspondingly higher quality. The best Scrum teams in the world average 75% gains over the velocity of waterfall teams with much higher quality, customer satisfaction, and developer experience."

Pretty impressive, but I am curious what the improvement is measured against. Going deeper into the Shock Therapy paper, the results from their MySpace experience say that:

"The baseline velocity (100%) is established for a team during the first Sprint. The Product Owner presents the prioritized Product Backlog in the Sprint Planning meeting. This is estimated using Planning Poker and story points. The team selects what can be accomplished during the first Sprint and the Product Owner determines exactly what is "Done" at the end of the Sprint. The number of points completed is the baseline velocity."

When I first read that, I was like "what am I missing here". Hyper-productivity is defined as improvement over the first iteration of a brand new Scrum team. That didn't strike me as a very fair metric, you'd think *most* brand new teams would get dramatically better after a few weeks of working together, using Scrum or not.

But here is their key assertion that supports the measurement:

"At MySpace, the baseline velocity is often significantly higher than their previously chaotic implementation of waterfall, so the baseline is conservative"

I don't know about you guys, but I'm kind of bothered by the assumptions, and the numbers that support them. Can we always assume the first sprint of a Scrum team is better than the same team doing Waterfall? I'm not sure. Are all Waterfall projects chaotic? I can tell you first hand they aren't. It might also be interesting to see the team stabilize first, and then see how much better they'd get by systematically removing impediments.

These numbers have been nagging me ever since I read this paper and first heard Jeff explain them. Is it only me? What am I missing here?

BTW - I loved the image here... seemed pretty much with the Shock Therapy theme. Check out the artists website at http://llynhunter.blogspot.com/2008/05/electricity-if.html

References

Reference: 

Hyperproductivity in Scrum