In this interview, recorded at the recent Agile 2009 conference in Chicago, Scott Ambler, Chief Methodologist for Agile at IBM Rational, shares his findings from a recent Agile Practices survey he conducted this past summer. One of the most interesting observations that came out of the survey, according to Ambler, was the disparity between what people are talking about doing and what they're actually doing. He shares his findings on how people are approaching Agile in terms of requirements, design, and testing and highlights the need for modeling across the lifecycle vs. 'Big Design Up Front.' Ambler also talks about his findings around Agile project initiation and estimation, and describes the Agile Scaling Model, a framework that describes eight techniques for scaling agile within the organization.
The slides from the presentation that Scott gave at Agile 2009, "Agile by the Numbers: What Organizations are Really doing out there," can be downloaded here.
The complete transcript of the interview has been provided below.
DZone: Scott, can you tell us a little bit about your role at IBM?
Scott Ambler: Yes. I'm the Chief Methodologist, and my job is basically to help people, particularly customers, as well as the internal Rational people to understand Agile, to determine how we could help customers to become more effective at what they do. In many ways, it's more of a process improvement role than it is a strictly Agile role.
DZone: You've been doing a lot of research around Agile adoption. Based on the surveys you've conducted, have you observed any interesting trends that you can share with us?
Scott: Yes. I think that the most interesting trend is that the Agile community really seems to struggle to communicate what they do and practice, as opposed to what they claim to be doing in the blogs, and in some of the books, and in some of the mailing lists. For example, about a week ago I was involved in an interesting conversation on one of the mailing lists about Big Design Up Front. Somebody's asking “If I'm doing TDD, should I not be doing a little bit of modeling beforehand?”
As soon as you ask a question like that in the Agile community, instantly people start shooting their mouths off about the evils of Big Design Up Front and all that good sort of stuff. The reality is that surveys show that the vast majority of Agile teams do in fact do some up front modeling. That's not necessarily Big Design Up Front. We've got this challenge where technical people tend to see things in black and white.
What will happen is, a lot technical people have been told “Oh, Big Design Up Front's a bad thing; therefore, let's not do any modeling.” Well, no. We are doing a lot of modeling, but we're not talking about it. This confuses the heck out of people. It's a serious challenge. This is just one of many examples where we really do struggle to communicate what we do.
DZone: Is there a big disparity between how business people think about Agile and how technical people think about Agile?
Scott: In many ways, yes. I think there are different issues that the technical people are worried about. Technical people are often interested in the technology. They want to talk about different languages, and different tools, and different techniques. They'll get down to the technical weeds, which is perfectly fine for people in those roles. But the business people are really interested in far more important issues, in many ways. Are we improving the time-to-value? We just happen to be doing that. Are we producing better quality? Which we also happen to be doing. Are we producing stuff that people want to use?
A lot of the business needs are being met by the Agile community far better than the traditional community, but we often don't recognize and don't understand - technical people don't understand how to communicate that to managers.
I think that's a huge frustration for the technical people, but also for the business people, because I run into organizations all the time where the technical people are down in the trenches. They're doing their jobs. They're good at it. They're trying to bring Agile into the organization. The business folks aren't listening to them, and it's because they're not speaking the language of the business.
The business doesn't want to hear about testing and development. They don't want to hear about the neat, new tool you're doing or the new book you've read. They want to hear about what business impact can you have. You've got to be able to talk to the business in business terms, and a lot of technical people don't seem to be good at that.
DZone: Would you say there's the need for an Agile framework that helps us better measure ROI?
Scott: I think so. We're starting to see that in the Agile community. At the conference here this week, we're seeing some talks on Agile metrics, a little bit on Agile governance, project management, stuff like that. How do you report in terms that the business understands, and technical management as well? There are metrics for certain purposes and certain groups of people. I think over the next couple of years, we're going to start seeing a better understanding and a better appreciation growing of the need for more intelligent reporting and hopefully for more automated reporting.
I've been working with organizations for a bit on this, and some of my advice around this is that I simply don't trust metrics anymore that are generated manually, because the interesting information gets left out. There's some fudge factors involved, stuff like that, and they're expensive. They're inaccurate, and they're expensive, and they're often late.
I prefer metrics that are being gathered by my tools in real time, being done basically for free, requiring no extra effort on the part of the technical person at all. As a technical person, I love that because I don't have any of this extra bureaucracy. As a business person, I also love that because I'm getting accurate, real-time results. This is something that you need integrated tooling for.
I don't know if you've talked with Erich Gamma at all, but he's heading up the Jazz team. Alistair Cockburn was talking about that earlier today. One of the things that the Jazz producs suites do is the generated metrics. It's sort of an interesting thing to see.
DZone: Based on the survey data you collected, how would you describe the reality of Agile adoption within organizations today?
Scott: A couple of things. First, the majority of organizations are now doing Agile from what I can tell. I've been running surveys on this for several years. VersionOne has also been doing surveys; a bunch of us have been doing surveys. The overwhelming evidence seems to show that the majority are doing Agile. I recently did a survey that showed, within the organizations doing Agile, very soon it appears that the majority of projects within those organizations will be doing Agile. So we've very clearly crossed the chasm in my mind, which is a good thing.
Now different organizations adopt Agile in different ways. This is something we're seeing at the conference this week. Different speakers have got different ways of communicating what they're doing because they're in different situations.
A very clear message that's been coming through is that you need to focus on repeatable results and not on repeatable processes, because different people are in different situations, so they're going to be working in different ways.
DZone: How are people approaching Agile adoption in terms of requirements, design, and testing?
Scott: They're doing a lot of it, actually. This is one of the things we're not good at communicating. We do up front modeling because most project teams have to justify what they're going to do. They've got to answer questions like “What do you think it's going to cost”, and “How long do you think it's going to take”, and “What are you going to build? How are you going to build it?” If you can't answer those questions, you're not going to get funding. To answer those questions, you've got to do some up front modeling. But what's going on is, that doesn't mean that you need to do detailed modeling. It doesn't mean you need to write big, huge documentation.
So this I think is the difference with what we see in the traditional world, where they do far too much up front work. They actually increase their risk, increase their cost, and stretch out the timeline of their projects by doing this up front bureaucracy work.
What we're doing in the Agile world by following Agile modeling techniques, is that we're getting the benefits of modeling, which is to communicate and to think things through, but we're not taking the hit of needless documentation. We're not taking the hit of making decisions too early in the project when we really don't have good information.
I think this is a big cultural shift for a lot of organizations that they're struggling with, and particularly for the professional modelers, who for the most part have been trained to do all this up front modeling, do lots of detail, and do the best that you can. Well, the best that you can means that you need to spread the modeling effort, the analysis, the design effort out throughout the life cycle, not just at the beginning.
DZone: So modeling should then, again, be iterative, like many other things within the Agile universe.
Scott: That's exactly what we do. For example, user stories have taken off within the Agile community. People describe user stories with the three Cs, so you're having a card, then having a conversation, and then confirming that the story works. Well, that second C, the conversation, that's just-in-time analysis. What's happening is the Agile people are doing this analysis. It's a common thing that they do throughout the life cycle, but because they're not using the old terms, the traditional people, they're not hearing “Oh, you're doing analysis, you're doing design.” They're hearing “We're having a conversation.” What the heck is that?
So this gets back to the communication challenges that we see. We need to speak in the language of the people that we're trying to interact with not in our own made up language.
DZone: In terms of project initiatition, what challenges are there around getting funding and approval for Agile projects?
Scott: Yeah. I recently did a survey on Agile project initiations. I wanted to find out what people were actually doing, and sure enough some project teams are doing big detail, up front estimates because they are being forced to. Some of them are doing very loose things. A very small percentage are not doing any up front estimating at all. I think it was about 7% or 8%, so not a lot of teams are getting away with not doing some up front estimation.
So, yes, we're getting the funding. The funding effort is really paradigm neutral in many ways and this setting is an important thing for the business community to understand. And when they try to manage their financial risk by asking project teams to give a detailed, precise estimate up front, they're really increasing their financial risk because they don't understand the behavior that that's going to motivate on the part of the development team.
So business, if they really want to manage the projects, if they really want to reduce their financial risk, they need to loosen up the reins a bit, and they need to manage throughout the lifecycle as opposed to insisting these up front estimates. The up front estimate stuff is interesting theory, but it's just not working out.
Internals at IBM, I'm not sure if this is published yet but enough of it has been presented it that it might as well be almost official now. We were on a study a year or two ago where we looked at our existing database. We've done thousands of projects over the years, right? So, we've got this huge database of information showing what the actuals are and what the initial models were on all sorts of stuff.
Two researchers went into that database, and they took a subset of projects where they could produce what the original estimate would have been using Kokomo, using Slim, using a couple other methodologies. And then they compared what would have been predicted to what actually happened.
What they found was of all these estimation methodologies none were really all that accurate. So, for 70% of the projects you get plus or minus 30%, and then the other 30% of the projects were just all over the board. As long as you are willing to have that level in that range, it's fine.
A recent survey that I did shows that the average project is required to give an estimate which was on average, I think, plus or minus 11%.
So the expectations of business are completely out of whack with reality. And then what happens is you end up with an escape from Gilligan's Island situation where all these crazy things start to happen to try to get out of that bad situation. In many ways the business increases their risk by doing it. I would invite the business to understand the implications of what they are asking for, and that requires knowledge and experience and requires them to actually observe what's happening around them as opposed to what they want to be happening.
DZone: Business people are always going to ask for predictable financial models. Are there any alternatives? The 30% discrepency sounds almost unacceptable.
Scott: Well, it's interesting. In the keynote, Alistair Cockburn was talking about adaptive versus predictable, and that's the wrong question to be asking because the fact is that these Agile adaptive approaches are far more predictable than the bureaucratic, traditional, trying to plan approaches and the reason why. This sounds strange. People are going to go, "Scott is a heretic. He doesn't know what he's talking about." But if you step back and think about that, if I predict that this project is going to cost a million dollars, I've got a much better chance of hitting that prediction if I steer the project and adapt what I'm doing based on what I'm learning as I go. But if I try to predict up front exactly what I'm going to do to get there, I'm got much less chance of hitting that target because I really don't understand. It's really difficult to predict the future.
So we need to observe these things, and this goes against the dogma of the traditional community. In many ways the traditional dogma is theory, and we've forgotten it because it's been around for so long now. It's been around for like 30-40 years. Most of it is forgotten. It was just theory, and we need to start observing -- now that we've had 30-40 years of experience -- we should observe what really works as opposed to what we want to work.
DZone: Scott, can you talk a little bit about the Agile scaling model?
Scott: Yes. Myself and a bunch of others at IBM are working on a framework or a model to describe how to scale Agile. Right now we're calling it the One Plus Seven Framework, and that name could change. The idea here is that there are these eight scaling factors that we're interested in. So, the first scaling factor is just scaling to make sure that you address the full delivery lifecycle as opposed to the real cool construction stuff that everyone talks about. Because we know it takes time to start up a project. We know that release and stuff in your production can be interesting. We know that operations and support once it's in production is also interesting. So we need to look at the full lifecycle. That's the One.
The Seven are a bunch of scaling factors that may or may not apply to you. When people think about scaling, they usually think of a team size which is fair. So that's one of the scaling factors, but it becomes more complex when your team is distributed. And the majority of Agile teams are distributed.
This is one of the dirty secrets of the Agile community. We're constantly talking about co-located teams -- which is a good thing -- but that's not the majority of them. The majority of teams have some sort of distribution. So, that's one of the scaling factors.
Regulatory compliance -- are you FDA compliant? You have to be Sarbannes-Oxely compliant. There are several talks to conference this week on exactly that.
Are you organizationally distributed? Do you have multiple companies involved? Do you have technical complexities? It's very easy to put together a single technology, doing a stand alone system which we often talk about. Are you doing a stand alone website?
But the vast majority of Agile teams have to deal with legacy systems, have to do with legacy data and trying to integrate either with existing stuff or help the existing stuff integrate with what they're building. So that brings some complexity in.
There's organizational complexity. There's an entire track on cultural challenges and good stuff like that. That can really harm you and do wonderful things for you when you're trying to adapt Agile. We consider that a scaling factor.
The last one, which is not coming to mind -- enterprise disciplines which is another almost theme that we're seeing at the conference this year where you're considering cross project issues like product line management, enterprise architecture, strategic re-user, asset management, or what we're going to call that, portfolio management.
These are all cross-system issues which bring some complexity into your process. There is significant value if you're doing it right, but it's also going to change the way you work, and it's going to change the way you govern the projects and the way that the development teams act.
These are topics that are not often discussed in the Agile community, often because we've got a very significant development focus within this community as well as a significant leadership focus. But it's almost always at the project level, not at the enterprise level. So, that can bring some complexities in.
There's a little more to scaling than just dealing with team size, not that dealing with team size can be the easiest thing in the world either. That's what the Agile scaling model is all about, and once it makes these scaling factors explicit, then we start talking with things: what practices do you need to adopt if you are in certain situations, or how does a practice like doing a daily stand up meeting change if you've got a large team, or if you've got a distributed team in a regulatory environment.
We're seeing entire talks at the conference on these very issues because if you've got a distributed team in a regulatory environment, your daily stand-up meeting is going to be significantly different than if you're a small, co-located team in a non-regulatory environment. So, we need to change our strategies. It's the same practice, but it's implemented in a different way. This is something that can be difficult in a lot of companies.
DZone: IBM itself underwent a very radical shift in its development processes across the different development groups, moving them all over to Agile. Some 20,000 people were affected by this transition, led by IBM's Sue McKinney. What was your involvement in this project?
Scott: I was involved when it first started off. I'm still semi-involved, I guess, in a supporting role. Sue's group has been doing a good job for a few years now. They're still in the middle of transition. It'll never end, I suspect, in the process. You can only get better, so the process never really ends, but I don't know if Sue was talking about that. She understands that. It can be a frustrating thing to observe sometimes. At the beginning I helped out a lot to provide some ideas. I did some internal training. I'm now more focused on Rational and more focused particularly on customers as opposed to internal stuff. But I help out whenever I can.
I'm based in Toronto, near the Toronto Labs, which is 2,500 or 3,000 people, so it's a good sized group. They've got an amazing group; evangelistas there now, so there's very little need for me to be involved other than to bounce ideas around, I guess. I have interesting meetings with them. We share ideas and we share strategies, and some of the people at the lab in Sue McKinney's group have been providing input into the ASM. So we're learning from each other. It's a lot of interesting stuff going on there.
DZone: You were involved with the Agile Manifesto in its early days. How would 'refresh' the manifesto to reflect some of the needs of today's environment?
Scott: In the beginning of the Agile movement when the manifesto was published, I was intrigued by it, and I jumped on that bandwagon almost instantly because it really sort of spoke to me. I wasn't one of the original writers, but I was definitely one of the very first supporters of it. I've been pretty impressed with it. I think it has stood the test of time. There's a good argument that maybe it's missing a principle or two. I think the four value statements are pretty solid. What's interesting though is the manifesto for software craftsmanship has extended the Agile Manifesto by extending each of the four values.
I think there's some pretty good stuff there. I think it's a great step in the right direction. To me it shows a maturing of the industry, and people are starting to look at becoming more professional which I think is a good thing. We're still missing, in my mind, at least, some sort of recognition of enterprise issues. I don't see enough governance being brought into the discussion.
We talk about self-organization, but at IBM we talk about self-organization with an appropriate governance framework because you have limits and you have constraints that you must actually reflect. This is something that is missing.
There's a lot of Agile developers out there that are really novices. They are very smart people doing great work, but they don't get the big picture, and it really hobbles them. We still have some challenges with bringing the data community into Agile. I've been working a bit on that topic. I sort of let it go aside for the last couple of years, but I think the data community is finally starting to recognize that they need to take Agile seriously which is a real shame.
At the conference this week there was only one data talk, which was a good one I attended earlier today. But the usability committee caught on a few years ago. So, there's a very strong usability track at the conference. There's been some really great talks here. So for whatever reason, that community has seen the light and seen how Agile is growing. The majority of organizations are doing it, but the data community is still struggling to understand it. They're still struggling to see where they fit in.
So, I would invite everybody in the Agile community, everybody in the data community to start having conversations and start talking. I think the data community has a long way to go. They've got some cultural challenges. Then again, too, the Agile community needs to appreciate the data stuff. It's amazing how naive some of the advice I hear in some of these Agile talks is when it comes to the data issues. They just don't get it, and that's a real problem.
DZone: It sounds like an effective reporting strategy then, is elemental to Agile success.
Scott: Exactly. Everyone's talking refactoring, and refactoring is a great process. But you've got to be able to refactor throughout the entire technology stack. It's nice that we've gotten very good at refactoring the UI. It's nice that we've gotten good at refactoring the middle tier, but if you can't also refactor the database, you've got some challenges. Myself and Pramod Sadalage wrote a book about this a couple of years ago, so the techniques are there. Some of the tools are there now, but if the data community is in denial of it, then... I had a conversation today with somebody, and they were adamant that you couldn't possibly refactor a database. It just couldn't be done. I said, “I wrote a book on this, and if you would be willing to type in code from a book, you could refactor your database.” So it's not impossible. Sorry.
But because the data community hasn't really picked up on this and it doesn't really match their theory very well -- you know, practice and theory sometimes don't match -- they've really missed the boat on this sort of thing. So, they are really struggling right now to overcome some real cultural challenges and to overcome some pretty huge misconceptions.
One of the reasons why I run the surveys that I do is, I want to have fact-based arguments. I don't want to have these religious arguments with people. I want to have real discussions. I want to say, "Yeah, I appreciate your point of view, but the data seems to show that the Agile community is producing better quality. It seems to show we are getting things out the door quicker, and it seems to show that we are better at delivering the function that people want. So, maybe, you should pay attention to that because it seems to be a good thing".
It's interesting. They'll come along. It'll take time, but they'll come along.
DZone: What advice would you give developers and architects, as well as those at the executive level for adopting Agile?
Scott: Take Agile seriously. It works. It's here to stay. Also, recognize that just because it's working for a lot of people, it might not work for you. It might not be the right thing. So, be open minded and be willing to work with others, learn from others. Always be trying to improve. I get worried when I hear about people who have been doing the same thing for years and years and haven't really picked up new skills. Don't get pigeonholed into a single career so don't be just a programmer or just be a DBA or a tester. Those are all valuable things, but I'd rather see a programmer with some data skills and some testing skills or a tester with some business analyst skills and some programming skills and, maybe, some data skills.
You can interact with people much better that way. You'll have a much more fulfilling career, and you'll be much more employable. So, you'll be more efficient, more employable. What's not to like about that? So, get out of the ghettoes of wherever your career is right now and expand your horizons a bit. I think that's the best thing.
At the executive level the best advice is to support your staff and to create a safe environment for them to experiment, for them to learn. Push them to learn as much as possible and to break out of the mold. I think that's the best thing that executives can do.
DZone: Scott, on behalf of the DZone community, I would like to thank you for your time today.
Scott: My pleasure. Thank you.