Agile Zone is brought to you in partnership with:

Jared Richardson works at Logos Technologies As a recognized expert in the software industry, Jared has worked with both start-ups and software giants. He's been involved with various open source projects, with roles from contributor to founder. Jared co-authored the best selling book Ship It! and Career 2.0, and founded the Agile RTP user group as a local outlet for the agile community in North Carolina. His personal blog is Agile Artisans Jared has posted 52 posts at DZone. You can read more from them at their website. View Full User Profile

Agile Versus Waterfall: Part One

05.14.2010
| 8088 views |
  • submit to reddit
The agile crowd always bashes waterfall. It's silly. It's dumb. It's guaranteed to fail. But waterfall is responsible for most of the software available today. There must have been something good going on there. What was it?

In my opinion, it was the illusion of control. Managers, often non-technical, had to manage teams of very technical developers. It was their job to report on the progress of the teams, but they couldn't. The technical work was so, well... technical, and so heavily based in the developer's heads, that it was very difficult to evaluate or understand progress.

The result is fairly easy to understand. A manager felt a strong need to understand what was going to happen, so he or she tasked their teams to do a great deal of analysis before the project started. This didn't work, so like violence and XML, they reasoned if it's not working, add more! This became what is often referred today as Big Design Up Front (BDUF).

So developers were tasked with doing very detailed studies and analysis when they knew the least about the product, and the least about the solution. And the customer they were trying to interview knew the least about the solution as well. It seemed that no matter how well the teams did their Big Design Up Front (BDUF), the customer was never happy. They either changed their minds or wanted a completely different product? The solution? Add more design!

Waterfall projects tended to last a long time... years in some cases. Of course, the bigger the project, the more work had to be done up front when everyone was in their optimum state of ignorance. However, the one thing that waterfall had going for it was the long time frames. It gave the teams a relative period of quiet, uninterrupted work. They could make mistakes, recover from them, and move on towards success. And this happened often enough that it took hold.

Unfortunately, it more frequently failed. That quiet period could also be used to hide problems. Issues ranging from developer apathy to incompetence weren't surfaced very often. If the team misunderstood the requirements, or missed a key point, no-one noticed until the end of the waterfall product cycle.

The first advantage of agile over waterfall is the short iteration. Developers focus on completing tasks on a weekly or bi-weekly rhythm. This short cadence has many benefits, the two that spring to mind at the moment are the surfacing of issues, and the ability to change directions.

If you have four teams, and three of them finish their short iterations very well on a regular basis, and one can't complete half of their work on most iterations, you know which team has problems. We don't know what the problem is, but we know where to start looking. A waterfall project can hide the problem teams, or the less than competent developer, until about the 75% mark, when teams start trying to integrate their work. At this point it's often too late to recover.

The other benefit is the ability to show work to the customer after an iteration is done and get feedback. The problem with getting feedback is that it might change what you're doing. The problem with not getting feedback is your project could fail, the customer might withhold payment, and the company could go broke. I think I'd prefer to change what I'm doing from time to time.

The world we live in is extremely dynamic and your customers may not be very good at describing their problems (assuming they understand them at all!) If you don't get feedback as you go, the product is at risk. Your success as a team is at risk. The only way to manage or minimize that risk is to show the customer your work as often as possible. The iteration provides a great stopping point for those demos. In reality most teams don't demo for a customer after each iteration, but they could. And some do.

Another agile technique for minimizing risk is the continuous integration system. After every code push, the entire project is compiled and every automated test is run. The alternative is waiting until "later" to merge everyone's code, and that's both painful, and risky. What do you do when people have been in the same projects, changing the same code? Frequent integration catches those problems quickly, minimizing the impact of the collision.

Which is better? I'd have trouble picking waterfall for most projects. Even a project for a government entity that had very stable requirements can still benefit from the frequent iterations as a risk management tool.

I'll post more on this topic next week... for now, what are your thoughts?

Published at DZone with permission of its author, Jared Richardson.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Jared Richardson replied on Fri, 2010/05/14 - 2:30pm

I got a reply to the article via Twitter from Dan Puckett

Waterfall advantage: it makes intuitive sense that if code changes are costly, lots of early brain work will pay off later. - link

My reply?

Absolutely! We do need up front design work. It's a myth that Agile forbids any up front design. However, Agile recognizes that you can't do ~all~ the design up front.

Waterfall tries to separate the work into discrete phases. Requirements, design, code, and maintenance. Agile recognizes that even with lots of up front analysis and design (and some projects need more than others) we can't be comprehensive. There are too many unknowns to be that precise, so we do as much as we can at each iteration, then move forward. But we can always revisit decisions if it makes sense.

One might say it's best to push decisions back until the last responsible moment. ;)

Dan Puckett replied on Fri, 2010/05/14 - 3:07pm in response to: Jared Richardson

Devil's advocate foiled again. When will that pesky devil win?

Jared Richardson replied on Fri, 2010/05/14 - 3:20pm in response to: Dan Puckett

If at first you don't succeed, try, try again! ;)

Ketav Kshatriya replied on Sun, 2010/05/16 - 5:26pm

How do you maintain the quality of Code in Agile. We have 2 week iterations and all the time the whole team is busy finishing work that is supposed to be delivered. Most of the time the best approach a team takes is lets get it done, which sacrifices reusability, code quality or componentization (Not that its anybody's fault, thats the nature of iterative and especially short deliveries).

 

Also whats your take on architectural decision. I mean deciding technologies for the project should be done iteratively? Which means that team works on some technologies which they know will be replaced down the line. I am curious to know how are big projects handling selection of technologies in iterative environment?

Jerry Gulla replied on Sun, 2010/05/16 - 6:14pm in response to: Ketav Kshatriya

Doing "Agile" - whatever your flavor (Scrum, etc.) is asking for trouble if you don't build in the time to write *quality* code. The way to think about it, Agile without some form of XP - test driven design, code metrics (and making that they're telling you good things) - is a way of accumulating technical debt (or as I've heard Jared call it "Credit Card" software development). These same things happen in waterfall, of course (sacrificing reusability, code quality or componentization as you say) but you tend not to notice for awhile. With 2-4 week iterations, it exposes that problem much sooner. Now the team/organization/company needs the discipline to make sure the dev teams have time to build things the "right way" otherwise, well, you end up with a mess. Jerry

Jared Richardson replied on Mon, 2010/05/17 - 7:17pm in response to: Ketav Kshatriya

As Jerry said, not writing good code isn't caused by iterations. It's exposed by iterations. I strongly suspect your team was already writing bad code before, but now it's not hidden.

And as David Bock says, don't turn off the fire alarm just because the sound hurts your ears. Put out the fire!

If you're going to write code, whether iterative or not, you've got to include the time to wrap that code with automated tests and run them in continuous integration. Not doing this is incurring mounting technical debt. I call this type of development "credit card technical debt" because you end up paying for it every month... forever! In the technical space it's not monthly payments but manual testing, endless debugging, and painful maintenance.

As for architecture, I'm not a fan of emergent design for large-scale systems. I prefer tracer bullet software development (link). Tracer bullets allow for large scale architectural decisions to be made up front, while still allowing plenty of freedom for the teams within each area to innovate.

The goal is to make the decisions when you need to make them, not at the beginning of the project just because it's at the beginning. The later you can push a decision back, the more information you'll have, and that additional information can change your decision. This concept is fleshed out much more deeply in the Lean Software area.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.