Agile Zone is brought to you in partnership with:

Alberto has posted 4 posts at DZone. View Full User Profile

Waterfall vs. Agile: Can they be Friends?

06.02.2010
| 46392 views |
  • submit to reddit

There has been a lot of discussion lately regarding which approach to software development is better; some may say Waterfall, some may say Agile. The truth is that there isn’t an absolute answer.

Agile and Waterfall have a different set of characteristics that make them better depending on many external and internal factors related to every project: time to deliver, amount of change expected in the specifications, nature of requirements, type of technical employees…etc.

It is uncommon to find projects that should be purely Agile oriented or Waterfall oriented, that’s why companies usually have to combine practices from both of them.

In these series of articles we are going to clarify what the main characteristics of Agile and Waterfall are, and how they apply to the classic areas of software development: Development, QA, Management, and Business.

Read the other parts in DZone's 'Agile vs. Waterfall' series -

 

Agile

Agile’s most important practical characteristic is that it is based on iterations. An iteration in Agile is a small period of time, usually from 1 week to 4 weeks, where some functionality from the application is built from end to end.

Agile has its foundation in the Agile manifesto which only focuses on how to implement Agile, but never mentions anything about its advantages and disadvantages, or what kind of software development it is better for.

Agile is best suited to projects that:

  • Focus on time to market - Time to market measures how fast a company can have a product out in the market from the moment they start developing. A fast time to market allows the company to have its product available long before its competitors. Agile is a sure bet to achieve very fast times to market as at the end of each iteration the application should be production ready.
  • May require a high degree of change - Requirements can and do change for projects as they progress. Agile provides a flexible process that optimizes feedback so changes can be introduced reliably during development.
  • Have one unique important deliverable: the product - There are many projects that only have one important output, the final product. Agile focuses on the product, almost ignoring other artefacts, such as documentation.

 

Agile is not a well defined, closed process; it is more like a philosophy or a family of related specific processes. Some common Agile specific processes are:

  • XP - Focused on engineering practices.
  • Scrum - Focused on management practices.
  • Lean - Focused on 7 principles that mainly apply to management.

 

In summary, due to its characteristics Agile is well suited for commercial applications where the focus is on moving from concept to software product without having to fully develop the original idea up front.

 

Waterfall

Waterfall is the classic approach to software development. In Waterfall the phases of software development are consecutive and always in the same order Analysis – Design – Coding – Testing – Deployment. Waterfall considers the analysis and the design the most important phases of them all. Waterfall is best suited for projects which:

  • Are contract based - The customer requires the company providing the software to commit in writing to fulfil a series of requirements. Since Waterfall is document driven, it lends itself to contracts that are heavily based on requirements. This helps to guarantee that everything specified on the contract is complete.
  • Are focused on analysis - Some software development projects require the analysis to be completed beforehand, this would be the case of very complex or critical systems that require many validation steps or approvals. Being a sequential process Waterfall is naturally suited to this purpose.
  • Have more than one deliverable - Not just the product, but also the user manual, the architecture …etc. Waterfall produces documents and artefacts other than the software itself. For some projects, these artefacts are considered almost as important as the final product.

 

Some examples of software development projects suited for Waterfall are:

  • Heavy engineering projects, for instance NASA.
  • Projects that require external certification.
  • Closed requirements projects.

In summary, Waterfall is best suited for projects where the specification and the design are better created upfront. These could include: engineering projects, large public institutions, software that has to comply with strict regulations…etc.

 

Mixing Agile and Waterfall

The wrong thing to do in software development is to pick only Agile or Waterfall and completely discard the other one for your project. Given your circumstances and the previous considerations, you should be able to identify which you are going to pick as your main philosophy. But, there will be exceptions where it will be better suited to mix and match from both philosophies. For example, many Agile projects still require a lot of documentation so you may have to tweak your Agile process and introduce some Waterfall principles to generate the documentation.

No matter if the project is considered an Agile or a Waterfall one, the main activities that are necessary to perform don’t change. What changes are “when” they are performed, and “how” they are performed. In summary there are 6 main activities in software development:

In the next 2 articles we will continue to look at the differences between Agile and Waterfall from the perspective of the 4 classic structural areas of software development: Business, Development, Management, and QA. Please stay tuned!! Same Bat-channel, same Bat-Time!

AttachmentSize
waterfallAgile.png56.6 KB
Published at DZone with permission of its author, Alberto Gutierrez.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Mike Bria replied on Wed, 2010/06/02 - 6:47pm

On the surface, I like your idea of contrasting Agile and Waterfall. In the details (as where the devil lives, eh), I'm not so enthused.

First, I'll challenge your opening remark:
There has been a lot of discussion lately regarding which approach to software development is better; some may say Waterfall, some may say Agile.


Is this true, in a general enough sense to state it this way? Do many people really still say "Waterfall is better"? I suppose this part is a bit of a rhetorical question, so no need to reply.

Moreover though...
Your basic premise states that agile is better for environments where speed is important, change is evident, and the product is the important deliverable. And, conversely, Waterfall is better for circumstances where the client wants to contract requirements up-front, heavy analysis is required before development, and/or documents about the software are more valued than the software itself.

What you state isn't false. What I question though, heavily, is what you don't state. Specifically, my question being: which environments truly should value the items in the waterfall-friendly list, and likely shouldn't be valuing the items in the agile-friendly list?

I'm pretty sure that implied in the manifesto is that "you, big bloated company, are likely overvaluing getting the requirements/analysis "right" up-front, and it's likely the cause of much of your dysfunction and failure". And "you, big document heavy process, don't you realize that change is the surest reality, to be expected not controlled." And that getting something to see and evaluate sooner than later is really the best way to manage the project's risk, let alone possibly make you quicker money.

In a nutshell, I worry that your post doesn't highlight the fact that much of what agile can do is break waterfall-heavy organizations out of their perceived chains. IE. better understanding their true value sources.

Agile doesn't say "no documents". It does challenge you to find out which ones are truly necessary. Same goes for the other items. I believe you imply this sentiment in your closing remarks, which I support.

Cheers,
MB

Alberto Gutierrez replied on Thu, 2010/06/03 - 5:29am

Hi Mike!

I think yours is an excellent point... 

What you state isn't false. What I question though, heavily, is what you don't state. Specifically, my question being: which environments truly should value the items in the waterfall-friendly list, and likely shouldn't be valuing the items in the agile-friendly list? 

So, I assume we agree in that there are projects wich are better suited for Agile, and projects that are better suited for Waterfal... But! How do we know? ... Yeah! That's the million dollar question. As you say:

"you, big bloated company, are likely overvaluing getting the requirements/analysis "right" up-front, and it's likely the cause of much of your dysfunction and failure". And "you, big document heavy process, don't you realize that change is the surest reality, to be expected not controlled."

In general, I would say most of the commercial applications are better suited for agile, with some exceptions, as I said, I don't think NASA is ever going to switch to Agile.

In my opinion, what's really important, is to realize that no matter what approach you pick, there are going to be areas in your project where you will still probably will have to switch your approach.

 

Thanks!

D Taylor replied on Thu, 2010/06/03 - 6:14pm

Great start!  I'll be very interested to see what you have to write in your next few articles.  I work in the government space and I would say that not only can Agile & Waterfall be friends, in my space, they HAVE to be most of the time.  Many companies (including mine) here inside the beltway use hybrid approaches, the most prevalent being Agile practice being iterative development, and within each interation you have a kind of mini-waterfall process, so you still have checkpoints/gate reviews, etc.. but gain the advantage of continuous feedback so you don't have customers unhappy with the end result by the time you get to the end.

David Bland replied on Fri, 2010/06/04 - 9:29am

In my humble opinion, it is quite simple (adapted from Eric Ries)

Waterfall = Known Problem / Mostly Known Solution

Agile = Known Problem / Mostly Unknown Solution

I say mostly because I feel as though we typically have some degree of certainty & uncertainty within all of our software development projects.

Agile makes the most since in web development where you have a rapidly changing ecostystem. Any company that is leveraging Waterfall for web development risks becoming obsolete in today's world. By the time you've finished revising your functional requirements document on your next big thing, others have released it and are learning from real world usage. Sure it may not have all of the bells and whistles, but if your solution is mostly unknown it doesn't make sense to pretend you know it. Besides the technology & user base has advanced enough so that you can literally continuously deploy code and measure business value metrics in real time. This approach lines up well with iterative development practices, as it doesn't make sense to batch up all of these learned metrics into a waterfall lifecycle (it'll be too late).

Waterfall makes sense when the solution is mostly known, and from my experience that lies outside the web development space in other areas such as biotech etc.

Attempts to blend Waterfall & Agile together generally loses quite a bit of effectiveness and spirit from both, and I've witnessed counterproductive things such as:

 

Analysis Sprint

Design Sprint

Coding Sprint

Testing Sprint

Launch & Cleanup Sprint

 

Throw in a command & control Project Manager as ScrumMaster, and you have the recipe for disaster.

 

I'd rather they not be friends, have them be acquaintences who step into a software development project when it makes the most sense...

 

-David

Loren Kratzke replied on Sat, 2010/06/05 - 1:23pm

I think that the most striking part of this article is the table of Waterfall and Agile. It clearly illustrates how perpendicular they are to each other.

It is clear to see the logical steps of waterfall, one can not test that which is not coded, one can not code that which is not designed, etc. It is clear to see the potential risks that change can incurr, and deadlines can be missed.

It is also clear to see the the absurdity of extreem Agile. QA has nothing to do at the begining of a sprint, programmers have nothing to do at the end of a sprint, yet everybody is somehow supposed to be busy during the whole sprint. Testing the previous sprints work in the next sprint is considered and antipattern in Agile. Yet there is Agile value in decomposing the project into smaller chunks giving business much better visibility and control compared to Waterfall.

Agile claims that it can adapt, but Agile coaches freak out when you explain that square pegs do not fit into round holes. You must deliver finished products at the end of every iteration, period. Otherwise you are not doing Agile.

Waterfall simply states that one foot must go before the other. It is not a Waterfall antipattern to create proofs of concept early in the process (in fact it is encouraged) or revise specifications late in the process when discoveries are made (a simple reality of any project).

My prediction for the future is that projects will transition between phases that incorporate different methodologies in each phase and perhaps even different blends of multiple methodologies. My concern is that the Agile community will not be receptive to this concept in the short term as it will be viewed as a concession, as if Agile is not perfect. Waterfall however is always looking to improve, and aside from aspiring to maintain a logical (i.e. sane) sequence of steps is more flexible than the greater Agile community is willing to admit.

Some may claim that I am not doing Agile right, otherwise why would I say this. Agile is good, but if all you have is a hammer then everything starts looking like nails. It is better to use all of the tools in your toolbox. The right tool for the job, I say.

Loren Kratzke replied on Sat, 2010/06/05 - 1:18pm in response to: David Bland

I differ with your assertion about the counterproductive sequence. I would prefer to work like this:

  • Analysis Sprint (produces functional documents)
  • Design Sprint (produces functional designs)
  • Code/Test Sprint (produces functional unit)
  • Code/Test Sprint (produces functional unit)
  • Code/Test Sprint (produces functional unit)
  • Launch and Cleanup Sprint (produces final integration and supports production rollout)

 
This is the perfect blend - the best of both worlds.

If you back off, you see Waterfall. If you drill down, you see Agile. I call it Wagile.

In order for those three functional units to work well together, they will require that a design be created in advance (frameworks, services, interfaces, etc). Just hoping that they will work together is not sufficient. The fatal flaw of Agile is that it attempts to swallow the entire project in every single iteration regardless of the project. You end up with parts that don't work together because nobody was ever given the opportunity to think everything through. The benefits of Agile over Waterfall are lost when this happens.

Juha Majuri replied on Mon, 2010/06/07 - 4:35am in response to: Loren Kratzke

I differ with your assertion about the counterproductive sequence. I would prefer to work like this:

  • Analysis Sprint (produces functional documents)
  • Design Sprint (produces functional designs)
  • Code/Test Sprint (produces functional unit)
  • Code/Test Sprint (produces functional unit)
  • Code/Test Sprint (produces functional unit)
  • Launch and Cleanup Sprint (produces final integration and supports production rollout)

 
This is the perfect blend - the best of both worlds.

 

I'm sorry, this is not agile development. This is iterative development, which has been utilized on waterfall before agile went to main stream. People seem to think that when they have sprints they're doing agile. Your sprints sound just like RUP.

Loren Kratzke replied on Mon, 2010/06/07 - 11:52am in response to: Juha Majuri

If I need to develop 5 new modules in 5 sprints that share a common service (a service that runs as a huge cluster), do I develop 1 module plus 1/5 of the service in each iteration? Is this even possible?

Each module needs the entire service, and the service must be capable of facilitating all 5 modules. I have three teams totalling roughly 20 developers (backend, service, and user interface). Do they all start coding on the same day? What are they coding against? Do they all finish on the same day and everything magically just works?

Admit it, extreem Agile does not answer these questions. It just says that I am not doing Agile if I try to design something before I turn my teams loose on coding a solution. This tells me that Agile simply does not work if you have a complex interrelated problem to solve, and an existing revenue stream that MUST NOT be disrupted.

By the way, I have millions of users. It is not an option to release piece meal. The whole thing must launch at one time and be precisely integrated with the existing site. There are hundreds of potential regressions to track. This is a real situtation, not an acedemic discussion for me. I remain unconvinced that Agile is good for anything beyond little atomic sized chunks of work that people choose to call projects.

If Agile requires a finished and shippable unit of functionality with every iteration, then I'm sorry, pure Agile sucks. Go build a house on a vacant lot using Agile. Start with the upstairs bathroom and then build it room by room. Be sure to paint and furnish each room as part of each iteration. Don't worry about electricity, sewage, or plumbing, it will just work. If you lay the foundation for the entire house first (or god forbid you make blueprints before starting) then you are not doing Agile.

If that's wrong, then I don't want to be right.

David Bland replied on Tue, 2010/06/08 - 11:40am

Loren,

Sounds like you've already decided on what you will use, but if you are curious:

- Continuous Integration is your friend. There are companies out there that deploy code every few minutes and manage when it is "turned on" based on what makes sense for their architecture. Flickr is a good example of this, and it pairs well with agile software development.

- Agile software development can be used for mission critical stuff, such as financial institutions and such. I know because I've personally worked on them, and they were also service based.

- Dependencies between User Stories can and should be identified.

I'm somewhat confused about the house example, since I've not seen anyone preaching that you build every little unit to the paint and finish & pray it fits together in the end.

Instead think of building a cake in slices rather than layers, as that seems more accurate a description in my opinion.

Good luck with your service deployment, it sounds important.

-David

Loren Kratzke replied on Tue, 2010/06/08 - 1:20pm in response to: David Bland

Continuous Integration - Agreed.

Agile - The part that bites me in the ass is having to deliver "potentially shippable" product at the end of each iteration. This conflicts with my desire to create documentation, architecture, analysis, service layers, and other things that Agile labels as "waste" because it has no "value" and is not "shippable". Logically, this is nonoptimal.

When you need a layer, you need a layer. Attempting to slice a layer into vertical slices is not wise.

Dependencies - We identify them, much like one identifies a train coming at you. Then we get hit.

Lastly, it is exactly like trying to build a cake in slices instead of layers. We should stop trying to slice everything just for the sake of slicing it.

Very frustrating, but I have learned alot by thinking everything through while trying to understand why software layers are somehow wrong when I know they are right. I have concluded that Agile is a luxury afforded to fortunate projects that slice easily into verticle (ideally identicle) slices. For these projects, Agile is the correct choice - instant gratification without casualty. For the other types of projects, not so much.

Stephane Vaucher replied on Thu, 2010/06/10 - 5:51pm

Waterfall is best suited for projects [...]

I'll stop you there. Pure waterfall has been abandoned by NASA. And if their projects do not qualify as document-driven, fully specified, then I don't know what projects are. Why do I state that they are not waterfall? because there is a separation between the development phase and its activities. Design activities can continue during the implementation phase. Actually, requirement gathering activities continue throughout the development process [1].

[1] SEL-81-305 (1991) : Recommended Approach to Software Development

Also, I agree with Juha, you seem to be proposing a meta-process like RUP. (I say meta because the specific activities for different phases are not defined by the process).

Loren Kratzke replied on Fri, 2010/06/11 - 2:16pm in response to: Stephane Vaucher

Two questions:

  1. When (if ever) do you create design specifications for a service layer (while remaining "Agile")? I remind you that a design specification is waste because it is not shippable by definition in Agile.
  2. How do you concurrently create service and client layers in vertically sliced sections if each slice of the client layer requires the entire service layer? Here I remind you that you must have shippable product at the end of every iteration.

I am not an advocate of "pure" anything. In fact "pure" anything is exactly what I oppose (be it Waterfall or Agile). To declare that "pure anything" is the only solution is to have tunnel vision. I am an advocate of phased dynamic hybrids that are actually (lower case) agile (e.g. flexible, adjustable, adaptable, shapable) by definition as opposed to being (upper case) Agile (which ironically has zero tolerance for deviation from pure Agile) by declaration.

In other words, each phase of the project should use the tools best suited for that phase with all factors considered. To say that a hammer is better than a screwdriver without consideration of the task at hand is the insanity that I am trying to address.

Stephane Vaucher replied on Sat, 2010/06/12 - 1:08pm

When (if ever) do you create design specifications for a service layer (while remaining "Agile")? I remind you that a design specification is waste because it is not shippable by definition in Agile.
Really, I guess we do not have the same definition of agile. You can do specification and modeling in agile. Sometimes, it is implicit, sometimes it isn't. Agile is a philosophy based on a manifesto, you need to look at specific methodologies. For example, in XP, you would define your specification as acceptance tests. This is an extreme case, but there are specifications produced with every methodology.
How do you concurrently create service and client layers in vertically sliced sections if each slice of the client layer requires the entire service layer? Here I remind you that you must have shippable product at the end of every iteration.
Shippable product does not necessarily mean a system that is in production. It means having a system you can use for early client-validation. That means you do not have to have a fully implemented service-layer. Focus on the slices of the client layer that the client prioritises, and implement the service-layer according to the functions that the client layer requires. If the client uses all possible server-side functionality, then use stubs. Ex: we do not access a real DB yet, but we use sample data to validate the approach. What I mentioned is how I would do things in an agile way.
The RUP way is iterative as well, so what I mentioned would probably hold, except the development is generally less incremental.
Using the Waterfall (or variations), you would need to define the communication first and then you would write the service/client code. Finally, you would verify the code, then you would verify the communication, and finally, you would check to see if the customer is happy. At that point, the client would validate what was produced. The Waterfall is an aberation that was created accidentally because of an article written by Royce.
Go and talk to a client, tell him that you will need a few months to do the specification, and he will see a product in year or two. Business-wise, a client might accept, but it is extremely risky for him unless you are a large development firm.

Andy Leung replied on Mon, 2010/06/14 - 7:36am

Hi,

I worked in a big well known American company before. The department that I worked in was using ASP.NET and their approach was Agile. However, I wasn't enjoying it very much. The reason was that everybody felt the rush and so requirements, design and testing were not done thoroughly. Coders were patching everywhere and eventually, 9/10 releases or updates during the year were bug fixing of code introduced in the beginning of the cycle.

I guess this is not related to Agile approach, it could be the management's mistake but I definitely didn't enjoy this approach.

Andy

Stephane Vaucher replied on Mon, 2010/06/14 - 9:50pm

@Andy Leung - Agile methodologies are feature-based rather than artifact-based (waterfall). For a agile process to be successful, IMO, developers need to be rigorous and follow the process rules. This is harded than in an artifact-based process. You didn't mention any of the components of your process like pair-programming or TDD. If your team did not follow any agile rules, then it was undisciplined cowboy programming.

Loren Kratzke replied on Tue, 2010/06/15 - 12:51pm in response to: Stephane Vaucher

Feature based development is not suitable for service layer projects. I don't understand why Agile fanatics just assume that everything will work out on its own. And your comment regarding acceptance tests representing a specification made me dizzy and I smacked my head on the desk when I passed out.

 What do you tell the DBA's when they ask how many transactions/second you require? How fast is your database going to grow in the next hour, day, year, decade? What are your hardware requirements? Do you have race conditions during replication? How are you handling these race conditions. Is your architecture compatible with legacy systems? BANG! I just smacked my head on the desk again.

...because you don't have an architecture. You have acceptance tests. And you insist on unit testing and pair program your way out of having a specification. Good luck with that.

Acceptance tests are not specifications. Period.

And another thing, why do Agile fanatics say things like, "If you are doing this or that then you are not doing Agile." as if Agile is what everybody should strive to do. And if you don't follow Agile rules then you are "undiciplined" and practicing "cowboy programming". It seems downright arrogant. I just don't think there is room in the Agile church for free thinking secular/infidels such as myself that see Agile for what it is - roughly 10 pounds of shit in a 5 pound sack.

I no longer think that waterfall and agile can be friends. I am throwing my hands in the air on this one.

Stephane Vaucher replied on Tue, 2010/06/15 - 11:35pm

Loren, you can vent if you wish, but I'm certainly not an agile fanatic.
What you have to realise that a few things about the majority of existing systems is that they lack a definable architecture and were developed in a haphazard way. If you've used a phone in the US, you've interacted with such systems. Furthermore, to avoid bad maintenance, telcos enforced strict rules prohibiting developers from against removing code from these systems. That is the result of good ol' cowboy programing. My question to Andy was to clarify if his team was following a process or not. A problem with agile is that many people say they do agile but are in fact doing the cowboy thing. It's like people doing waterfall when they say they are doing RUP. That is why I wanted clarification.
IMHO, Agile has helped development as a whole by emphasizing the importance of testing. That being said, agile has its share of problems with its integration in an existing organisation. What do you do with traditional roles? If you have a dedicated IVV team, what do you have them do? IMO, these are major issues with scaling agile. Another is, as you said, that agile methodologies tend to focus on the construction of software and can ignore other activities (that don't provide functionality/value).
All this to say, choose the process that floats your boat. If you need tight integration in an large traditional infrastructure that is, then agile might be hard to use. You could consider RUP instead, but avoid the waterfall unless it is imposed by the industry.

Loren Kratzke replied on Mon, 2010/06/28 - 2:33pm

One could argue that dividing everything into vertical slices for the sake of doing so results in "cowboy" programming. No forsight, no hindsight, just a continuous stream of vertical slices, each independent of the others. Like cowboys in the wild west.

Regarding telcos, one could not build that system "from the ground up" in vertical slices. It is highly stratified in nature. Not possible. Perhaps that is why they did what you said they did, because everything would break with one bad line of code. That is critical risk management of a highly stratified infrastructure composed of many interrelated systems from different vendors and even different countries, not cowboy programming. Agile is not the answer there either.

William Martinez replied on Wed, 2010/07/14 - 6:53pm

Hello. I just want to post a link to my take on Waterfall and Agile. The Agile missing point and the Waterfall Illusion.

Basically, Waterfall was a faulty methodology used to describe a problem. Agile may not be solving the problem but helping a little. The post describes the problem is actually the distance between design phase and testing phase, not about being phase related at all.

 

Cheers.

Loren Kratzke replied on Wed, 2010/07/14 - 9:32pm in response to: William Martinez

I enjoyed your post, William. I had several epiphanies while reading it.

  1. How did Agile "evangelists" make the leap from the mellow Agile Manifesto to the extreme Agile "processes" that we see in place today?
  2. When in the history of humans has Agile-like practices ever been successful? In this regard, it seems much like the hippie free love and hard drugs movement of the sixties. If it worked, everybody would have already be doing it for hundreds or thousands of years. In this respect I consider extreme Agile as the decay of society, resting on the laurels of Moore's Law.
  3. The Agile Manifesto is nothing but false choices. Personally, I value big muscles over working out. That doesn't give me big muscles. All of the points in the manifesto read like that to me. Especially resonding to change over following a plan. Any half wit knows that most battle plans do not survive first contact with the enemy, duh. Of course you need to respond to change, but valuing such over the other is simply pointless at best, ignorant at worst. It's a false choice in every respect.
  4. TDD really ticks me off.

Agile has value, but so does hot sauce. Best used sparingly.

Alan Dayley replied on Fri, 2010/07/16 - 4:21pm

Loren, you have quite a chip on your shoulder.

1. I'll answer in two parts here:

First, Agile "evangelists" have been pushing against a culture of rejection and aggressive undermining of their goals for a long time. Most of this resistance is not malicious, though it is pervasive. This can have an effect of hardening the resolve of the Agile people to push harder. A human reaction.

Second, the Agile Manifesto does not define specific processes. And just as with someone new to waterfall or any other framework does not fully understand at first, everyone does not have the ability to live principles first. We need steps and guides until the principles become part of our actions. Thus, the various Agile frameworks and practices help people act like they understand the manifesto until they really do understand. Along with this, a great many people don't want to learn principles and make choices, only wishing to be told what to do. These people also need to be brought along if the organization is to progress toward Agile practices. So these practices must be enforced, maybe even strongly, so that the benefits of them can be realized. Or, at least so that an experienced choice to abandon them can be made. So many people use the terms of Scrum or XP without actually changing the way they do things and then declare Agile a failure. This is not right. If it is to fail, let it fail in truth, not in name.

2. Successful human use of Agile-like practices: directing a battle, parenting, scientific research, invention, natural disaster response, crime investigation, a four-year-old building with wooden blocks, etc. Should I go on?

Moore's Law does not apply to humans. Smarter humans are needed to make better software at a faster pace. Creating software faster and better cannot depend on Moore's Law because that surplus of power in chips only goes so far to enhance the ability of humans. Agile is, in part, a reaction to this lack of surplus in "human processing power." We have learned that scaling humans does not work and more GHz and RAM does not solve the problems of bad software. Agile does not rest on Moore's Law, bloated software projects do!

3. These choices are real and happen all the time. For example, the traditional software development world chooses following a plan over responding to change all the time. At one company I worked the most expensive meeting ever was the Change Control Board. Six executives and at least that many engineers or other functional people spending three hours a week trying to prevent and control change. Most places spend weeks or months making a plan and then spend vast amounts of energy enforcing compliance to the plan. This one and the other manifesto values are choices, made every day.

4. TDD "ticks [you] off." A development practice makes you angry. I don't have a response to that. Don't use TDD, then. And hope your competitors don't either.

Stephane Vaucher replied on Sun, 2010/07/18 - 1:01am

At least, we can safely assert that no one believes that waterfall and agile can co-exist :p People tend to go with what works for them based on their experiences. Loren has obviously had a bad experience with agile development (probably bad coaches, ref 2010/06/05). If what he states as "obvious" agile practices are what a coach told him, then he had a bad coach. Honestly, I could only laugh when he said that he considers Agile as the decay of society... Some people should lighten up. You can't have a discussion when confronted by an emotional response.

Loren Kratzke replied on Sat, 2010/07/24 - 1:24am in response to: Alan Dayley

4. Not TDD specifically - evangilism of TDD ticks me off. It is one tiny tool in a giant box.

3. Agile is neither the answer nor the cure for change control. The wages of six executives for 3 hours per week should in fact pale in comparison to the damage that can be caused in their absence. Exactly how large of a revenue stream were they looking after?

2.  Various points: None of these things are Agile specific. Do you have values in a battle? Does not waterfall require planning and patience? Research is key to success in general. Everything was invented before Agile. Shit happens. Figuring out what happened. A shitty design (key word - design). No, don't go on.

My reference to Moore's Law was vague and you missed it. Business has been wondering why productivity has not increased with the advent of computers (think of stuff we built before computers if you are that old, no offense intended). My answer is simple. Business pocketed the difference - a different business - IT. Not everybody realizes this. And hell, the Boy Scouts have said "always be prepared"since 1910. Actually "being" agile is not a new invention.Putting a capital A in front does not change this.

1. Well spoken. Your post in general is very well worded. But you are making leaps from unstable platforms. I have done, and am doing Agile but unless I am blind, I am doing a process. Unless I am blind, I have deadlines. Unless I am blind, the dollar rules the day. Agile results in nothing more than another process that I consider nothing more than a stepping stone. It is healthy for an old school business to bite the bullet on Agile. But Agile is a lesson, not a destination in my opinion. A good lesson, but a bad destination. How can you tell when there is really a fire when the fire alarm is on all the time? Give it time and you will see Agile rot set in in the form of sprint depth techinal designs running into their golden years (or months as the case may be). May time prove me wrong. May time prove me wrong...

Loren Kratzke replied on Sat, 2010/07/24 - 1:29am in response to: Stephane Vaucher

Perhaps decay of society was a little over the top. But again it was a vague reference to money going from one pocket into another. There are big bucks in convincing the right people that you can add a few points to the bottom line. And I see Agile evangelists as snake oil salesmen in this regard. As the old saying goes, if you're not part of the solution, then you are a consultant.

Loren Kratzke replied on Sat, 2010/07/24 - 2:03am in response to: Loren Kratzke

Oddly I would like to reiterate that I actually like Agile, just not "100% Agile".

Jonathan Bunting replied on Tue, 2010/07/27 - 12:48pm

I wonder how many of you guys have been on the receiving end when a manager decides [s]he has to implement Agile or Waterfall or anything similar. I'm a developer in a small group of 7. We often work on crash projects - something has to be done fast and on time, it has to work because we have large customer base, and most importantly - it _has_ to be one step ahead of the competition.

When working on a project we have to work tightly together. I'm doing something, I get stuck at some place because, let say, I'm missing a piece that somebody else is working on and this got delayed for some reason. Or maybe I finished my work early. I give this guy a call, let him know I'm waiting and I switch to some other task, knowing that the other guy is meanwhile working on what I need. Yes, there is a plan what is going to be done and when, but this plan changes very dynamically - sometimes several times each day. When any of us works on a feature or a new concept, there is no telling in advance _exactly_ how much time is it going to take. Unless the task is extremely routine and does not require any innovation (just dumb coding) a developer or architect _has_ to come up with this flash of insight - it may happen immediately, it may happen after a couple of days.

One example: We were working on something (I can't disclose what it is, sorry) that had to do with recognition of the shape of a complex curve. There is no known standard algorithm for that. There is no library to just get and integrate. Nobody knows if it is possible at all. If we go with any of these strict capital-letter strategies, we could not plan on including this feature at all, but it's a killer feature, really. So, while I'm working on something else, I'm thinking about this. Coming with an idea, switching for half an hour or so to this, trying - nope, doesn't work this way. Continuing working on something else. Suddenly - Aha! - a flash of insight. The new feature is there. In the next day I wrap it with convenient API and other developers start integrating it, with the condition that if there is an issue with it it should be easy to turn off. I tell QA to start testing it and they starting doing so immediately.

Issues keep coming and are resolved as they come. People switch tasks many times a day. The whole project is done in less than half the time planned and with a couple of competition killers in it.

Now, once my manager decided to go Agile on us. Sure, it worked. Everything was very nice and planned and orderly and timely. The disadvantage - maybe we got 1/5th of the work we would have done without being Agile. Everything that was unconventional was pushed away - there is no place for "if"-s and "possibly this is doable"-s with Agile. Things like this got discarded in the early planning phases. Agile is not very agile when it comes to that. With Agile when I'm done with something that was expected to take a week, but somehow (because I had a really great day) was finished in a day and a half, what I'm doing the next 3.5 days? Pretending to work. That's what. And believe me, that's tougher than it sound - not because you get especially tired, but because you think what I could have been doing with all this time instead of sitting bored and waiting for the next scheduled meeting.

For me, from the perspective of a lowly programmer, Agile is a toy for the management to play with. It works _if_ the task at hand is dumb designing (i.e. no unknowns of the type "is it possible at all"), dumb programming (i.e. just coding something that is already decided and when the coding does not require any thought at all), and dumb testing - you know exactly what the features will be and QA can make rigid plans months in advance. Maybe this is the only way to work if the team is huge. Don't know - my experience is with development teams of up to 10 people. It may work if the team members are not motivated and not especially competent, but otherwise I don't see any reason whatsoever to have all this rigid structure imposed from the top.

Stephane Vaucher replied on Tue, 2010/08/03 - 11:11am

@Loren - I understand where you are coming from. Many consultants try to sell a packaged solution. I can easily imagine how you might have had a bad experience. Of course, most bad process-level consultant will probably cause chaos in a dev team. If you add the fact that agile is significantly different from traditional processes, yeah, I can understand the decay comment.

@Jonathan - Your manager is not promoting agile. Self-managing teams is at the core of an agile process. A comment like "Things like this got discarded in the early planning phases" indicates that something is odd. Agile was created by programmers, not by managers. These programmers encourage challenges instead of discouraging them. Its a question motivating the team. Motivation is a core agile principle. Your initial situation actually sounds more agile than the "agile" one

Mike Dobbles replied on Fri, 2010/10/29 - 8:20pm

Alberto, you mention NASA projects as something that will and likely should never go to Agile development. A lot of times people say the same about medical device projects. I disagree completely and I argue as much in my post "Can and should agile be used for medical device development? Absolutely!". The problem with waterfall even on these mission critical projects, is that it is based on the typically false premise that people can write decent specs at the beginning of a project. I describe the beginning of a project as the point of maximum ignorance.

Liezel Jandayan replied on Tue, 2012/05/22 - 9:47pm

Does this mean we can be certain there are more Java developers than there were a decade ago? Well, it could just reflect the immense changes in the Internet and in search engines in the past decade.-Steven Delarge

Matt Trousdale replied on Fri, 2014/08/01 - 4:09am

 I have been developing software for 16 years and over those years worked solidly in both. The biggest difference I've seen (in simple term) is the upfront costs are much larger with more risk for waterfall. However it's a bit unfair to hold this up alone to the light. As I am noticing more and more - with Agile systems and continued sprinting the architectural decay "can be" much faster with Agile. I believe this comes from shifting, in a lot of cases, the focus wholesale from tech to biz. By this I mean the business keeps pressing changes (feature changes) while the developers keep taking short cuts to get showcases working in time allotted.

This has happened in a few places I've worked. Is the answer to stop revisit design in a big way and refactor, telling the business they need to wait for more showcases? Does this then lead to risky redesign? Is this then more like waterfall? Is this where they end up meeting sometime in the future?

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.