Agile Zone is brought to you in partnership with:

Ted Neward is the Principal at Neward & Associates, a developer services company. He consults, mentors, writes and speaks worldwide on a variety of subjects, including Java, .NET, XML services, programming languages, and virtual machine/execution engine environments. He resides in the Pacific Northwest. Ted is a DZone MVB and is not an employee of DZone and has posted 50 posts at DZone. You can read more from them at their website. View Full User Profile

"Agile is treating the symptoms, not the disease"

12.15.2009
| 9642 views |
  • submit to reddit

The above quote was tossed off by Billy Hollis at the patterns&practices Summit in Redmond. I passed the quote out to the Twitter masses, along with my +1, and predictably, the comments started coming in shortly thereafter. Rather than limit the thoughts to the 120 or so characters that Twitter limits us to, I thought this subject deserved some greater expansion.

But before I do, let me try (badly) to paraphrase the lightning talk that Billy gave here, which sets context for the discussion:

  • Keeping track of all the stuff Microsoft is releasing is hard work: LINQ, EF, Silverlight, ASP.NET MVC, Enterprise Library, Azure, Prism, Sparkle, MEF, WCF, WF, WPF, InfoCard, CardSpace, the list goes on and on, and frankly, nobody (and I mean nobody) can track it all.
  • Microsoft released all this stuff because they were chasing the "enterprise" part of the developer/business curve, as opposed to the "long tail" part of the curve that they used to chase down. They did this because they believed that this was good business practice—like banks, "enterprises are where the money is". (If you're not familiar with this curve, imagine a graph with a single curve asymptotically reaching for both axes, where Y is the number of developers on the project, and X is the number of projects. What you get is a curve of a few high-developer-population projects on the left, to a large number of projects with just 1 or 2 developers. This right-hand portion of the curve is known as "the long tail" of the software industry.)
  • A lot of software written back in the 90's was written by 1 or 2 guys working for just a few months to slam something out and see if it was useful. What chances do those kinds of projects have today? What tools would you use to build them?
  • The problem is the complexity of the tools we have available to us today preclude that kind of software development.
  • Agile doesn't solve this problem—the agile movement suggests that we have to create story cards, we have to build unit tests, we have to have a continuous integration server, we have to have standup meetings every day, .... In short, particularly among the agile evangelists (by which we really mean zealots), if you aren't doing a full agile process, you are simply failing. (If this is true, how on earth did all those thousands of applications written in FoxPro or Access ever manage to succeed? –-Me) At one point, an agilist said point-blank, "If you don't do agile, what happens when your project reaches a thousand users?" As Billy put it, "Think about that for a second: This agile guy is threatening us with success."
  • Agile is for managing complexity. What we need is to recognize that there is a place for outright simplicity instead.

By the way, let me say this out loud: if you have not heard Billy Hollis speak, you should. Even if you're a Java or Ruby developer, you should listen to what he has to say. He's been developing software for a long time, has seen a lot of these technology-industry trends come and go, and even if you disagree with him, you need to listen to him.

Let me rephrase Billy's talk this way:

Where is this decade's Access?

It may seem like a snarky and trolling question, but think about it for a moment: for a decade or so, I was brought into project after project that was designed to essentially rebuild/rearchitect the Access database created by one of the department's more tech-savvy employees into something that could scale beyond just the department.

(Actually, in about half of them, the goal wasn't even to scale it up, it was just to put it on the web. It was only in the subsequent meetings and discussions that the issues of scale came up, and if my memory is accurate, I was the one who raised those issues, not the customer. I wonder now, looking back at it, if that was pure gold-plating on my part.)

Others, including many people I care about (Rod Paddock, Markus Eggers, Ken Levy, Cathi Gero, for starters) made a healthy living off of building "line of business" applications in FoxPro, which Microsoft has now officially shut down. For those who did Office applications, Visual Basic for Applications has now been officially deprecated in favor of VSTO (Visual Studio Tools for Office), a set of libraries that are available for use by any .NET application language, and of course classic Visual Basic itself has been "brought into the fold" by making it a fully-fledged object-oriented language complete with XML literals and LINQ query capabilities.

Which means, if somebody working for a small school district in western Pennsylvania wants to build a simple application for tracking students' attendance (rather than tracking it on paper anymore), what do they do?

Bruce Tate alluded to this in his Beyond Java, based on the realization that the Java space was no better—to bring a college/university student up to speed on all the necessary technologies required of a "productive" Java developer, he calculated at least five or six weeks of training was required. And that's not a bad estimate, and might even be a bit on the shortened side. You can maybe get away with less if they're joining a team which collectively has these skills distributed across the entire team, but if we're talking about a standalone developer who's going to be building software by himself, it's a pretty impressive list. Here's my back-of-the-envelope calculations:

  • Week one: Java language. (Nobody ever comes out of college knowing all the Java language they need.)
  • Week two: Java virtual machine: threading/concurrency, ClassLoaders, Serialization, RMI, XML parsing, reference types (weak, soft, phantom).
  • Week three: Infrastructure: Ant, JUnit, continuous integration, Spring.
  • Week four: Data access: JDBC, Hibernate. (Yes, I think you need a full week on Hibernate to be able to use it effectively.)
  • Week five: Web: HTTP, HTML, servlets, filters, servlet context and listeners, JSP, model-view-controller, and probably some Ajax to boot.

I could go on (seriously! no JMS? no REST? no Web services?), but you get the point. And lest the .NET community start feeling complacent, put together a similar list for the standalone .NET developer, and you'll come out to something pretty equivalent. (Just look at the Pluralsight list of courses—name the one course you would give that college kid to bring him up to speed. Stumped? Don't feel bad—I can't, either. And it's not them—pick on any of the training companies.)

Now throw agile into that mix: how does an agile process reduce the complexity load? And the answer, of course, is that it doesn't—it simply tries to muddle through as best it can, by doing all of the things that developers need to be doing: gathering as much feedback from every corner of their world as they can, through tests, customer interaction, and frequent releases. All of which is good. I'm not here to suggest that we should all give up agile and immediately go back to waterfall and Big Design Up Front. Anybody who uses Billy's quote as a sound bite to suggest that is a subversive and a terrorist and should have their arguments refuted with extreme prejudice.

But agile is not going to reduce the technology complexity load, which is the root cause of the problem.

Or, perhaps, let me ask it this way: your 16-year-old wants to build a system to track the cards in his Magic deck. What language do you teach him?

We are in desperate need of simplicity in this industry. Whoever gets that, and gets it right, defines the "Next Big Thing".

Published at DZone with permission of Ted Neward, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Christophe Hanon replied on Tue, 2009/12/15 - 11:10am

Simply said: you right! We simply continue adding layers of software, architectures and methodologies and as we add these layers we build a Software Pisa Tower Project...

Scott Hickey replied on Tue, 2009/12/15 - 3:09pm

Five to six weeks isn't realistic in my experience. Starting from scratch, I think it takes several months for a the typical developer (whatever that means), fresh out of school, new to all the technologies you mentioned to become mildly productive. For the typical legacy corporate developer, the learning curve is closer to six months in my experience. Unless you have an exceptional learner, when you try to show someone all of those topics at once,  their head explodes.

I'm teaching my 9 year old and 11 year old PLT-Scheme.

Josh Marotti replied on Tue, 2009/12/15 - 4:35pm

"A lot of software written back in the 90's was written by 1 or 2 guys working for just a few months to slam something out and see if it was useful. What chances do those kinds of projects have today? What tools would you use to build them?"

 

First of all, that very situation IS agile.

When I teach Scrum, I use this story:

The best softare is made by a 2 person company.  The boss, and me, the IT department.  He comes up with an idea, I start coding, he checks in on that idea, makes changes, I code the changes, and the cycle completes and he sells the final product.  When we have success and scale to 1000 employees, you put a waterfall process, a giant PMO, and countless middle management between me and the customer (aka 'the boss').  The idea of Scrum (and agile, for that matter) is to get that out of the way, or at least emulate it as if we are just 1 or 2 developers working for the boss.

 

So your/Billy Hollis's statement is actually asking where the agile is ;)

Having said that, those projects exist at google.  People making software off the side of their desk has become: GMail, Crome, CromeOS, etc...  I'll admit that agile isn't needed 100% of the time, but please don't apply waterfall and failed processes to those 'access/foxpro/gmail' projects, either!

 

Abdul Habra replied on Tue, 2009/12/15 - 4:40pm

It is true that the traditional Java stack wil take weeks (if not months) to master. But we do have alternatives specially for web development: rails and grails come to mind. Learning Ruby/Rails or Groovy/Grails will take significantly less time.

Now for teaching programming to young kids, I found Scratch to be very helpful.

David Noble replied on Tue, 2009/12/15 - 6:11pm

These are two different questions:
  1. "Where is this decade's Access?"
  2. "What language do you teach him?"
There are plenty of platforms on the web: Zoho, DabbleDB, and RollBase to name a few. None of them have established complete dominance of the marketplace, but they seem useful and effective.

I agree that the software world could benefit from more simplicity and elegance. Rails and the "Getting Real" book have had an influence throughout the web application landscape. Spring and Hibernate (among others) have helped to nudge the enterprise Java standards away from a precipice. But we do need to continuously oppose the tide of overcomplexification.

Rainer Eschen replied on Wed, 2009/12/16 - 3:23am

I'm sorry, but this is comparing apples and oranges. You start to have a look at the tools and blame the process that is does not reduce the complexity of the tools. Do it the other way around: define a useful agile process and select suitable tools for it. I don't know if Access has the quality today to create rock-solid solutions. But, looking back a decade ago I can't recognize this. A lot of those solutions (the same with Excel) where rewritten with tools that scale better and have a better maintenance. Your "Decade's Access" demand doesn't work in context to agile. Agile processes are useful if you have to build big solutions. If you talk about agile and enterprise level have a look at Oracle, SQL Server and the like ;-).

Scot Mcphee replied on Wed, 2009/12/16 - 3:41am

First thing, I think it takes years - not weeks or months - to learn Java. To master it ... well at least 3 years I might even say 5.

Second thing, I agree with Rainer, in that the tool platform and technology is a different thing to that of the methodology. Although they exist in symbioisis, agile addresses organisational issues, which are a different class of problem from the technology platforms.

Also I think that technologies like PHP, or ruby,  and things like Content Management Systems (the user's view thereof) often take the place of the 80s and 90s foxpro and access style technology. And let's not forget Excel and other spreadsheets. I use Excel or Apple's Numbers for quick and dirty personal numerical applications and even sometimes for simple lists.

Alex(JAlexoid) ... replied on Wed, 2009/12/16 - 8:46am in response to: Rainer Eschen

 

 I don't know if Access has the quality today to create rock-solid solutions. But, looking back a decade ago I can't recognize this. A lot of those solutions (the same with Excel) where rewritten with tools that scale better and have a better maintenance. Your "Decade's Access" demand doesn't work in context to agile.

 Yet, the whole point of the article is about fast development of usable applications. Where Access absolutely shines, even by todays standards.

And what is this "rock-solid" thing you are talking about? Because this looks like the mumbo-jumbo architects and salespeople like to throw at clueless clients. There is no such thing as rock-solid.

The fact that those were rewritten, just shows that those were good solutions, but the technology changes made them inadequate.

Jan Dockx replied on Wed, 2009/12/16 - 8:55am

I largely agree. To answer the questions: today's Access is WaveMaker, Sharepoint (lists!) and Confluence.

Frank Silbermann replied on Wed, 2009/12/16 - 12:10pm

The web is a kludge that was not designed for applications; that's why we have no tool simplicity. There are ways to shrink the necessary tool stack while maintaining scaleability, but only with nontraditional approaches to web development. That is because embedding logic into HTML is inherently not scaleable. Thinking along that line, if you want to use Java with HTML/CSS while keeping it simple, you can use just Wicket to replace XML, Servlets, JSP, JSTL, custom tag API, Struts, and Java Expression Language. (On the back end, you can be as simple as you please; if you don't need EJB or Hibernate/Spring you can use simple JDBC, which is part of standard Java.)

Bob Lauer replied on Thu, 2010/06/17 - 7:48am

We should not blame agile followers for creating complexity. After all, according to the agile old testament (the "agile manifesto")
Simplicity--the art of maximizing the amount of work not done--is essential.
The problem is all this Amblerism out there - folks that think that calling yourself agile *makes* you agile.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.