Agile Zone is brought to you in partnership with:

Ilya Sterin is a software engineer with Nextrials, a clinical trials data management software company and also consults for a variety of startups, specifically dealing with scalable and distributed system architectures. Ilya’s also a book and blog author and avid user and contributor to open source software. When not hacking on yet another software project, Ilya enjoys spending his time coaching his 10 year-old son’s ever growing amount of sports teams. Ilya is a DZone MVB and is not an employee of DZone and has posted 5 posts at DZone. View Full User Profile

Agile Intifada

09.19.2010
| 7326 views |
  • submit to reddit

This blog post was inspired by a link my friend sent me titled “Agile Ruined My Life” as well as a conversation with my friends/co-workers and just wanting to clear up a few things about my previous “Agile Dilemma” post.

Some of this is taken verbatim from an email exchange between myself and a few friends I greatly respect.

I agree/like agile in theory and practice. I agree and like TDD in theory and practice. Anything that makes it mainstream can be scathed and unfairly criticized. So now that I have this out of the way, and I will clarify it later, the rest of the post won’t be so nice for some. (WARNING: It will sound a bit harsh to some ears, with the hope of being constructive.) With that I wish that if nothing else, the few that read this blog post will reflect upon it and critique constructively.

I wrote the Agile Dilemma post mostly while I was letting off steam about computer science itself becoming extinct in lieu of a bunch of consultants pushing agile and TDD 90% of the time without teaching people how to actually design and write good software. I think lots of failures in our industry are due to the fact that most folks don’t know anything about programming and computer science, they just know a language or few and eventually learn how to express some intention utilizing these languages. That’s like comparing a native speaker or someone who’s learned and practiced the art of a foreign language for years, with someone who’s learned how to ask for directions after listening to RosettaStone audios. From a business perspective most pointy-haired bosses don’t give a crap. It works and the business development teams can do their “magic”, but from a perspective of a computer scientist (the real software developer) it stinks. It smell of amateur manure (no matter how many tests your write to prove that the action button works).

Now, there are those that go even as far as saying that non-TDD written code is “stone age” or that programmers not practicing agile, TDD, pair-programming and all the other process crap are not “good” programmers or shouldn’t be programming. Well, then shut down your linux/unix operating systems, stop using emacs or most other editors, as a matter of fact, stop using 90% of the stuff that you’re currently using, because most of it was written by these “stone age” programmers who didn’t give a crap about formalities of agile or TDD, but created masterpieces because they were smart, motivated, and knew how to program with common sense.

Before I get into detail, I’d love for people to stop petitioning for turning opinions formed by so called “experts”, into commandments. Just because Martin Fowler, Robert Martin, or name your own Agile Mythology Deity say it is so, doesn’t mean much more than anyone else in the field with experience. They are human just like me and you and form their own opinions just like me and you. The only thing they are better than the average at, is marketing. Yes, they are sales folks with large investments in agile, TDD, etc… in their consulting business, so I don’t think I go overboard by saying that they are a bit biased. Spreading FUD brings them more business. As one of the above links pointed out, Peter Norvig, Linus Torvalds, and hundreds of other brilliant programmers (far more brilliant individually than Fowler and Martin combined), have been programming successfully for decades not using any formal methodologies and techniques and not following TDD and have succeeded beyond their wildest dreams. I’ll take one Norvig over 50 Fowlers any day.

Writing tests is not innovative, it’s been around for decades. It was just never formalized. Yes, there weren’t as many tests written or sometimes none, programmers actually got to decide whether a test was needed or not. Programmers are sometimes overly optimistic, so yes, mistakes were made, code had bugs, stuff was rewritten. Decades later, TDD and Agile at hand, mistakes are made, code still has bugs, stuff still gets rewritten. Now put that on your t-shirt along with your favorite TDD blurb.

So I had an interview about two years ago, where I was asked two questions that after reading this post, folks should immediately take off their interview questionnaire if they ever want to hire someone good. The first question, wasn’t as bad, but it wanted me to recite some design patterns. I’m all too well familiar with those, probably more than I want to as this point, but familiarity with patterns doesn’t in any way exclude anyone from the “good” programmer group. The second question was the worst, they wanted to know how many lines of test code I write per week. I had to pause and then ask them to ask again, as I was puzzled. WTF does that mean? I don’t know how many lines of non-test code I write per week, you want me to count/average the amount of tests I write? I mean, I see if the question was if I practice TDD, but lines of f***ing test code? Either way, just to clarify that I’m not bitter and that’s why I’m mentioning this, the interview went very well, we just couldn’t come to terms on salary.

Agile is also not innovative. I’d like to think of agile as an abstract set of empirical ideas (patterns), with implementations left to the people/companies. Lately though, because there is really not much more one can write about the few agile principles, most literature about agile is about concrete agile practices through author’s experiences. These are all great reads only if people would read and take them for what they are “experiences”. We should learn from experiences, not try to recreate them. Agile approach is common sense that has been practiced for decades in different circles. Iterations are common sense, they were practiced for as long as programming existed, tests are also common sense, actually most of agile is just good common sense approaches to building products. Formalizing it actually helped quite a bit, as the industry was able to reason about it and transfer the empirical knowledge to others with less experience. But then, as with anything mainstream, the bureaucrats took over, and it’s IMO been down the hill ever since. Actually, some recent attempts at quantifying agile’s success have failed to show it to be any better (within a margin of error) than any other process or no process at all. That’s not to say that it’s not successful, it’s been very successful for many, including me and the companies I worked with. The failures that folks are seeing are due to many reasons, but partly because in most companies agile is practiced as a bureaucratic rule of thumb, without any common sense. Folks are forced to write comprehensive test suites. The “comprehensive” is something inferred and quantified by quality control tools, that apply pointless heuristics. But managers love it, it gives them numbers and pie charts. So programmers (even the ones that love programming as more than just a day job), do what ever it takes to keep the job, they write comprehensive tests and crap software. At the end of the day, who gives a crap if your algorithm is exponential complexity and mine is logarithmic, the tests pass and the sales can go on. But what about folks that actually love what they do (computer science)?

So let’s get back to basics. Learn computer science, algorithms, data structures, language theory and practice. You’re never done learning. Go out and create your own masterpieces. Don’t let the agile Deities full you into thinking that your software isn’t worthy if you didn’t pass a TDD heuristic or if you don’t hold daily standup meetings. No one knows you and your team better than you. DO WHAT MAKES SENSE FOR YOU AND YOUR TEAM!

DISCLAIMER: No agile enthusiasts were harmed in this experiment, including myself.

Ilya blogs at: http://ilyasterin.com
Published at DZone with permission of Ilya Sterin, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Dalibor Ružić replied on Sun, 2010/09/19 - 5:03am

Greatest programmesrs do the greatest program without much formality and process,for the rest of us some methodoly helps. :)

Alexandru Repede replied on Sun, 2010/09/19 - 6:41am

first things first, from what i have learned and read, agile is not a religion and you don't have to follow it by the book. you have to take from the agile practices what best suites you.

this means that if you don't like something or just don't want it or it screws up your work more then it helps you, then don't embrace that practice. no one forces you to do that. the manifesto says something like "people over tools", right ?

i personally do like agile and many of their practices, but because each of them gives me leverage over something: tests lets me modify code know exactly what i broke; this allow me to refactor (it opens the refactoring door much more easy); once the refactoring door is open, i can clean stuff and redesign it (and the list goes on)

in my opinion, a methodology is better than no methodology; just make it flexible

second, agile doesn't stop you from learning design patterns, design principles... it doesn't really have an explicit practice for that, true.

but doing test for things will get you wondering "did i write this class right? is there a better way to do it? cause sure as hell, it seems hard to test and think about what it's supposed to do"

it would be nice indeed if there was something explicit about your learning "procedure" in an agile team. so far, there are some practices ("let's do workshops and presentations"), but those seems a bit too abstract for me. no one is leading the direction in those practices

referring to what you said in the end, maybe a personal project should be an agile practice (with feedback about it during the retrospective meeting; you know, to share knowledge with your colleagues; it would be like the rss feeds, but live and from your team).

John J. Franey replied on Sun, 2010/09/19 - 9:49am

Might be something interesting here, but I get distracted by the gratuitous 'f-bombs'.

Alessandro Santini replied on Sun, 2010/09/19 - 10:08am

Ilya, I tend to concur with John - there might be a few valid points in your reasoning, but the "jargon" was way unsuitable.

Having said that, I think that Agile is no different from other methodologies in that they offer a framework of roles, processes and deliverables; it is up to the practitioner to tailor it to the specific context.

When you say it is just common sense - again - you are right - people were building bridges and towers without knowing the static equations. Nevertheless, many of those bridges are still there; many others have collapsed, despite using the best modern methodologies.

Formalization helps conveying information quicker and more efficiently; how easier is to say "use a visitor" instead of explaining the whole thing all the time? Methodologies also encompass sets of best practices (again, common sense) that are proven to work together.

Methodologies won't guarantee a project to succeed, but they will certainly contribute.

Thomas Eichberger replied on Sun, 2010/09/19 - 3:39pm

Thanks for this blog post, I love it:-)

Tim Boudreau replied on Sun, 2010/09/19 - 10:26pm

Amen. I'm forever annoyed by the idea that making software should be like making ketchup, and if we could only find the right methodology (an oxymoron), then it would be.

I am, however, in favor of one methodology. I call it the hire smart people methodology. It's worked every place I've seen it tried.

Balázs Bessenyei replied on Sun, 2010/09/19 - 11:09pm

Thanks for the article. This one just got me to register.

han boon kiat replied on Mon, 2010/09/20 - 6:58am

i used to work with this moron who did some lame youtube knockoff

called his project agile and has been going around bragging about great an agile practitioner he is

 

agile is a good ideal to work towards, it shouldn't be some kind of badge equating to software quality

scrum, TDD, XP, Clear etc... may be 'thicker' methodologies that can fit your organization

most of what agile is saying : test first, continous, people>process, collab. with customer, flexi-to-change

few will challenge these are beneficial, following very naturally in the scheme of things of present IT landscape

and can be adopted in simple form, in parts, and with ease

 

it only takes the opening of mind, and willingness to forsake past invested stubborn ways of thinking

 

John J. Franey replied on Mon, 2010/09/20 - 7:52am

In general, I'm with you, but I have a three nits.

Thomas Edison said: "Don't learn from your own mistakes, learn from others'." I think that is a driving principal for developers. Find other peoples' mistakes and try to avoid them. Find people who seem successful and emulate them is the inverse of that. That's why we look for people like Fowler, Martin, Torvalds and Korvig. I don't agree that Fowler and Martin are sought out solely for the sake of good marketing. If there is some experience in Fowler and Martin to learn from, we ought to consider it, regardless of whether they are selling a book or not. Right?

Software developers should not be encouraged to develop like Korvig and Torvalds and the same time discouraged from emulating Fowler/Martin:

  • Almost everybody cannot. The skills are not there.
  • A few people can tightrope without a net but that doesn't mean everyone can or should.
  • The project space of Korvig, Torvalds and Fowler/Martin are very different. Most application/business development (Fowler/Martin) struggles with discovering and mapping user level requirements to running implementation. This is not what AI (Korvig) and kernel (Torvalds) developers do. Its a different problem space entirely.
  • Fowler/Martin focus on problems of software development process methodologies which are much more common than the problems of AI and kernel development.

My last point is on education and team-work. The content of common sense is not universal. Everybody agrees to do things with common sense, but the difficulty is the 'common' part. Its hard to get 'common' understanding of the process and tools. I've had many managers/bosses and team members with common sense, but we did not easily agree on how software is best developed by appealing to our common sense.

Software development is a social, communitive event. Abstract ideas must be translated from the originator's brain to the outside world and into the listeners' brains. This is an amazingly complex human process and is part of every project, except the ones that failed.

Tim Blackler replied on Mon, 2010/09/20 - 8:59am

I think what he's trying to get at is as Han said the in Agile Manifesto:

people>process

whereas now Agile/TDD or whatever has hit the big consultancies it become

process > people

Formalised, codified and (re)applied without thought...

Philopator Ptolemy replied on Mon, 2010/09/20 - 9:05am

Great post!

It names the names!

Nate Buwalda replied on Mon, 2010/09/20 - 9:34am

Fundamentally I agree with the problems that the codification of many agile practices has caused.  I think if you ask any of the major players in the agile arena they would agree as well.  Namely because the first tenet of the manifesto is people over process.  I often get very frusterated in meetings where we discuss the process involved in making people important. 

Where I find problems with this article lies in the assumption that people can be trained to work at the level of programmers like Torvalds, et al.  This is false.  The large majority of people in our field simply do not have the intellectual capacity to function at that level.  I know for sure that I do not.  As such, I find methodologies such as pair programming and TDD to be preferable.  They help me write better code and they help my team bring underskilled developers up to speed.  There is only a small amount of academia that will produce a better coder.   Arming underskilled developers with powerful knowledge is dangerous at best, especially if you are not going to have some kind of methodology safety net to catch them when they fail.

 

Abel Morelos replied on Mon, 2010/09/20 - 11:04am

Whoa, great post! Mostly I agree with you, in special I am against those folks/companies that feel like they are the "chosen ones" because they do TDD, Agile, Scrum, etc.

 I actually follow a lot of the "Agile" practices, but for me these practices are just that, practices, they are unrelated to the coolness factor of your work and they don't make you code great code instantly, it is about keeping balance and choose the right set of tools for the work at hand.

 

Laurent Cohen replied on Tue, 2010/09/21 - 1:00am

Ilya, thank you very much for this article, your arguments resonate in me.

I have always thought that dogmatism at any level of the development cycle was like giving cruches to people who can walk. As can be seen throughout history, (methodology) dogma has always been a (computer) science killer. One is based mostly on belief ("our way is the only real one"), the other on experimental discovery ("I want to see it for myself"). Which one appeals most to us?

Indeed, the marketing hype on many practices has done them a lot of harm from a computer science perspective. I believe the outcome could be hugely different if they'd been marketed as an additional string to the programmer's bow, something that can help move things along, when it is needed.

Emma Watson replied on Fri, 2012/03/30 - 5:15am

Great post.

Agile, XP, TDD, Scrum, Waterfall...

All means to an end. A way, methods, to get goals accomplished.

Agile vs Waterfall, pair programming vs solo, vi vs emacs, nano vs gedit, new line for if statement brackets versus inline...

JDBC

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.