Agile Zone is brought to you in partnership with:

I am an experienced software development manager, project manager and CTO focused on hard problems in software development and maintenance, software quality and security. For the last 15 years I have been managing teams building electronic trading platforms for stock exchanges and investment banks around the world. My special interest is how small teams can be most effective at building real software: high-quality, secure systems at the extreme limits of reliability, performance, and adaptability. Jim is a DZone MVB and is not an employee of DZone and has posted 95 posts at DZone. You can read more from them at their website. View Full User Profile

7 Agile Best Practices that You Don’t Need to Follow

05.24.2013
| 35904 views |
  • submit to reddit

There are many good ideas and practices in Agile development, ideas and practices that definitely work: breaking projects into Small Releases to manage risk and accelerate feedback; time-boxing to limit WIP and keep everyone focused; relying only on working software as the measure of progress; simple estimating and using velocity to forecast team performance; working closely and constantly with the customer; and Continuous Integration – and Continuous Delivery – to ensure that code is always working and stable.

But there are other commonly accepted ideas and best practices that aren’t important: if you don’t follow them, nothing bad will happen to you and your project will still succeed. And there are a couple that you are better off not following at all.

Test-Driven Development

Teams that need to move quickly need to depend on a fast, efficient testing safety net. With Test First Development or Test-Driven Development (TDD), there’s no excuse for not writing tests – after all, you have to write a failing test before you write the code. So you end up with a good set of working automated tests that ensure a high level of coverage and regression protection.

TDD is not only a way of ensuring that developers test their code. It is also advocated as a design technique that leads to better quality code and a simpler, cleaner design.

A study of teams at Microsoft and IBM (Realizing Quality Improvement through Test Driven Development, Microsoft Research, 2008) found that while TDD increased upfront development costs between 15-35% (TDD demands developers change the way that they think and work, which slows developers down, at least at first), it reduced defect density by 40% (IBM) or as much as 60-90% (Microsoft) over teams that did not follow disciplined unit testing.

But in Making Software Chapter 12 “How Effective is Test-Driven Development” researchers led by Burak Turhan found that while TDD improves external quality (measured by one or more of test cases passed, number of defects, defect density, defects per test, effort required to fix defects, change density, % of preventative changes) and can improve the quality of the tests (fewer mistakes in the tests, tests that are easier to maintain), TDD does not consistently improve the quality of the design. TDD seems to reduce code complexity and improve reuse, however it also negatively impacts coupling and cohesion. And while method and class-level complexity is better in code developed using TDD, project/package level complexity is worse.

People who like TDD like it a lot, so if you like it, do it. And even if you are not TDD-infected, there are times when working test first is natural – when you have to solve a specific problem in a specific way, or if you’re fixing a bug where the failing test case is already written up for you. But the important thing is that you write a good set of tests and keep them up to date and run them frequently – it doesn't matter if you write them before, or after, you write the code.

Pair Programming

According to the VersionOne State of Agile Development Survey 2012, almost 1/3 of teams follow pair programming – a surprisingly high number, given how disciplined pair programming is, and how few teams follow XP (2%) or Scrum/XP Hybrid (11%) methods where pair programming would be prescribed.

There are good reasons for pairing: information sharing and improving code quality through continuous, informal code reviews as developers work together. And there are natural times to pair developers, or sometimes developers and testers, together: when you’re working through a hard design problem; or on code that you’ve never seen before and somebody who has worked on it is available to help; or when you’re over your head in troubleshooting a high-pressure problem; or testing a difficult part of the system; or when a new person joins the team and needs to learn about the code and coding practices.

Some (extroverted) people enjoy pairing up, the energy it creates and the opportunities it provides to get to know others on the team. But forcing people who prefer working on their own or who don’t like each other to work closely together is definitely not a good idea. There are real social costs in pairing: you have to be careful to pair people up by skill, experience, style, personality type and work ethic. And sustained pair programming can be exhausting, especially over the long term – one study (Vanhanen and Lassenius 2007) found that people only pair between 1.5 and 4 hours a day on average, because it’s too intense to do all day long.

In Pair Programming Considered Harmful? Jon Evans says that pairing can have also negative effects on creativity:

Research strongly suggests that people are more creative when they enjoy privacy and freedom from interruption … What distinguished programmers at the top-performing companies wasn’t greater experience or better pay. It was how much privacy, personal workspace and freedom from interruption they enjoyed,” says a New York Times article castigating “the new groupthink”.

And in “Still Questioning Extreme Programming” Pete McBreen points out some other disadvantages and weaknesses of pair programming:

  • Exploration of ideas is not encouraged, pairing makes a developer focus on writing the code, so unless there is time in the day for solo exploration the team gets a very superficial level of understanding of the code.
  • Developers can come to rely too much on the unit tests, assuming that if the tests pass then the code is OK. (This follows on from the lack of exploration.)
  • Corner cases and edge cases are not investigated in detail, especially if they are hard to write tests for.
  • Code that requires detail thinking about the design is hard to do when pairing unless one partner completely dominates the session. With the usual tradeoff between partners, it is hard to build technically complex designs unless they have been already been worked out in a solo session.
  • Personal styles matter when pairing, and not all pairings are as productive as others.
  • Pairs with different typing skills and proficiencies often result in the better typist doing all of the coding with the other partner being purely passive.
And of course pairing in distributed teams doesn't work well if at all (depending on distance, differences in time zones, culture, working styles, language), although some people still try.

While pairing does improve code quality over solo programming, you can get the same improvements in code quality, and at least some of the information sharing advantages, through code reviews, at less cost. Code reviews – especially lightweight, offline reviews – are easier to schedule, less expensive and less intrusive than pairing. And as Jason Cohen points out even if developers are pair programming, you may still need to do code reviews, because pair programming is really about joint problem solving, and doesn’t cover all of the issues that a code review would.

Back to Jon Evans for the final word on pair programming:

The true answer is that there is no one answer; that what works best is a dynamic combination of solitary, pair, and group work, depending on the context, using your best judgement. Paired programming definitely has its place. (Betteridge’s Law strikes again!) In some cases that place may even be “much of most days.” But insisting on 100 percent pairing is mindless dogma, and like all mindless dogma, ultimately counterproductive.

Emergent Design and Metaphor

Incremental development works, and trying to keep design simple makes good sense, but attempting to define an architecture on the fly is foolish and impractical. There’s a reason that almost nobody actually follows Emergent Design: it doesn't work.

Relying on a high-level metaphor (the system is an "assembly line" or a "bill of materials" or a "hive of bees") shared by the team as some kind of substitute for architecture is even more ridiculous. Research from Carnegie Mellon University found that

… natural language metaphors are relatively useless for either fostering communication among technical and non-technical project members or in developing architecture.
Almost no one understands what a system metaphor is any ways, or how it is to be used, or how to choose a meaningful metaphor or how to change it if you got it wrong (and how you would know if you got it wrong), including one of the people who helped come up with the idea:
Okay I might as well say it publicly - I still haven't got the hang of this metaphor thing. I saw it work, and work well on the C3 project, but it doesn't mean I have any idea how to do it, let alone how to explain how to do it.
Martin Fowler, Is Design Dead?

Agile development methods have improved development success and shown better ways to approach many different software development problems – but not architecture and design.

Daily Standups

When you have a new team and everyone needs to get to know each other and more time to understand what the project is about; or when the team is working under emergency conditions trying to fix something or finish something under extreme pressure, then getting everyone together in regular meetings, maybe even more than once a day, is necessary and valuable. But whether everyone stands up or sits down and what they end up talking about in a meeting should be up to you.

If your team has been working well together for a while and everyone knows each other and knows what they are working on, and if developers update cards on a task board or a Kanban board or the status in an electronic system as they get things done, and if they are grown up enough to ask for help when they need it, then you don’t need to make them all stand up in a room every morning.

Collective Code Ownership

Letting everyone work on all of the code isn't always practical (because not everyone on the team has the requisite knowledge or experience to work on every problem) and collective code ownership can have negative effects on code quality.

Share code where it makes sense to do so, but realize that not everybody can – or should – work on every part of the system.

Writing All Requirements as Stories

The idea that every requirement specification can be written as User Stories in 1 or 2 lines on cards, that requirements should be too short on purpose (so that the developer has to talk to someone to explain what’s really needed) and insisting that they should all be in the same template form

“As a type of user I want some goal so that some reason…”
is silly and unnecessary. This is the same kind of simple minded orthodoxy that led everyone to try to capture all requirements in UML Use Case format with stick men and bubbles 15 years ago.

There are many different ways to effectively express requirements. Sometimes requirements need to be specified in detail (when you have to meet regulatory compliance or comply with a standard or integrate with an existing system or implement a specific algorithm or…). Sometimes it’s better to work from a test case or a detailed use case scenario or a wire frame or some other kind of model, because somebody who knows what’s going on has already worked out the details for you. So pick the format and level of detail that works best and get to work.

Relying on a Product Owner

Relying on one person as the Product Owner, as the single solitary voice of the customer and the “one throat to choke” when the project fails, doesn't scale, doesn't last, and puts the team and the project and eventually the business at risk. It’s a naïve, dangerous approach to designing a product and to managing a development project, and it causes more problems than it solves.

Many teams have realized this and are trying to work around the Product Owner idea because they have to. To succeed, a team needs real and sustained customer engagement at multiple levels, and they should take responsibility themselves for making sure that they get what they need, rather than relying on one person to do it all.

Published at DZone with permission of Jim Bird, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Oleksandr Alesinskyy replied on Wed, 2013/05/29 - 6:19am

Nice article - one of the few in the weeklies  (not only this weekly- but weeklies overall) that have some substantial meat in it.

Few remarks:

  • To further support the author's point on TDD - "found that while TDD increased upfront development costs between 15-35% (TDD demands developers change the way that they think and work, which slows developers down, at least at first), it reduced defect density by 40% (IBM) or as much as 60-90% (Microsoft) over teams that did not follow disciplined unit testing."  - a classical case of biased researches based on "apple-pear comparison". TDD should be compared to other approaches to the disciplined unit testing, not to the absence of disciplined unit testing.
  • As for collective code ownership - the author "tossed the baby out with the bath water", the 2 main reasons for such code ownership.
    •  to keep things running in case some team members become unavailable (temporary or permanently). So, depending on the team size, each piece of code should be owned with the whole team or its sizable part. 
    • To pass knowledge inside a team (especially effective with pair programming or code reviews).




Arthur Bechstein replied on Wed, 2013/05/29 - 10:05am

 RE: Pair Programming

I can see why so many programmers like pair programming. I've been programming professionally for over 30 years, and it can be damn lonely sometimes. Pair programming turns programming into a social activity.

But it sickens me that so many have jumped on the pair programming bandwagon as if it is some kind of cult. I'm looking at you Ruby on Rails.

The truth is that all of the peer-reviewed literature I've read, and I've read a lot of it, finds that pair programming really only has advantages in training. When you have two developers that are already experienced there is really no benefit at all, but you do double your costs.

For an industry that should be based on science and mathematics to ignore those findings is troublesome.

Jim Dejonge replied on Wed, 2013/05/29 - 10:15am

The UML Use Case diagram was never intended to capture detailed requirements. It simply showed which use cases were relevant to which actors. That way, if you've got a question about a use case, you know who to talk to.

Florin Jurcovici replied on Thu, 2013/05/30 - 7:41am in response to: Oleksandr Alesinskyy

I find collective code ownership useful for one more reason: it favors a clean and comprehensible design. If you design with the goal of having others easily maintain your code, you're more likely to come up with an easier to understand design, and certain architectural patterns tend to get established across the team. Which leads to a product more uniform in design than if strong, individual code ownership was employed.

As for TDD, I disagree that it increases complexity at package and application level. Some might think that doing TDD makes design and architecture obsolete. But not all programmers are equally good designers. Therefore, relying on design being done by each individual team member and not employing at least (pre-commit) code reviews to make sure the design of an app is uniform and good across the entire codebase is the mistake that leads to bad design. But if this is the case, it's not the fault of TDD. Remember, TDD's promise is a better code structure, not a better architecture.



Raging Infernoz replied on Fri, 2013/05/31 - 8:08am

Pair Programming and Ruby-on-Rails; maybe that is why it is so insecure, because good developers can't get their heads down to crack it and fix it!  As an INTJ, I just hate the idea of Pair Programming, I'll only seek group advice when I want to talk through and clear up how best to solve knotty problems which other /specific/ people may have more insight on.

TDD sounds like a retrogression to Waterfall development; some complex design has to be refined by throw away or refactored prototypes, and TDD would really get in the way of that!  TDD could also become very brittle, enforce broken design, and make later refactoring quite painful, especially for automated IDE refactoring.

Collective ownership should be less of an issue if a project is componentised (e.g. as libraries and frameworks with Unit Tests), and with appropriately and proper separation of concerns e.g. for MVC.  Usable code naming, code comments, and at least outline documentation help too!


Bernhard Bockelbrink replied on Sat, 2013/06/22 - 3:54pm

You should do whatever works for you. I the end it's perfectly fine for teams not to do agile at all.

As I see it, all of the practices you quote have a few things in common: it takes a while to understand them, and even longer to do them well. Each of these practices may or may not work for a  given team, but you will only know after the team really tried them. 

This is, by the way, one of the reasons why agile teams greatly benefit from being coached: these experiments will be faster, better, and less painful.

The main issue I have with this article is that it is written in a way that may deter people who have just a vague idea about what these practices actually are from actually experiencing them.

Just one example: you almost make it sound like all that user stories are is the template "As a … I want … so that …". Defining requirements just like that would be stupid. 

The template just tells you "who" wants to do "what" and "why". For all those features where there is no actor, no action, or no reason, you can, of course skip that. 

For me a user story is the Card, the Conversation and the Criteria: the physical card on the wall is constantly visible to the team to remind them of a series of conversations they had about the problem. The shared understanding that resulted from these conversations is condensed into a set of acceptance criteria which can be automated and will help them to implement the story as well as document and maintain the value of that story over time, because these tests will fail when some of that value is taken away. 

From my experience this works really well in most cases. And nobody ever said you're not allowed to use sketches, legal texts, UML diagrams or whatever as supporting documents.

Just my 2c…

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.