DevOps Zone is brought to you in partnership with:

Dawn Cannan is a software test evangelist who has been working to integrate testers as members of software development teams and improving the working relationships between testers, developers, and everyone else for the past 9 years. When not speaking at testing and agile conferences and user group meetings, she spends her time working in the open source community on the .NET version of SeleNesse, a plugin making it possible to create Selenium tests in FitNesse. She also writes actively, publishing articles and posting to her blog at passionatetester.com. Dawn is a DZone MVB and is not an employee of DZone and has posted 8 posts at DZone. You can read more from them at their website. View Full User Profile

What do you mean, don't submit bugs?

09.29.2011
| 10948 views |
  • submit to reddit
Of all of my "aha!" moments in transitioning from a waterfall-style QA person to an agile tester, the one I am about to talk about is *definitely* in the top 3. Top 5, most .... *Definitely* top 10. :)

I have to give my credit to Janet Gregory, yet again ... during her presentation at STAREast that I named in my last post. This one, in particular, hit me personally, rather than being a suggestion for the whole team.

Janet talked about how as an agile tester, the "Quality Police" mindset must be avoided. I have to admit, I am *really* guilty of playing "quality police". *REALLY* guilty .... However, I was *really shocked* when she said that one of the indicators that you may be playing "quality police" is that you insist that all bugs go into a bug tracking system.

"Well, DUUUUUUUH," I thought, "OF COURSE all bugs go into the bug tracking system!"

(pause)

Wait. What?

As this discussion went on, I learned that there was a practice that I not only had never thought of, but actually made sense! It went something like this: When testing in the scope of a user story inside a sprint, DON'T SUBMIT BUGS. Just go talk to the developer and have him fix it!

I brought this idea back to my team. It was easy to sell this one ... I was able to play the humble one, and show them Janet's slide that shows the indicators for "quality police". We all had a big laugh while the list was read ... describing me, just about to a tee.

The amazing part, however, was that within hours, I could hear testers going to developers and talking about things they found, and I could hear developers yelling, "Hey Sally! I checked in that change. Get latest and let me know." It was almost like this team was waiting for exactly this kind of idea.

What we decided follows roughly this pattern: If a tester is testing a user story card within the scope of a sprint and finds a small problem that can be fixed easily, they just go talk to the developer (maybe send an email ... ). Bugs are submitted if issues are found relating to work done in a previous sprint, or if an issue is so big that it can't easily be fixed within the scope of the current sprint, or if we hear about issues from our customers (our product owners or beta users, for example).

I monitor the bug list and watch for developer activity during a sprint that relates to issues in the bug list. If I see that a developer is working in a certain section of code, I ask if he can look into the related issue. In this way, I still get to play quality police *a little* ....

Benefits that we see from this process:

  • fewer bugs (which means less of everyone's time reviewing bugs) !!
  • more developer/tester interaction ... better relationships
  • better tester insight into developer processes (they talk about where problems are in the code)
  • less email traffic
  • cards marked as 'done' earlier

Pitfalls:

  • ? I haven't yet seen any direct pitfalls .... the one concern I have is not being able to analyze issues for patterns. I have in the past, seen a bug crop up several times, and being able to review previous discussions and/or fixes can help reduce troubleshooting/fixing time. Instinct tells me that I can look through checkin comments just as easily ..... but this is an edge case, nonetheless ......
Personally, I was amazed at how willing I was to give up the "quality police" mindset. I simply had not realized how much that mindset was interfering with the team integration, and just how much closer testers and developers were when this barrier was removed. I realized that in order to be a better agile tester, I had to remove my own barriers. This was a big one, and I have been watching, first-hand, just how much better the team operates with this one removed.
References
Published at DZone with permission of Dawn Cannan, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Gábor Lipták replied on Thu, 2011/09/29 - 7:31am

I have to disagree. You will regret this decision later.
Why:
  • Developers are lazy like everyone. They often change the code without proper code comments, they will often miss the version control comments as well.
  • The developer will go away as soon as he cannot tell from his own code after few years, what it should do. And in this case you will be in trouble.
  • The changes will not be documented, which is a problem in case of a product. The issues are important to generate release note and documentation from
I absolutely agree with you in the thing that you have to go to the developer to see if he understands the problem, and to ask him or her how it will be solved. And yes, you can understand development this case which is also important. BUT having a date with the developer, the tester should go back, file the bug, send the link or the number to the developer through skype or mail, and the developer should mention the bug id in comments and versioning system.
What you can do to make things faster and nicer is to integrate tools. Versioning and issue tracking with development environment. For example use Jira, Eclipse+mylyn and SVN. This is comfortable and fast. Developers do not need to write a single line into svn comments, since mylyn fills it for them.
So my basic message is go to the developer, but still create bugs, and ease the pain with tools.

Mikael Couzic replied on Thu, 2011/09/29 - 7:47am

Makes perfect sense ! Thanks for the insight sharing.

Dave Rooney replied on Thu, 2011/09/29 - 8:33am

...the one concern I have is not being able to analyze issues for patterns.

Waaaay back when I first learned about XP, it was suggested that the team should perform Root Cause Analysis for each defect found. This meant:

  • Finding and fixing the defect, of course;
  • Looking at the type of defect and determining if there are any other places in the system where that sort of defect could occur;
  • Examining the team's process to try to understand how the defect was introduced in the first place, and how to prevent it from being introduced in the future.

James Shore included this in The Art of Agile Development.

This could take 5 minutes or it could take hours depending on the situation, but it helps provide the analysis and pattern detection that you mention. It also eliminates the need for a defect tracking tool *and* addresses Gabor's concern about "lazy developers", provided that the whole team participates.

There are situations where it doesn't make sense to do this, such as a typo on a screen or web page - just fix it and move on. However, the investment of the time to perform RCA will pay off in the future as the team doesn't submit the same types of bugs over and over and over.

In the end, this is as much a mindset change as anything else - anything greater than 0 defects is unacceptable!

Krystian Kaczor replied on Thu, 2011/09/29 - 9:16am

What if:

  • bug is not fixed until end of the Sprint,
  • bug relates to functionality built in previous Sprints,
  • customer found bug
  • you forgot to run a test, but task is DONE and Story closed?

Liam Knox replied on Thu, 2011/09/29 - 10:06am

Look. we can improve. No we will never get rid of bugs. FACT. But this Developers are lazy like everyone. They often change the code without proper code comments, they will often miss the version control comments as well. Yes this is soceity But its not Greece.

Gábor Lipták replied on Thu, 2011/09/29 - 10:12am in response to: Liam Knox

I am a developer ;) So I am lazy as well. But If I can click on mylyn to see my outstanding tasks, and before commit I get the comment out of the box, I will use it, since I understand its advantages. After 3 months nobody remembers what was the problem and the solution exactly.

Mladen Girazovski replied on Thu, 2011/09/29 - 12:12pm in response to: Gábor Lipták

Gabor, might have missed the point,

this is about nothing else than about small bugs:

If a tester is testing a user story card within the scope of a sprint and finds a small problem that can be fixed easily,

Some things are faster corrected than you can file a bug.

Silly little bugs, wrong labels, mispelled labels, etc. pp.

Developers are lazy like everyone. They often change the code without proper code comments, 

Personally, i can not think of what a "proper code comment" should be, or why you would want to write inline comments, be it to describe bug fixes or anything else, write a bug report for a real bug and write a test.

Gábor Lipták replied on Thu, 2011/09/29 - 4:17pm in response to: Mladen Girazovski

For cosmetical bugs, like labels, its fine. The problem is that lets assume for example some change, that affects the business logic, is 10 minute for a good programmer, since he/she knows the code and can change it really fast (maybe it is not corrected nicely, but works). This guy leaves, and the next one will have no idea, why is the logic completely different than it is described in the obsolote specification. He thinks this part is wrong and "corrects" it. So I think if the programmer says that "yes, its a piece of cake", and the tester or the customer do not make an issue or note about it, it will be forgot. As for proper comment I would say something from which I understand what is happening and why :) I know it is not objective answer, but there is no such thing in this case.

Liam Knox replied on Thu, 2011/09/29 - 5:19pm

Just get your Unit testing in order. Cosmetic, I do not care.

Kelvin Meeks replied on Thu, 2011/09/29 - 8:30pm

If this approach is applied in some isolated, small-scale, internal company development teams - then I would be supportive of that approach.

However, let me offer a few comments on why even that special case may not be a supportable scenario.

- Developers Leave.

- Developers sometimes change teams / projects

- Knowledge gained about what caused the bug is lost

- Depending on word-of-mouth to share lessons learned is inherently inefficient

- The "how it was fixed" is lost

- Defects have a funny way of showing up again - memories are weak mechanisms for retrieving knowledge

I  will leave enumeration of arguements against this proposed approach, for larger scale projects, that often involve legally binding contracts with 3rd parties, as an exercise for the reader.

Dave Rooney replied on Thu, 2011/09/29 - 8:53pm in response to: Kelvin Meeks

- Developers Leave.

Automated tests don't.

- Developers sometimes change teams / projects

Automated tests don't.

- Knowledge gained about what caused the bug is lost

Automated tests aren't.

- Depending on word-of-mouth to share lessons learned is inherently inefficient

Automated tests aren't word of mouth, and they can be execute as often as you wish.

- The "how it was fixed" is lost

Automated tests capture this.

- Defects have a funny way of showing up again - memories are weak mechanisms for retrieving knowledge

Root Cause Analysis. On every non-trivial bug (and even some of the trivial ones). And, of course, automated tests should ensure that the same bug doesn't come back.

Just 'cus you fixed it doesn't mean that you learned anything that will prevent it from happening again. I'm not talking about the Spanish Inquisition, but rather investing a small amount of time to learn how to prevent defects rather than trying to find them after the fact.

Mark Something replied on Thu, 2011/09/29 - 9:38pm

Why: Developers are lazy like everyone. They often change the code without proper code comments, they will often miss the version control comments as well. The developer will go away as soon as he cannot tell from his own code after few years, what it should do. And in this case you will be in trouble. The changes will not be documented, which is a problem in case of a product. The issues are important to generate release note and documentation from
I think these are seperate concerns. A lazy developer who writes bad code that requires comments will continue to work that way whether a bug is logged in an issue tracking system or not. In this case they are fixing bugs found in the current round of development, so there should be no need to generate release notes for these bugs. i.e. they are not in production yet! The longer a bug stays unresolved the larger the "cost" to fix it. I agree that issues should only be logged for bugs that either can't or won't be fixed in the current sprint.

dieter von holten replied on Fri, 2011/09/30 - 6:04am

dawn,

believe me - the bugtracker is your friend.

you work as 'serious tester' (lets say) as a quality gate after your programmers. fine. 

now someone whispered in your ear that bug-reports are bad - makes too much work - having an eye-to-eye coffee with the developer chatting over some anomalies in his artifacts is more fun..

when you do that for a while, some cost-sensitive senior shows up at your place and asks 'what exactly are you doing?' - is it effective? - is it efficient? - do we need you? 

how do you prove that you ARE important ? without numbers? 

No bug-reports means EITHER: developers are perfect (like me) and make very few mistakes (like me) - then your testing-job is not needed. OR: you test but dont really find bugs - maybe you are not right for the job??

 the bugtracker is your friend!

 

Jörg Buchberger replied on Fri, 2011/09/30 - 6:49am

Thanks a lot for sharing, Dawn.
Very nice reading. Although, I don't agree, I gained from your article and very much enjoyed (sincerely!) reading it and reasoning about it...


I guess, you will have a lot more regressions soon - and won't even notice(!) it.

What's worse, you will have no history about the regressions (issue comments).

If using the bug tracker becomes an excuse not to go and talk with each other about issues, then the cause _might be_ not(!) the bug tracker.

If you fell omitting posting small bug reports and reading those assigned to you and commenting them is too much hassle - perhaps you are using an "unfriendly" bug tracker?

Cheers.
Jörg

Loren Kratzke replied on Fri, 2011/09/30 - 5:55pm in response to: Mladen Girazovski

A proper code comment looks like this:

// If this is a root element then the parentId will be null.

...or this:

//  We must check to make sure that the cron has finished before committing updates to the database or else we will corrupt 95 terabytes of data and go out of business over night.

It is easy to see what good code does, it is not always so easy to understand why it does things in a particular way.

Lund Wolfe replied on Sat, 2011/10/01 - 1:31am

It seems to me that if you don't like finding bugs or being "quality police" you may not be well suited to testing. I essentially agree with the cooperative and trust relationship of getting bugs fixed sooner with less confrontation and minimal overhead. Quality software is all that really matters and this depends on building quality in the first place, finding bugs faster, and fixing them properly and completely. Bug tracking can't give you this quality.

I have to agree with Gabor on the many benefits of bug tracking, even if they are peripheral, though. Trust is good when it works with your team, but bugs can be more difficult or misunderstood than when first "fixed" and may resurface as regressions. It may be worthwhile to have a history from the developer and user perspectives until it is completely understood and resolved. On the other hand, the code is the documentation, and a good developer will track what worked and what didn't and why in the code comments, regardless of any bug tracking. There may be far more bugs reported than can or should be fixed. Priorities can change as more users report on the same issue and more details are collected and shared.

From a management perspective it is nice to have a more objective view of how much time, effort, expense is going into features and bug fixes on a project to track the progress of a project and the increase or decrease in quality based on the quantity, seriousness, and quality of defect fixes over time. The developers and testers don't have to recall what they did for the last x months. Managers can just check the bug tracking/time tracking system.

Depending on the situation, I think it's easy to argue for or against bug tracking. Bug trackers themselves, in terms of quality, have a pretty bad reputation, but that by itself doesn't justify not using one under any circumstances.

Mladen Girazovski replied on Sat, 2011/10/01 - 2:49am in response to: Loren Kratzke

// If this is a root element then the parentId will be null.

Lets say you're putting this inline comment over a method like this:

private boolean isRoot(final Node node) {
    return null == node.getParent().getId();
}

or even

public class Node {
...
    public boolean isRoot() {
        return null == this.parent().getId();
    }
...
} 

Would you still think it is necessary?

I'd say it is superflous and leave it out.(first one is breaking the law of demeter btw.)

The trick is to write your code in a way that it becomes expressive, everytime you write an inline comment, think again. Keep your methods short and sweet so you'll have plenty of opportunities to use expressive and intention revealing names.

OTOH, i you have this if condition inside of a 200 line method

if (null == node.getParent().getId()) {
.. another 50 lines of code here
}

you cannot make up for it by using an inline comment, it stinks IMO.

As for you second example, this should be part of the documentationen and not just buried somewhere in code/implementation details, at least put it into the JavaDoc.

Loren Kratzke replied on Mon, 2011/10/03 - 2:05pm in response to: Mladen Girazovski

No, any moron could deduce my comment from your method (isRoot()). You missed my point.

Inline comments are meant to give context as to WHY something is being done in a particular way, not WHAT is being done, and are only useful when WHAT is being done does not necessarilly shed light on WHY it is being done that way.

Anybody can recognize a test for null. Whether you get an extra gumball out of a machine or blow up a rocket on the launch pad, well, that's more a question of context now isn't it?

Too exaggerate your example, some of the worst code I have ever seen looks like a method farm - hundreds of virtually pointless tiny little methods making it nearly impossible to attain any sense of context in the bigger picture.

Regarding my second example, I assume that would be in the documentation, email, wiki, sung in 5 part harmony at company meetings, and carved on the inside of the bathroom stall in addition to being mentioned as an inline comment at the single point where it actually matters, in the source code.

Funny you mention "in documentation" as opposed to "burried somewhere in code". I would reverse those to be "burried in documentation" as opposed to "in code". You are speaking directly to a developer when using an inline comment about a specific line or block of code that you have created and that they are now reading. Perhaps you are reading it to remind yourself in the future of WHY you did something in a particular way. Or perhaps you are reading it to understand WHY somebody else did it in a particular way.

The comment may be it a warning, appology, plea for better code, or challenge for the same. You can't do that in Javadocs or specs.

 

 

Mladen Girazovski replied on Mon, 2011/10/03 - 3:37pm

No, any moron could deduce my comment from your method (isRoot()).

Exactly,if even morons could understand code that was written this way, the comment is redundant, it won't add any additional information.

You missed my point. 

Whats your point?

Restating obvious facts that become clear when reading the code in a redundant way by using inline comments?

 Inline comments are meant to give context as to WHY something is being done in a particular way, not WHAT is being done, and are only useful when WHAT is being done does not necessarilly shed light on WHY it is being done that way.

 No, this is what documentaiton if for, like JavaDoc. What and how it is done must be clear from the code itself, it's called "implementantion details".

Too exaggerate your example, some of the worst code I have ever seen looks like a method farm - hundreds of virtually pointless tiny little methods making it nearly impossible to attain any sense of context in the bigger picture.m

Again, you cannot make up for bad code by using inline comments, inline comments are a sign for bad code. By writing small methods you gain a lots of advantages, like burying dirty implemenation details in private methods, using public methods to give an coarse grained overview of whats gonna happen.

Worst code i've ever seen was made up of few classes with huge methods and many inline comments (many of them outdated or wrong due to c&p) and no unit tests at all.

 Regarding my second example, I assume that would be in the documentation, email, wiki, sung in 5 part harmony at company meetings, and carved on the inside of the bathroom stall in addition to being mentioned as an inline comment at the single point where it actually matters, in the source code.

Funny you mention "in documentation" as opposed to "burried somewhere in code". I would reverse those to be "burried in documentation" as opposed to "in code". You are speaking directly to a developer when using an inline comment about a specific line or block of code that you have created and that they are now reading. Perhaps you are reading it to remind yourself in the future of WHY you did something in a particular way. Or perhaps you are reading it to understand WHY somebody else did it in a particular way. 

JavaDoc is also part of the source, but readable without having to read the source.

Do you expect a developer always to read the implementaton when he is calling a method? If the dev doesn't read the source, he would miss your hint if it was written as inline comment.

The comment may be it a warning, appology, plea for better code, or challenge for the same. You can't do that in Javadocs or specs. 

Inline comments are a sign of bad code, usually because the devloper didn't know any better. You can warn/apologies etc in JavaDoc, or why wouldn't you be able to do that?

OTOH, you could write clean code and forget about the apology ;)

Dave Newton replied on Mon, 2011/10/10 - 3:44pm

The pitfall is that you have no accountability regarding who's doing what and when, what components are "attracting" the most churn w/o digging through commit logs, and so on. I actively encourage wide-open communication lanes between departments, *particularly* dev and test, but that doesn't mean that issues shouldn't be tracked--two completely separate things.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.