What do you mean, don't submit bugs?
Of all of my "aha!" moments in transitioning from a waterfall-style QA
person to an agile tester, the one I am about to talk about is
*definitely* in the top 3. Top 5, most .... *Definitely* top 10. :)
I have to give my credit to Janet Gregory, yet again ... during her presentation at STAREast that I named in my last post. This one, in particular, hit me personally, rather than being a suggestion for the whole team.
Janet talked about how as an agile tester, the "Quality Police" mindset must be avoided. I have to admit, I am *really* guilty of playing "quality police". *REALLY* guilty .... However, I was *really shocked* when she said that one of the indicators that you may be playing "quality police" is that you insist that all bugs go into a bug tracking system.
"Well, DUUUUUUUH," I thought, "OF COURSE all bugs go into the bug tracking system!"
(pause)
Wait. What?
As this discussion went on, I learned that there was a practice that I not only had never thought of, but actually made sense! It went something like this: When testing in the scope of a user story inside a sprint, DON'T SUBMIT BUGS. Just go talk to the developer and have him fix it!
I brought this idea back to my team. It was easy to sell this one ... I was able to play the humble one, and show them Janet's slide that shows the indicators for "quality police". We all had a big laugh while the list was read ... describing me, just about to a tee.
The amazing part, however, was that within hours, I could hear testers going to developers and talking about things they found, and I could hear developers yelling, "Hey Sally! I checked in that change. Get latest and let me know." It was almost like this team was waiting for exactly this kind of idea.
What we decided follows roughly this pattern: If a tester is testing a user story card within the scope of a sprint and finds a small problem that can be fixed easily, they just go talk to the developer (maybe send an email ... ). Bugs are submitted if issues are found relating to work done in a previous sprint, or if an issue is so big that it can't easily be fixed within the scope of the current sprint, or if we hear about issues from our customers (our product owners or beta users, for example).
I monitor the bug list and watch for developer activity during a sprint that relates to issues in the bug list. If I see that a developer is working in a certain section of code, I ask if he can look into the related issue. In this way, I still get to play quality police *a little* ....
Benefits that we see from this process:
Published at DZone with permission of Dawn Cannan, author and DZone MVB. (source)I have to give my credit to Janet Gregory, yet again ... during her presentation at STAREast that I named in my last post. This one, in particular, hit me personally, rather than being a suggestion for the whole team.
Janet talked about how as an agile tester, the "Quality Police" mindset must be avoided. I have to admit, I am *really* guilty of playing "quality police". *REALLY* guilty .... However, I was *really shocked* when she said that one of the indicators that you may be playing "quality police" is that you insist that all bugs go into a bug tracking system.
"Well, DUUUUUUUH," I thought, "OF COURSE all bugs go into the bug tracking system!"
(pause)
Wait. What?
As this discussion went on, I learned that there was a practice that I not only had never thought of, but actually made sense! It went something like this: When testing in the scope of a user story inside a sprint, DON'T SUBMIT BUGS. Just go talk to the developer and have him fix it!
I brought this idea back to my team. It was easy to sell this one ... I was able to play the humble one, and show them Janet's slide that shows the indicators for "quality police". We all had a big laugh while the list was read ... describing me, just about to a tee.
The amazing part, however, was that within hours, I could hear testers going to developers and talking about things they found, and I could hear developers yelling, "Hey Sally! I checked in that change. Get latest and let me know." It was almost like this team was waiting for exactly this kind of idea.
What we decided follows roughly this pattern: If a tester is testing a user story card within the scope of a sprint and finds a small problem that can be fixed easily, they just go talk to the developer (maybe send an email ... ). Bugs are submitted if issues are found relating to work done in a previous sprint, or if an issue is so big that it can't easily be fixed within the scope of the current sprint, or if we hear about issues from our customers (our product owners or beta users, for example).
I monitor the bug list and watch for developer activity during a sprint that relates to issues in the bug list. If I see that a developer is working in a certain section of code, I ask if he can look into the related issue. In this way, I still get to play quality police *a little* ....
Benefits that we see from this process:
- fewer bugs (which means less of everyone's time reviewing bugs) !!
- more developer/tester interaction ... better relationships
- better tester insight into developer processes (they talk about where problems are in the code)
- less email traffic
- cards marked as 'done' earlier
- ? I haven't yet seen any direct pitfalls .... the one concern I have is not being able to analyze issues for patterns. I have in the past, seen a bug crop up several times, and being able to review previous discussions and/or fixes can help reduce troubleshooting/fixing time. Instinct tells me that I can look through checkin comments just as easily ..... but this is an edge case, nonetheless ......
(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)
Tags:






Comments
Gábor Lipták replied on Thu, 2011/09/29 - 7:31am
Why:
- Developers are lazy like everyone. They often change the code without proper code comments, they will often miss the version control comments as well.
- The developer will go away as soon as he cannot tell from his own code after few years, what it should do. And in this case you will be in trouble.
- The changes will not be documented, which is a problem in case of a product. The issues are important to generate release note and documentation from
I absolutely agree with you in the thing that you have to go to the developer to see if he understands the problem, and to ask him or her how it will be solved. And yes, you can understand development this case which is also important. BUT having a date with the developer, the tester should go back, file the bug, send the link or the number to the developer through skype or mail, and the developer should mention the bug id in comments and versioning system.What you can do to make things faster and nicer is to integrate tools. Versioning and issue tracking with development environment. For example use Jira, Eclipse+mylyn and SVN. This is comfortable and fast. Developers do not need to write a single line into svn comments, since mylyn fills it for them.
So my basic message is go to the developer, but still create bugs, and ease the pain with tools.
Mikael Couzic replied on Thu, 2011/09/29 - 7:47am
Dave Rooney replied on Thu, 2011/09/29 - 8:33am
Waaaay back when I first learned about XP, it was suggested that the team should perform Root Cause Analysis for each defect found. This meant:
James Shore included this in The Art of Agile Development.
This could take 5 minutes or it could take hours depending on the situation, but it helps provide the analysis and pattern detection that you mention. It also eliminates the need for a defect tracking tool *and* addresses Gabor's concern about "lazy developers", provided that the whole team participates.
There are situations where it doesn't make sense to do this, such as a typo on a screen or web page - just fix it and move on. However, the investment of the time to perform RCA will pay off in the future as the team doesn't submit the same types of bugs over and over and over.
In the end, this is as much a mindset change as anything else - anything greater than 0 defects is unacceptable!
Krystian Kaczor replied on Thu, 2011/09/29 - 9:16am
What if:
Liam Knox replied on Thu, 2011/09/29 - 10:06am
Gábor Lipták replied on Thu, 2011/09/29 - 10:12am
in response to:
Liam Knox
Mladen Girazovski replied on Thu, 2011/09/29 - 12:12pm
in response to:
Gábor Lipták
Gabor, might have missed the point,
this is about nothing else than about small bugs:
Some things are faster corrected than you can file a bug.
Silly little bugs, wrong labels, mispelled labels, etc. pp.
Personally, i can not think of what a "proper code comment" should be, or why you would want to write inline comments, be it to describe bug fixes or anything else, write a bug report for a real bug and write a test.
Gábor Lipták replied on Thu, 2011/09/29 - 4:17pm
in response to:
Mladen Girazovski
Liam Knox replied on Thu, 2011/09/29 - 5:19pm
Kelvin Meeks replied on Thu, 2011/09/29 - 8:30pm
If this approach is applied in some isolated, small-scale, internal company development teams - then I would be supportive of that approach.
However, let me offer a few comments on why even that special case may not be a supportable scenario.
- Developers Leave.
- Developers sometimes change teams / projects
- Knowledge gained about what caused the bug is lost
- Depending on word-of-mouth to share lessons learned is inherently inefficient
- The "how it was fixed" is lost
- Defects have a funny way of showing up again - memories are weak mechanisms for retrieving knowledge
I will leave enumeration of arguements against this proposed approach, for larger scale projects, that often involve legally binding contracts with 3rd parties, as an exercise for the reader.
Dave Rooney replied on Thu, 2011/09/29 - 8:53pm
in response to:
Kelvin Meeks
Automated tests don't.
Automated tests don't.
Automated tests aren't.
Automated tests aren't word of mouth, and they can be execute as often as you wish.
Automated tests capture this.
Root Cause Analysis. On every non-trivial bug (and even some of the trivial ones). And, of course, automated tests should ensure that the same bug doesn't come back.
Just 'cus you fixed it doesn't mean that you learned anything that will prevent it from happening again. I'm not talking about the Spanish Inquisition, but rather investing a small amount of time to learn how to prevent defects rather than trying to find them after the fact.
Mark Something replied on Thu, 2011/09/29 - 9:38pm
dieter von holten replied on Fri, 2011/09/30 - 6:04am
dawn,
believe me - the bugtracker is your friend.
you work as 'serious tester' (lets say) as a quality gate after your programmers. fine.
now someone whispered in your ear that bug-reports are bad - makes too much work - having an eye-to-eye coffee with the developer chatting over some anomalies in his artifacts is more fun..
when you do that for a while, some cost-sensitive senior shows up at your place and asks 'what exactly are you doing?' - is it effective? - is it efficient? - do we need you?
how do you prove that you ARE important ? without numbers?
No bug-reports means EITHER: developers are perfect (like me) and make very few mistakes (like me) - then your testing-job is not needed. OR: you test but dont really find bugs - maybe you are not right for the job??
the bugtracker is your friend!
Jörg Buchberger replied on Fri, 2011/09/30 - 6:49am
Very nice reading. Although, I don't agree, I gained from your article and very much enjoyed (sincerely!) reading it and reasoning about it...
I guess, you will have a lot more regressions soon - and won't even notice(!) it. What's worse, you will have no history about the regressions (issue comments). If using the bug tracker becomes an excuse not to go and talk with each other about issues, then the cause _might be_ not(!) the bug tracker. If you fell omitting posting small bug reports and reading those assigned to you and commenting them is too much hassle - perhaps you are using an "unfriendly" bug tracker? Cheers.
Jörg
Loren Kratzke replied on Fri, 2011/09/30 - 5:55pm
in response to:
Mladen Girazovski
A proper code comment looks like this:
// If this is a root element then the parentId will be null.
...or this:
// We must check to make sure that the cron has finished before committing updates to the database or else we will corrupt 95 terabytes of data and go out of business over night.
It is easy to see what good code does, it is not always so easy to understand why it does things in a particular way.
Lund Wolfe replied on Sat, 2011/10/01 - 1:31am
I have to agree with Gabor on the many benefits of bug tracking, even if they are peripheral, though. Trust is good when it works with your team, but bugs can be more difficult or misunderstood than when first "fixed" and may resurface as regressions. It may be worthwhile to have a history from the developer and user perspectives until it is completely understood and resolved. On the other hand, the code is the documentation, and a good developer will track what worked and what didn't and why in the code comments, regardless of any bug tracking. There may be far more bugs reported than can or should be fixed. Priorities can change as more users report on the same issue and more details are collected and shared.
From a management perspective it is nice to have a more objective view of how much time, effort, expense is going into features and bug fixes on a project to track the progress of a project and the increase or decrease in quality based on the quantity, seriousness, and quality of defect fixes over time. The developers and testers don't have to recall what they did for the last x months. Managers can just check the bug tracking/time tracking system.
Depending on the situation, I think it's easy to argue for or against bug tracking. Bug trackers themselves, in terms of quality, have a pretty bad reputation, but that by itself doesn't justify not using one under any circumstances.
Mladen Girazovski replied on Sat, 2011/10/01 - 2:49am
in response to:
Loren Kratzke
Lets say you're putting this inline comment over a method like this:
private boolean isRoot(final Node node) { return null == node.getParent().getId(); }or even
public class Node { ... public boolean isRoot() { return null == this.parent().getId(); } ... }
Would you still think it is necessary?
I'd say it is superflous and leave it out.(first one is breaking the law of demeter btw.)
The trick is to write your code in a way that it becomes expressive, everytime you write an inline comment, think again. Keep your methods short and sweet so you'll have plenty of opportunities to use expressive and intention revealing names.
OTOH, i you have this if condition inside of a 200 line method
if (null == node.getParent().getId()) { .. another 50 lines of code here }you cannot make up for it by using an inline comment, it stinks IMO.
As for you second example, this should be part of the documentationen and not just buried somewhere in code/implementation details, at least put it into the JavaDoc.
Loren Kratzke replied on Mon, 2011/10/03 - 2:05pm
in response to:
Mladen Girazovski
No, any moron could deduce my comment from your method (isRoot()). You missed my point.
Inline comments are meant to give context as to WHY something is being done in a particular way, not WHAT is being done, and are only useful when WHAT is being done does not necessarilly shed light on WHY it is being done that way.
Anybody can recognize a test for null. Whether you get an extra gumball out of a machine or blow up a rocket on the launch pad, well, that's more a question of context now isn't it?
Too exaggerate your example, some of the worst code I have ever seen looks like a method farm - hundreds of virtually pointless tiny little methods making it nearly impossible to attain any sense of context in the bigger picture.
Regarding my second example, I assume that would be in the documentation, email, wiki, sung in 5 part harmony at company meetings, and carved on the inside of the bathroom stall in addition to being mentioned as an inline comment at the single point where it actually matters, in the source code.
Funny you mention "in documentation" as opposed to "burried somewhere in code". I would reverse those to be "burried in documentation" as opposed to "in code". You are speaking directly to a developer when using an inline comment about a specific line or block of code that you have created and that they are now reading. Perhaps you are reading it to remind yourself in the future of WHY you did something in a particular way. Or perhaps you are reading it to understand WHY somebody else did it in a particular way.
The comment may be it a warning, appology, plea for better code, or challenge for the same. You can't do that in Javadocs or specs.
Mladen Girazovski replied on Mon, 2011/10/03 - 3:37pm
Exactly,if even morons could understand code that was written this way, the comment is redundant, it won't add any additional information.
Whats your point?
Restating obvious facts that become clear when reading the code in a redundant way by using inline comments?
No, this is what documentaiton if for, like JavaDoc. What and how it is done must be clear from the code itself, it's called "implementantion details".
Again, you cannot make up for bad code by using inline comments, inline comments are a sign for bad code. By writing small methods you gain a lots of advantages, like burying dirty implemenation details in private methods, using public methods to give an coarse grained overview of whats gonna happen.
Worst code i've ever seen was made up of few classes with huge methods and many inline comments (many of them outdated or wrong due to c&p) and no unit tests at all.
JavaDoc is also part of the source, but readable without having to read the source.
Do you expect a developer always to read the implementaton when he is calling a method? If the dev doesn't read the source, he would miss your hint if it was written as inline comment.
Inline comments are a sign of bad code, usually because the devloper didn't know any better. You can warn/apologies etc in JavaDoc, or why wouldn't you be able to do that?
OTOH, you could write clean code and forget about the apology ;)
Dave Newton replied on Mon, 2011/10/10 - 3:44pm