Agile Advisor

Agile Bug Management: Not an Oxymoron

An agile team's goal is to eliminate bug debt. These techniques can help.

Agile has a refreshing approach to bugs, especially when contrasted with traditional waterfall practices. Historically, teams would postpone bugs to a stabilization phase. This stabilization phase was long and tedious, typically with high stress and long hours. Agile, on the other hand, promotes the following thought processes:

  • Carrying bug debt is unhealthy and sometimes, even destructive. We should fix bugs, not track them.
  • A feature isn't done until the bugs are fixed. Quality is part of the overall cost.
  • Bugs are prioritized against new features. A bug needs to be more important than the next new feature.

I think most agree with the principles behind these statements. They sound great when reading them in a book, but things can be different in the real world of development. This is what I've found:

  • Nearly all teams have bug debt. Bug debt is bad, but having no plan to manage bugs is even worse.
  • Teams held hostage by the one or two bugs on a newly-developed user story. They've just spent hundreds of hours developing a high-quality story, but they get zero credit in that sprint, because it's not "done". This is especially an issue if management is watching that team's velocity as a measure of productivity. Ugh.
  • Having bugs appear on the product backlog along with new features always felt awkward. The bug's granularity is so different from user stories, and we prioritize them differently. For example, do I really care that Bug No. 9 is completed before Bug No. 10?

I've talked with several other agile teams who share the same struggles. We all want to do the right thing with bugs, but still feel this pain.

For me, it's helpful to remember that all new features have a "bug tail." The bug tail is the diminishing number of open bugs on the new functionality. When a feature is added, bugs are introduced. Most are discovered and fixed immediately by the team as a part of feature development: some a few days later, others not until long after a story is considered "done".

If you graphed it out, a bug tail might look like Figure 1. Most bugs are found and fixed immediately within the sprint. However, there are always a few that trickle in later on.

Figure 1. A typical bug tail.

An unhealthy team's bug tail might look like Figure 2. This team is likely not finding most of the bugs within the sprint.

Figure 2. An unhealthy bug tail.

Figure 3 shows what the dreaded stabilization phase might look like: I think we can all agree this is bad.

[Click on image for larger view.]
Figure 3. A stabilization phase, typical of waterfall development methods.

Even a healthy agile team that discovers and fixes bugs while the feature is being developed will still have a bug tail; and that bug tail can overlap the start of the next feature, as shown in Figure 4.

Figure 4. Bug tails overlapping feature development.

Teams should understand what their bug tail is, and strive to reduce the length of it. While the tail may diminish exponentially, it doesn't reach zero very quickly. Bugs are discovered days or even weeks later. Therefore, a team needs a plan to manage its bug debt.

One technique I've found useful is to use a separate bug backlog, and align bug debt reduction with sprint boundaries.

The Bug Backlog is Separate From the Product Backlog
The bug backlog includes all bugs you want to fix on previously-implemented features. It does not include bugs found within a sprint, or on features under development. Bugs found within a sprint are considered work not completed, like tasks. The bug backlog only includes bugs on features previously marked "done".

Before the beginning of a sprint, the product owner must triage all the bugs in the backlog. That is, they have to say "I want all these bugs fixed before starting on new functionality". This is tough, because they have to let go of bugs. This forced trade-off, however, is healthy because it requires the product owner to ask the question: "What would the customer want?" Sometimes a bug is just cut. Sometimes a group of bugs are moved to the backlog as a single story, to prioritize against other features. Regardless, what's left in the bug backlog is prioritized above any features on the product backlog.

The bug backlog isn't sequentially ordered, but sorted by a field such as Priority, which defines classes of prioritization. All bugs in the bug backlog are expected to be fixed. The bug priority informs a developer on how important a bug is, so they can escalate if the actual effort becomes disproportionate to the bug's priority.

Every Sprint, Address Any Known Bug Debt
Every sprint, the team looks at any bugs remaining in the bug backlog and allocates capacity to get that known set of bugs down to zero, or near-zero. Whether this is one day, one week or the entire sprint, they fix the bugs first. Bugs found later, within the sprint, are not considered part of that initial commitment. Unless they're very high priority, they're put on the bug backlog for the next sprint.

Many teams work in a commitment-based organization, where management places a high value on a team's ability to meet their commitments. Doing capacity planning against a known set of bugs makes sprint planning more deterministic, increasing their chance to meet commitments. Any new bugs discovered during the sprint are not a part of the initial commitment, and will be tackled next sprint.

Managing Bug Debt Across An Enterprise
An organization transitioning to a culture where debt is continually eliminated likely is dealing with the following question: How do you get teams to reduce their bug count without telling them exactly what to do? Leadership wants the team to change, yet gives the team autonomy to determine how they change. One option is to use a bug cap.

For example, consider a bug cap of three bugs per engineer. This means a team of 10 people should not have more than 30 bugs in its bug backlog. If the team is over its cap, it's expected to stop work on new features and get under the bug cap. A team is expected to be under its cap at all times, but the team decides how it wants to do that. The bug cap ensures that bug debt is never carried for too long, and the team can learn from the mistakes that causes the bugs to be injected in the first place.

Remember that the bug cap represents the bugs in the bug backlog. It does not include bugs found and fixed within the sprint in which a feature is developed. Those bugs are considered undone work, not debt.

A team always struggling to stay under the cap may have a bug trend similar to that shown in Figure 5.

[Click on image for larger view.]
Figure 5. A team struggling to keep its bugs under the bug cap.

That same team, several sprints later, may have improved its bug trend, as seen in Figure 6. It has hopefully learned ways to shorten its bug tail and eliminate most of it within the sprint. This team doesn't really need a bug cap anymore.

[Click on image for larger view.]
Figure 6. A team with its bugs under control.

An organization with a lot of bug debt can start with a higher bug cap, and slowly lower it with each sprint, until a healthy level is reached. The bug cap can be useful for transitioning an entire organization to a more agile culture. The bug cap, however, is not the goal. It's a transition tool. The goal is to implement a culture where the bug cap is irrelevant.

A healthy team that eliminates its bug tail almost immediately, and carries negligible debt, may no longer need the practices outlined in this article. Awesome! However, moving to a "no debt" culture is likely to be one of the largest paradigm shifts for teams implementing agile. These teams need tools to help them make that transition. The practices discussed in this article are just that: more tools in a team's toolbox.

About the Author

Gregg Boer is a Principal Program Manager at Microsoft with 25 years of experience in software. Over his career, Gregg has worked as a Project Manager, Program Manager, Requirements Lead, Software Engineer, Analyst, QA Lead, and Software Designer. Gregg joined Microsoft in 2005 because he believed in the vision of Team Foundation Server. Currently, Gregg is working on the team developing a set of world-class Agile Tools built on top of the TFS Platform.

comments powered by Disqus

Reader Comments:

Tue, Oct 16, 2012 Kannan Chennai, INDIA

This is a very nice article. Bug debt,if captured as a key metric to be monitored at project, program and enterprise level against set goals would be another great indicator of the how reliable the deliveries of the teams were. I look forward to explore leveraging this metric in future assigments.

Tue, Oct 16, 2012

This is not really in line with current industry agile trends.

Tue, Oct 16, 2012 Raymond Neville Pacific NW

Great topic and summarizaiton article. I appreciate the graph examples and recommendations to use these trends as indicators.

Tue, Oct 16, 2012 JimL

These ideas sound good if you are management and you want to treat your developers like assembly line workers. Just go to code at the start of iteration 1. Do not think, talk, or draw out design. Therefore, more bugs. The software developers and engineers have more experience than the usual product owner (which is usually a person from marketing). Why does the product owner only triage the bugs? The developers should be allowed to triage the bugs. Which would be easier (less time consuming) to complete? For each story, the developers should have a design in place that matches the architecture of the project and include possible bugs or contingencies. Also, design and code reviews must be mandatory involving people who are close to and may be users of the developer’s code. Also, under agile, there is a sense that the results must be complete ASAP. Just get it done. This comes from on high, so that the commitments can be made. (This sometimes sacrifices code quality and produces more bugs.) When tasks and bugs are handed out, the engineer must step back and take some time to understand the requirement and how to make the best design and fit it into the project architecture. What I have seen lately in our agile is a mentality of the old “cowboy” coder multiplied by the number of software developers on the project. Each doing his own thing, slinging code to get his part done. Yes, there is communication between the engineers and our daily standup meetings usually last ½ to 1 hour to resolve issues. But, bugs persist. When the engineer finishes the design for a task, he should consider possible bugs and if necessary any conflicts that will be created. And, sometimes, bugs arise because a requirement is not fully understood by the engineer. In the end, Agile is much better than the waterfall model because the software people get almost immediate feedback from the customer. Even if a requirement was requested one way and the customer changes his mind, it can be changed quickly. Building contractors work the same way where they allow the customer to inspect the work and make changes. It just costs the customer more.

Add Your Comments Now:

Your Name:(optional)
Your Email:(optional)
Your Location:(optional)
Please type the letters/numbers you see above

.NET Insight

Sign up for our newsletter.

I agree to this site's Privacy Policy.