Agile Advisor

Agile Bug Management: Not an Oxymoron

An agile team's goal is to eliminate bug debt. These techniques can help.

• By Gregg Boer
• 10/12/2012

Agile has a refreshing approach to bugs, especially when contrasted with traditional waterfall practices. Historically, teams would postpone bugs to a stabilization phase. This stabilization phase was long and tedious, typically with high stress and long hours. Agile, on the other hand, promotes the following thought processes:

• Carrying bug debt is unhealthy and sometimes, even destructive. We should fix bugs, not track them.
• A feature isn't done until the bugs are fixed. Quality is part of the overall cost.
• Bugs are prioritized against new features. A bug needs to be more important than the next new feature.

I think most agree with the principles behind these statements. They sound great when reading them in a book, but things can be different in the real world of development. This is what I've found:

• Nearly all teams have bug debt. Bug debt is bad, but having no plan to manage bugs is even worse.
• Teams held hostage by the one or two bugs on a newly-developed user story. They've just spent hundreds of hours developing a high-quality story, but they get zero credit in that sprint, because it's not "done". This is especially an issue if management is watching that team's velocity as a measure of productivity. Ugh.
• Having bugs appear on the product backlog along with new features always felt awkward. The bug's granularity is so different from user stories, and we prioritize them differently. For example, do I really care that Bug No. 9 is completed before Bug No. 10?

I've talked with several other agile teams who share the same struggles. We all want to do the right thing with bugs, but still feel this pain.

For me, it's helpful to remember that all new features have a "bug tail." The bug tail is the diminishing number of open bugs on the new functionality. When a feature is added, bugs are introduced. Most are discovered and fixed immediately by the team as a part of feature development: some a few days later, others not until long after a story is considered "done".

If you graphed it out, a bug tail might look like Figure 1. Most bugs are found and fixed immediately within the sprint. However, there are always a few that trickle in later on.

Figure 1. A typical bug tail.

An unhealthy team's bug tail might look like Figure 2. This team is likely not finding most of the bugs within the sprint.

Figure 2. An unhealthy bug tail.

Figure 3 shows what the dreaded stabilization phase might look like: I think we can all agree this is bad.

[Click on image for larger view.]

Figure 3. A stabilization phase, typical of waterfall development methods.

Even a healthy agile team that discovers and fixes bugs while the feature is being developed will still have a bug tail; and that bug tail can overlap the start of the next feature, as shown in Figure 4.

Figure 4. Bug tails overlapping feature development.

Teams should understand what their bug tail is, and strive to reduce the length of it. While the tail may diminish exponentially, it doesn't reach zero very quickly. Bugs are discovered days or even weeks later. Therefore, a team needs a plan to manage its bug debt.

One technique I've found useful is to use a separate bug backlog, and align bug debt reduction with sprint boundaries.

The Bug Backlog is Separate From the Product Backlog
The bug backlog includes all bugs you want to fix on previously-implemented features. It does not include bugs found within a sprint, or on features under development. Bugs found within a sprint are considered work not completed, like tasks. The bug backlog only includes bugs on features previously marked "done".

Before the beginning of a sprint, the product owner must triage all the bugs in the backlog. That is, they have to say "I want all these bugs fixed before starting on new functionality". This is tough, because they have to let go of bugs. This forced trade-off, however, is healthy because it requires the product owner to ask the question: "What would the customer want?" Sometimes a bug is just cut. Sometimes a group of bugs are moved to the backlog as a single story, to prioritize against other features. Regardless, what's left in the bug backlog is prioritized above any features on the product backlog.

The bug backlog isn't sequentially ordered, but sorted by a field such as Priority, which defines classes of prioritization. All bugs in the bug backlog are expected to be fixed. The bug priority informs a developer on how important a bug is, so they can escalate if the actual effort becomes disproportionate to the bug's priority.

Every Sprint, Address Any Known Bug Debt
Every sprint, the team looks at any bugs remaining in the bug backlog and allocates capacity to get that known set of bugs down to zero, or near-zero. Whether this is one day, one week or the entire sprint, they fix the bugs first. Bugs found later, within the sprint, are not considered part of that initial commitment. Unless they're very high priority, they're put on the bug backlog for the next sprint.

Many teams work in a commitment-based organization, where management places a high value on a team's ability to meet their commitments. Doing capacity planning against a known set of bugs makes sprint planning more deterministic, increasing their chance to meet commitments. Any new bugs discovered during the sprint are not a part of the initial commitment, and will be tackled next sprint.

Managing Bug Debt Across An Enterprise
An organization transitioning to a culture where debt is continually eliminated likely is dealing with the following question: How do you get teams to reduce their bug count without telling them exactly what to do? Leadership wants the team to change, yet gives the team autonomy to determine how they change. One option is to use a bug cap.

For example, consider a bug cap of three bugs per engineer. This means a team of 10 people should not have more than 30 bugs in its bug backlog. If the team is over its cap, it's expected to stop work on new features and get under the bug cap. A team is expected to be under its cap at all times, but the team decides how it wants to do that. The bug cap ensures that bug debt is never carried for too long, and the team can learn from the mistakes that causes the bugs to be injected in the first place.

Remember that the bug cap represents the bugs in the bug backlog. It does not include bugs found and fixed within the sprint in which a feature is developed. Those bugs are considered undone work, not debt.

A team always struggling to stay under the cap may have a bug trend similar to that shown in Figure 5.

[Click on image for larger view.]

Figure 5. A team struggling to keep its bugs under the bug cap.

That same team, several sprints later, may have improved its bug trend, as seen in Figure 6. It has hopefully learned ways to shorten its bug tail and eliminate most of it within the sprint. This team doesn't really need a bug cap anymore.

[Click on image for larger view.]

Figure 6. A team with its bugs under control.

An organization with a lot of bug debt can start with a higher bug cap, and slowly lower it with each sprint, until a healthy level is reached. The bug cap can be useful for transitioning an entire organization to a more agile culture. The bug cap, however, is not the goal. It's a transition tool. The goal is to implement a culture where the bug cap is irrelevant.

A healthy team that eliminates its bug tail almost immediately, and carries negligible debt, may no longer need the practices outlined in this article. Awesome! However, moving to a "no debt" culture is likely to be one of the largest paradigm shifts for teams implementing agile. These teams need tools to help them make that transition. The practices discussed in this article are just that: more tools in a team's toolbox.