Inside VSTS

So You Want To Customize a Process Template

Process for process' sake is a waste of time. Jeff gives some pointers on how to create something more useful.

• By Jeff Levinson
• 09/23/2008

One of the most powerful and unique features of Team Foundation Server (TFS) is the ability to customize the process to work the way you work. Most companies can't, and don't, use the process templates included with TFS out of the box. There are lots of reasons for this but it's mostly because every company (and even teams within companies) works differently. Virtually every article on customizing the process templates tells you how to customize the template, but not why. Before you ever get to customizing the process template, you have to understand your process.

This is something that many companies have difficulty with. TFS is a fantastic tool, but it's only as good as the processes which underlie it. The tool isn't the process. The process exists independently of the tool. The tool helps implement the process. These are key distinctions. The tool can only help you when you understand what you want from the tool.

So what should you want when creating the process? The most important thing is information. Process for process' sake is a waste of time and tends to get everyone upset. Every step in the process has to have a purpose (say, to communicate status) and every step has to provide relevant information.

But what information is relevant? Any information which can help you improve your process! Let's take a look at the built in bug process (for the CMMI template) shown in Figure 1.

What information does this process provide you? Table 1 describes the state transitions and the information provided by those transitions. The items with question marks in front of them provide information that's either not important or you need to ask yourself how a bug could get to a given state and have that action taken.

Looking at some of these steps these may seem a bit puzzling and even downright impossible to do easily. The process guidance provides detailed information on how and why a bug transitions between each of these states but it doesn't tell you what information you're supposed to glean from the state transitions.

For instance, the guidance tells you not to open a new bug until you check to see if it has already been reported. But is that helpful? Work is being done against something but it isn't being recorded. That's not guidance that I would personally follow or recommend. Use the Active/Investigate state for that since it's already there for you. Or how about going from Resolved to Closed? The description states that this is done after the person who reported the bug has verified that it has been fixed. But what about the quality assurance team? Did they ever receive this bug to test? There are numerous questions that you can, and should, ask yourself before accepting any process that you haven't thought through.

What a Process Needs To Tell You
So, why might you want to change this particular process? It's a sound process but as I've just shown, it may work in a way that you don't and it may not provide information that's important to you. There are two categories of information that you need to think about: customer-facing information and internal information. You want to be able to tell your customer what's going on with the bug they reported, and you want information which will help you improve your process.

A customer wants to know:

• Is my bug being looked at? (What's the status of my bug?)
• What caused my bug?
• When will the fix be released? (This is different than knowing what build a bug was fixed in.)

You want to know a great deal more:

• How much time am I spending on this bug? (How long are the developers working on this? How long are the analysts working on this? How long are the testers working on this?)
• Who has been working on this bug?
• Do I keep getting the same bug reported (duplicates)?
• How many times do I report a bug as fixed to the QA group and it isn't fixed (re-activations)?
• Are we catching the bugs in Dev, Test or Production?
• How long is the bug sitting with no work being done on it (waste)?
• Is this behavior by design?
• Is this already "fixed" in an upcoming release?

These are a lot of questions and most of them aren't answered by the CMMI process template. So how do you answer these questions? I've proposed a sample bug process that answers most of these questions but which may not be right for you (see Figure 2). You may not need all of this information or all of the activity streams that these require. In addition, one of the benefits of Team System is that you can change the process template as you go and add states as you determine that you have questions that need to be answered.

An Improved Bug Process
This particular process (which may or may not be right for you) provides information I would find important when trying to improve an organizations bug handling process. Let's look at what this process provides.

Creating a new bug and knowing who reported it is critical. If dev is catching bugs, that's great. If QA is catching bugs, then that's good -- at least they aren't making it to production and there's a sound testing process. If users are reporting bugs, that means that somehow I had a bug slip all the way through the process. This is important information for being able to improve the process.

Under investigation is not only a status (I'm telling whoever is interested that work is in progress on the bug) but it also tells me how long my delay between a bug report being filed and triage being performed on it is taking. The investigation period is when research should be done to determine if the reported bug is a duplicate or fits other criteria. This is work and the time spent doing the work should be noted. For instance, if a team is taking a long time to determine that bugs are duplicates, maybe the application architecture needs to be updated to provide more pinpoint information or error numbers to help me trace down problems more quickly.

The time between Investigation Complete and Active is the time the bug is sitting idle. Maybe a number of high priority bugs are not being fixed quickly enough. This will give you a good indication. In addition notice that there is no deferred reason. To defer a bug, move it to a new Iteration and the problem is solved. That way the bug is not prematurely closed (and possibly forgotten).

"Fixed, unverified" very clearly states that the bug is out of development and awaiting testing. Resolved does not say that quite as clearly (also this is terminology that many dedicated bug tracking systems use). The time between this state and "In Testing" again shows a delay of work where the bug is simply idle.

Going from "In Testing" to "Active" shows waste. The developer has to re-do the work. Why didn't they get it fixed the first time? Was there no test case? Was the bug more complicated than was previously thought? This information is critical to understanding how to avoid wasting time.

You may in fact want to add additional steps into this process to track time spent in UAT or to note that a bug is fixed but has not been deployed yet. Which release is it scheduled to be put out in and do you need to track that specifically? Maybe the Iteration classification is enough but maybe not.

The important take away from this is that while the tool matters, the process matters more. And if you don't know why you are doing something, you probably shouldn't be doing it. Always try to understand how the process will help you. If it doesn't, then it's just process for process' sake -- and it will hinder you.