DevDisasters

The Cost of Static

Offshore application development has its merits, but sometimes, as one dev team found, it's just not worth it.

• By Alex Papadimoulis
• 07/01/2009

Steve's development team was meeting with the CTO to discuss the benefits of offshore application development. Specifically, how the company would be piloting it in their group.

"Well, of course you get what you pay for," the CTO agreed, "but in this case, we're paying for the same thing. An hour in this country has just as many minutes as an hour overseas."

Aside from the CTO and some other higher-ups, no one liked the idea of offshoring, but they agreed to try it out. Besides, it was only for one small project.

Fast forward a few months and the "small project" had ballooned into a 10,000-manhour rewrite of the document management subsystem, which was a major module in their flagship, employee performance management application. Not that it mattered, though. Offshore development was very inexpensive.

Of course, a project with such a wide scope required an extensive amount of testing. Steve's group, however, simply didn't have the resources to create test cases, set up test plans and execute the thousands of necessary test scripts. Not that it mattered, though. As it turned out, offshore testing was very inexpensive, too.

After deploying the latest release to their customers, Steve's group ran into another resource problem. With the vast number of changes to the document management subsystem, there were a lot of confused users who needed help with the new software. Steve's group simply couldn't keep up. Not that it mattered, though. Offshore support was also very inexpensive.

Offshore Debugging
Despite the resources available overseas, there were two bugs that plagued countless companies that used the app. First, when users would save a document, sometimes the changes wouldn't "stick." The offshore support team attributed this to "user error" and resolved it by reminding users how to save correctly.

The other issue was that some documents would have "mixed" data. Employees would have strange performance appraisals tacked on to their records from managers in entirely different departments. The support team attributed these problems to "random data anomalies" and fixed them by manually correcting the database.

While most users just learned to deal with the bugs, their largest customer threatened to switch to a new vendor. Steve's team was notified about the issue.

Working in parallel with another developer, Steve stumbled across a rather troublingly named file: globals.cs. After digging through hundreds of poorly implemented "helper" functions, Steve discovered the root of the problem:

public static string UserID;
...
public static int DocumentID;

The "static" keyword ensured that the user- and document-identifier were always available, without ever needing to create an instance of an object. In a multi-user, Web-based system, having these globally defined variables didn't work out so well.

If two different documents were worked on at the same time, the first's modifications would be saved to the second's and the second's would be lost. The same thing happened with user names.

Steve looked through the customers' data files and noticed that the problem had been occurring since the rollout, causing nearly six months of data to be incorrect. But, because users generally don't double-check everything they enter, only a handful of bad data was ever noticed by the customers.

Steve's finding didn't go over so well with their largest customer, or, really, any of their customers. Quite a few left immediately, and many more demanded refunds.

So, while the company saved a tremendous amount of money through offshore application development, after all was said and done the cost of the statically defined variables was estimated in the millions-in lost revenue, troubleshooting and court costs.