DevDisasters

Coding Catastrophes

A painful legacy and death by aptitude -- software development gone wrong.

• By Alex Papadimoulis
• 04/01/2009

Experience may be the best teacher, but failure too often is its bitter reward. After all, what software developer worth his salt hasn't committed some code-bound atrocity or embarrassed himself with an ill-advised kludge? The thing is, good developers learn from these failures to become better developers.

I should know. As the author of DevDisasters and founder of The Daily WTF Web site, I've made it my personal mission to stand witness to the painful shortcomings of software developers. The whole thing started back in 2004, when I noticed how frequently programmers tend to discuss and share anecdotes about the poorly written code they encounter every day. It was clear to me that developers could learn an awful lot by reading about some other guy's ugly flameout, rather than experiencing it themselves. And who knows, we all might even enjoy a laugh or two while we're at it.

So, welcome to the first installment of DevDisasters in Visual Studio Magazine. This regular department, which has appeared in VSM's sibling publication Redmond Developer News since 2007, will offer readers a chance to learn the hard lessons without taking the hard hits.

A Painful Legacy
The fourth time will be the charm. That's what Martin F. told himself on the first day of a new assignment. He was the fourth consultant sent from his IT firm to work with the HR department at Pan Euro Investments (PEI), a major European banking and insurance conglomerate. Apparently, the first consultant walked off the job. The second ran. And the third sprinted like a bat out of hell.

"Just to warn you," Martin's new boss started, "I've been told that our database is a bit of a mess. Ever since Rob -- the guy who built the system -- left, it's changed quite a few hands and has experienced a few 'growing pains' over the year."

Growing pains was a gross understatement. Back in the early '90s, PEI purchased an incredibly inefficient and unreliable enterprise resource planning (ERP) system to help HR manage employees. Because the system lacked so many features, the HR department had no choice but to supplement it with applications of its own. That's where Rob came in. He was an HR specialist who "dabbled a bit" in programming and took it upon himself to build the HR department's supplemental systems.

A decade or so later, Rob's supplemental applications were central to the business and the ERP system had faded out. By the time Rob left, the HR department worked with an amalgamation of Microsoft Access databases, VB5 applications and macro-powered Excel spreadsheets, strewn across different workstations and servers.

At the heart of this "system" lay the central database. It held monthly snapshots of all 50,000-plus employees' data going back about four years. Over that period of time, the database had grown to an impressive 2 million records. And for an Access database on a file share, that meant a typical "Select and Group By" query took a good eight minutes.

Diving into the HR-Access Pool
Robert's first task was to correct a few bugs in the "provider importer." It was an Excel-based "application" that PEI's 100-plus daughter companies would use each month. After the appropriate data was filled in, the spreadsheet would generate a few .CSV files and e-mail them to a manager in PEI's HR department.

In turn, that HR manager would forward the e-mail to the appropriate subordinates, who would then do the following:

1. Process every .CSV through an Access-based data-verification program that output "Verified .CSVs"
   
2. Consolidate all the verified files into another Access database
   
3. Find and remove any duplicate employee records
   
4. Export the data into "Duplicate Verified .CSVs" and import it into temporary tables of the central database
5. Perform the final checks and corrections against temporary tables and then consolidate to master tables
6. Run the appropriate reports and send PDF-formatted documents to appropriate managers at daughter companies

This song and dance took two people a full week to accomplish. After sending the reports to the 100-plus managers, they set aside three days for feedback and corrections from the daughter companies, and then went back to step one to repeat the process for the "final" version.

Ol' Teller
Martin had little faith that the code powering this ridiculous process would make any more sense than the process itself. Peering under the veneer of the ordinary-looking spreadsheet, Martin uncovered a writhing heap of undocumented VBA code that made him shudder.

When Martin completed the requested bug fixes, his boss thanked him for the prompt turnaround. Then he quietly added: "Now, ahem, you just need to fix all of the daughter companies' importers."

As Martin begrudgingly edited each and every one of the 100 or so different "provider importer" spreadsheets, he plotted the ways that he could tastefully flee like there was no tomorrow. And who knew, maybe the fifth time would be the charm.

Death by Aptitude
Nothing screams sexy like an ad for a C# developer job at a well-funded start-up. In fact, when Frank's recruiter presented the opportunity to him, Frank knew it would be a long shot. After all, he had no commercial software experience and had only limited Web development experience. But he did know .NET pretty well, so he figured it'd be worth a try. Maybe his skills could translate to a Web-based software company.

His first interview went surprisingly well. They asked a lot of "aptitude" questions, as they were firm believers that "experience can be gained and skills can be learned, but attitude never changes." Apparently, Frank showed good aptitude and was asked back for a second interview.

When Frank came back for round two, he was given the grand tour. It was everything start-up dreams were made of: posh decor, shiny new Macbooks everywhere, an air-hockey table, free catered meals and Aeron chairs as far as the eye could see. After the interview, his future boss Derrick made Frank an offer he couldn't refuse.

Frank's first day consisted mostly of introductions. When he finally made it back to his desk, he couldn't help but wonder about some things. Things like, why would a Web-based software company with 20 employees have only three developers on staff? And why would two of those developers be part-time interns?

The more he thought about it, the more he worried. Frank counted four different program managers. What tasks at the small company could possibly keep the New Product Introduction Manager or Geospatial Systems Program Manager busy all day?

"Did the company really need 10 'manager' level positions?" Frank asked himself. "Shouldn't some of these people have software backgrounds?" But Frank shrugged his concerns off, reasoning that this was the type of team needed to build a successful start-up.

Wait … What?
Once Frank finally settled into his comfy Aeron chair and fired up his dual-monitor workstation, he was ready to learn the code. This, he figured, is where it would all come together and make sense. He heeded his fellow developers' warning that things "were a little beta-ish" and dove right in. After all of two minutes, he wanted to jump back out.

The pages were riddled with race conditions and generally only supported one request at a time. The developers didn't seem to know the difference between Cache, ViewState and Session objects, so they just used all three simultaneously to "make things work." As for the"helper functions" -- they were perhaps the most unique take on .NET coding Frank had ever seen. In one baffling case, the developers created their own areValuesDifferent method to act as a wrapper for object.Equals():

             public bool areValuesDifferent

                (object x, object y) {

                ... snip ...

                if (0 == x.GetType().FullName

                .CompareTo("System.String"))

                if (0 != ((string)x)

                .CompareTo((string)y))

                diff = true;

                if (0 == x.GetType().FullName

                .CompareTo("System.Int64"))

                if (0 != ((long)x)

                .CompareTo((long)y))

                diff = true;

                ... snip ...

                }

As the lead developer later told Frank,"Well, this way, it only checks the value."

Frank took a few minutes to explain the whole concept of "virtual" methods and how Equals() does exactly the same thing. His colleague shrugged his shoulders and replied, "Hmm, I guess that would work, too."

The really good news came later that day, when Frank learned that the company would be hiring some new intern developers. Well, not just any interns, only the "most aptitudinal" that money could buy.