DevDisasters

The Defect Black Market

Software development gone wrong.

• By Alex Papadimoulis
• 02/15/2008

When Damon's coworker stopped by his cubicle for a chat, Damon wondered if they should have met in a dark alley somewhere. Damon, a developer, and his coworker in quality assurance were meeting to trade bugs on the newly created Defect Black Market.

It all started a week before, when the CTO of Damon's midsize warehousing and transportation company in Northern California announced an innovative program to motivate employees and boost the quality of their logistics software. For every bug found by a tester and fixed by a programmer, both would get $10.

Bonus Plan in Operation
The CTO hoped to improve code quality in the firm's far-reaching overhaul of its procurement, purchasing, inventory, warehousing, distribution, transportation, customer support and financial systems. The development team was initially skeptical, but by the end of the incentive program's first day, both coders and testers alike were convinced. After an average day of finding and fixing bugs, they had each earned between 30 and 40 extra dollars.

Everyone worked a bit harder the next day. Testers made sure to check and double-check every test case they ran, while developers worked through lunch to fix their assigned bugs. And it paid off. On that second day each had earned an average bonus of $50.

Everyone worked even harder on the third day. On the fourth day, however, the well had started to dry up. The testers ran, re-ran, and re-ran again the test cases, but they could only find a handful of issues. The developers strained the issue-tracking system, constantly reloading the "unassigned bugs" page and rushing to self-assign anything that appeared.

Lunch Dates
And then something strange happened at lunch. Instead of going out to eat with his usual teammates, one of the developers went out with a tester. Soon after, another developer went out with another tester. Within a few minutes, almost all of the developers had paired up with testers.

As the developers returned from lunch, they immediately got to work. Instead of scavenging for newly found bugs, they worked on "code refactoring" and new functionality. And as soon as they deployed their changes, testers found bugs -- minor, obscure bugs that a developer could easily overlook. And just as quickly as testers found bugs, the developers were able to fix them and re-deploy. By the end of the day, developers and testers had earned an average of $120.

But not all developers made out so well. Some, like Damon, didn't feel comfortable "cheating" by overlooking minute requirements. It just didn't seem right.

Cheating Doesn't Pay
By the sixth day, however, Damon had changed his mind. Sure, it was wrong to cheat. But it was worse that everyone else was doing it and getting away with it and bragging about it. And this brings us back to Damon and his coworker, who had just sat down to discuss how they'd profit on the Defect Black Market.

"So in the new communications module," his coworker started, "if a certain developer was to, say, not fully implement the 'sender address validation' requirement, that'd cause at least eight test cases to fail."

Eight test case failures meant eight bugs, which meant eighty bucks a piece. All Damon would have to do is misplace an "And" with an "Or" in the validation logic.

"Oh and check this out," the tester said, showing Damon the test scripts, "if, *ahem*, someone neglected to use the 'validated' address instead of the 'original' address, there'd be another six failures."

It doesn't feel right, Damon thought, but for a $140, I can overlook one or two simple mistakes.

Tell Us Your Tale Each issue Alex Papadimoulis, publisher of the popular Web site The Daily WTF, recounts first-person tales of software development gone terribly wrong. Have you experienced the darker side of development? We want to publish your story. Send us your 300- to 600-word tale-if we print it, you'll win $100 and an RDN T-shirt! E-mail your story to Senior Editor Kathleen Richards at [email protected] and use "DevDisasters" as the subject line.

Shortly thereafter, Damon found himself feverishly writing code, eager to get his changes -- along with the bug he'd write -- deployed for testing. But when the time came to misplace that "And," he hesitated. Before he plunged headfirst into the Defect Black Market, he decided to take a walk around the office and evaluate if he really wanted to cheat.

"Hey Damon," a fellow developer said as he walked past his cube, "did you hear? They're stopping the accuracy bonuses! The CTO is writing up the announcement now."

And like that, the Defect Black Market was closed. It lasted a grand total of five days.