DevDisasters

Crouching Manager, Hidden Database

If there's one life lesson to be learned here, it's that just because a manager knows how to write code, it doesn't mean that they should be allowed to write code.

• By Mark Bowytz
• 03/02/2015

Blake felt an intense pressure behind his eyes. It migrated up into his forehead, but he kept it in check. He had made his case to his boss in this, his weekly one-on-one update meeting, but it was obvious -- it wasn't going to do any good.

Lately, everything in Blake's life revolved around Asset Trackmaster (Trackmaster, for short) -- the year's biggest project for his team. It made everyone's lives easier having a single system to track assets corporate-wide. The Classic ASP site that queried an IBM AS/400, two Microsoft Access Databases, and two lonely Microsoft SQL Server databases was finally put to bed.

"I understand that you're frustrated with David, but he brings a lot to the table. If you give him a chance, you'll see why he's leading the project," Blake's manager assured.

Blake wanted to scream, releasing all the tension that had built up in one massive eruption, perhaps as a primal yell. Instead, his professionalism downgraded this rage to a sigh. "Alright. I'll keep working with David."

The conversation continued for a little bit longer but was interrupted by an unexpected visitor. It was David.

"Heyyyyyy, boss! I have some information on remote-site performance statistics. Things are looking much better after we deployed the latest fix late yesterday. Still some ways to go but ... Oh! Hey Blake-man! Talking me up? Heh heh."

Blake chuckled politely, imagining how he might talk David "up" with a punch in the nose.

There's Something About David
It wasn't necessarily that David was a bad person in general -- he was just a bad person to work with as a project lead.

Somehow, he had leveraged his years of experience consulting at different companies for a software vendor that Blake's employer used for managing enterprise software licensing into a senior business analyst position in Blake's group. Basically, he would talk to the users, gather requirements, and make sure the developers delivered on promised features.

At first, nobody balked when he asked for a Visual Studio license. A strange request to be made by someone in a strict non-developer role, but Trackmaster was a huge project and it was approved. Soon, things got weird when David insisted that he be allowed Read/Write privileges in Team Foundation Server allowing him to contribute to the project. "I did this all the time when I was at my old job. Believe me, it'll reduce our deployment cycle and you guys will be able to focus on the really tricky parts! Kid gloves ... I swear!" he had assured everybody in many meetings.

Working on his own, he would implement small features, providing only the hooks to access the new functionality, which worked fine but was kept outside of the main branch. Until they broke. Usually on a Friday. Often around 4:30 p.m.

After each crisis was resolved, he'd round up the team and apologize for his "oversight" and if anybody asked for details on what he did, "It's still not quite documented -- I don't want to introduce any noise into the main branch. Believe me, it's for everyone's own good," was his usual reply.

Thankfully, Blake had "seen the light of day" and promised to keep his hands out of the "cookie jar" leaving the actual coding to the development team. This led to fewer issues and, best of all, no more calls in the middle of the night!

However, good things were never meant to last.

"SURPRISE!!" -- 11 p.m. Saturday, Blake's phone unexpectedly went off.

"Hi Blake! This is Jeff from the help desk, we've been getting tickets saying that the TrackMaster2 database is down. We don't have any knowledge articles about this particular database. Can you help out?"

"Wait ... the TrackMaster2 database?! Are you sure?"

After confirming the database name, Blake triaged with the DBA team on-call administrator and, after receiving the all clear, Blake connected to the SQL box on which the mysterious "2" database lived.

Though he had never heard of it, sure enough, Trackmaster had a secondary SQL Server database.

Upon inspection, there wasn't much to say about the database and its contents. In fact, he was nearly ready to lay the issue to rest and move on with his life, however, it did have one notable feature -- it didn't seem to have any stored procedures. Or functions. Or any real tables, for that matter. What was the database used for? The truth was nearly surreal and Blake probably wouldn't have believed it had he not seen it first hand.

It was used for the single purpose of generating GUIDs in possibly the kludgiest way imaginable:

public static string getNewGuid()
{
  SqlConnection connection = new SqlConnection(connectionString);
  connection.Open();
  SqlCommand command = connection.CreateCommand();
  command.CommandText = "select newid()";
  SqlDataReader reader = command.ExecuteReader();
  reader.Read();
  string result = reader.GetValue(0).ToString();
  connection.Close();
  return result;
}

Blake immediately thought back to the project planning meetings and searched his e-mail. No mention of it anywhere.

Thinking it over more, Blake also noticed that the code results in a slow memory leak because the SqlDataReader is not disposed of. Could this be the root cause of the poor client performance and -- maybe -- the reason why the database went down?

Who on earth would pick this method over the tried-and-true call to Guid.NewGuid? Blake reviewed the check-in history of this particular module and ... the culprit was clear, but was not what Blake expected.

Come morning, Blake related the night's events and brought up his observations of newGetGuid with the team, and David beamed at the news, even though he had been the perpetrator in the first place.

"Wow! Well done, Blake-man! Heh, yeah, definitely not one of my best coding decisions. Guess it's best that I'm out of that game! Given you're most familiar with this I'm going to let you take all the glory on this one. I'll book a few hours for us to work together in one of the conference rooms. I want to make sure I really understand the scope of your changes."

At this, Blake found that headache was quickly returning. Thankfully the changes were a series of one-liners. Unfortunately, the problems were scattered across several modules and would take him hours to resolve.