DevDisasters

Dev Disasters: Rounding Errors

• By Mark Bowytz
• 12/01/2011

Usually, when a multi-megabyte e-mail lands in Jerry's inbox, it means that someone tried to be fancy by including some silly photos or graphic images in intra-office correspondence. Jerry would receive one or two of these e-mails a day, prompting an immediate delete in response to Microsoft Outlook's warning that his available e-mail space was running low, but today's big e-mail was different. It was 10MB, but it was all text.

One of the plant managers started the long chain with an angry e-mail to his shipping company. "You should be ashamed of yourselves. Four times now, you've left our chemical plant with only 1,800 gallons of our product in your tank wagons. I know margins are tight, but I'm paying to ship full loads!"

The e-mail thread continued.

A vice president jumped on, pointing out that in the tight economy, the company couldn't afford to waste money shipping anything less than a full load. The operations manager at the plant replied that they were shipping full loads. They measured batches so that one batch exactly filled a tank wagon.

Jerry skipped up to the middle of the thread where some IT people had joined in. "From what I see in the material setup," one of his colleagues explained, "the output density is 4.5 lbs./gal, and they run batches of 16,200 lbs. That's a tank wagon. The data upstream from us must be incorrect."

At that point, Jerry realized why he was in the e-mail chain. He was in charge of supporting a slew of Visual Basic .NET console applications responsible for moving data in and out of the mainframe.

Density Data
After reading through the e-mail chain, he made an educated guess about which console app was probably moving density information around and then started walking through the code. Jerry checked the data at the source, the data at the destination and the data in the console application. Nothing jumped out, but there was something about the DeDecimalize function that struck him as funny:

Public Function DeDecimalize(ByVal val As Object, 
  ByVal preDecimal As Integer, ByVal postDecimal As Integer) As String
  Dim x As String = String.Empty
  Dim y As Double = 0
  Dim z As String = String.Empty
  Dim arrVals() As String
  Try
    x = Convert.ToString(val) 'Turn the number into a string
    arrVals = x.Split(CChar(".")) 'Break it on its decimal point
    If UBound(arrVals) = 0 Then 'handle a null
      ReDim Preserve arrVals(1)
      arrVals(1) = "0"
    End If
    x = arrVals(0).PadLeft(preDecimal, 
      CChar("0")) 'Grab the whole number part and pad it out
    y = Convert.ToDouble("." & arrVals(1)) 
      'Grab the decimal part and turn it back into a decimal
    y = Math.Round(y, postDecimal) 
      'Round it off to the appropriate number of significant figures
    z = Convert.ToString(y) 'Turn it back into a string
    z = Mid(z, 3, postDecimal + 2) 'Grab characters 3-5 
    z = z.PadRight(postDecimal, CChar("0")) 'Pad it out
    x = x & z 'Mash the strings together
  Catch ex As Exception
    Throw
  End Try
  Return x
End Function

Half Its Size
Jerry had a hunch. He tried passing 8.9995 to the function. When it returned "0008000" he was rewarded. The logic never computed properly unless the last three digits rounded off to 1.000.

This meant that it only failed in cases where the decimal part was greater than or equal to 0.9995; or, statistically, roughly one in every 2,000 batches. It seemed insignificant, but as it turned out, the product that the plant manager complained about included most of the ingredients that had the rounding issue, resulting in the tank amount being rounded down to about half of its normal size.

Jerry tweaked the code, tested it and then triumphantly clicked "Reply All" to send around the good news -- but only after removing most of the e-mail thread. After fixing the problem, the last thing he wanted was to be responsible for blowing out anyone's e-mail account.