Dev Disasters: Rounding Errors

Usually, when a multi-megabyte e-mail lands in Jerry's inbox, it means that someone tried to be fancy by including some silly photos or graphic images in intra-office correspondence. Jerry would receive one or two of these e-mails a day, prompting an immediate delete in response to Microsoft Outlook's warning that his available e-mail space was running low, but today's big e-mail was different. It was 10MB, but it was all text.

One of the plant managers started the long chain with an angry e-mail to his shipping company. "You should be ashamed of yourselves. Four times now, you've left our chemical plant with only 1,800 gallons of our product in your tank wagons. I know margins are tight, but I'm paying to ship full loads!"

The e-mail thread continued.

A vice president jumped on, pointing out that in the tight economy, the company couldn't afford to waste money shipping anything less than a full load. The operations manager at the plant replied that they were shipping full loads. They measured batches so that one batch exactly filled a tank wagon.

Jerry skipped up to the middle of the thread where some IT people had joined in. "From what I see in the material setup," one of his colleagues explained, "the output density is 4.5 lbs./gal, and they run batches of 16,200 lbs. That's a tank wagon. The data upstream from us must be incorrect."

At that point, Jerry realized why he was in the e-mail chain. He was in charge of supporting a slew of Visual Basic .NET console applications responsible for moving data in and out of the mainframe.

Density Data
After reading through the e-mail chain, he made an educated guess about which console app was probably moving density information around and then started walking through the code. Jerry checked the data at the source, the data at the destination and the data in the console application. Nothing jumped out, but there was something about the DeDecimalize function that struck him as funny:

Public Function DeDecimalize(ByVal val As Object, 
  ByVal preDecimal As Integer, ByVal postDecimal As Integer) As String
  Dim x As String = String.Empty
  Dim y As Double = 0
  Dim z As String = String.Empty
  Dim arrVals() As String
    x = Convert.ToString(val) 'Turn the number into a string
    arrVals = x.Split(CChar(".")) 'Break it on its decimal point
    If UBound(arrVals) = 0 Then 'handle a null
      ReDim Preserve arrVals(1)
      arrVals(1) = "0"
    End If
    x = arrVals(0).PadLeft(preDecimal, 
      CChar("0")) 'Grab the whole number part and pad it out
    y = Convert.ToDouble("." & arrVals(1)) 
      'Grab the decimal part and turn it back into a decimal
    y = Math.Round(y, postDecimal) 
      'Round it off to the appropriate number of significant figures
    z = Convert.ToString(y) 'Turn it back into a string
    z = Mid(z, 3, postDecimal + 2) 'Grab characters 3-5 
    z = z.PadRight(postDecimal, CChar("0")) 'Pad it out
    x = x & z 'Mash the strings together
  Catch ex As Exception
  End Try
  Return x
End Function

Half Its Size
Jerry had a hunch. He tried passing 8.9995 to the function. When it returned "0008000" he was rewarded. The logic never computed properly unless the last three digits rounded off to 1.000.

This meant that it only failed in cases where the decimal part was greater than or equal to 0.9995; or, statistically, roughly one in every 2,000 batches. It seemed insignificant, but as it turned out, the product that the plant manager complained about included most of the ingredients that had the rounding issue, resulting in the tank amount being rounded down to about half of its normal size.

Jerry tweaked the code, tested it and then triumphantly clicked "Reply All" to send around the good news -- but only after removing most of the e-mail thread. After fixing the problem, the last thing he wanted was to be responsible for blowing out anyone's e-mail account.

About the Author

Mark Bowytz is a contributor to the popular Web site The Daily WTF. He has more than a decade of IT experience and is currently a systems analyst for PPG Industries.

comments powered by Disqus


  • What's New in Visual Studio 2019 v16.5 Preview 2

    The second preview of Visual Studio 2019 v16.5 has arrived with improvements across the flagship IDE, including the core experience and different development areas such as C++, Python, web, mobile and so on.

  • C# Shows Strong in Tech Skills Reports

    Microsoft's C# programming language continues to show strong in tech industry skills reports, with the most recent examples coming from a skills testing company and a training company.

  • Color Shards

    Sharing Data and Splitting Components in Blazor

    ASP.NET Core Version 3.1 has at least two major changes that you'll want to take advantage of. Well, Peter thinks you will. Depending on your background, your response to one of them may be a resounding “meh.”

  • Architecture Small Graphic

    Microsoft Ships Preview SDK, Guidance for New Dual-Screen Mobile Era

    Microsoft announced a new SDK and developer guidance for dealing with the new dual-screen mobile era, ushered in by the advent of ultra-portable devices such as the Surface Duo.

  • How to Create a Machine Learning Decision Tree Classifier Using C#

    After earlier explaining how to compute disorder and split data in his exploration of machine learning decision tree classifiers, resident data scientist Dr. James McCaffrey of Microsoft Research now shows how to use the splitting and disorder code to create a working decision tree classifier.

.NET Insight

Sign up for our newsletter.

Terms and Privacy Policy consent

I agree to this site's Privacy Policy.

Upcoming Events