debugging services

Wintellect is unique in that we have our a separate practice with the skills on staff to debug the nastiest and meanest bugs plaguing our industry. We have solved hundreds of “show stopper” bugs that have meant the difference between successfully shipping a product, and the company going out of business or losing an important contract. When you add up what you lose in development time, client dissatisfaction, and the cost on your reputation, Wintellect’s debugging services pay for themselves. Read some of our Wintellect debugging “war stories” below and check it out.

debugging case file

client
CONFIDENTIAL
status
RESOLVED
the challenge

This customer in the insurance industry came to Wintellect with a major problem. They had a large C++ application that was randomly crashing on customer machines. Having spent the last two months trying to find the problem, they hoped Wintellect could finally get this issue solved.

the result

This application was several hundred thousand lines of code, and had been created by many different developers working on it over many years. None of the current developers on the project had worked with the application for more than a year or two, so the domain knowledge was light, and there were many areas of the code base which no one truly understood well. With this daunting challenge, the company management was extremely price conscious, and did not like the idea of bringing in outside consultants on this bug. A determined development manager convinced them to use Wintellect, but he had to prove value or he was in trouble.

Wintellect’s initial hypothesis was that the team was looking at an uninitialized pointer problem. When a pointer is uninitialized in a release build, the pointer value contains whatever value happened to be there at the time, so programs crash randomly. As there’s no way to know what value is in the memory, other than setting the memory to read only or to no access, the write to the random address occurs and the program crashes differently each time.

The first thing the Wintellect team did was to ensure that all code was compiled at warning level four (/W4), and to treat all warnings as errors (/WX). The team hadn’t compiled with the advanced compiler flags turned on in the past because the legacy code produced so many errors that they gave up. Wintellect wasn’t looking to compile the code, but rather to get the best error checking turned on as was possible.

Wintellect began with the list of C4700 and C4701 errors, which both report uninitialized pointers, and started fixing all those errors one by one. The second step that Wintellect took was a reading of every line of source code looking for pointer allocations, accesses, and reuses. The idea was to find all pointer problems and to ensure each pointer was initialized and properly “de-initialized” by being set back to NULL after use. Through this exercise, which took roughly five hours, we identified another 40-50 places where there were problems with pointer accesses.

After fixing these errors, the client embarked on a major test pass to ensure that nothing was broken by these changes. The test pass finished successfully and the client never got a random crash report again.

the lesson

A pointer in C++ is an accident waiting to happen. Whenever you declare and use them, you need to ensure that you have them initialized and “de-initialized” so that you know exactly what values they have; this is especially true with those pointers declared on the stack.

You might be wondering why Wintellect took a brute force approach to finding this problem. When it comes to pointer problems, it’s the only approach you have. There are no shortcuts to finding these pointer problems.

testimonials

  • As a leader in the statistical software market, Minitab leverages .NET to its fullest in order to pr

    Charles Snellgrove, Chief Technology Officer, Minitab
  • At Montefiore, our Clinical Looking Glass (CLG) appl

    Navneet Srivastava, Montefiore, Manager of Software Engineering