Symbol Servers are one of the best things that have ever happened in Windows debugging. Having a central store for all PDB files makes the debugging experience so much better because you’ll always get exact right symbols with essentially no effort on your part. This means perfect call stacks on native code and if you’ve done your job correctly by using the Source Server indexing tools, you’ll have the exact version of the source code used for the build automatically show up in the debugger. As the Symbol and Source Servers are trivial to set up, whenever someone tells me they are having trouble getting symbols and source, I do have to wonder if they are serious about developing software.
Probably the most common question I get about the debugger is “Why does the debugger hang on me when I start debugging or occasionally in the middle of debugging?” As wonderful as the Symbol and Source Servers are, I hate to say this, but they are at the root of the problem. In fact, I have a very special extra sensory perception (ESP) skill. Even if I’m not in the office, I can always tell when a company’s Source Server has gone down. It’s all those emails going back and forth saying “Hey, what happened? This morning I was debugging just fine, but every time I start this afternoon, the debugger hangs. Visual Studio’s got a bug.” While I’m proud of my ESP skills, I wish they would tell me the winning lottery numbers instead.
As the Symbol and Source Server are utilizing network file shares all it takes is the server going down or an invalid share specified in either the _NT_SYMBOL_PATH environment variable or Visual Studio Options dialog. With the network timeout blocking the user interface, you have two choices. Either wait it out or kill the debugger with Task Manager. Obviously, the first trouble shooting step is to see if you can ping the Symbol Servers you’ve told the debugger to use. If there’s no response, you’ve solved the problem. With a down Symbol Server, I temporarily work around the problem by starting a Command Prompt or PowerShell window, disable the _NT_SYMBOL_PATH environment variable and start DEVENV.EXE (Visual Studio) or WINDBG.EXE from that shell. When Visual Studio starts, I go into the Options dialog and disable the symbol server there as well.
The other debugger slowdown you run into is when you first start debugging on a new operating system installation, after a major gaggle of updates on Patch Tuesday, or your company’s Symbol Server is based in another country. In all those cases, the debugger slowdown is because of downloading all the symbols into your local cache. As some of these PDB files can be large and networks spotty, it’s a pain.
Fortunately, there’s an easy remedy in the SYMCHK.EXE program that comes with the Debugging Tools for Windows (AKA WinDBG). Instead of having the debugger download the PDB files, the trick is to fill your cache with them before you need them. Of course, if you like having that extra-long debugging session once a month for an extended coffee break you may want to forget all about SYMCHK.EXE.
To make SYMCHK.EXE easier to run always ensure you have the _NT_SYMBOL_PATH environment variable set so you don’t have to specify giant sets of command line parameters. (Previously on my blog, I’ve written PowerShell scripts to make this setup easy for VS 2008 and VS 2010.) To fill your local symbol cache, run the following two commands to get the .NET and operating systems DLLs:
symchk -r C:\windows\Microsoft.NET\*.dll
symchk -r C:\windows\System32\*.dll
Once you get your cache filled, you can keep running those two commands so they pick up any changed files. So I don’t have to remember to run the above commands after, on my machines I set up a Scheduled Task to run the two commands in a batch file every Wednesday morning at 5:00 AM. Obviously, you can run SYMCHK.EXE against your product’s installation directories to get those builds into your cache as well.
Now debugger hangs and stutters should be a thing of the past for you. Less time waiting on the debugger means faster debugging, which means more time writing code. As always if you have any questions about SYMCHK.EXE or symbols in general, let me know in the comments.
Oh! I’m feeling an ESP moment… Numerous of you are thinking “This is all great but what if the debugger hangs on just one or two symbols every time? Is there a way to tell the debugger to skip loading those symbols?” I knew you were going to ask that question, so I wrote about it back on May 29, 2009. How’s that for preparation!?