The other day I wrote about using SYMCHK.EXE from the Debugging Tools for Windows (WinDBG) install to pre-populate your symbol server cache. That prompted a question I got in email:

Our test lab machines are isolated from the rest of our network and have no internet access. How can I get symbols and source over to those machines?

For private builds, those builds you do on your development machine, it’s as simple as copying the source tree to portable storage and putting the PDB files in the same directories as the binaries. Once you copy everything onto the isolated machines, the debugger appropriately finds everything. Life gets a lot more interesting for public builds; those builds done on build machines. Also, the operating system symbols from Microsoft count as a public build machine.

To get all the right PDB files, you’ll need to first run SYMCHK.EXE on the unconnected machine with the /om switch on all the directories you want symbols for. This will include the operating system as well as your products directories. The /om switch builds a text file of all the information necessary to download the symbols but does not download them. You’ll copy that text file to a machine connected to the internet and your company’s symbol server. On that machine, you’ll run SYMCHK.EXE with the /im switch and pass in the text files you created on the unconnected machine. That will populate the connected machine’s cache directory with all the PDB files needed on the unconnected machine.

The PDB files are easy. The more interesting issue is getting the public sources. I’m assuming that everyone reading this has set up source indexing for all your builds, also known as source server, so you can debug all public builds with the right source code. If you don’t know what a source server is, head over here to read an article I wrote about what source server is and how to get it set up on your build server. If you’re using TFS 2010 support for symbol and source servers are included in TFS build.

While SYMCHK.EXE makes getting the PDB files into the local cache easy, we need a similar tool that will do the same for source files. Fortunately, there is such a tool: SRCTOOL.EXE. It’s in the Debugging Tools for Windows location in the SRCSRV directory. By specifying the –x switch and the PDB file you want, SRCTOOL.EXE will execute all the version control commands embedded in the PDB file and get the source for you.

There are two caveats with SRCTOOL.EXE. The first is that it only runs with on single PDB file at a time so if you need to grab the source files for 30 different of your DLLs, you have to run it 30 times. The second is that it doesn’t put the source in your cache directory unless you specify the –d command line option. Obviously, you and I don’t want to manually run SRCTOOL.EXE on each PDB file in our symbol cache so I whipped up a PowerShell script, Get-SourceServerFiles.PS1, at the bottom of this blog entry that will do the work for you. Simply pass in your cache directory and the script will do all the work as well as ensure the source files are placed in the cache directory.

After running SYMCHK.EXE and Get-SourceServerFiles.PS1, you’ll have all the PDB files and source code in the connected machine’s cache. You’ll just need to copy the whole cache directory to a portable drive and take that to the unconnected machine and copy the directory to the local drive. On the unconnected machine, you’ll set up the machine to use your symbol server. When you start debugging, what happens? The debuggers always look in the cache directory for symbols and source so if you have put everything in the cache, there’s no network access.

With a little setup debugging with full source and correct call stacks on unconnected machines is now nearly as easy as debugging locally on your development computer. Go forth and debug!

#requires -version 2

# (c) 2010 by John RobbinsWintellect – Do whatever you want to do with it
# as long as you give credit.

Prepopulate your symbol cache with all your Source Server extracted source code.

Recurses the specified symbol cache directory for PDB files with Source Server sections
and extracts the source code. This script is a simple wrapper around SRCTOOl.EXE from
the Debugging Tools for Windows (AKA WinDBG). You must have SRCTOOL.EXE in the path. It
is in the SRCSRV directory under the Debugging Tools for Windows installation directory.

.PARAMETER cacheDirectory
The cache directory for the local machine.

Param([Parameter(Mandatory=$true)] $cacheDirectory)

Set-StrictMode –version Latest

# Verify SRCTOOL.EXE is in the path.
if ((Get-Command srctool.exe -ErrorAction SilentlyContinue) -eq $null)
    throw “SRCTOOL.EXE does not appear to be in the PATH.”

# Verify the cache directory exists.
if ((Test-Path $cacheDirectory) -eq $false)
    throw “The specified directory does not appear to exist”

# Get all the PDB files, execute SRCTOOL.EXE on each one.
Get-ChildItem -Recurse -Include *.pdb -Path $cacheDirectory | ForEach-Object { SRCTOOL.EXE -d:$cacheDirectory -x $_.FullName }

  • Interesting Finds: June 6, 2010

  • wds_admin

    Thanks for the information about the /om and /im options. Unfortunately, this knowledge comes a bit late, since about 5 years ago (after reading your magnificent ‘Debugging Applications’ book) I wrote my own utility to list all the DLL’s used by the application, logging them in a file, and then calling SYMCHK on a connected machine. That’s for getting the PDB files. Ok, nice. But what do you have to do if you don’t even have the .DLL files?Imagine the following situation: At the customer your application crashes. Luckily you have an unhandled-exception-handler that uses DBGHELP.DLL and its MiniDumpWriteDump function, so you can investigate the .DMP file in the debugger (again a nice tip from your ‘Debugging Applications’ book). Unfortunately the customer is running a Windows flavour, service pack or patch that you don’t have installed at your company (every patch seems to change kernel32.dll, user32.dll, gdi.dll, …). Getting the symbols is then not enough. You also need the correct DLL’s or the debugger will not load the symbols. Our current solution is to ask the customer for the DLL’s, but I think it’s a bit ‘shameful’ to ask customers for DLL’s of their system. Isn’t there a better way? Isn’t there a possiblity to also get the DLL’s from Microsoft’s symbol server?

  • The ApiChange tool does automate at least the download thing a bit.

    I found that symchk will not download the pdbs again if one does already exist although it is the wrong one.

    Alois Kraus

  • John Robbins a publier 2 posts à propos de l’utilisation de SymChk (Symchk.exe, livré avec les Debugging

  • Hi John,
    It would seem that you only need to index source files using srctool.exe if you are using a symbol server. You should have access to the source and line numbers as long as you generate the .pdb files – even without using srctool.exe. Am I correct?
    Thank you for your time.