With Paraffin 3.5, I included this document, but thought it would be worth posting as well for those of you that aren't familiar with Paraffin. You can download the lastest release here: http://www.wintellect.com/CS/files/folders/18310/download.aspx

Introduction

When building WiX-based installers, you quickly run into the problem of maintaining the list of files you want to install. The HEAT tool that comes with WiX does a great job creating the initial fragments, but doesn't have any support for what happens during normal development: you're adding and removing files all the time. For many installs, HEAT works fine, but if you have an installer that's installing hundreds to hundreds of thousands of files, the thought of maintaining those fragments of XML by hand would fill anyone with dread.

A few years ago, I was running into that same trouble with my WiX installers right around the time .NET 3.0 and LINQ were starting their beta testing. Seeing a need to scratch my itch, I dove in and created Paraffin. The idea was simple, you could maintain your fragments without breaking component rules as files were added and removed from your installer. Paraffin has grown as more people have started using it and suggest features. Thanks to everyone who's used Paraffin!

This guide is to help you get more out of Paraffin. Obviously, it assumes you are conversant with WiX and Windows Installer. If you're new to WiX, you should head over to Gábor DEÁK JAHN's excellent tutorial (http://www.tramontana.co.hu/wix/).

TWO SUPER IMPORTANT NOTES ABOUT PARAFFIN!

The first key to maximizing Paraffin usage is that you always run Paraffin updates from the same relative directory where you first created the initial WiX fragment. For example, if you change to C:\BUILD\IMAGES in PowerShell and create the initial WiX fragment for the IMAGES directory, all updates have to be run from the IMAGES directory. You can hard code starting directory paths in Paraffin, but that will probably break your builds. By running from the IMAGES directory, you are in the same place relative to the rest of the code in your WiX project every time. This allows more flexibility.

The second key is that in Windows Installer, component rules are sacred. Paraffin works very hard to avoid issues, but if you are not careful you can make your life super difficult. This warning is mainly around patching issues. Patching is far more difficult to get right than it should be. From my experience, you would be much better off sticking to making everything a major upgrade for every install. Based on many requests, I've made Paraffin much more usable in patching scenarios (minor upgrades), but you really do need to know what you're doing and have planned your install completely.

Creating Your First Fragments

Paraffin supports three modes of execution, creating fragments, updating fragments for major upgrades, and updating fragments for minor upgrades. Obviously, you first have to create a fragment in order to upgrade that fragment.

Required Parameters for Initial File Creation

Optional Parameters for Initial File Creation

Deprecated Optional Parameters for Initial File Creation

Updating Fragments for Major Upgrades

As I mentioned earlier, major upgrades are really the way to go with Windows Installer. They alleviate you of worrying very much at all about what happens when you install a new version. Before the new bits are installed, Windows Installer uninstalls the old version so there are no problems with component rules, left over goo, or anything else associated with minor upgrades. If you don't have major prerequisites, such as SQL Server, specific versions of .NET, or anything else complicated, you don't even need a boot strapper, like the long dreamed about Burn, everything can be done with just the .MSI file.

When using Paraffin to update an existing fragment for a major upgrade, any new files found in the directory are added as appropriate. The Components elements for files that no longer exist are removed. For files that haven't changed the Component element GUID and Id attributes are properly transported to the output file so the component rules don't break.

It's important to realize that Paraffin does not overwrite the existing <input>.WXS file but writes out the changes to <input>.PARAFFIN. Numerous Paraffin users have asked for Paraffin to overwrite the file, but we're dealing with Windows Installer here. It's a brittle and scary API in Windows so I'm loath to go down that route.

Required Parameters for Major Upgrade Processing

Optional Parameters for Major Upgrade Processing

Updating Fragments for Minor Upgrades

Adding files is trivial in a minor upgrade. The problem becomes when you need to remove a file during that minor upgrade. According to the Window Installer rules, that's requires you go directly to a major upgrade. Many people have asked me to implement the scheme described by Vagmi Mudumbai in his blog: http://geekswithblogs.net/Vagmi.Mudumbai/archive/2006/06/11/81426.aspx, which will remove files during minor upgrades.

In WiX, the above would look like the following (word wrapped for this document). The trick is to add the Transitive attribute to the Component element, and add a nonsensical condition. The transitive forces the condition to be reevaluated and the condition is false. Thus, the file is uninstalled in a minor upgrade.

<Component Id="comp_F5B392C05B5249C2AB34810BE4A6163F"
           Guid="FD9D7BF4-550A-47E2-85A3-634D77F609FB"
           Transitive="yes">
  <File Id="file_81DD236ED61B4BBEB445FAB9E9C2DAB2"
        Checksum="yes"
        KeyPath="yes"
        Source=".\Temp\a.exe" />
  <Condition>1 = 0</Condition>
</Component>

There's one more step that's involved. The file must be there when building your install and it's also best if that file is only zero bytes long so it doesn't take up space in the .MSP. Don't worry Paraffin takes care of all of this mess for you. All you have to do is to delete the files you used to install before running Paraffin to build the minor upgrade ready file.

Once you've deleted a file for a minor upgrade, Paraffin assumes it's deleted forever. If you try to add a file with the same name and directory back, Paraffin will abort processing because you are going to break the component rules. If you need to add a file with a previously deleted name you need to use a major upgrade. While I'm sure someone will ask that I make Paraffin handle that situation, I won't because you're probably screwing up your installation.

With the minor upgrade checking, Paraffin is only looking at files it does not look at directories. If you delete or rename a directory, you need to use major upgrades. The trick of using the transitive attribute only applies to the Component element. There's no way for a minor upgrade to remove a directory so there's no need for me to implement support for it.

Required Parameters for Minor Upgrade Processing

Optional Parameters for Minor Upgrade Processing

Creating the Removed Zero Byte Files

If you're like me, you probably don't want to have these odd zero byte files necessary for the minor upgrade file removal littered through your version control system. Paraffin offers a separate command line option to go create all those zero byte files on the fly. The idea with this command line option is you'll run Paraffin with it as part of your master builds so you don't have to worry about those long ago deleted files ever again.

Required Parameters for Zero Byte File Creation

Optional Parameters for Zero Byte File Creation

Paraffin Exit Codes

Paraffin .ParrafinMold files

In order to support more customization of the Paraffin output, Paraffin supports the concept of injectable data through files that have the .ParrafinMold extension. For example, since Paraffin does not handle COM and .NET assembly registration, which HEAT does, you may want to put the COM information into a .ParaffinMold file so it can be injected automatically.

As Paraffin is processing a directory and finds any files matching *.ParaffinMold, it will inject the contents of each .ParaffinMold file into the output file under the current element (normally Directory, or DirectoryRef if -norootdirectory is used).

All .ParaffinMold files are WiX fragment files must have a DirectoryRef element, but the Id value is ignored. Any elements under the DirectoryRef are placed directly in the output file. There is no checking done on thevalidity of any of those child nodes. Here is an example .ParaffinMold file to insert a Registry element.

<?xml version="1.0" encoding="utf-8"?>
<Wix xmlns="http://schemas.microsoft.com/wix/2006/wi">
  <Fragment>
    <DirectoryRef Id="INSTALLDIR">
      <Component Id="INSTALLDIR"
              Guid="PUT-GUID-HERE">
        <RegistryKey Root="HKLM"
                   Key="SOFTWARE\My Company\My Product"
                   Action="createAndRemoveOnUninstall">
          <RegistryValue Name="InstallRoot" Value="[INSTALDIR]"
               Type="string" KeyPath="yes"/>
        </RegistryKey>
      </Component>
    </DirectoryRef>
  </Fragment>
</Wix>

Notes for Upgraders to Paraffin 3.5

Prior to Paraffin 3.5, Paraffin supported multiple files per component. Since one file per component is much preferred for Windows Installer resiliency, and I couldn't find anyone using multiple files per component, I removed support for it. By doing so it made my development life easier and greatly simplified the new -PatchUpdate feature. If you happen to need multiple files per component, please keep using Paraffin 3.13 you can find here: http://www.wintellect.com/CS/files/folders/8198/download.aspx.

Paraffin used to put the KeyPath attribute on the Component. It looks like now it's much better to have that on the File element itself. If you use Paraffin 3.5 to update a .WXS file created with a previous version, I swap the KeyPath attribute as appropriate. That shouldn't cause any problems for anyone, but I wanted to give you notice of the change. You'll also see yellow warning text on the screen when you run Paraffin when the attribute swap occurs.

Slowly but surely I'm getting through the list of feature requests everyone's been asking for. This release of Paraffin brings support for the trick outline by Vagmi Mudumbai (http://geekswithblogs.net/Vagmi.Mudumbai/archive/2006/06/11/81426.aspx) of allowing minor upgrades (AKA patches) to remove files by setting the transitive bit on the component and adding a conditional element that evaluates to false.

<Component Id="comp_F5B392C05B5249C2AB34810BE4A6163F"
           Guid="FD9D7BF4-550A-47E2-85A3-634D77F609FB"
           Transitive="yes">
  <File Id="file_81DD236ED61B4BBEB445FAB9E9C2DAB2"
        Checksum="yes"
        KeyPath="yes"
        Source=".\Temp\a.exe" />
  <Condition>1 = 0</Condition>
</Component>

So if you need to delete A.EXE on a minor upgrade, delete A.EXE from disk and run Paraffin with the new –PatchUpdate switch. The output .PARAFFIN file will have the following in it for A.EXE

Those of you who've looked at Vagmi's trick are wondering about the other piece that needs to be there; the file. But you deleted the file. If you build with WiX, you'll get an error because the File element must point to a file to put in the output .MSP. To keep the patch file small, you should have a zero byte file called A.EXE to make everything work.

Fortunately, Paraffin has you covered there as well. When you specify the –PatchUpdate switch, you can add the new –PatchCreateFiles switch and Paraffin will create the zero byte files for you. Don't worry, when you rerun Paraffin again with –PatchUpdate, if the zero byte file is there, Paraffin checks if the Transitive attribute is set and makes sure to keep everything straight.

While you could keep those zero byte files in your version control system, which seems a little odd to me since they are, well, no longer needed. Because Paraffin knows which files have been deleted with the Transitive attribute on the Component element you can recreate the zero byte files at any time by using –PatchUpdate file.wxs as the command line. The idea is that you'll run this as part of your build so you never have to worry about those zero byte files.

While I believe major upgrades are really the way to go since they are much easier to manage, I know a lot of you using Paraffin are doing minor upgrades. Hopefully this will make your WiX life a little easier.

A second new feature I added is the –ReportIfDifferent switch. Several people told me they wanted a way they could tell if when updating, through –Update (for major upgrades) and the new –PatchUpdate switch, if the output .PARAFFIN file was different than the input file. As one person told me they wanted to run the update during a build and if the files were different, to fail the build in case someone forgot to update the .WXS file to reflect changes. With –ReportIfDifferent, the exit code for Paraffin will be four if the input .WXS and output .PARAFFIN files are different. If they are the same, the exit code is zero.

I did remove a feature from Paraffin, support for multiple files per component. All the people I talked to using Paraffin were using one file per component because it offers better resiliency with Windows Installer. By removing this support, it made my development life easier as well. If you need multiple files per component support, keep using Paraffin 3.13 (http://www.wintellect.com/CS/files/folders/8198/download.aspx).

Another change for upgraders is I changed to adding the KeyPath on the File element instead of the Component element. If you use Paraffin 3.5 to update a .WXS file created with a previous version of Paraffin, I swap the attributes around.

While I never cared what you did with Paraffin or its source code, not having a license in the download was freaking out many of your lawyers. Without legal language somewhere lawyers head's explode. I threw in the CPL license file to make your life easier. However, I still don't care what you do with the code. All I ask is that you let me know about any bugs you find or features you want in Paraffin.

Here's the list of minor changes I made in Paraffin 3.5.

• Added a Zen of Paraffin document to help explain how to best use Paraffin.
• Added the -verbose command line option for verbose output.
• Now the DiskId attribute is only added if the value is not the default of 1.
• Refactored the code into three different files for ease of development.
• Removed deprecated command line switches and the code reporting their usage.
• Warnings are written in yellow text and errors are now written in red.

Here's the program and the source: http://www.wintellect.com/CS/files/folders/18310/download.aspx.

Every once in a while you run across an undocumented trick that totally and completely makes your day. A great example of that is the ability to debug MSBuild scripts in Visual Studio. Today I ran across another amazingly useful feature in WinDBG and SOS for .NET 4.0 that will save you countless hours of typing when analyzing minidumps. If you thought using .cmdtree to click your way to SOS happiness was amazing, you're going to love this one.

The next time you start a WinDBG debugging session with SOS 4.0, issue the following undocumented command before you run any SOS commands:

.prefer_dml 1

If you've been using WinDBG for a while, you might have heard of DML (Debugger Mark Up Language). It's a feature of WinDBG where you can get hyperlinks in specific command output, such as LM (Loaded Modules) where you can click on a module to get more information about that module. To see DML in action, execute the lm /D command and you'll see the hyperlinks in the Command window like the following:

Clicking on one of the links will execute the lmDvm<module> command which not only shows the version, but allows you to dump all the native symbols using the X command.

So what does DML have to do with SOS? It turns out everything! Once you've issued the .prefer_dml 1 command some serious magic happens. For example, here's the output of the !dso (Dump Stack Objects) command:

You've never seen !dso look so sexy, have you? If you click on one of those object addresses, that will issue the !dumpobj /d <address> command for that address:

With SOS 4.0 Microsoft has implemented DML in most of the commands! If you want the clickable output on an SOS command, add the /d command line option. Here's an example with !threads /d:

Even the most useful command, !dumpheap, supports the /d option:

Get in the habit of adding /d to SOS 4.0 commands so you can debug faster by clicking your way to happiness. As I mentioned, not all commands support /d, but most do. Note that I'm using the latest WinDBG, 6.12.002.633, from the Windows SDK. I haven't checked if the SOS DML works in previous versions of the debugger but suspect it will.

A huge part of the joy of our business is discovering new features even if they haven't been fully supported that make your life easier. It's like the developers were anticipating your needs and thinking about what would make your life better. The SOS 4.0 developers deserve a ton of credit for bringing a huge smile to my face and everyone who needs to debug those tough to solve problems!

See the corrected entry with pictures http://www.wintellect.com/CS/blogs/jrobbins/archive/2010/07/14/a-cool-windbg-sos-hidden-feature.aspx.

Before I get into the quick new features for Paraffin 3.13 (download here), I need to ask for your help. Pretty much from the first day I released Paraffin, I've been consistently getting the following question:

I want to be able to uninstall obsolete components during minor upgrades.

For this I would like to have an additional option, which does not remove missing/deleted files from the .Paraffin file. Instead it should generate the code according to http://geekswithblogs.net/Vagmi.Mudumbai/archive/2006/06/11/81426.aspx.

Paraffin should set component to transitive and use a false condition. If a file does not exist it should use a 0-byte dummy file.

I think that's a great feature, but being mostly a Windows Installer neophyte, I have no idea what that's supposed to look like in actual WiX format. If one of you WiX ninjas can give me a functioning sample of an installer with a functioning patch using the above trick in WiX, I'll get that feature worked into Paraffin.

While I think major upgrades are the way to go because they are easier to manage, I know there are some of you using Paraffin where patching is a requirement. Shoot that sample to john @ this company's domain.

Paraffin 3.13 now supports a new command line option –regExExclude. Now you can exclude files and directories based on regular expressions to give you much more control. The new switch works for both creating files as well as when updating files to add new exclusions after you've already created your initial file. The –ext switch still excludes by file extension and –direXclude continues to exclude directories based on a value contained in the string. How the implementation work is that for both files and directories, I process the –ext and –direXclude switches before I process the –regExExclude. I thought that would be better for backwards compatibility.

In the Paraffin download, I updated the example installer to show using the new switch when building the WiX fragment for the output Debug directory. All you want included from the compiled output is the .EXE and .PDB files, but Visual Studio has a bunch of other files like Paraffin.VSHOST.EXE and with Code Analysis turned on, those *.CodeAnalysisLog.xml and *.lastcodeanalysissucceeded files. With the new –regExExclude switch, you can specify the following on the command line to avoid those files

-rex ".*\.vshost\.exe.*" -rex ".*codeanalysis.*"

If you're not familiar with regular expressions, now you have a real reason to learn them. One final note is that for files the regular expression is tested against just the filename. For directories, your regular expression is tested against the complete drive and path. This new trick should make Paraffin easier to control for many of you.

The second thing I did for version 3.13 was remove something, the file creation node in the comment section. Many people wanted to be able to do automatic diffs on a .WXS and the updated .PARAFFIN file and the creation node messed those up. Removing the node means you can automate running Paraffin with the –update switch as part of your build and if your diff tool reports a difference, you can halt the build until the installer gets properly fixed.

My plan is to have another release of Paraffin soon that adds some major features. Hisham Zreiq sent me some awesome extensions to Paraffin that incorporates COM registration, by using the same code Heat does. I'll make this an optional flag to ensure it works with existing Paraffin uses. In addition to the above mentioned possible patching support, I also want to incorporate the latest WiX 3.5 language changes for the Component element, which will make my code drastically simpler.

As always, please don't hesitate to let me know you have ideas for Paraffin because I'd love to hear about them.

Have you ever tried to figure out why your MSBuild-based build is failing? While the diagnostic output is OK, it really isn't an each task (no pun intended) to debug what's going on in an MSBuild file with the traditional means of <Message> tasks. Fortunately, Dan Mosely, one of the MSBuild developers, had some time on vacation and got MSBuild debugging working in the IDE! This is a secret feature is in the VS 2010 IDE, but not enabled for the RTM bits. Dan has finally blogged about how to enable the awesomeness he wrote on his own time. When Dan first showed me how easy it was to debug an MSBuild file, I shouted with joy because this is the one thing that's stopped people from taking the MSBuild automation approach to the next level. The only problem with my shout of joy is that I had to keep quiet about it until Dan shared it with the world. I've been debugging my custom MSBuild files with the IDE for a while and it works like a dream.

Remember, this is an unsupported feature, but for those of us doing more than just straight project files, meaning everyone that does real software development builds, Dan's debugging engine is a super nice treat. Maybe we should ask Microsoft to let Dan go on vacation more often!

Here’s my latest script for setting up a user’s_NT_SYMBOL_PATH environment variable as well as Visual Studio 2010’s symbolsettings. In a previous version of this script, I wasn’t setting the publicsymbol server cache directory in Visual Studio so you could end up with downloadedsymbols in C:\SYMBOLS\PUBLIC instead ofC:\SYMBOLS\PUBLIC\MICROSOFTPUBLICSYMBOLS.

#requires -version 2.0

#Wintellect .NET Debugging Code
#(c) 2009-2010 by John Robbins\Wintellect - Do whatever you want to do with it
# aslong as you give credit.

<#.SYNOPSIS
Setsup a computer with symbol server values in both the environment and in
VS 2010.
.DESCRIPTION
Sets up both the _NT_SYMBOL_PATH environment variable and Visual Studio 2010
to use a common symbol cache directory as well as common symbol servers.
.PARAMETER Internal
Sets the symbol server to use to \\SYMBOLS\SYMBOLS. Visual Studio will not use
the public symbol servers. This will turn off the .NET Framework SourceStepping
You must specify either -Internal or -Public to the script.
.PARAMETER Public
Sets the symbol server to use as the two public symbol servers from Microsoft.
All the appropriate settings are configured to properly have .NET Reference
Source stepping working.
.PARAMETER CacheDirectory
Defaults to C:\SYMBOLS\PUBLIC\MicrosoftPublicSymbols for -Public and
C:\SYMBOLS\INTERNAL for -Internal. If you specify a different cache directory
with -Public, MicrosoftPublicSymbols will always be appended. This is to
avoid issues with Visual Studio downloading the symbols to a differentlocation.
.PARAMETER SymbolServers
A string array of additional symbol servers to use. If -Internal is set, these
additional symbol servers will appear after \\SYMBOLS\SYMBOLS. If -Public is
set, these symbol servers will appear after the public symbol servers so both
the environment variable and Visual Studio have the same search order
#>
[CmdLetBinding(SupportsShouldProcess=$true)]
param ( [switch]   $Internal      ,
              [switch]   $Public         ,
              [string]   $CacheDirectory ,
              [string[]] $SymbolServers   )
       
#Always make sure all variables are defined.
Set-PSDebug -Strict        

#Creates the cache directory if it does not exist.
function CreateCacheDirectory ( [string] $cacheDirectory )
{
       if ( ! $(Test-path $cacheDirectory -type "Container" ))
       {
              if ($PSCmdLet.ShouldProcess("Destination:$cacheDirectory" ,
                                    "CreateDirectory"))
              {
                     New-Item -type directory -Path $cacheDirectory > $null
              }
       }
}

function Set-ItemPropertyScript ( $path , $name , $value , $type )
{
    if ( $path -eq $null )
    {
        Write-Error "Set-ItemPropertyScriptpath param cannot be null!"
        exit
    }
    if ( $name -eq $null )
    {
        Write-Error "Set-ItemPropertyScriptname param cannot be null!"
        exit
    }
       $propString = "Item: " + $path.ToString() + " Property:" + $name
       if ($PSCmdLet.ShouldProcess($propString ,"SetProperty"))
       {
        if ($type -eq $null)
        {
               Set-ItemProperty -Path $path -Name $name -Value $value
        }
        else
        {
               Set-ItemProperty -Path $path -Name $name -Value $value -Type $type
        }
       }
}

# Dothe parameter checking.
if ( $Internal -eq $Public )
{
    Write-Error "You must specify either -Internal or-Public"
    exit
}

#Check if VS is running.
if (Get-Process 'devenv' -ErrorAction SilentlyContinue)
{
    Write-Error "Visual Studio is running. Pleaseclose all instances before running this script"
    exit
}

$dbgRegKey = "HKCU:\Software\Microsoft\VisualStudio\10.0\Debugger"

if ( $Internal )
{
       if ( $CacheDirectory.Length -eq 0 )
       {
       $CacheDirectory = "C:\SYMBOLS\INTERNAL"
       }

    CreateCacheDirectory $CacheDirectory
   
    # Default to \\SYMBOLS\SYMBOLS and addany additional symbol servers to
    # the end of the string.
    $symPath = "SRV*$CacheDirectory*\\SYMBOLS\SYMBOLS"
    $vsPaths = ""
    $pathState = ""

       for ( $i = 0 ; $i -lt $SymbolServers.Length ; $i++ )
       {
        $symPath += "*"
        $symPath += $SymbolServers[$i]
       
        $vsPaths += $SymbolServers[$i]
        $vsPaths += ";"
        $pathState += "1"
       }
    $symPath += ";"
   
    Set-ItemPropertyScript HKCU:\Environment _NT_SYMBOL_PATH $symPath
   
    # Turn off .NET Framework Sourcestepping.
    Set-ItemPropertyScript $dbgRegKey FrameworkSourceStepping 0 DWORD
    # Turn off using the Microsoft symbolservers.
    Set-ItemPropertyScript $dbgRegKey SymbolUseMSSymbolServers 0 DWORD
    # Set the symbol cache dir to the samevalue as used in the environment
    # variable.
    Set-ItemPropertyScript $dbgRegKey SymbolCacheDir $CacheDirectory
    # Set the VS symbol path to anyadditional values
    Set-ItemPropertyScript $dbgRegKey SymbolPath $vsPaths
    # Tell VS that to the additional serversspecified.
    Set-ItemPropertyScript $dbgRegKey SymbolPathState $pathState
   
}
else
{
       if ( $CacheDirectory.Length -eq 0 )
       {
       $CacheDirectory = "C:\SYMBOLS\PUBLIC"
       }
      
       # For -Public, we have to putMicrosoftPublicSymbols on the end because
       # Visual Studio hard codes that on forsome reason. I have no idea why.
       if ( $CacheDirectory.EndsWith("\") -eq $false )
       {
              $CacheDirectory += "\"
       }
       $CacheDirectory += "MicrosoftPublicSymbols"

    CreateCacheDirectory $CacheDirectory
   
    # It's public so we have a littledifferent processing to do. I have to
    # add the MicrosoftPublicSymbols as VShardcodes that onto the path.
    # This way both WinDBG and VS are usingthe same paths for public
    # symbols.
    $refSrcPath = "$CacheDirectory*http://referencesource.microsoft.com/symbols"
    $msdlPath = "$CacheDirectory*http://msdl.microsoft.com/download/symbols"
    $extraPaths = ""
    $enabledPDBLocations ="11"
   
    # Poke on any additional symbol servers.I've keeping everything the
    # same between VS as WinDBG.
       for ( $i = 0 ; $i -lt $SymbolServers.Length ; $i++ )
       {
        $extraPaths += ";"
        $extraPaths += $SymbolServers[$i]
        $enabledPDBLocations += "1"
       }

    $envPath = "SRV*$refSrcPath;SRV*$msdlPath$extraPaths"
   
    Set-ItemPropertyScript HKCU:\Environment _NT_SYMBOL_PATH $envPath
   
    # Turn off Just My Code.
    Set-ItemPropertyScript $dbgRegKey JustMyCode 0 DWORD
   
    # Turn on .NET Framework Source stepping.
    Set-ItemPropertyScript $dbgRegKey FrameworkSourceStepping 1 DWORD
   
    # Turn on Source Server Support.
    Set-ItemPropertyScript $dbgRegKey UseSourceServer 1 DWORD
   
    # Turn on Source Server Diagnostics asthat's a good thing. :)
    Set-ItemPropertyScript $dbgRegKey ShowSourceServerDiagnostics 1 DWORD
   
    # It's very important to turn offrequiring the source to match exactly.
    # With this flag on, .NET ReferenceSource Stepping doesn't work.
    Set-ItemPropertyScript $dbgRegKey UseDocumentChecksum 0 DWORD
   
    # Turn on using the Microsoft symbolservers.
    Set-ItemPropertyScript $dbgRegKey SymbolUseMSSymbolServers 1 DWORD
   
    # Set the VS SymbolPath setting.
    $vsSymPath ="$refSrcPath;$msdlPath$extraPaths"
    Set-ItemPropertyScript $dbgRegKey SymbolPath $vsSymPath
   
    # Tell VS that all paths set are active(you see those as check boxes in
    # the Options dialog, Debugging\Symbolspage).
    Set-ItemPropertyScript $dbgRegKey SymbolPathState $enabledPDBLocations
   
    # Set the symbol cache dir to the samevalue as used in the environment
    # variable.
    Set-ItemPropertyScript $dbgRegKey SymbolCacheDir $CacheDirectory
   
}
""
"Pleaselog out to activate the new symbol server settings"
""

AT&T has finally enabled internet tethering so you can use your iPhone as a 3G connection. Those of you outside the US that have had iPhone tethering for years can feel free to laugh all you want now. Anyway, the downside to using an iPhone on a PC is that you need to install iTunes. While Apple makes wonderful software for OSX, iTunes on the PC is well, how can I say this, umm, kind of big and fat. My wife has iTunes on her PC and it's been a hassle at times. As I use OSX for all my music, photo, and video needs, I've never needed to run iTunes on my Windows machines. With my upgrade to iPhone 4 (which I can't rave enough about because it fixed all my dropped call problems), I realized to use the iPhone for tethering to get an internet connection, I was looking at installing iTunes, which was filling me with dread.

As iTunes was downloading, I was thinking I was going to be installing a truck load of programs and services just to get an attachment to the iPhone. All I needed were the device drivers for the iPhone, not the rest of the stuff in the package. As I've hacked through Apple's installers before, and have been doing Windows development far too long, I set out to see if I could get just the hardware drivers without the rest of iTunes. It turned out to be far easier than I ever would have guessed! I'm fully tethered and nary an iTunes or Apple service in sight. Even better, I didn't have to jail break or hack the iPhone at all.

Obviously you need to be set up with AT&T (or your service provider if outside the US) and have purchased the tethering plan before attempting these steps. The following instructions might work with phones other than the iPhone 4, but I haven't tested any other combinations. My operating system is Windows 7 x64 but I suspect this might work for other operating systems but don't know for sure. One major point is that the iTunes-less internet tethering requires a Bluetooth connection. Since most modern laptops have Bluetooth, that's not much of a problem. If you don't have a Bluetooth connection, there's spiffy $20 Bluetooth USB dongles that should work. I spent no time trying to get a USB connection to work so you're on your own there. Another item I need to point out is that by following these steps, you'll lose the ability to use your iPhone as a USB file drive. I'm sure there's a way to get that capability back, but I haven't looked at what it would take.

Finally, this is obviously not supported at all by Apple or AT&T nor am I an employee of either company. This works for me but it might your fish to swim upside down, your kids to dye their hair purple with yellow racing stripes right before graduation pictures, or blow up your machine so I proclaim myself blameless for any bad things that could happen. With that disclaimer, here are the steps that worked for me.

This blog entry may prove popular with people outside my normal hardcore software developer audience so I've made the steps very explicit in order to help avoid bricked phones and laptops. All disclaimers still apply!

1. Disconnected your iPhone from the computer if it's connected with a USB cable.
2. Download iTunes. It is critical you select the right version for the "bitness" of your operating system installation. If you're running an x64 operating system, you have to get the 64-bit version. If you're not sure what you have, Microsoft has a Knowledge Base article for you.
3. Open Explorer. Enter "%TEMP%" (without quotes) in the address bar to navigate to your TEMP directory. For the curious, %TEMP% is the environment variable expansion Explorer supports.
4. Run the iTunes64Setup.exe or iTunesSetup.exe program. DO NOT CLICK ANY BUTTONS ON THE SETUP USER INTERFACE.
5. With the Explorer window still open to the %TEMP% directory, look for a directory with the most current time stamp. In my case, the directory was names IXP686.TMP, but the name is generated randomly. If you're having trouble finding the directory search for AppleMobileDeviceSupport64.MSI or AppleMobileDeviceSupport.MSI for 32-bit machines.

   

6. Once you've found the directory, copy AppleMobileDeviceSupport64.MSI or AppleMobileDeviceSupport.MSI to another location.
7. Cancel the iTunes installation and make sure it's user interface is gone before continuing.
8. In the copied directory, double click AppleMobileDeviceSupport64.msi or AppleMobileDeviceSupport.msi to install the Apple device support. This will not require a reboot.
9. On the iPhone do the following steps
   1. Go into Settings->General->Network->Internet Tethering and turn on tethering.
      
   2. In Settings->General->Bluetooth and turn Bluetooth and leave the Bluetooth screen active.
      
10. On your Windows 7 machine, start Devices and Printers. Click the Add device button.
    
11. Follow the steps to pair your iPhone with your Windows 7 PC. Your iPhone will show up in the Devices and Printers as connected. However, the installation of the device drivers will fail reporting that the Bluetooth Peripheral Device failed. That's not a problem as you don't need this way of connecting. You just need the device paired at this point.
    
12. Start the Control Panel and in the search box, type "Bluetooth". In the Devices and Printers section, click "Change Bluetooth settings." In the Bluetooth Settings dialog, check the "Show Bluetooth icon in the notification area." Click the OK button. I had you do this as this is the fastest way to get to your Bluetooth devices to initiate a tethered connection.
    

To connect to the internet with your iPhone, here's what you do.

1. In the tray area, double click on the Bluetooth icon to bring up the Bluetooth Devices window.
2. Right click on your iPhone and from the context menu, select Connect using, Access point.
   
   You'll see the Bluetooth connection dialog pop up and after about 15 seconds or so you'll be fully tethered to your iPhone.
   Note that you'll always see the "Needs troubleshooting" for the status for your iPhone in the Bluetooth Devices window. You can ignore that because the magic is being done through the Bluetooth Network Connection in your network adapters.
3. To check the connection on your iPhone, look for the top bar on the iPhone to turn blue and say Internet Tethering.
   
4. To disconnect from the iPhone tethering, bring up the Bluetooth Devices window and select "Disconnection from device network" from the context menu after right clicking on your iPhone.    

Before the iPhone tethering option, I was using a second 3G wireless modem for my laptop. With the new tethering support, I'm now saving $65 a month with a cheaper data plan and canceling my other line, while getting the same level of service. Throw in a massively better phone with iPhone 4 and it's a total win all around especially with the minimal software installed.

If you're having trouble making the iTunes-less tethering to work drop a question in the comments. If you get these instructions to work on different Windows operating systems or iPhone model combinations, please let us know in the comments as well.

As I was waiting for a minidump I was grabbing on a very large and busy server application to finish writing, my mind wandered and I realized there were quite a few ways to grab a minidump today. Back in the old Windows days, when we had to program up hill in the snow both ways, there was only WinDBG. Now it seems like an application isn't complete unless it produced a minidump. I thought I'd throw out all the ways I know off the top of my head. Of course, I'm sure there are other ways so please add them in the comments!

Note: I'm only using Windows 7 and Server 2008 R2 these days so some of these might not work on legacy Windows operating systems.

ADPlus/WinDBG's .dump Command

The original heavy duty way to create a minidump. To create a full memory minidump, which you'll need in order to work with .NET's SOS/SOSEX/PSSCOR2 extensions or for native code to follow all memory like linked lists, use the /ma option.

Visual Studio

With Visual Studio 2010, the wonderful "Save Dump As…" menu option now appears on the Debug menu for both native and managed debugging. Any time you need a minidump while debugging, just grab one. Add in the fact that Visual Studio 2010 has the awesome minidump reading capabilities, especially for .NET code, we can now spend way less time in WinDBG.

TaskManager

Many people don't realize that TaskManager knows all about minidumps. When you're either in the Applications or the Processes tabs, right click on the process and select Create Dump File.

After the minidump is finished, you'll see the dialog showing you where the dump was created. A nice hidden treat is that the path shown is a read only edit control you can select and copy so you don't have to try to remember a long path.

You might be wondering what type of minidump TaskManager makes. How about I leave that as an exercise for the reader? OK, that's cruel. All TaskManager created dumps are full memory minidumps.

Process Explorer

TaskManager is fine, but real developers use Process Explorer to fulfill our task management needs. Right clicking on a process lets you choose a minidump or a full memory minidump.

MiniDumpWriteDump

As I am trying to create a complete list I do need to include the Windows API that actually creates the minidumps themselves: MiniDumpWriteDump. There's nothing stopping you from writing your own program that creates minidumps.

ProcDump

The sweet SysInternals ProcDump tool is designed to get you a minidump when specific nasty issues happen to your processes. It's great for snagging dumps when you have intermittent CPU spikes or memory usage. I find that I'm using this tool constantly on production servers to get minidumps of those hard to reproduce problems. Everyone using computers needs to know about this tool, even your grandmother!

DebugDiag

Got IIS problems? DebugDiag is for you. The ability to script when the dump occurs is pretty interesting. The always brilliant Tess Ferrandez has a great blog post that helps you to decide when you should use ADPlus/WinDBG vs. DebugDiag that's worth reading.

Windows Error Reporting (WER)

If your company signs up for Windows Error Reporting you'll get the same minidumps Microsoft gets. For native developers, WER is a wonderful resource but for .NET developers you only get basic minidumps so it's not as useful.

Do you know of any other ways to capture a minidump?

Remote debugging in Visual Studio works great if both machines are on the same domain and/or workgroup. It also works dreamily if you're doing straight native C++ where you can use the TCP/IP as the debugging transport. The problem comes in when you need to do remote debugging for managed code across domains or workgroups. It doesn't work because .NET remote debugging relies on DCOM, which as a transport protocol does not jump workgroup or domain boundaries.

When I have to remote debug across domain/workgroup boundaries, here's what works for me. It's not an ideal solution but until Microsoft allows us to debug .NET code with pure TCP/IP this will get you at least started. Again, your mileage may vary and your network admins may not let you perform the following so you're on your own with these steps. Finally, I'm assuming you know how to set up remote debugging for the supported same domain/workgroup scenarios. If you don't you should read the documentation to understand what I'm talking about here.

The issue with DCOM is there's a security ramification where an account one domain does not have rights on the other workgroup or domain. The big trick is to do your remote debugging with local machine accounts. DCOM first attempts to make the connection with machine\username account and if that does not work, it falls back to the username using the hashed password. As long as the username and password on the both machines is the same, DCOM can make the connection.

On the machine where Visual Studio runs, called the local machine, open up an elevated command shell or PowerShell window with administrator rights and execute the following command. Yes, you can do the same through the GUI but this is much faster. Don't fear the command line!

net user username password /add

That creates the user account you're going to use for Visual Studio.

On the machine where your application runs, called the remote machine, open up an elevated command shell or PowerShell window with administrator rights and execute the following commands. Obviously you'll be using the same user name and password you entered on the local machine.

net user username password /add

net localgroup administrators remotecomputername\username /add

With the same computer accounts on both machines account life is easy. On the remote machine, you'll need to login with the new account or if you're logging in with a different account, use RUNAS.EXE to fire up MSVSMON.EXE like the following.

runas /user:remotecomputername\username "<full path>\msvsmon.exe"

With the remote debugging stub waiting for connections, you'll need to start Visual Studio on the local machine with the account as well.

runas /user:localcomputername\username "<full path>\devenv.exe"

With both sides running with the same account and password, you'll work around the DCOM bumps and get your remote debugging across domains and/or workgroups functioning. As I mentioned, this approach isn't ideal because you're now running Visual Studio in a different account so all your settings, extensions, and drive mappings aren't there. The good news is that those are pretty simple to set up especially since you can export and import your Visual Studio settings.

One problem you'll run into with this approach is when you need to attach to a process running with administrator rights.

The issue is that MSVSMON.EXE itself is not running with administrator rights. One option is to log into the machine with the local username, right click on MSVSMON.EXE in Explorer and choose "Run as administrator." Another option, which I do instead, is to use my Elevate program that starts processes as an administrator at the command line. When I need MSVSMON.EXE running with administrator rights, I simply do the following.

runas /user:remotecomputername\username "<full path>\elevate <full path>\msvsmon.exe"

Right now some of you are sitting there thinking that instead of creating the completely separate account to run everything in, why not just create a machine account on the remote machine that matches the name of the domain account you want to use? That seems to make sense, but you'll have a small little problem. The DCOM connection only works one way and you'll get an error.

While not a perfect solution, using machine accounts with the same usernames and passwords on both machines will at least let you do remote debugging. In some cases this may cause more trouble than it's worth, but I thought it was worth mentioning the approach. One of these days full remote debugging with complete security will be built into the operating system. A boy can dream, right?

If you haven't guessed by now from my blog, I have a little bit of a passing interest in PowerShell these days. Hal Rottenburg from PowerScripting Podcast will be interviewing me about PowerShell and how it applies to developers. Of course I'll have to talk about debugging in there somehow. I was honored to be invited as the PowerScripting Podcast is generally geared at super hard core PowerShell users on the IT Pro side. For those of us not so familiar with PowerShell, consider this a great opportunity to ask questions about PowerShell from a developer's perspective as you'll have some great PowerShell users on the call. I figure they must be having me on either because of Start-PowerShellPoint (my one feeble contribution to the PowerShell community) or for the comic relief.

Check it out live at 9:30 PM EDT on June 17^th at the streaming site. If you can't make the live show, but have questions about PowerShell from a developer's perspective, put them in the comments or email them to me and we'll get them answered!

The other day I wrote about using SYMCHK.EXE from the Debugging Tools for Windows (WinDBG) install to pre-populate your symbol server cache. That prompted a question I got in email:

Our test lab machines are isolated from the rest of our network and have no internet access. How can I get symbols and source over to those machines?

For private builds, those builds you do on your development machine, it's as simple as copying the source tree to portable storage and putting the PDB files in the same directories as the binaries. Once you copy everything onto the isolated machines, the debugger appropriately finds everything. Life gets a lot more interesting for public builds; those builds done on build machines. Also, the operating system symbols from Microsoft count as a public build machine.

To get all the right PDB files, you'll need to first run SYMCHK.EXE on the unconnected machine with the /om switch on all the directories you want symbols for. This will include the operating system as well as your products directories. The /om switch builds a text file of all the information necessary to download the symbols but does not download them. You'll copy that text file to a machine connected to the internet and your company's symbol server. On that machine, you'll run SYMCHK.EXE with the /im switch and pass in the text files you created on the unconnected machine. That will populate the connected machine's cache directory with all the PDB files needed on the unconnected machine.

The PDB files are easy. The more interesting issue is getting the public sources. I'm assuming that everyone reading this has set up source indexing for all your builds, also known as source server, so you can debug all public builds with the right source code. If you don't know what a source server is, head over here to read an article I wrote about what source server is and how to get it set up on your build server. If you're using TFS 2010 support for symbol and source servers are included in TFS build.

While SYMCHK.EXE makes getting the PDB files into the local cache easy, we need a similar tool that will do the same for source files. Fortunately, there is such a tool: SRCTOOL.EXE. It's in the Debugging Tools for Windows location in the SRCSRV directory. By specifying the –x switch and the PDB file you want, SRCTOOL.EXE will execute all the version control commands embedded in the PDB file and get the source for you.

There are two caveats with SRCTOOL.EXE. The first is that it only runs with on single PDB file at a time so if you need to grab the source files for 30 different of your DLLs, you have to run it 30 times. The second is that it doesn't put the source in your cache directory unless you specify the –d command line option. Obviously, you and I don't want to manually run SRCTOOL.EXE on each PDB file in our symbol cache so I whipped up a PowerShell script, Get-SourceServerFiles.PS1, at the bottom of this blog entry that will do the work for you. Simply pass in your cache directory and the script will do all the work as well as ensure the source files are placed in the cache directory.

After running SYMCHK.EXE and Get-SourceServerFiles.PS1, you'll have all the PDB files and source code in the connected machine's cache. You'll just need to copy the whole cache directory to a portable drive and take that to the unconnected machine and copy the directory to the local drive. On the unconnected machine, you'll set up the machine to use your symbol server. When you start debugging, what happens? The debuggers always look in the cache directory for symbols and source so if you have put everything in the cache, there's no network access.

With a little setup debugging with full source and correct call stacks on unconnected machines is now nearly as easy as debugging locally on your development computer. Go forth and debug!

#requires -version 2

# (c) 2010 by John Robbins\Wintellect – Do whatever you want to do with it
# as long as you give credit.

<#.SYNOPSIS
Prepopulate your symbol cache with all your Source Server extracted source code.

.DESCRIPTION
Recurses the specified symbol cache directory for PDB files with Source Server sections
and extracts the source code. This script is a simple wrapper around SRCTOOl.EXE from
the Debugging Tools for Windows (AKA WinDBG). You must have SRCTOOL.EXE in the path. It
is in the SRCSRV directory under the Debugging Tools for Windows installation directory.

.PARAMETER cacheDirectory
The cache directory for the local machine.
#>

Param([Parameter(Mandatory=$true)] $cacheDirectory)

Set-StrictMode –version Latest

# Verify SRCTOOL.EXE is in the path.
if ((Get-Command srctool.exe -ErrorAction SilentlyContinue) -eq $null)
{
    throw "SRCTOOL.EXE does not appear to be in the PATH."
}

# Verify the cache directory exists.
if ((Test-Path $cacheDirectory) -eq $false)
{
    throw "The specified directory does not appear to exist"
}

# Get all the PDB files, execute SRCTOOL.EXE on each one.
Get-ChildItem -Recurse -Include *.pdb -Path $cacheDirectory | ForEach-Object { SRCTOOL.EXE -d:$cacheDirectory -x $_.FullName }

Symbol Servers are one of the best things that have ever happened in Windows debugging. Having a central store for all PDB files makes the debugging experience so much better because you'll always get exact right symbols with essentially no effort on your part. This means perfect call stacks on native code and if you've done your job correctly by using the Source Server indexing tools, you'll have the exact version of the source code used for the build automatically show up in the debugger. As the Symbol and Source Servers are trivial to set up, whenever someone tells me they are having trouble getting symbols and source, I do have to wonder if they are serious about developing software.

Probably the most common question I get about the debugger is "Why does the debugger hang on me when I start debugging or occasionally in the middle of debugging?" As wonderful as the Symbol and Source Servers are, I hate to say this, but they are at the root of the problem. In fact, I have a very special extra sensory perception (ESP) skill. Even if I'm not in the office, I can always tell when a company's Source Server has gone down. It's all those emails going back and forth saying "Hey, what happened? This morning I was debugging just fine, but every time I start this afternoon, the debugger hangs. Visual Studio's got a bug." While I'm proud of my ESP skills, I wish they would tell me the winning lottery numbers instead.

As the Symbol and Source Server are utilizing network file shares all it takes is the server going down or an invalid share specified in either the _NT_SYMBOL_PATH environment variable or Visual Studio Options dialog. With the network timeout blocking the user interface, you have two choices. Either wait it out or kill the debugger with Task Manager. Obviously, the first trouble shooting step is to see if you can ping the Symbol Servers you've told the debugger to use. If there's no response, you've solved the problem. With a down Symbol Server, I temporarily work around the problem by starting a Command Prompt or PowerShell window, disable the _NT_SYMBOL_PATH environment variable and start DEVENV.EXE (Visual Studio) or WINDBG.EXE from that shell. When Visual Studio starts, I go into the Options dialog and disable the symbol server there as well.

The other debugger slowdown you run into is when you first start debugging on a new operating system installation, after a major gaggle of updates on Patch Tuesday, or your company's Symbol Server is based in another country. In all those cases, the debugger slowdown is because of downloading all the symbols into your local cache. As some of these PDB files can be large and networks spotty, it's a pain.

Fortunately, there's an easy remedy in the SYMCHK.EXE program that comes with the Debugging Tools for Windows (AKA WinDBG). Instead of having the debugger download the PDB files, the trick is to fill your cache with them before you need them. Of course, if you like having that extra-long debugging session once a month for an extended coffee break you may want to forget all about SYMCHK.EXE.

To make SYMCHK.EXE easier to run always ensure you have the _NT_SYMBOL_PATH environment variable set so you don't have to specify giant sets of command line parameters. (Previously on my blog, I've written PowerShell scripts to make this setup easy for VS 2008 and VS 2010.) To fill your local symbol cache, run the following two commands to get the .NET and operating systems DLLs:

symchk -r C:\windows\Microsoft.NET\*.dll
symchk -r C:\windows\System32\*.dll

Once you get your cache filled, you can keep running those two commands so they pick up any changed files. So I don't have to remember to run the above commands after, on my machines I set up a Scheduled Task to run the two commands in a batch file every Wednesday morning at 5:00 AM. Obviously, you can run SYMCHK.EXE against your product's installation directories to get those builds into your cache as well.

Now debugger hangs and stutters should be a thing of the past for you. Less time waiting on the debugger means faster debugging, which means more time writing code. As always if you have any questions about SYMCHK.EXE or symbols in general, let me know in the comments.

Oh! I'm feeling an ESP moment… Numerous of you are thinking "This is all great but what if the debugger hangs on just one or two symbols every time? Is there a way to tell the debugger to skip loading those symbols?" I knew you were going to ask that question, so I wrote about it back on May 29, 2009. How's that for preparation!?

Pretty much any time I write about a cool .NET only feature here on the blog I get emails or comments about how the native C++ developers are left out in the cold. There are still a lot of applications out there using native C++ and it feels like those developers don't get the love like .NET developers do so here's a post only for the C++ developers out there.

In keeping up with C++ development, I was looking at the Boost Libraries Conference 2010 site to see what was discussed at BoostCon 2010. There was a wonderful presentation from Stephan T. Lavavej of Microsoft, Data Structure Visualizers in Visual Studio 2010. That's the direct link to the PDF of the slides, but make sure to check out the rest of the conference sessions here. If you ever wanted to know how to hack the AUTOEXP.DAT to show anything you wanted, this is the session for you. It's advanced and undocumented, but that never stopped a C++ developer before!

Wintellect is so much more than just me posting a random blog entry every once in a while. This year our big marketing focus is to offer unique and interesting content for developers. To that end we've set up our Tech Focus which you can sign up for here. This gets you special articles each month on all sorts of topics relevant to anyone doing .NET and Windows development.

For May and June the Tech Focus is on Debugging and Tuning, a topic near and dear to my heart. I've written the first article about debugging and I think it's pretty cool because it's something that helped me out immensely. Also, our Tech Focus offers great tips and tricks over Twitter at the Wintelletuals feed. I'm super proud to say that all the Debugging and Performance tips for the next two months are all exactly 139 characters long! I figured 140 characters meant they were too verbose and a real developer starts counting at zero, right? Follow us on Twitter to get the total Ninja Zen of killer debugging and performance tips that make haiku's look relatively easy.

I have to honestly admit that I do not get Twitter at all and will probably never get an account but I loved the challenge of doing two months of real world tips in 139 characters. Perusing through Twitter I had to wonder why anyone would bother with writing any tweets that are less than 139 characters because it shows laziness. I guess it's like what Mark Twain said "I didn't have time to write a short letter so I wrote a long one." While I certainly don't care what you had for breakfast, I greatly appreciate the effort to craft your message to maximize the medium. I should create a web site that shows just tweets that are 139 characters and DO NOT have any goo from that bit.ly abomination in them. Maybe I can start a revolution that focuses on creativity and uniqueness on Twitter instead of the banality that seems to plague it. Are you with me!? Viva la 139 character revolution! Tweet creatively!