One of the projects I am working on as of late involves hosting a DLR language (IronPython) in a managed application. Our application has dual build targets – one for WPF and one for Silverlight, with nearly identical feature sets.
When embedding IronPython in an application there are multiple things to consider from a packaging perspective. The DLR itself lives in a pair of open-source assemblies (Microsoft.Dynamic.dll and Microsoft.Scripting.dll). The IronPython runtime is also in a pair of open-source assemblies (IronPython.dll and IronPython.Modules.dll). All of these are available from the IronLanguages codeplex project in precompiled form (http://ironpython.codeplex.com/), or as source code if you prefer from the working repository at github (https://github.com/IronLanguages/main). There is support for both the desktop CLR and the Silverlight CLR.
Those assemblies provide the managed scripting runtime needed to run Python scripts inside of an application. And of course there are the python scripts themselves, which can be stored on the local file system, a web server, a database, or pretty much anyplace you can dream up. But there is a third part to this picture – and that is the Python Standard Library. Full documentation of this library can be found at http://www.python.org/. It is the Python equivalent of the .NET Framework Library, including modules for a wide range of features from xml processing to threading to math functions. This library is distributed in source code form – it is entirely written in Python. Typical applications would update the sys.path system value to point to a local directory (usually something like C:\Python27\) that contains the full suite of .py files in the standard library (or some subset). The runtime will find them when needed, and compile them just-in-time as scripts declare references to them via “import” statements.
But our application is not typical, and we wanted to exercise very tight control over that library. The last thing we want is for a user to start mucking around in C:\Python27\ and break something. So we decided to embed the entire set of Python Standard Library source files as resources. The IronPython module loader supports this technique through custom Platform Abstraction Layer classes (a topic for a different post). The problem here is that there are quite a few files in a hierarchical folder structure. Adding them all to our .csproj file manually would be cumbersome, and maintenance would be a real headache when we want to update to a newer Python library release. And so this is where the wildcard inclusions feature comes in handy.
Most developers have no idea you can do this. Visual Studio does not directly offer a way to add items as wildcards (if you specify a wildcard in the Add Files dialog, it will add individual items for each file). It actually turns out to be very easy. You need to open the csproj file in a text editor (you can do this in Visual Studio by selecting “Unload Project” from the context menu in Solution Explorer, and then “Edit [project name].csproj”).
In the csproj file you will find individual items listed in <ItemGroup> sections:
A “Compile” item indicates a source file that is compiled with the C# compiler into the target assembly. An “EmbeddedResource” item indicates a file that will be embedded as a resource in the target assembly. With 482 files in the standard library for the current release of IronPython (2.7), that is a lot of EmbeddedResource entries!
However, we can use wildcards when specifying these files. Here is an example that includes all .py files from a folder (and all subfolders):
Once you save the project file and reload it in Visual Studio, you will see that the Solution Explorer understands the wildcard and will populate with all files that match it. You can copy new .py files into the target directory and Solution Explorer will update accordingly (you need to click the “Refresh” button for this). And while it shows the results of the wildcard inclusion, it does not alter the entries in the project file unlike the behavior of the Add Files dialog.
This is a nice feature of the project file specification, and it really helped with this particular situation, but there are some gotchas to be aware of. First of all, Visual Studio doesn’t provide a UI for creating or editing the wildcards – you need to do it by hand. Secondly, the Version Control hooks will simply ignore it, and you will need to add/manage the included files directly in your Source Control Manager if your code is affected by an SCM such as TFS. Lastly, you cannot combine this feature with the “link to item” feature that allows you to add an existing item as a link. Those two features are exclusive of each other. I mention this last limitation because it is common practice to support dual-target Silverlight/Desktop codebases by making heavy use of linked files. There are ways around this limitation, but that too is an entire blog post subject in itself.