Recently, I was involved in a Wintellect consulting engagement where a customer had some class library code that was created years ago. The code in this class library was not designed to work well in a multithreaded environment. Specifically, this means that if two or more threads called into the class library code at the same time, the data got corrupted and the results could not be trusted. Non-concurrent-ready code is easy to obtain with frequent use of mutable static fields. But, there are other common coding practices which can result in code that is not safe with multiple threads passing through it at the same time.

This customer wanted to increase the performance of their code and the best way to accomplish this is to have multiple threads processing their own data independently of each other. But, again, the class library code base would produce errant results if we did this. To demonstrate the problem, imagine this very simple static class which is indicative of the non-thread safe class library that I'm referring to:

internal static class NonConcurrentAlgorithm {

private static List<String> s_items = new List<String>();

public static void Add(String item) { s_items.Add(item); }

public static String[] GetItems() { return s_items.ToArray(); }

}

If we have two threads using this class simultaneously, the results are unpredictable. For example, let's say we have this code:

ThreadPool.QueueUserWorkItem(o => NonConcurrentAlgorithm.Add("Item 1"), null);
NonConcurrentAlgorithm.Add("Item 2");
Thread.Sleep(1000);  // To demonstrate the problem consistently
foreach (var i in NonConcurrentAlgorithm.GetItems())
   Console.WriteLine(i);

When this code runs, the console could show:

• "Item 2" by itself
• "Item 2" followed by "Item 1"
• "Item 1" followed by "Item 2"

Clearly, this is not desireable and the problem all stems from the threads sharing the one List<String> object. In fact, the List<T> class is not itself thread-safe and so it is even possible that having multiple threads accessing it simultaneously could result in items disappearing or a single item being inserted multiple times, or worse. It depends on how the List<T> class is internally implemented which is not documented and is subject to change.

Now, one way to improve performance for the customer would be for Wintellect to take the non-concurrent code base and make it thread safe. This is the best thing to do; however, the code base was very large, the changes to the code base would have been substantial and a lot of testing would have had to be done. The customer was not too excited by this recommendation. So, what we did instead was use the CLR's AppDomain feature. This works out great because an AppDomain's state is completely isolated from all other AppDomains. To communicate across AppDomains, you must create a class derived from MarshalByRefObject. So, I created a NonConcurrentAlgorithmProxy class that wraps the methods of the static class. The NonConcurrentAlgorithmProxy class looks like this:

internal sealed class NonConcurrentAlgorithmProxy : MarshalByRefObject {

   public void Add(String item) { NonConcurrentAlgorithm.Add(item); }
   public String[] GetItems()   { return NonConcurrentAlgorithm.GetItems(); }
}

Then we modified the code that uses this class library code so that each thread creates its own AppDomain and creates an instance of the NonConcurrentAlgorithmProxy class in each AppDomain. The static field in the NonConcurrentAlgorithm class is now one-per-AppDomain and since each thread gets its own AppDomain, there is now one List<String> object per thread. Here is the new code that uses the class library code via the proxy class:

// Create Thread 1's AppDomain & proxy
var thread1AppDomain = AppDomain.CreateDomain(String.Empty);
var thread1Proxy = (NonConcurrentAlgorithmProxy)

   thread1AppDomain.CreateInstanceAndUnwrap(

      typeof(NonConcurrentAlgorithmProxy).Assembly.FullName,

      typeof(NonConcurrentAlgorithmProxy).FullName);

					
// Create Thread 2's AppDomain & proxy
var thread2AppDomain = AppDomain.CreateDomain(String.Empty);
var thread2Proxy = (NonConcurrentAlgorithmProxy) 

   thread2AppDomain.CreateInstanceAndUnwrap(

					typeof(NonConcurrentAlgorithmProxy).Assembly.FullName, 

					typeof(NonConcurrentAlgorithmProxy).FullName);

					
// Have 2 threads execute the same code at the same time but in different AppDomains
ThreadPool.QueueUserWorkItem(o => thread2Proxy.Add("Item 1"), null);
thread1Proxy.Add("Item 2");
Thread.Sleep(1000);  // To demonstrate the problem has been fixed

					
// Show Thread 1's list of items; guaranteed to be "Item 2"
foreach (var i in thread1Proxy.GetItems())
   Console.WriteLine(i);

					
// Show Thread 2's list of items; guaranteed to be "Item 1"
foreach (var i in thread2Proxy.GetItems())
   Console.WriteLine(i);

					
// Cleanup each thread's AppDomain
AppDomain.Unload(thread1AppDomain);
AppDomain.Unload(thread2AppDomain);

This is substantially less coding, testing, and effort than what would have been involved with making the customer's current code base thread safe! And, this technique can scale very nicely for many threads as well. Of course, there is a slight performance penalty when creating/destroying AppDomains and calling methods through the proxy. This is why it would be ideal to make the existing code base thread safe. But, this is a great compromise and the customer was quite happy with the results and the time/cost it required to improve their application's performance substantially.

In my previous blog post, I discussed how to build a .NET component that works with different versions of a particular library. In that post, I demonstrated a technique for accessing instance methods that get added in a later version of a library. In this post, I’ll modify the technique some to make it work reasonably well for static methods that get added in a later version of a library.

Let’s starts by looking at Version 1.0 of some library class:

public sealed class VersioningType {    // v1.0
   // Other methods not shown
}

And, let’s say that version 2.0 of the same library class looks like this:

public sealed class VersioningType {     // v2.0
   public static String NewStaticMethod(String arg) {// New STATIC method introduced in v2.0
      return "Real static method: " + arg; 
   }
   // Other methods not shown
}

Now, we want to write some code in a component that calls VersioningType’s NewStaticMethod if it exists at runtime. To do this, I would add the following VersioningTypeExtensions class into my component code:

internal static class VersioningTypeExtensions {
   private static dynamic s_NewStaticMethodFound =
      new StaticMemberDynamicWrapper(typeof(VersioningType));   // Assume method exists

   /// <summary>The stub for VersioningType's static NewStaticMethod method</summary>
   public static String NewStaticMethod(String arg) {
      if (s_NewStaticMethodFound != null) {
         try {
            // We think the method exists, try to invoke it
            return s_NewStaticMethodFound.NewStaticMethod(arg);
         }
         catch (RuntimeBinderException) {
            // The method doesn't exist; in the future, avoid the dynamic/exception perf hit
            s_NewStaticMethodFound = null;
         }
      }

      // This is the default implementation we want if the method doesn't exist
      return "Stub static   method: " + arg;
   }
}

This code introduces a static method called NewStaticMethod that matches the same signature as the NewStaticMethod introduced in v2.0 of the library. Now, in my component’s code, I can call this static stub method and I get compile-time type safety at all the call sites. This code uses a helper class that I wrote, called StaticMemberDynamicObject. I show the code for this class at the bottom of this blog post. You construct an instance of my StaticMemberDynamicObject class by passing a type to its constructor. In short, by casting a reference to a StaticMemberDynamicObject object to C#’s dynamic type, you can invocate a type’s static methods, properties, or fields dynamically at runtime. It makes the type look like an object and this class for use with dynamic types. The StaticMemberDynamicObject class is useful in many programming scenarios.

Inside VersioningTypeExtension’s NewStaticMethod stub method, I use the s_NewStaticMethodFound field (which is of the dynamic type) to try to call VersioninType’s static NewStaticMethod. If our component is running against v2.0 of the library, then its NewStaticeMethod will be invoked. But, if our component is running against v1.0 of the library, then attempting to call NewStaticMethod will throw a RuntimeBinderException. My stub method catches this exception, sets the static field to null and then my code that simulates the default behavior of this method. In this case, I return a string indicating that my stub method handled the call.

The static s_NewStaticMethodFound field exists to improve performance. Once the stub determines that VersioningType doesn’t offer the desired method, this field is set to null and future invocations of the stub method avoid the performance hit of entering/leaving a try block, attempting to dynamically invoke the NewStaticMethod and the performance hit of processing the thrown RuntimeBinderException. In addition, the StaticMemberDynamicObject object can be garbage collected now.

If in the future, we decide that our component no longer needs to work with v1.0 of the library, we can build our component code against v2.0 of the library and we can leave all our code alone and it will just suffer a slight performance penalty whenever calling NewStaticMethod. Or, we can delete our stub method and then modify all of the call sites to call VersioningType’s static NewStaticMethod method directly.

The instance method technique shown in my previous blog posting used C#’s extension methods to implement the stub. This had two significant benefits. First, the code at the call site looked just like the code you would have to call the instance method meaning that no source code would have to change in order to call the V2.0 library. Second, if we build the component against v2.0 of the library, then the compiler would bind to the instance method instead of the extension method resulting in a performance improvement by not calling the stub method. Unfortunately, we don’t get these benefits with static methods because extension methods don’t allow you to extend a type with a static method. Extension methods can only extend objects with instance methods.

Below is the code for my useful StaticMemberDynamicObject class that allows you to dynamically invoke static members of a type. This class is derived from the .NET Framework’s System.Dynamic.DynamicObject class which allows instance of the class to control how its members are invoked when invoked using C#’s dynamic type.

/// <summary>Construct an instance of this class and cast the reference to 'dynamic' 
/// to dynamically invoke a type's static members</summary>
internal sealed class StaticMemberDynamicWrapper : DynamicObject {
   private const BindingFlags c_bf = BindingFlags.Public | BindingFlags.Static;
   private readonly Type m_type;
   public StaticMemberDynamicWrapper(Type type) { m_type = type; }

   public override IEnumerable<String> GetDynamicMemberNames() {
      return m_type.GetMembers(c_bf).Select(mi => mi.Name);
   }

   public override bool TryGetMember(GetMemberBinder binder, out object result) {
      var members = m_type.GetMember(binder.Name, MemberTypes.Field | MemberTypes.Property, c_bf);
      if (members.Length != 1) { result = null; return false; }
      result = (members[0].MemberType == MemberTypes.Field) 
         ? ((FieldInfo)members[0]).GetValue(null) 
         : ((PropertyInfo)members[0]).GetValue(null, null);
      return true;
   }

   public override bool TrySetMember(SetMemberBinder binder, object value) {
      var members = m_type.GetMember(binder.Name, MemberTypes.Field | MemberTypes.Property, c_bf);
      if (members.Length != 1) return false;
      if (members[0].MemberType == MemberTypes.Field)
         ((FieldInfo)members[0]).SetValue(null, value);
      else 
         ((PropertyInfo)members[0]).SetValue(null, value, null);
      return true;
   }

   public override Boolean TryInvokeMember(InvokeMemberBinder binder, 
      Object[] args, out Object result) {

      MethodInfo method = m_type.GetMethod(binder.Name, c_bf);
      if (method == null) { result = null; return false; }
      result = method.Invoke(null, args);
      return true;
   }
}

It’s common to want to build a .NET component that works with different versions of a particular library. It is also common for newer versions of the library to introduce new methods that your component might want to call. However, if you build your component against the oldest version of the library you support, then your component cannot access the new methods unless you add code to your component that uses reflection to look for the new methods at runtime. In this blog post, I’d like to share a technique for using C# 4.0’s new dynamic type and C# 3.0’s extension methods feature to simplify the building of components that leverage newer library features if the library is available at runtime and gracefully falls back to some default implementation if the newer library is not available at runtime.

Let’s starts by looking at Version 1.0 of some library class:

    public sealed class VersioningType {    // v1.0
      // Other methods not shown
   }

And, let’s say that version 2.0 of the same library class looks like this:

   public sealed class VersioningType {    // v2.0
      public String NewInstanceMethod(String arg) { // New method introduced in v2.0
         return "Real instance method: " + arg; 
      }
      // Other methods not shown
   }

Now, we want to write some code in a component that calls VersioningType’s NewInstanceMethod if it exists at runtime. To do this, I would add the following VersioningTypeExtensions class into my component code:

   internal static class VersioningTypeExtensions {
    private static Boolean s_NewInstanceMethodFound = true;   // Assume the method exists

      /// <summary>The stub for VersioningType's instance NewInstanceMethod method</summary>
      public static String NewInstanceMethod(this VersioningType vt, String arg) {
         if (s_NewInstanceMethodFound) {
            // We think the method exists, try to invoke it
            try {
               return ((dynamic)vt).NewInstanceMethod(arg);
            }
            catch (RuntimeBinderException) {
               // The method doesn't exist.
            // In the future, avoid the dynamic/exception perf hit
            s_NewInstanceMethodFound = false;
            }
         }

       // This is the default implementation we want if the method doesn't exist
     return "Stub instance method: " + arg;
      }
   }

This code introduces a C# extension method called NewInstanceMethod that matches the same signature as the NewInstanceMethod introduced in v2.0 of the library. Now, in my component’s code, I can call this extension method and I get IntelliSense support and compile-time type safety at all the call sites. Inside this stub method, I cast the vt variable to C# 4.0’s new dynamic type and then I try to call NewInstanceMethod. Casting vt to dynamic lets the code compile and, at runtime, reflection will be used to determine if the VersioningType actually offers a NewInstanceMethod. If our component is running against v2.0 of the library, then its NewInstanceMethod will be invoked. But, if our component is running against v1.0 of the library, then attempting to call NewInstanceMethod will throw a RuntimeBinderException. My stub method catches this exception, sets the static Boolean field to false and then my code that simulates the default behavior of this method executes. In this case, I return a string indicating that my stub method handled the call.

The static s_NewInstanceMethodFound field exists to improve performance. Once the stub determines that VersioningType doesn’t offer the desired method, this field is set to false and future invocations of the stub method avoid the performance hit of entering/leaving a try block, attempting to dynamically invoke the NewInstanceMethod and the performance hit of processing the thrown RuntimeBinderException.

There is an additional benefit that comes from writing the code this way. Let’s say that in the future, we decide that our component no longer needs to work with v1.0 of the library. If we build our component code against v2.0 of the library (instead of against v1.0), then the C# compiler will see that VersioningType offers a NewInstanceMethod method and the compiler will bind all of the calls to this method. The compiler will simply ignore the stub method entirely. This improves performance. Of course, it would also be good to just delete the stub method now since it is serving no purpose anymore.

The technique I just described for version works great for instance methods. However, it does not work for static methods that are added to later versions of a library. This is because there is no object (so you can’t cast to dynamic) and extension methods only work for instance methods too. However, I do have a variation of this technique that will work for static methods. I’ll share the static-method technique in a future blog posting.

If you have never used my AsyncEnumerator class (which is part of my free Power Threading library) to simplify writing code that performs asynchronous operations, then you can ignore the rest of this blog posting.

I have been working with Microsoft in an attempt to add my AsyncEnumerator (or something like it) into a future version of the .NET Framework.
The good folks at Microsoft would like to gather more information how people have actually used the AsyncEnumerator in their own projects.
To that end, I have created a short, 3 question survey which I would like all users of my AsyncEnumerator class to answer.

If you have ever used my AsyncEnumerator class, please take a few minutes out of your day and go here: http://www.surveymonkey.com/s/FTZPD8Y

Please give as much information as you can and feel comfortable with. The more information we get, the better MS and I can accommodate your asynchronous programming needs in a future version of the .NET Framework.

MS (and I) would also love to see some actual code that leverages the AsyncEnumerator. If you feel comfortable sharing some of your source code, please ZIP it up and e-mail it to me directly at this e-mail address: JeffreyR at Wintellect dot com. Please put “AsyncEnumerator Sample Code” in the subject line and do not expect a reply from me. MS and I will review this code in the next couple of weeks to help make decisions for a future version of the .NET Framework.

Again, thank you all very much for your help and support.

--Jeffrey Richter

You can find Part 1 here: http://blogs.msdn.com/devpara/archive/2010/03/16/jeffrey-richter-at-devweek-2010-part-1.aspx

and Part 2 here: http://blogs.msdn.com/devpara/archive/2010/03/16/jeffrey-richter-at-devweek-2010-part-2.aspx

Hello all, I just wanted to tell everyone that my book, CLR via C# 3rd Edition, went to the printer this week and should be in stores in early February!
The book has been updated for Visual Studio 2010, CLR 4.0 and C# 4.0.
The source code for the book is available now and can be downloaded from here: http://wintellect.com/Books.aspx
Also, an excerpt from the book's Introduction can be found here: http://blogs.msdn.com/microsoft_press/archive/2010/01/21/rtm-d-today-clr-via-c-third-edition.aspx

While creating the 3rd Edition of my CLR via C# book (http://www.amazon.com/CLR-via-C-Third-Pro-Developer/dp/0735627045/ref=dp_ob_title_bk), I came up with a cool little class that will raise an event after a collection of Generation 0 or Generation 2 occurs. Here is the code for the class:

public static class GCNotification {
   private static Action<Int32> s_gcDone = null; // The event’s field
   public static event Action<Int32> GCDone {
      add {
         // If there were no registered delegates before, start reporting notifications now 
         if (s_gcDone == null) { new GenObject(0); new GenObject(2); }
         s_gcDone += value;
      }
      remove { s_gcDone -= value; }
   }
   private sealed class GenObject {
      private Int32 m_generation;
      public GenObject(Int32 generation) { m_generation = generation; }
      ~GenObject() { // This is the Finalize method
         // If this object is in the generation we want (or higher), 
         // notify the delegates that a GC just completed
         if (GC.GetGeneration(this) >= m_generation) {
            Action<Int32> temp = Interlocked.CompareExchange(ref s_gcDone, null, null);
            if (temp != null) temp(m_generation);
         }
         // Keep reporting notifications if there is at least one delegate
         // registered, the AppDomain isn't unloading, and the process 
         // isn’t shutting down
         if ((s_gcDone != null) && 
            !AppDomain.CurrentDomain.IsFinalizingForUnload() && 
            !Environment.HasShutdownStarted) {
            // For Gen 0, create a new object; for Gen 2, resurrect the
            // object & let the GC call Finalize again the next time Gen 2 is GC'd
            if (m_generation == 0) new GenObject(0);
            else GC.ReRegisterForFinalize(this);
         } else { /* Let the objects go away */ }
      }
   }
}

And here is some code to see it in action:

public static void Main() {
   GCNotification.GCDone += g => Console.Beep(g == 0 ? 800 : 8000, 200);
   var l = new List<Object>();
   // Construct a lot of 100-byte objects.
   for (Int32 x = 0; x < 1000000; x++) {
      Console.WriteLine(x);
      Byte[] b = new Byte[100];
      l.Add(b);
   }
}

This gives you a very small taste of what you should expect to see in the next edition of my book. Hope you enjoy.

Last week I submitted the reaming chapters for my new book. It is now being edited and should be available right around the time that Visual Studio 2010 launches (March 22, 2010).

One place you can order it is here: http://www.amazon.com/CLR-via-C-Third-Pro-Developer/dp/0735627045

I know that many people will ask me what are the differences between the 2nd edition and the 3rd edition and so I thought I'd create this blog post to address this.

Overall, every chapter has been modified making the text clearer, fixing any known mistakes and I’ve added more 64-bit coverage as this hardware is becoming more commonplace. I' ve also embellished a lot of text to reflect new things that I've learned in the last 5 years since the previos edition of the book was published. In addtion, since the 2^nd Edition of the book covered version 2.0 of the .NET Framework and C#, the new book adds coverage of versions 3.0, 3.5 and 4.0.

Also, I always thought I’d write a Threading Book showing how to properly architect software to build responsive and scalable applications and components in today’s world of multi-core computers. However, I decided to just include this other book’s content in the 3^rd Edition of CLR via C# and so Part 5 of the book has five pretty lengthy chapters related to Threading. These chapters (like all chapters in the book) are very prescriptive. That is, I don’t just explain what is in the .NET Framework and how to use it. I explain when to use it and why as well as pitfalls associated with various constructs. I have written a lot of threading material over the past 20+ years and this is all new material presented in an all new way that I think will resonate well with software developers. The 2nd edition of CLR via C# had two chapters related to threading; the five new chapters contain a small part of that material but the new chapters are basically rewritten and add all of the new stuff that is being introduces with .NET 4.0.

Below is the Table of Contents for CLR via C#, 3^rd Edition and a brief description of what has been added to each chapter since the 2^nd Edition.

Part I – CLR Basics

Chapter 1-The CLR’s Execution Model

Added about discussion about C#’s /optimize and /debug switches and how they relate to each other.

Chapter 2-Building, Packaging, Deploying, and Administering Applications and Types

Improved discussion about Win32 manifest information and version resource information.

Chapter 3-Shared Assemblies and Strongly Named Assemblies

Added discussion of TypeForwardedToAttribute and TypeForwardedFromAttribute.

Part II – Designing Types

Chapter 4-Type Fundamentals

No new topics.

Chapter 5-Primitive, Reference, and Value Types

Enhanced discussion of checked and unchecked code and added discussion of new BigInteger type. Also added discussion of C# 4.0’s dynamic primitive type.

Chapter 6-Type and Member Basics

No new topics.

Chapter 7-Constants and Fields

No new topics.

Chapter 8-Methods

Added discussion of extension methods and partial methods.

Chapter 9-Parameters

Added discussion of optional/named parameters and implicitly-typed local variables.

Chapter 10-Properties

Added discussion of automatically-implemented properties, properties and the Visual Studio debugger, object and collection initializers, anonymous types, the System.Tuple type and the ExpandoObject type.

Chapter 11-Events

Added discussion of events and thread-safety as well as showing a cool extension method to simplify the raising of an event.

Chapter 12-Generics

Added discussion of delegate and interface generic type argument variance.

Chapter 13-Interfaces

No new topics.

Part III – Essential Types

Chapter 14-Chars, Strings, and Working with Text

No new topics.

Chapter 15-Enums

Added coverage of new Enum and Type methods to access enumerated type instances.

Chapter 16-Arrays

Added new section on initializing array elements.

Chapter 17-Delegates

Added discussion of using generic delegates to avoid defining new delegate types. Also added discussion of lambda expressions.

Chapter 18-Attributes

No new topics.

Chapter 19-Nullable Value Types

Added discussion on performance.

Part IV – CLR Facilities

Chapter 20-Exception Handling and State Management

This chapter has been completely rewritten. It is now about exception handling and state management. It includes discussions of code contracts and constrained execution regions (CERs). It also includes a new section on trade-offs between writing productive code and reliable code.

Chapter 21-Automatic Memory Management

Added discussion of C#’s fixed state and how it works to pin objects in the heap. Rewrote the code for weak delegates so you can use them with any class that exposes an event (the class doesn’t have to support weak delegates itself). Added discussion on the new ConditionalWeakTable class, GC Collection modes, Full GC notifications, garbage collection modes and latency modes. I also include a new sample showing how your application can receive notifications whenever Generation 0 or 2 collections occur.

Chapter 22-CLR Hosting and AppDomains

Added discussion of side-by-side support allowing multiple CLRs to be loaded in a single process. Added section on the performance of using MarshalByRefObject-derived types. Substantially rewrote the section on cross-AppDomain communication. Added section on AppDomain Monitoring and first chance exception notifications. Updated the section on the AppDomainManager class.

Chapter 23-Assembly Loading and Reflection

Added section on how to deploy a single file with dependent assemblies embedded inside it. Added section comparing reflection invoke vs bind/invoke vs bind/create delegate/invoke vs C#’s dynamic type.

Chapter 24-Runtime Serialization

This is a whole new chapter that was not in the 2^nd Edition.

Part V – Threading

Chapter 25-Threading Basics

Whole new chapter motivating why Windows supports threads, thread overhead, CPU trends, NUMA Architectures, the relationship between CLR threads and Windows threads, the Thread class, reasons to use threads, thread scheduling and priorities, foreground thread vs background threads.

Chapter 26-Performing Compute-Bound Asynchronous Operations

Whole new chapter explaining the CLR’s thread pool. This chapter covers all the new .NET 4.0 constructs including cooperative cancelation, Tasks, the aralle class, parallel language integrated query, timers, how the thread pool manages its threads, cache lines and false sharing.

Chapter 27-Performing I/O-Bound Asynchronous Operations

Whole new chapter explaining how Windows performs synchronous and asynchronous I/O operations. Then, I go into the CLR’s Asynchronous Programming Model, my AsyncEnumerator class, the APM and exceptions, Applications and their threading models, implementing a service asynchronously, the APM and Compute-bound operations, APM considerations, I/O request priorities, converting the APM to a Task, the event-based Asynchronous Pattern, programming model soup.

Chapter 28-Primitive Thread Synchronization Constructs

Whole new chapter discusses class libraries and thread safety, primitive user-mode, kernel-mode constructs, and data alignment.

Chapter 29-Hybrid Thread Synchronization Constructs

Whole new chapter discussion various hybrid constructs such as ManualResetEventSlim, SemaphoreSlim, CountdownEvent, Barrier, ReaderWriterLock(Slim), OneManyResourceLock, Monitor, 3 ways to solve the double-check locking technique, .NET 4.0’s Lazy and LazyInitializer classes, the condition variable pattern, .NET 4.0’s concurrent collection classes, the ReaderWriterGate and SyncGate classes.

I’ve been doing a lot of work with the new Task class that ships with .NET 4.0 as I’ve been revising my CLR via C# book (due out in early 2010).

Task are really good for performing asynchronous compute-bound work and while my AsyncEnumerator was really designed for performing I/O-bound work using the CLR’s APM, it is possible to use Tasks with the AsyncEnumerator giving you the ability to easily perform I/O-bound as well as compute-bound work and use the AsyncEnumerator to coordinate it all.

For example, you might want to read a bitmap image into memory from disk or network asynchronously (this is I/O bound) and then use Tasks to produce one or more thumbnail images (this is compute-bound) and then write the thumbnail images back to disk/network (I/O-bound again).

To use Tasks within an AsyncEnumerator, just create the Task(s) as you normally would and add a ContinueWith task that simply calls AsyncEnumerator’s End method to get a delegate. Then, invoke this delegate passing the task itself (since Tasks implement the IAsyncResult interface). Or, if you don’t need to identify the task, you can simplify the code some and just pass null. On the other side of the yield return statement, you need to call AsyncEnumerator’s DequeueAsyncResult to remove the entry from the AsyncEnumerator’s IAsyncResult collection but then there is no EndXxx method to call.

Here is a simple example, that starts a Task that sleeps for 10 seconds and then returns the current DateTime.

private static IEnumerator<Int32> AsyncEnumeratorAndTasks(AsyncEnumerator ae) {

   var t = new Task<DateTime>(() => { Thread.Sleep(10000); return DateTime.Now; });

   t.Start();

   // The Task tells the AsyncEnumerator when it is done

   // If you don’t need to identify the Task, you can pass ‘null’ instead of ‘task’

   t.ContinueWith(task => ae.End()(task));

   yield return 1;      // Waits for the 1 Task to complete

   // You MUST call DequeueAsyncResult to Remove the entry form the AsyncEnumerator object’s collection

   // Casting the return value and assigning to ‘t’ is not necessary; since ‘t’ already refer to the same Task object

   t = (Task<DateTime>) ae.DequeueAsyncResult();        

   Console.WriteLine(t.Result);     // Shows the DateTime when the Task completed
}

-- Jeffrey Richter (http://Wintellect.com)

During my CLR/BCL/C# 4.0 talk, a few people asked me questions for which I didn’t have the answer. I have now researched these questions and have the answers.

Question #1: Are the new concurrent collection classes in .NET 4.0 serializable?

Answer: Yes, ConcurrentStack, ConcurrentQueue, ConcurrentBag, and ConcurrentDictionary are all serializable. However, the BlockingCollection class (which can wrap any of these except for ConcurrentDictionary) is not serializable.

Question #2: How do you set multiple fallback languages with the ResourceManager class in Silverlight and in .NET 4.0?

Answer: The ResourceManager calculates the full fallback sequence as follows:

   1. Requested culture (argument to GetString or GetObject), if provided, or current thread UICulture (let’s call this C1) – as of .NET 4.0, only one culture can be specified this way.

   2. C1’s parent; repeat up parent chain but stop before invariant culture

   3. Cultures returned by the Win32 GetThreadPreferredUILanguages (passing the MUI_MERGE_USER_FALLBACK | MUI_MERGE_SYSTEM_FALLBACK flags), removing duplicates from 1 & 2 – this is the new addition in Silverlight & .NET 4.

   4. Invariant culture

I have been asked by many people if I will be updating my CLR via C# book for .NET 4.0.

Well, I'm happy to report that the answer is YES! I have already signed the contract with Microsoft Press and have been busy writing.

The new edition keeps its focus on the CLR and the C# language. And, since I didn't update the book for C# 3, I will actually be adding C# 3.0 content as well as C# 4.0 content. And, of course, I'll be covering the new features/changes between CLR 2.0 and 4.0.

The goal is to get the book out by the time .NET 4.0 ships. All looks good and I should easily be able to make this deadline.

Go here: http://channel9.msdn.com/shows/This+Week+On+Channel+9/This-Week-on-C9-Bing-Changes-to-NET-FX-40--play-Apple-IIe-games-on-your-Xbox/

And at about 23 minutes in, they talk briefly about the Wintellect Power Live Framework library.

Since late 2008, I have been spending a lot of time focusing on Microsoft’s Live Framework and Mesh technologies. Towards this end, I have produced a Power Live Framework library that simplifies coding against the Live Framework. And, I have also put together some sample applications that leverage my library. Furthermore, I have set up a Yahoo news group in order to support people that wish to use my library and sample code. The files and news group can be found here: http://tech.groups.yahoo.com/group/PowerLiveFx/.

The purpose of this blog post is to help you get started with programming for the Live Framework using my class library. For more information on the Live Framework in general, what it offers, and the concepts that surround it, I encourage you to see some of the videos from Microsoft’s PDC. Also, I will be speaking about Live Framework programming at Wintellect’s own Devscovery conference.

To develop against the Live Framework, you must:

·                     Provision your account:

o   Go to https://lx.azure.microsoft.com/Cloud/Provisioning/Default.aspx

o   From “New Project” page, click "Activate Live Framework CTP“

·                     If you want to access the Live Framework locally (not via the Internet), then download & run the developer client Live Operating Environment:

o   Go to https://developer.mesh-ctp.com/

o   Sign In, select “Add Device” and then install the client Live Operating Environment.

o   Then run “C:\Users\Jeffrey\AppData\Local\Microsoft\Live Framework Client\Bin\MeshAppMonitor.exe”.

o   Sign in with the client (see tray icon), and add your computer to your developer mesh.

·                     Download Microsoft’s developer SDK (& Tools)

o   Go to https://developer.mesh-ctp.com/Developers/Developers.aspx

o   Then, in your Visual Studio projects, reference Microsoft.Web.dll and Microsoft.LiveFx.ResourceModel.dll. These DLLs can be found in “C:\Program Files (x86)\Microsoft SDKs\Live Framework\v0.91\API Toolkits\.Net Library”. Do NOT add a reference for Microsoft.LiveFx.Client.dll as my Wintellect.LiveFx.dll is a replacement for this library.

o   For some projects, you will also need to add System.ServiceModel.Web.dll (included with the normal .NET Framework).

Some words about building you code with my Wintellect.LiveFx.dll library:

·                     I produced my library because I wanted a lightweight, simple, stateless, and RESTful API for programming against the Live Framework. Microsoft’s Microsoft.LiveFx.Client.dll is stateful, does not allow multiple asynchronous operations to execute concurrently and is not in keeping with the RESTful nature of the Live Framework itself. Microsoft and I have been communicating about this and they have plans to change their API and possibly adopt my API directly or something like it.

·                     My library can be used for all kinds of rich .NET applications including Console, Window Forms, Windows Presentation Foundation, ASP.NET, WCF, etc. At this time, my library will not work with Silverlight due to limitations of Silverlight’s HttpWebRequest class (for example, it only supports GET & POST methods and I need PUT & DELETE too). I am monitoring this closely and fully expect to bring my library to Silverlight in the future.

·                     Since my Wintellect.LiveFx.dll library offers and implements many asynchronous operations, it depends on Wintellect’s Power Threading library (Wintellect.Threading.dll). In fact, if you start accessing the Live Framework asynchronously, then you may also find your own code benefitting greatly from the classes contained inside my Power Threading library. The Power Threading library can be downloaded from http://Wintellect.com/PowerThreading.aspx and there is a separate Yahoo news group available that supports this library: http://tech.groups.yahoo.com/group/PowerThreading/.

An introduction to libraries and type for programming against the Live Framework:

·                     The Live Framework uses a RESTful model where applications make HTTP POST (create), GET (read), PUT (update), & DELETE (delete) operations against the Live Framework servers in the cloud or against a local (or client machine installed) Live Framework “server”. The information interchanged using the AtomPub protocol.

·                     The Microsoft.Web.dll (included with the Live Framework SDK) contains types for low-level AtomPub data processing, serialization, and deserialization. For most people, this is too painful a level to program at.

·                     The Microsoft.LiveFx.ResourceModel.dll (also included with the Live Framework SDK) contains types that mirror the Resources that the cloud/client Live Framework “servers” expose. You should think of the classes defined in this DLL as being data containers. That is, they mostly contain properties. There are some methods that deserialize the wire-protocol text to an instance of a .NET type and there are also methods that serialize an instance of a .NET type to the wire-protocol text required by AtomPub. Effectively, the .NET types provide developers with IntelliSense support while writing code and also compile-time type-safety when using the properties.

·                     The Wintellect.Net.HttpRestClient class (provided by my Wintellect.LiveFx.dll) is a class that simplifies communication with any REST server. This class is not specific to Microsoft’s Live Framework at all. In essence, each method on the class internally does the following:

o   Creates an HttpWebRequest

o   Initializes the HTTP method and headers (which can have some specified defaults)

o   Initiates a Create/Read/Update/Delete request & sends a payload synchronously or asynchronously

o   Returns the HttpWebResponse

·                     Note that the state of an HttpRestClient object can change immediately after a request has been initiated; there is no need to wait for a response. This allows a single HttpRestClient object to be used for multiple asynchronous operations that execute concurrently. However, a single HttpRestClient object should not be used by multiple threads at the same time without performing your own thread synchronization. If you want to have multiple threads concurrently accessing a REST server, then create multiple HttpRestClient objects (they are very lightweight in terms of resource consumption).

·                     The Wintellet.LiveFx.LiveFxClient class (also provided by my Wintellect.LiveFx.dll) is derived from HttpRestClient. The LiveFxClient class is specific to Microsoft’s Live Framework. This class knows about Base Uris and authorization tokens. This class also knows how to perform CRUD operations while converting the wire-protocol data to an instance of a resource class and vice versa.

·                     The Wintellet.LiveFx.LiveFxExtensions class (also provided by my Wintellect.LiveFx.dll) is a static class (and therefore has no state). This class defines a bunch of methods that extend the resource classes defined in the Microsoft.LiveFx.ResourceModel.dll. These methods add the behavior to the data, so to speak. In effect, the extension methods, offer IntelliSense thereby suggesting what actions can be performed on a resource and what additional arguments you must pass in order to perform the action. You must always pass a LiveFxClient object as this contains the default request information (like authorization). Most of the extension methods return a deserialized resource or resource collection. If you desire, you can call HttpRestClient’s SveNextRequestResponse method and/or its SetNextRequestQuery method before making a request to store the HTTP response code/headers in a HttpRestClientResponse object or to set query filter information.

The sample applications:

·                     The LiveFxPatterns project is a console application that demonstrates how to perform various synchronous and asynchronous operations against Microsoft’s Live Framework using my Power Live Framework library. The operations include:

o   Creating mesh objects, data feeds, and data entries (with and without media resources),

o   Creating a directory structure with folders & files visible via the Live desktop (running in a browser)

o   How to update a data entry.

o   How to use query operators when reading resource data.

o   How to add your own custom links to a resource.

o   How to add and read custom data associated with a resource.

o   How to map a mesh object to a device.

o   How to invite a member to a mesh object.

o   How to create a custom news item.

o   How to subscribe to a resource and process change notifications for it.

·                     The LiveFxBrowser project is a Windows Presentation Foundation application that shows you the contents of a user’s Mesh using a tree hierarchy which allows for easy navigation. When you select a node in the tree, you get a window showing you the contents of the selected resource. The view is “cooked” to show you the data in a human readable format: simple data is shown first, followed by collections, followed by link Uris. If you prefer, you can see the same information in raw wire-protocol format (XML, RSS, Atom, or JSON). I've included a screen shot showing the “cooked” view of the data as an attachment.

·                     I am working with Wintellect’s own Andy Hopper to produce a real Live Framework application called “Wintellect Cloudboard.” This application allows you to copy and paste clipboard items to your mesh effectively putting them in the cloud (see how we got the name?). You can then invite others to your Cloudboard and share each other’s clipboard items. You will also receive notifications when new items appear in the Cloudboard. We have set up yet another Yahoo news group for this application (http://tech.groups.yahoo.com/group/Cloudboard/) and I encourage you to subscribe to it if you’d like to be notified when the application becomes available. Our goal is to eventually share the code for Cloudboard and discuss our programming efforts it in articles.

I posted a new version of my Power Threading Library today on http://Wintellect.com/PowerThreading.aspx

This version has some minor bug fixes and a few new (minor) classes. See the Power Threading Overview.pdf file for change log information.

I have recently been spending a great deal of time writing code that communicated with RESTful services (Microsoft's Live Framework, in particular). Towards this end, I'm using the HttpWebRequest class to communicate and, for scalability and responsivenss, I want to perform my I/O operations asynchronously. This has caused me to look into the [Begin]GetRequestStream methods. At first it seemed quite odd to me that there would be a need to get the request stream asynchronously. Afterall, getting the stream just didn't seem like an I/O operation to me. However, via experimentation, I did notice that acquiring the stream did cause some I/O operations. This led me to start some communication with people at Microsoft to really get to the bottom of what is happenning. The information they shared was certainly news to me and I have never seen it documented anywhere. And so now, I'd like to share this information with you too as I think it is very useful for anyone using the HttpWebRequest class to send request data.

There are basically 3 ways to make an HTTP request that contains a payload:

1. If the payload is amll, you can call [Begin]GetRequestStream and it will immediately return a memory-backed stream that buffers the request payload. Then, when you call [Begin]GetResponse, the HttpWebRequest object calculates the ContentLength header based on how much data was written to the stream and then it initiates the I/O.
   
2. If the payload is large (2GB, for example), then you shouldn't cache the whole payload in memory before sending it. In this case, you'd set the ContentLength property explicitly to the payload size. Then when you calll [Begin]GetRequestStream, it now performs all of the I/O up to the point of sending the payload. You now write your payload to the stream which now sends its directly to the network (it does not get sent to an in-memory stream). When you call [Begin]GetResponse, it checks that the amount of data sent matches what you specified for the ContentLength property and does no additional I/O in order to start receiving the response.
   
3. If the payload is potentially large, generated at runtime, and you don't know the its size ahead of time then you set HttpWebRequest's SendChunked property to true (and do not set the ContentLength property). When you call [Begin]GetRequestStream, all of the I/O is performed up to the point of sending the payload data (including a "Trasnsfer-Encoding: Chunked" header). You now write to the stream in chunks and these writes go directly to the network. When you close the stream, a special terminating chunk is sent to notify the server that the payload is done. When you call [Begin]GetResponse, no I/O is performed and you can start receiving the response.

I hope you find this as useful as I did. Now, I can make intelligent decisions about my payload data, its size, how to buffer it, how to set headers, and when I can perform I/O synchronously as opposed to asynchronously. I really feel empowered with this knowledge.