I've been doing quite a bit with Reactive Extensions (Rx) for Silverlight lately. One idea that I keep exploring is the concept of creative intuitive sequential workflows for asynchronous operations. You can read about my explorations using Wintellect's own Power Threading Library in this post along with a simple solution using an interface and enumerators in part 2 of that series. I'm tackling the problem from a different angle.

First, my disclaimers: this is more of an exploratory, "Here's what can be done" post. Please don't take this as best practice or guidance, but more of another way of looking at the solution as well as an opportunity to dive deeper into Reactive Extensions. Also understand I am by no means an expert with Rx so I welcome feedback related to this experiment.

The concept is straightforward: there are often times we want an asynchronous set of operations to perform sequentially. Perhaps you must load a list from a service, then load the selected item, then trigger an animation. This can be done either by chaining the completed events or nesting lambda expressions, but is there a cleaner way?

To me, a cleaner solution would make it easy to see what happens sequentially, and also easy to fire the sequence. If it becomes too complex to force the sequential workflow, it buys us nothing from the traditional methods of wiring into completed events or nesting lambda expressions.

So, I started with the idea of having a helper object that I could feed operations to in sequence, then fire it off. What would that look like?

First, I made an interface to keep track of operations as they run and complete. This is by no means a perfect design but for now it's a sort of "recrusive container" for work done. The interface looks like this:

public interface ISequentialResult 
{
    List<ISequentialResult> Results { get; set; }
    object Value { get; set; }        
}

The result is simple: it aggregates all previous results, and contains a pointer to whatever is processing the current action in the sequence. Simple enough. Now on to a base class:

public abstract class BaseResult<T> : ISequentialResult 
{
    protected IObserver<ISequentialResult> Observer { get; set; }

    protected BaseResult(IObserver<ISequentialResult> observer)
    {
        Results = new List<ISequentialResult>();
        Observer = observer;
    }

    public List<ISequentialResult> Results { get; set;}

    public object Value { get; set; }       

    public T TypedValue
    {
        get { return (T) Value; }
        set { Value = value; }
    }
}

The base class does a little more for us. First, it introduces the concept of an IObserver. The observer is simply a class that receives notifications, and the type is the type of the notification. In this case, we allow for a derived type that will create the observer as well as take our "object" value and type it to a specific value. You'll see how the observer works with our sequences in a bit.

Next, I created three implementations. One is just a default that does nothing, and the other two handle Click events and Storyboard interactions. Let's take a look at the handler for storyboards:

public class StoryboardResult : BaseResult<Storyboard>  
{
    public StoryboardResult(IObserver<ISequentialResult> observer, Storyboard sb) : base(observer)
    {
        TypedValue = sb;
        sb.Completed += SbCompleted;
        sb.Begin();
    }

    void SbCompleted(object sender, EventArgs e)
    {
        TypedValue.Completed -= SbCompleted;
        Observer.OnNext(this);
        Observer.OnCompleted();
    }       
}

You'll notice a few interesting things. We take in the story board, wire into its Completed event, then fire it off (we could have changed the interface to have an Execute method to give this more flexibility). The story board completion is the important piece to consider. First, the event is unhooked. Then, we pass the result to the observer. The observer is waiting for ISequentialResult notifications. We provide it with one, but only after the story board is completed. We also tell the observer "we're done" (an observer can listen for multiple push notifications, so we could have created a handler that never completed, or one that aggregated multiple storyboards and only completed when all story boards were done).

With that in mind, take a look at the Click handler:

public class ClickResult : BaseResult<ButtonBase>
{
    public ClickResult(IObserver<ISequentialResult> observer, ButtonBase button) : base(observer)
    {
        TypedValue = button;
        button.Click += ButtonClick;
    }

    void ButtonClick(object sender, System.Windows.RoutedEventArgs e)
    {
        TypedValue.Click -= ButtonClick;
        Observer.OnNext(this);
        Observer.OnCompleted();                 
    }
}

This is very similar to the storyboard. There is nothing to "kick off" but instead this result simply registers itself whenever the target is clicked. Why would we do that? I'll get to the example in a minute.

The most interesting part of workflow is the sequencer itself. This is the class that will manage the asychronous events and coordinate them. The key is that while they run sequentially, they also run asynchronously so there is no blocking.

Here is the sequencer class:

public class Sequencer : IDisposable 
{
    private IDisposable _sequence;
    private ISequentialResult _result;

    private readonly List<Func<IObserver<ISequentialResult>,ISequentialResult,ISequentialResult>> _sequences
        = new List<Func<IObserver<ISequentialResult>, ISequentialResult,ISequentialResult>>();

    public void AddSequence(Func<IObserver<ISequentialResult>, ISequentialResult,ISequentialResult> sequence)
    {
        _sequences.Add(sequence);
    }
        
    public void Run(Action<Exception> exception, Action<ISequentialResult> completed)
    {
        _sequence = Observable.Iterate(_Sequencer)
            .Subscribe(
                next => { },
                exception,
                ()=>completed(_result));
    }

    private IEnumerable<IObservable<Object>> _Sequencer()
    {
        var x = 0;

        _result = new DefaultResult(null) {Value = this};

        while (x (
                observer =>
                    {
                        var newResult = sequence(observer, _result);
                        newResult.Results.Add(_result);
                        newResult.Results.AddRange(_result.Results);
                        _result = newResult;
                        return () => { };
                    }
                ).Start();

            yield return step;                

            x++;
        }

        _result.Results.Add(_result);
    }

    public void Dispose()
    {
        _sequence.Dispose();
    }
}

There's a lot going on here, so let's break it down.

First, you'll notice two references: one to a disposable object, and one to a result. More on those in a bit.

The list may seem confusing at first, but it's only because of all of the type specifiers. Each "step" is represented by a function. The sequence will call the function with an observer and the previous result, and expect to get a new result back. Think of it as "here's who is watching, and what happened the last time ... now give me what you have."

A method is provided to add these steps to the sequence.

The run method is interesting. This is where we set up our disposable reference because we're creating a long-running observer that we want to dispose of when the sequence itself is disposed. The iterate function takes a list of observable sequences and observes them sequentially. We provide an enumerator that feeds the observable sequences, and for each sequence in the "outer loop" (the main algorithm that drives the sequential work flow) we simply drive through the collection. If an exception is encountered, we'll call back with the exception. When completed, we call back with the final result.

To better understand what's going on, take a look at the enumerator. For each iteration, we create a new observable stream. We call the function I mentioned earlier and pass the observer in. When we receive the result, we stack it recursively and store it, then iterate to the next in line. The yield statement ensures the sequential operation completes (when we call the OnCompleted in our result) before the next step begins.

That's the complicated part: setting up the core framework. Now comes the easy part: plugging into it.

In the XAML I placed a large red rectangle. There are three storyboards tied to the rectangle. One changes the colors, one shrinks it, and one twists it using the plane projection. There are also three buttons. One button kicks off the story boards. One button kicks off a sequential workflow that will run the story boards in order. The final button resets the story boards by calling Stop on them.

Let's have some fun. First, kicking off the story boards is simple enough:

private void Button_Click(object sender, RoutedEventArgs e)
{
    StoryTwist.Begin();
    StoryShrink.Begin();
    StoryColor.Begin();
}

The reset button is also easy:

private void Button_Click_2(object sender, RoutedEventArgs e)
{
    StoryTwist.Stop();
    StoryShrink.Stop();
    StoryColor.Stop();
}

Now for the sequential storyboards. This is where things should get easier for us. Instead of having to wire in several completed events or nesting lambdas, let's see what it looks like to run them in order using our sequencer:

private void Button_Click_1(object sender, RoutedEventArgs e)
{
    if (_sequence != null)
    {
        _sequence.Dispose();
    }

    _sequence = new Sequencer();
    _sequence.AddSequence((o, r) => new StoryboardResult(o, StoryColor));
    _sequence.AddSequence((o, r) => new StoryboardResult(o, StoryShrink));
    _sequence.AddSequence((o, r) => new StoryboardResult(o, StoryTwist));
    _sequence.Run(ex => MessageBox.Show(ex.Message),
                    result =>
                        {
                            foreach (var storyboard in
                                result.Results.Select(sequence => sequence.Value).OfType<Storyboard>())
                            {
                                storyboard.Stop();
                            }
                            _sequence.Dispose();
                            _sequence = null;
                        });
}       

So what's going on? If we have a previous sequence running, we dispose it so we can start fresh. Then, we simply add each story board in the order we want it to fire (the observer is provided to us by the sequencer). Finally, we run it. If there is an error, show it. When the sequence is done, we iterate the results and find any result that was a story board and call the "stop" method. This means after the sequence completes, it will automatically restore the rectangle to its original state.

Finally, to show just how powerful it is to drive sequential workflows without blocking, I added one more method:

private void _WaitForThreeClicks()
{
    _buttonSequence = new Sequencer();
    _buttonSequence.AddSequence((o, r) => new ClickResult(o, ResetButton));
    _buttonSequence.AddSequence((o, r) => new ClickResult(o, ResetButton));
    _buttonSequence.AddSequence((o, r) => new ClickResult(o, ResetButton));
    _buttonSequence.Run(ex=>MessageBox.Show(ex.Message),
        result=>
            {
                MessageBox.Show("You clicked the Reset button 3 times!");
                _buttonSequence.Dispose();
                _WaitForThreeClicks();
            });
}

This is a fun method. It literally creates three sequences, all "waiting" for the reset button to be clicked. After the sequence completes, we show a message indicating that 3 clicks happened, then restart the sequence recursively. I'll call it the first time just after IinitializeComponent:

public MainPage()
{
    InitializeComponent();  
    _WaitForThreeClicks();
}      

There you have it! When you run the code, you'll get this:

Notice that you can keep clicking the sequence to start it over: there is no blocking. And whatever order you decide to click on other buttons, the reset button always faithfully shows a "3 click" message on the third click.

You can download the full source code for this solution here.

I received quite a bit of feedback related to my sequential asynchronous workflows post. Most people asked if I could simplify the example even more and/or show the inner workings of the coroutine engine itself. Because the source for the library I demonstrated in the last post is not available, I've put together a very simple solution to help you better understand and take advantage of coroutines.

First, the fine print. This framework is simply to help understand the concept and provide a starting point for building out coroutines yourself. I know some companies simple don't allow third-party code and it always helps to learn a framework by starting at the ground floor. Some things you won't find here that would belong in production code include thread-awareness (background or UI thread?), timeouts (i.e. if I call out and it never returns, does my workflow die?) and exception management. All very important but reasons why this framework is educational and not a "production-ready" implementation.

Download the source for this post

Remember the original interface we defined for illustration purposes, the ICoroutine interface? Here it is:

public interface ICoroutine
{
    void Yield();
    Action Yielded { get; set; }
}

Now, to drive a workflow, you simply need an engine. Let's build a quick and dirty engine to drive what we want. Last post I showed how to use an existing framework. This time we'll do it a little differently. Instead of queuing asynchronous calls using the asynchronous process model, we'll drive the process with the coroutine interface itself. This comes with some caveats. What I'm going to show you will work, but a production solution will have to take it a step further and deal with things like execution threads (background vs. UI) etc (this is all handled as part of the AsyncEnumerator class I showed you before).

So, here is our "bare bones" coroutine manager, and again, this is closer to Rob Eisenberg's implementation in his MIX session on building your own MVVM framework than the version I showed you last time, but I've taken it and dumbed it down as much as possible to make it easy to use and understand.

public class CoroutineManager
{
    private readonly IEnumerator<ICoroutine> _enumerator;

    public CoroutineManager(IEnumerable<ICoroutine> workflow)
    {
        _enumerator = workflow.GetEnumerator();
    }
    
    private void Yielded()
    {            
        if(!_enumerator.MoveNext())
            return;

        var next = _enumerator.Current;
        next.Yielded = Yielded; 
        next.Yield();
    }
    
    public static void Begin(object workflow)
    {
        if (workflow is ICoroutine)
        {
            workflow = new[] {workflow as ICoroutine};
        }
        
        if (workflow is IEnumerable<ICoroutine>)
        {
            new CoroutineManager(workflow as IEnumerable<ICoroutine>).Yielded();
        }            
    }     
}

Not much to it, is there? The class has an entry point that can take a single coroutine or group, instantiate the class, then kick off the workflow. The workflow simply takes the next coroutine from the stack, wires in the yielded event to call back into the coroutine manager, and then executes it. That's it!

So how does it look? Let's make two helper classes: one that is generic and can handle any type of action, and another that is specific to our random number service.

Here's what the generic action coroutine looks like:

public class ActionCoroutine : ICoroutine 
{
    private readonly bool _immediate;

    public Action Execute { get; set; }

    public ActionCoroutine()
    {
        
    }

    public ActionCoroutine(bool immediate)
    {
        _immediate = immediate;
    }

    public ActionCoroutine(Action action)
    {
        _immediate = false;
        Execute = action;
    }

    public ActionCoroutine(Action action, bool immediate)
    {
        _immediate = immediate;
        Execute = action;
    }

    public void Yield()
    {
        Execute();
        if (_immediate)
        {
            Yielded();
        }
    }

    public Action Yielded { get; set; }  
}

Again, not much to it. I can either pass an action to trigger immediately, or pass an action and set immediate to false. If I set immediate to false, then I need to wire in something to call the Yielded method (remember, our coroutine manager wires this up for us to re-enter the iterator state machine). I'll show you usage in a second. Finally, here is my random number service helper:

public class RandomResultCoroutine : ICoroutine
{
    private readonly RandomNumberService _service;

    public RandomResultCoroutine(RandomNumberService service)
    {
        _service = service;
    }

    public int Max { get; set; }

    public int Result { get; set; }

    public void Yield()
    {
        _service.GetRandomNumber(Max,
                                 result =>
                                     {
                                         Result = result;
                                         Yielded();
                                     });
    }

    public Action Yielded { get; set; }
}

Notice how this service "wires itself." It has a max setting, calls the random number service, and tells the random number service to call back into a lambda expression. The expression sets the return result, then fires the Yielded message to re-enter the state machine flow.

This is an example where we can make it work, but a more robust solution will have to handle the exceptions. For example, what if the service never calls my action? Then I'm in a bad state because the Yielded will never get executed. That's why having timeouts and other checks and balances are important for a production-ready solution.

OK, we've set up our simple helpers and coroutine manager, let's see it in action. I'm just going to do everything in the code-behind for the main page to keep it simple. I'll set up three workflows. Two will generate shapes (circles and squares) and then feed the shapes to the third workflow which animates colors. This means we'll actually have dozens of workflows running simultaneously, but they will still fire sequentially within the workflow.

Take a look at our color workflow (it has as many iterations as seconds are in a day, just to keep it going for you to watch):

private IEnumerable<ICoroutine> ColorWorkflow(Shape element)
{
    for (int x = 0; x 
                                    {
                                        element.Fill = new SolidColorBrush(color);
                                        ((Storyboard) o).Stop();
                                        storyboardAction.Yielded();
                                    };

        yield return storyboardAction; 
    }

    yield break; 
}

Notice how straightforward it is. With our direct random number service helper, we can keep yield returning results. We will only go past the yield statement when the service call actually returns, and we can inspect the result because of the field we added to the helper, which is only set when the result is received and before the state machine is re-entered by calling yielded.

What's nice about implementing the ICoroutine interface is that all you have to do to repeat the call and get a new result is simply yield the same helper class. The implementation ensures that the manager will call into it, block until a result is received, then continue execution with the new value available.

For the storyboard, we use the generic action coroutine. The begin action is set to kick off the storyboard and when it ends we set the new color and stop the storyboard. In this case I also wire in a call to yielded and set the "immediate" flag to false because we're depending on the story board completion to continue the workflow.

The square and circle workflows are exactly the same, so I'll just show the square one here (probably means they could be refactored to something simpler, too, but it works for this demonstration).

private IEnumerable SquareWorkflow()
{            
    var randomAction = new RandomResultCoroutine(_service) {Max = 20};

    randomAction.Yield();
    yield return randomAction;

    int iterations = randomAction.Result + 5;

    for (int x = 0; x  LayoutRoot.Children.Add(rectangle), false);                                   
        rectangle.Loaded += (o, e) => loadedSquare.Yielded();

        yield return loadedSquare;

        CoroutineManager.Begin(ColorWorkflow(rectangle));
    }

    yield break;
}

This time we get a random number of squares and begin setting them up. We have a random size for the squares. Note we use the generic action coroutine to fire adding the square and loading it, and only when it is loaded do we kick off the color workflow to begin animating the colors on the square. This same workflow is repeated for circles.

You see how easy it is to kick off the routine? In fact, to kick off the main workflows, we simply do this:

public MainPage()
{
    InitializeComponent();
    Loaded += MainPage_Loaded;
}

void MainPage_Loaded(object sender, RoutedEventArgs e)
{
    CoroutineManager.Begin(SquareWorkflow());
    CoroutineManager.Begin(CircleWorkflow());
}

With those kicked off, even though we have two coroutines, you'll see they run asynchronously. While the circles and squares are sequentially added and animated (as opposed to popping in immediately as would happen if they were asynchronous within the workflow), they do so in parallel with each other (and once loaded, all of the color workflows continue to animate the individual elements but don't kick off a new color until the old storyboard is complete).

Here it is to play with. Due to a bug (can you find it?) the storyboard animation runs and completes but we only see the color change when it's done. That's OK because you can pick any shape on the surface and count 2 seconds and you'll notice the color changes every 2 seconds ... for every shape, proof that we have simultaneous sequential workflows all running asynchronously with each other. We also may see some CPU and memory issues over time because I'm not unhooking events.

Hopefully this helped simplify it a bit for those of you who were having trouble with the last post or wanted to see the innards of a framework so you can begin to build your own infrastructure.

Download the source for this post

It all began with a post I read in the Silverlight.Net forums. A user, new to Silverlight, was frustrated with the asynchronous programming model required by Silverlight and wondered how to make all of the calls synchronous. I admit I heavily resisted the notion because I think the asynchronous programming model is one that developers should learn and be comfortable with. Forcing an asynchronous call to "look synchronous" seems to involve a lot of framework overhead just to keep the developer from having to learn something new, and would limit their ability to take advantage of the functionality available.

Then I watched Rob Eisenberg's Build your own MVVM Framework presentation and learned what he was doing with a concept known as coroutines. That was the "ah-hah" moment: some beginning programmers truly are looking for a crutch and should take some time to embrace the asynchronous model and learn it. Other developers have embraced the model but are faced with an interesting challenge: how to keep code maintainable and readable when it is littered with asynchronous calls?

Download the source for this post Note: this requires Wintellect's PowerThreading library. Download that here, then follow the instructions on the download link to add the reference.

Consider for a moment a typical workflow in a service-driven Silverlight application. The application must fetch information from a service, then perhaps take that data and push it to another service, then take even more data and bring it back. You might be Simplifying Asynchronous Calls using Action, but the code can start to look a little interesting.

You really end up with two solutions. One is to wire in the calls with methods, like this (consider a scenario where we get a number from a service, and the previous call result feeds into the next call):

private int n1, n2; 

private void FirstCall()
{
    Service.GetRandomNumber(10, SecondCall);
}

private void SecondCall(int number)
{
    n1 = number; 
    Service.GetRandomNumber(n1, ThirdCall);
}

private void ThirdCall(int number)
{
    n2 = number; 
    // etc
}

Or you can go lambda expression/delegate crazy like this:

private void FetchNumbers()
{
    int n1, n2; 

    Service.GetRandomNumber(10, result =>
                                    {
                                        n1 = result;
                                        Service.GetRandomNumber(n1, secondResult =>
                                                                        {
                                                                            n2 = secondResult;
                                                                        });
                                    });
}

That may be somewhat manageable, but what if your workflow chains, say, a dozen calls together?

You don't want your asynchronous calls to be synchronous. You just want to make them sequential.

Does that point make sense? If and when you become comfortable with asynchronous programming, you understand the issues with blocking threads and waiting for calls and everything else. That's not the issue. You just want clean code that does the job. Something like this:

private void FetchNumbers()
{
   int n1 = Service.GetRandomNumber(10); 
   int n2 = Service.GetRandomNumber(n1);
}

Let that run. We understand it's asynchronous and don't want to block everything else, but please, please, let's just do it sequentially and make it easy to read, mmm kay?

This is where the power of coroutines comes in.

C# does not have direct support for coroutines.

Let's make that clear up front. We have ways to build frameworks that make them happen, but it's not an "out-of-the-box" implementation. But let's start with the basics: what is a coroutine, anyway?

When I learned about coroutines, I understood the concept immediately. What frustrated me was understanding the mechanism to apply them in C#. Most of the posts and topics I uncovered assumed a fairly advanced base of knowledge and often jumped into a solution with a framework and didn't describe the framework itself. I consider myself a fast learner and the fact that I was hitting my head against a brick wall and failing to find some decent "101" tutorials put me on a mission: first, build my own framework from scratch to prove to me that I truly understand coroutines in C#, and second, blog about it so that you, too, can benefit from the learning process to receive better understanding and have an easy set of references to learn about them.

Please use the comments below to let me know if I succeeded!

Note: while I did build my own framework to gain understanding, the example here uses an existing, free library that has been maintained for years. It has been thoroughly tested for stability and performance and is chock full of additional features and benefits ... I'll share more on that in a bit.

In summary, a coroutine is a subroutine or method with multiple entry and exit points that maintains state between calls. That is me paraphrasing the various definitions there. The two keys to a coroutine are that you may enter it from places other than the beginning, and that it retains state when you enter it again after exiting.

When I mentioned that C# doesn't directly support coroutines, I wasn't 100% correct. C# has native support for a very specialized form of a coroutine, known as an iterator. You use these all of the time in your code. Traditional thought is that a foreach loop really just takes a list, like a piece of tape, and then spins through the tape, right? We really just take an index, then increment the index to look at item[0] and item[1] and we're on our merry way, right?

Not quite. The most common uses of iterators work that way, but behind the scenes something far more complex is happening. Whenever you use an iterator (which is really any time you use the keyword foreach), you are invoking a state machine under the covers.

If this is news to you, stop right now and head over to my article, What's in Your Collection Part 3 of 3. Don't worry, parts 1 and 2 just cover basic collections. This part focuses on iterators and shows how an iterator is really a state machine.

Those of you already familiar with the yield command know this. For an example of exploiting this knowledge, take a look at Pipeline and Yield in C#. Once you understand the iterator is a state machine, you begin to realize the possibilities. You don't always have to have the full list, for example. You can generate it on the fly! This way you only create objects in the list when and as you need them.

So this is great, we understand a little bit about coroutines and iterators in C#, but what does that have to do with "sequential asynchronous workflows in Silverlight using coroutines?" A lot, actually.

You see, we can take advantage of the iterator state machine to build our own coroutines. The way we do that is by taking over the IEnumerator implementation required to perform iterations.

Take a moment to look at the IEnumerator documentation. Now, stop thinking of foreach as a simple loop, and instead, consider it a state machine. You climb into the state machine, and the rules are simple:

1. We're currently at Current. In fact, we're stuck here until MoveNext is called.
2. Only a call to MoveNext will let us advance. If the call is true, we are now at a new Current state, so back to step 1. If the call is false, we're done. We can either go home, or Reset and start all over again.

The only caveat is that MoveNext can really do a number of things, until it hits a yield statement. Once the yield statement is reached, the process stops until the next MoveNext is called.

Those are some pretty simple rules, aren't they? Armed with that knowledge, you should now be able to predict what will happen in this simple little routine:

public void PrintAll()
{
   foreach(int x in IntegerList())
   {
      Console.WriteLine(x); 
   }
}

public IEnumerable<int> IntegerList()
{
   yield return 1;
   Thread.Sleep(1000);
   yield return 2; 
   yield return 3; 
}

In this program, you'll see 1, then wait a second, then see 2 and 3, then hang. Yes, hang: the program will not end. Remember our rule that we keep doing "something" until we hit a yield statement. In the state machine above, we return 1 and save our state. When the loop returns, we call move next. This results in jumping into our spot right after the yield, where we sleep, then return 2. After we return 3, the foreach loop still asks for something more. The state machine hits the end of the yield statement, and returns ... never finding anything. Because yield isn't hit, the machine just sits there, waiting.

To rectify the situation, we add a "yield break" to the end. This will result in the MoveNext call returning false and let our state machine know we're out of states. Then, unless we reset the enumerator, we're done.

Solving the Problem with Coroutines

Let me restate the problem: we have asynchronous calls that we want to process sequentially without having to chain dozens of methods or resort to complex lambda expressions. How do we do it?

The trick lies within our state machine. Because the state machine is linked to two key events: MoveNext and the yield keyword, we can hook into the enumerator and force the state machine to wait for our asynchronous actions to complete. We do this without blocking other threads: everything else still happens asynchronously, we just have a "sequential view" of our workflow.

The first step is to consider a generic contract we'll use to navigate the workflow. This is similar to Rob's but I chose to use a delegate instead of an event, as I don't need multiple subscribers to the workflow. This is a very simple interface, but it's important to note something: the Yield method is what is called for the class to start it's "life" in the state machine. The Yielded delegate should be called when it's done. If I have an instant action, then yield might be:

() => { action(); Yielded(); }

So you enter through one function, but call the other. Who sets the Yielded action? Our enumerator, or the "state machine manager" that is taking care of everything. Here's the interface:

public interface ICoroutine
{
    void Yield();
    Action Yielded { get; set; } 
}

Note: I'm using this interface to help conceptualize the model. We won't actually use this interface in the final solution. It helps here with understanding the general flow.

Before we worry about an implementation, let's conceptualize the engine that drives this, I'll call the engine CoroutineManager. This is the "master of the universe" that controls the state engine. Let me show the idea to you first, then we'll talk about what it does:

CoroutineManager

Begin: Yield()    

Yield: Move Iterator
       At End? 
          Yes: Done 
       No: Grab next item
       Set next item "Yielded" action to point to local "Yield" action
       Call "Yield" on item 

OK, let's break it down. This is driving a state machine. The method will encapsulate our workflow. Each item in the workflow gets passed into the coroutine engine, does it's thing, then exits and moves to the next workflow item.

To help us enter our workflow, we have a Begin method to start the process. This simply calls into the Yield, which is the main routine.

Now we play a game of back-and-forth using yield and yielded. The manager's yield method moves the iterator along. If there is nothing left, it stops. Our workflow is finished. Otherwise, it does two interesting things. First, it hooks up the next instance to call back to the engine by setting the Yielded action. While this is done sequentially, it also happens asynchronously. I prove that in the sample code by having several workflow happen simultaneously. We'll get to that in a second.

Once the callback is wired in, it calls into the item to execute. Note that our enumerator is now "hanging" and cannot go anywhere until we come back into the Yielded action, and then it will go to look for another yield in the code.

Conceptually, it looks something like this ... here our workflow has some miscellaneous code, then yield returns an ICoroutine instance, then has some more code, etc. Keep in mind the "hook" into yielded happens based on an asynchronous event completing, or a direct call:

So how do we use this engine, now that we have it defined?

The Wintellect PowerThreading Library

Wintellect's Jeffrey Richter has built a PowerThreading Library that does exactly what we needed. The Silverlight version is lightweight (under 30KB) but contains some powerful functionality. I'll let you drive through the full capabilities of the library; we're going to focus on two key aspects.

First, the AsyncEnumerator class is what helps us implement the ICoroutine concept. This class allows for asynchronous sequential enumeration of any process that implements the IAsyncResult interface. Next, we'll use the EventAPMFactory to cast event-driven processes into an asynchronous result. Because some of our code uses action callbacks instead of events, we'll also take advantage of a simple event wrapper that allows any process to fire a completed event.

You can find the download link for the PowerThreading library, complete with documentation and examples, here.

First, the class that allows any action to become an event:

public class EventWrapper
{        
    public EventWrapper()
    {
        
    }

    public EventWrapper(Action action)
    {
        Begin = action;
    }

    public Action Begin { get; set; }

    public void End()
    {
        if (Completed != null)
        {
            Completed(this, EventArgs.Empty);
        }
    }

    public event EventHandler Completed; 
}

Fairly straightforward: set the start action, and make sure you call End and it will raise the Completed event.

For the sample program, I included several layers so you can see both the sequential workflow, but understand it is still truly asynchronous because other workflows are also executing at the same time. Here's the premise:

First, I simulate a "random number" routine that gets passed a maximum value and returns a random number. We pretend this is a service call and even build in a random delay so it doesn't return immediately. In my main page, I'm going to do a few workflow tasks like animating a text box, etc. Then, I make a square 400 x 400. I get a random number from the "service" with a max of 400, and draw that square. If the square has a size of 320 x 320, I then get another random number between 0 and 320. The result is subsequently smaller squares until we hit zero.

The signature of the random number routine is:

...
void GenerateRandomNumber(int max, Action<int> result); 
...

To make it even more interesting, the squares themselves use a workflow to get random numbers to set their fill color. This is interesting because they use the same random number generator with the delay built in, so each square may take several seconds before it gets a color. The reason I did this is so you can see the squares being drawn and colored at the same time: proof we are asynchronously stepping through each workflow, however the individual workflows themselves are firing sequentially.

Here is the XAML for a square:

<UserControl x:Class="Coroutines.SquareView"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" 
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" 
    x:Name="SquareHost"
    >
    <Grid x:Name="LayoutRoot" Background="Black"
          HorizontalAlignment="Center"
          VerticalAlignment="Center">
        <Rectangle x:Name="Square" 
                   Width="{Binding Size}" 
                   Margin="3" 
                   Height="{Binding Size}" 
                   Fill="{Binding RectangleFill}"/>
    </Grid> 
</UserControl>

As you can see, it's a black grid with a rectangle that is bound to a size and a fill property. Here's the code behind:

[Export]
public partial class SquareView : IPartImportsSatisfiedNotification 
{
    [Import]
    public RandomNumberService Service { get; set; }

    public SquareView()
    {
        InitializeComponent();
        SetValue(NameProperty,Guid.NewGuid().ToString());
        LayoutRoot.DataContext = this;            
    }

    public static readonly DependencyProperty SizeProperty =
        DependencyProperty.Register(
            "Size",
            typeof (int),
            typeof (SquareView),
            null);

    public int Size
    {
        get { return (int) GetValue(SizeProperty); }
        set { SetValue(SizeProperty, value);}
    }

    public static readonly DependencyProperty RectangleFillProperty =
        DependencyProperty.Register(
            "RectangleFill",
            typeof(SolidColorBrush),
            typeof(SquareView),
            new PropertyMetadata(new SolidColorBrush(Colors.Gray)));

    public SolidColorBrush RectangleFill
    {
        get { return (SolidColorBrush)GetValue(RectangleFillProperty); }
        set { SetValue(RectangleFillProperty, value); }
    }

    private IEnumerator<Int32> ColorWorkFlow(AsyncEnumerator ae)
    {
        int alpha = 0, red = 0, green = 0, blue = 0;

        ae.SetOperationTag("Starting random number request.");

        var randomNumberWrapper = new EventWrapper();
        randomNumberWrapper.Begin =
            () => Service.GetRandomNumber(128,
                                          result =>
                                              {
                                                  alpha = result + 128;
                                                  randomNumberWrapper.End();
                                              });
        
        var eventArgsFactory = new EventApmFactory<EventArgs>();
        EventHandler serviceEnded = eventArgsFactory.PrepareOperation(ae.End()).EventHandler;
        randomNumberWrapper.Completed += serviceEnded;
        randomNumberWrapper.Begin();

        yield return 1;
        eventArgsFactory.EndInvoke(ae.DequeueAsyncResult());

        ae.SetOperationTag("Starting request for color red.");

        randomNumberWrapper.Begin =
            () => Service.GetRandomNumber(255,
                                          result =>
                                          {
                                              red = result;
                                              randomNumberWrapper.End();
                                          });

        randomNumberWrapper.Begin();
        yield return 1;

        eventArgsFactory.EndInvoke(ae.DequeueAsyncResult());

        ae.SetOperationTag("Starting request for color green.");

        randomNumberWrapper.Begin =
            () => Service.GetRandomNumber(255,
                                          result =>
                                          {
                                              green = result;
                                              randomNumberWrapper.End();
                                          });

        randomNumberWrapper.Begin();
        yield return 1;

        eventArgsFactory.EndInvoke(ae.DequeueAsyncResult());

        ae.SetOperationTag("Starting request for color blue.");

        randomNumberWrapper.Begin =
            () => Service.GetRandomNumber(255,
                                          result =>
                                          {
                                              blue = result;
                                              randomNumberWrapper.End();
                                          });

        randomNumberWrapper.Begin();
        yield return 1;

        randomNumberWrapper.Completed -= serviceEnded;
        eventArgsFactory.EndInvoke(ae.DequeueAsyncResult());

        RectangleFill = new SolidColorBrush(new Color { A = (byte)alpha, B = (byte)blue, G = (byte)green, R = (byte)red });

        ae.SetOperationTag("End of Color Workflow.");

        yield break; 
    }
   
    public void OnImportsSatisfied()
    {
        var ae = new AsyncEnumerator("Color Workflow");
        ae.BeginExecute(ColorWorkFlow(ae), ae.EndExecute);           
    }
}

The user control itself is exported. We have to import the service for the random number, so we wait until that is available using IPartImportsSatisfiedNotification. Once it is imported, we begin our workflow. You'll notice I'm using a helper class to call to the random number service and get the alpha, red, green, and blue values for my color. When I'm done, I set it and break out of the loop. In a nutshell, here are the steps we're taking:

1. Create an AsyncEnumerator and kick off the process
2. Implement IEnumerator<Int32> for the workflow. The documentation for the PowerThreading library explains this. Basically, the enumerator uses the yield return statements to keep track of how many pending asynchronous calls exist. We return 1 each time we start a process because were queuing up a single call. The yield statements cause the state machine to block until the asynchronous calls are complete.

With our ICoroutine concept, we "yield" or begin the process by kicking off the actual asynchronous call. The call's end is hooked into the Async Enumerator's End method (the "yielded" concept we discussed). The yield returns are what lock the state engine to wait for the operation to complete.

If you recall the diagram above, here is what the overall process looks like:

1. Kick it off by calling BeginExecute on the coroutine manager (our AsyncEnumerator)
2. Wire in a process so it completes by calling the coroutine manager's End method
3. Kick off the process
4. Yield
5. The engine now blocks until the asynchronous call completes and hooks back into the end method
6. Now we dequeue the result, and decide if we wish to continue the workflow or end it

By wrapping the random service call in the event wrapper, I can use the library's EventAPMFactory to convert the event to an asynchronous result that implements IAsyncResult. You'll notice we hook into the completed event by pointing to our enumerator, which will handle grabbing the result and implementing the interface for us. When it comes back, we dequeue the result and continue on, and unhook the event handler at the end.

The code behind does some other interesting things. You'll notice it exposes the dependency properties for the size and color so it sets the data context for the grid to itself. Not always the best practice, but it made this example a little more simple. You'll also notice we aren't doing anything with the size. Where does that come from?

The size is populated by the main workflow in the main page. The main page has a text box and some story boards. It dynamically inserts the squares onto the grid surface via the workflow. Take a look at the page XAML:

<UserControl x:Class="Coroutines.MainPage"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" 
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008" xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" 
    mc:Ignorable="d" d:DesignWidth="640" d:DesignHeight="480">
    <Grid>
        <Grid.Resources>
            <Storyboard x:Name="ScaleText">
                <DoubleAnimation Storyboard.TargetName="TextScale" Storyboard.TargetProperty="ScaleX"
                                 From="0.3" To="1.0" Duration="0:0:2"/>
                <DoubleAnimation Storyboard.TargetName="TextScale" Storyboard.TargetProperty="ScaleY"
                                 From="0.3" To="1.0" Duration="0:0:2"/>
            </Storyboard>
            <Storyboard x:Name="SlideText">
                <DoubleAnimation Storyboard.TargetName="TextSlide" Storyboard.TargetProperty="X"
                                 From="400" To="0" Duration="0:0:2"/>                
            </Storyboard>
            <Storyboard x:Name="HideText">
                <ObjectAnimationUsingKeyFrames 
                    Duration="0:0:2"
                    Storyboard.TargetName="Loading" Storyboard.TargetProperty="(UIElement.Visibility)">
                    <DiscreteObjectKeyFrame KeyTime="0:0:0">
                        <DiscreteObjectKeyFrame.Value>
                            <Visibility>Visible</Visibility>
                        </DiscreteObjectKeyFrame.Value>
                    </DiscreteObjectKeyFrame>
                    <DiscreteObjectKeyFrame KeyTime="0:0:2">
                        <DiscreteObjectKeyFrame.Value>
                            <Visibility>Collapsed</Visibility>
                        </DiscreteObjectKeyFrame.Value>
                    </DiscreteObjectKeyFrame>
                </ObjectAnimationUsingKeyFrames>
                <DoubleAnimation Storyboard.TargetName="Loading" Storyboard.TargetProperty="(UIElement.Opacity)"
                                 Duration="0:0:2" From="1.0" To="0"/>
            </Storyboard>
        </Grid.Resources>
        <Grid x:Name="LayoutRoot">
        
        </Grid>
        <Button x:Name="BeginButton" Content="Click to Begin" HorizontalAlignment="Center" VerticalAlignment="Center"/>
        <TextBlock FontSize="30" Foreground="White" FontWeight="Bold" x:Name="Loading" HorizontalAlignment="Center" VerticalAlignment="Center">
            <TextBlock.Effect>
                <DropShadowEffect/>
            </TextBlock.Effect>
            <TextBlock.RenderTransform>
                <TransformGroup>
                    <ScaleTransform x:Name="TextScale"/>
                    <TranslateTransform x:Name="TextSlide"/>
                </TransformGroup>                
            </TextBlock.RenderTransform>
        </TextBlock>
    </Grid>
</UserControl>

Notice the layout root is empty, and nested inside another grid. This allows us to layer the button to kick the workflow off and the text box we'll use to show some animations on top of everything.

The fun, of course, is in the code behind:

[Export]
public partial class MainPage : IPartImportsSatisfiedNotification
{
    [Import]
    public ExportFactory<SquareView> ViewFactory { get; set; }

    [Import]
    public RandomNumberService Service { get; set; }

    public MainPage()
    {
        InitializeComponent();            
    }
    
    public void OnImportsSatisfied()
    {
        BeginButton.Click += (o, e) =>
                                 {
                                     var asyncEnum = new AsyncEnumerator("Main Workflow");
                                     asyncEnum.BeginExecute(Workflow(asyncEnum), asyncEnum.EndExecute);
                                 };      
    }

    public IEnumerator Workflow(AsyncEnumerator ae)
    {
        BeginButton.Visibility = Visibility.Collapsed; // get rid of the button 
        
        Loading.Text = "Animating"; 

        SlideText.Completed += (o, e) => ((Storyboard)o).Stop();
        SlideText.Begin();
                    
        var eventArgsFactory = new EventApmFactory<EventArgs>();
        EventHandler storyboardEnded = eventArgsFactory.PrepareOperation(ae.End()).EventHandler;
        ScaleText.Completed += storyboardEnded;
        
        ae.SetOperationTag("Begin ScaleText Storyboard");
        ScaleText.Begin();

        yield return 1;
        ScaleText.Completed -= storyboardEnded; 
        ScaleText.Stop();
        
        eventArgsFactory.EndInvoke(ae.DequeueAsyncResult());
        
        Loading.Text = "Loading Squares";

        int max = 400; 
        
        while (max > 5)
        {
            var routedEventWrapper = new EventWrapper();

            EventHandler eventEnded = eventArgsFactory.PrepareOperation(ae.End()).EventHandler;
            
            ae.SetOperationTag("Adding a square.");

            var square = ViewFactory.CreateExport().Value;
            square.Size = max;

            routedEventWrapper.Begin = () => LayoutRoot.Children.Add(square);
            square.Loaded += (o, e) => routedEventWrapper.End();

            routedEventWrapper.Completed += eventEnded;
            routedEventWrapper.Begin();

            yield return 1;

            routedEventWrapper.Completed -= eventEnded;                
            eventArgsFactory.EndInvoke(ae.DequeueAsyncResult());

            ae.SetOperationTag("Getting a new random number.");

            var eventWrapper = new EventWrapper();
            
            int i = max;                

            eventWrapper.Begin =
                () =>
                    {
                        Service.GetRandomNumber(i,
                                                result =>
                                                    {
                                                        max = result;
                                                        eventWrapper.End();
                                                    });
                    };

            EventHandler randomCompleted = eventArgsFactory.PrepareOperation(ae.End()).EventHandler;
            eventWrapper.Completed += randomCompleted;

            eventWrapper.Begin();

            yield return 1;

            eventWrapper.Completed -= randomCompleted;
            eventArgsFactory.EndInvoke(ae.DequeueAsyncResult());
        }

        Loading.Text = "Goodbye";

        HideText.Begin();

        ae.SetOperationTag("End of main workflow.");
        
        yield break;
    }        
}

So the first thing you'll notice is my use of MEF's ExportFactory. This let's me get the "means to create" an export, rather than a single export value. This allows me to generate as many squares as I like, all using MEF's composition engine to inject the service so the workflow can create the random colors.

We wire the button click when the imports are ready, and that kicks off the workflow.

Notice how easy it is to read the sequential workflow. We hide the button and set the text up. I kick off one animation asynchronously (the expansion) but plug the other into the workflow. This is to show how you can mix the asynchronous and sequential calls: these are two different storyboards, but they run at the same time and only when the coroutine one ends does the text update. Our state engine allows us to have our own loops, so we continue to loop until the random value gets too small to draw a visible square. Notice I actually wait for the square to get loaded into the visual tree before I draw the next square. This means you'll get several squares with the default (gray) color visible while the color workflow fires and then eventually colors the squares when complete.

When the last square is drawn, it changes the text and fades it out and then the workflow ends.

Here is the actual application for you to play with:

There you have it! Feel free to pull down the source and kick the tires. I hope this article has done a decent job of explaining what coroutines are, how you can implement them in C# and what they are practical for. Please share your feedback and comments below as I believe this area is very interesting for discussion.

Download the source for this post Note: this requires Wintellect's PowerThreading library. Download that here, then follow the instructions on the download link to add the reference.