Wintellect Blogs

Pooling Buffers for Better Memory Management

Sergio Loscialo

9 Jun , 2015  

Occasionally, you need a more robust solution to solve a problem.

In my last post, I wrote about the horrors of this small code snippet:

public byte[] Serialize(object o)
{
    using (var stream = new MemoryStream())
    {
        MySerializer.Serialize(stream, o);
        return stream.ToArray();
    }
}

One way to alleviate the memory pressure that can be caused by frequent creation and destruction of large objects is to tell the .Net garbage collector to compact the Large Object Heap (LOH) using:

GCSettings.LargeObjectHeapCompactionMode = GCLargeObjectHeapCompactionMode.CompactOnce;

However, this solution, while it may reduce the memory footprint of your application, does nothing to really solve the initial problem of allocating all that memory in the first place. As I mentioned in my last post, one way to accomplish that goal is to use a buffer pool.

The Buffer Manager

Instead of writing my own buffer manager, I am going to use the implementation Microsoft provides in the Windows Communication Framework (WCF) called the BufferManager. Now that WCF is open source you can go look at the implementation of BufferManager here.

The BufferManager essentially allocates a large chunk of contiguous memory and sets it aside for later use. You can create an instance of a BufferManager by calling the CreateBufferManager() static method as follows:

BufferManager.CreateBufferManager(maxPoolSize, maxBufferSize);

The first parameter, maxPoolSize represents how much memory you want the BufferManager to allocate in total.… Read more

, ,

Wintellect Blogs

Hey, Who Stole All My Memory?

Sergio Loscialo

16 May , 2015  

Sometimes a little piece of seemingly innocuous code can cause a significant amount of trouble:

public byte[] Serialize(object o)
{
    using (var stream = new MemoryStream())
    {
        MySerializer.Serialize(stream, o);
        return stream.ToArray();
    }
}

Doesn’t look like much, but I’m sure we have all written something like this and paid it no mind. In fact, most of the time, code like this isn’t a problem.

This little method becomes a problem when that object o parameter is larger than 85k (actually 84,988 bytes) in contiguous memory. At that point, this object is considered to be a “large object.” That designation is significant since when you’re done with an object like this, the .Net garbage collector stores it on the Large Object Heap (LOH) rather than in the gen 0 heap.

A situation called LOH fragmentation arises when applications are frequently creating and disposing many large objects. A classic example is a file upload service that receives and processes large volumes of data in each request. The resulting fragmentation can lead to an OutOfMemoryException being thrown, even if the server still has plenty of memory and there’s plenty of space left in the process address space.

So, getting back to our little innocuous piece of code: we already know that if o is a large object, it will consume space in the LOH.… Read more

,

Wintellect Blogs

FileNotFoundException When Looking for YourAssembly.resources.dll? Here’s Your Fix!

John Robbins

13 Feb , 2015  

I happened to be working on a C# Roslyn code analyzer and being cognizant that not all the world speaks English, I went through the five minutes of work to internationalize the analyzer DLL. Add the .RESX file and you’re pretty much done. All my unit tests ran perfectly as did the tests of running the analyzer as a .VSIX. However, when I ran the tests with the analyzer in a NuGet package, I kept getting the compiler crashing with a FileNotFoundException trying to load the <assemblyname>.resources.dll file. After spending several hours trying different attempts at working around the problem, it finally dawned on me, I had run into this same bug many, many years ago.

Even though you have the resources in the assembly, that’s not the default place .NET looks. To tell .NET to look in the assembly, you have to specify in your Assembly.CS file the following:

[assembly: NeutralResourcesLanguage(“en-US”)]

That’s all there is to it. Why .NET resource searching doesn’t always look in the assembly by default is beyond me. Now that the Core Framework is open sourced, maybe I should submit a patch!… Read more

,

Wintellect Blogs

Coroutines for Asynchronous Sequential Workflows using Reactive Extensions (Rx)

Jeremy Likness

15 Apr , 2013  

I’ve been doing quite a bit with Reactive Extensions (Rx) for Silverlight lately. One idea that I keep exploring is the concept of creative intuitive sequential workflows for asynchronous operations. You can read about my explorations using Wintellect’s own Power Threading Library in this post along with a simple solution using an interface and enumerators in part 2 of that series. I’m tackling the problem from a different angle.

First, my disclaimers: this is more of an exploratory, “Here’s what can be done” post. Please don’t take this as best practice or guidance, but more of another way of looking at the solution as well as an opportunity to dive deeper into Reactive Extensions. Also understand I am by no means an expert with Rx so I welcome feedback related to this experiment.

The concept is straightforward: there are often times we want an asynchronous set of operations to perform sequentially. Perhaps you must load a list from a service, then load the selected item, then trigger an animation. This can be done either by chaining the completed events or nesting lambda expressions, but is there a cleaner way?

To me, a cleaner solution would make it easy to see what happens sequentially, and also easy to fire the sequence.… Read more

, ,

Wintellect Blogs

Entity Framework: Expressing the Missing LINQ

Jeremy Likness

18 Sep , 2012  

I have worked on quite a few projects that use the Entity Framework. It is a powerful ORM and does quite a lot out of the box. I’ve worked with code-first, database-first, and every other flavor in between. If I were to name the one reason I believe developers enjoy working with LINQ the most, it would be the support for LINQ via LINQ to Entities.

Download Source Code for this post

What’s not to like? You can query data in a very easy, straightforward manner that is consistent. Unfortunately, the Entity Framework can wreak havoc on an otherwise stable web application if it is not handled with care. There are a number of “gotchas” you will run into (updateable materialized views, anyone?) ranging from improper use of the context used to access the database to “features” of LINQ that can become defects in production. In this post I will focus on two very subtle LINQ problems I see people run into quite often.

It Will Be Deferred

The first is probably the easiest to understand and the fastest to catch in testing. Consider a very contrived model called Thing:

public class Thing
{
    public int Id { get; set; }
    public string Value { get; set; }
}

The code-first definition for a database of things is simple enough:

public class SampleContext : DbContext 
{
    public DbSet<ThingThings { get; set; }

    protected override void OnModelCreating(DbModelBuilder modelBuilder)
    {
        modelBuilder.Conventions.Remove();
    }
}

I could use an initializer to then seed the data but I got really lazy and created a controller instead that does this (not recommended for production code but will work perfectly well to illustrate the points in this blog post):

public class ThingsController : ApiController
{
    public ThingsController()
    {
        using (var context = new SampleContext())
        {
            if (context.Things.Any()) return;

            for (var x = 0; x < 1000; x++)
            {
                var thing = new Thing
                                {
                                    Value = Guid.NewGuid().ToString()
                                };
                context.Things.Add(thing);
            }
            context.SaveChanges();
        }

    }
}

As you can see, I’m ensuring we have at least 1000 things to deal with.… Read more

,

Wintellect Blogs

Reflection, Lambda, and Expression Magic

Jeremy Likness

29 Jul , 2011  

Sometimes I love how a little reflection can work magic. In this case I was building what I’ll call a “non-intrusive” validation system. The project contains entities that are generated by templates, and it would be extremely difficult to crack open those templates to put in a lot of custom validation code. So I decided to approach it using a helper class that would attach to the generated entities.

As always, I work backwards. I like to have readable, fluent interfaces, so I decide what I want them to look like and then implement the classes and interfaces to make them work. In this case the entities implemented INotifyPropertyChanged and IDataErrorInfo. This makes it easy on the validation class because if it needs to validate a property, it can simply hook into the property change notification and pick up the new value for validation. Problem solved.

The validation mechanism ended up looking like this:

myClass.GetValidationHelper()
   .WithValidation<FieldIsRequired(()=MyField)
   .WithValidation<FieldIsInRange(()=MyField,0,999) 

So, how do we get there? I’m not going to give you all of the code but I’ll touch on a few salient points. First, creating a common interface for validations makes it easy for the validation helper to parse through a dictionary (i.e.… Read more

Wintellect Blogs

A Fluent Backwards Design

Jeremy Likness

5 May , 2011  

Fluent interfaces make code easier to read, understand, and use. They can also provide a layer of separation and decoupling to make it easier to reuse components in a straightforward fashion. In this post I will walk through the scenario of creating a fluent interface with a “backwards design.” This means I’ll start with the end in mind: what I want my interface to look like, and build the API to support that.

The scenario is a common one that illustrates the point nicely: a feed reader. Regardless of using a fluent interface, you are likely to be following the MVVM model and practice SOLID principles. (If not, please read the linked article for a jumpstart).

The individual feed item can be modeled simply like this:

public class FeedItem
{
    public string Id { get; set; }

    public DateTime Posted { get; set; }

    public string Title { get; set; }

    public string Description { get; set; }
}

Knowing the definition of the feed item now makes it possible to create a view model to host it:

public partial class ViewModel
{
    public ViewModel()
    {
        FeedItems = new ObservableCollection<FeedItem();

        DesignData();

        if (DesignerProperties.IsInDesignTool) return;
    }

    public ObservableCollection<FeedItemFeedItems { get; private set; }
}

The design data can then populate the sample feeds:

public partial class ViewModel
{        
    [Conditional("DEBUG")]
    protected void DesignData()
    {
        if (!DesignerProperties.IsInDesignTool)
        {
            return;
        }

        for (var x = 0; x < 20; x++)
        {
            var feedItem = new FeedItem();
            feedItem.Id = Guid.NewGuid().ToString();
            feedItem.Posted = DateTime.Now.AddDays(-20 + x);
            feedItem.Title = "Blog Post Entry";
            feedItem.Description =
                "This is a sample blog post entry for the purpose of design-time data on the design surface.";
            FeedItems.Add(feedItem);
        }
    }
}

This simply wires up a short list for data-binding.… Read more

,

Wintellect Blogs

Parsing the Visual Tree with LINQ

Jeremy Likness

10 Feb , 2011  

I noticed a project on CodePlex that provides a “JQuery-like syntax” for parsing XAML. It looked interesting, but then I wondered why it would be needed when we have such powerful features available to us in the existing runtime. Silverlight provides a powerful VisualTreeHelper for iterating elements in the visual tree. It also provides a TransformToVisual function to find the offset of elements relative to other elements.

So first things first: create a function that parses all children in the visual tree. You can either parse specific types of children, or all children that derive from a type and then filter or group on those. The following snippet will handle this:

public static IEnumerable<TRecurseChildren<T(DependencyObject root) where T: UIElement
{
    if (root is T)
    {
        yield return root as T;
    }

    if (root != null)
    {
        var count = VisualTreeHelper.GetChildrenCount(root);

        for (var idx = 0; idx < count; idx++)
        {
            foreach (var child in RecurseChildren<T(VisualTreeHelper.GetChild(root, idx)))
            {
                yield return child;
            }                    
        }
    }
}

So why enumerable? Because sometimes we may want to only get the first item in the list. By making this enumerable, the iterator will stop once the query is done, avoiding the work of recursing n-levels deep.… Read more

Wintellect Blogs

Autofocus Nested Textboxes with Rx in Silverlight

Jeremy Likness

8 Sep , 2010  

The Problem

A common UI pattern is to expose text in a read-only container, then swap it for an editable input box based on a command. There are multiple approaches to doing this (including just changing the style of the same container). Swapping between a read-only TextBlock to a TextBox is easy enough, but what if you want to also focus and select the TextBox so the user can simply begin typing? And what if the UI elements are nestled deep with in a data template so there is no straightforward way to reference them?

A Solution

I say, “a solution” because there are probably other ones, but this is how I recently tackled the problem.

Unique Identifier

First, I figured no matter how nested the text box would be, it most likely is data bound to some data element. So, in order to uniquely identify the “transaction” I could expose a unique identifier on the bound object. Assume the bound field is “Name”:

public string NameIdentifier { get; private set; }

...

// constructor
NameIdentifier = Guid.NewGuid.ToString();

The Message

Using the Event Aggregator pattern, I created a message payload specifically for the “message” that the text item should receive focus:

public class TextFocusEvent
{
    public TextFocusEvent(string identifier)
    {
        Identifier = identifier;
    }
    public string Identifier { get; set; }
}

In the “edit command” we can now publish our message that the text box should receive focus (notice I’m using the DelegateCommand from Prism):

EditCommand = new DelegateCommand<object(
    obj ={
            if (!EditCommand.CanExecute(obj)) return;

            _oldName = _name;
            IsInEditMode = true;
            EventAggregator.Publish(new TextFocusEvent(NameIdentifier));
        },
    obj =!IsInEditMode);

In this case, you can infer we have an IsInEditMode flag exposed, which we can use to bind the visibility of the TextBlock and TextBox to swap them out.… Read more

,

Wintellect Blogs

Simplifying Silverlight Web Service Calls with Reactive Extensions (Rx)

Jeremy Likness

19 Aug , 2010  

I’ve been working with the Reactive Extensions (Rx) library quite a bit lately and am very impressed. While it is a new way of thinking about services, it certainly makes life much easier. In this example, I’ll show you a way to simplify your web service calls using Rx. In fact, even if you don’t use Reactive Extensions, you may benefit from the proxy wrappers that I’ll describe.

I’m assuming you are familiar with Silverlight, web services, and have some exposure to the Managed Extensibility Framework. You’ll also want to make sure you’ve got the latest version of Rx for Silverlight 4.

Let’s get started! First, create a new Silverlight 4 Application. Keep all of the defaults: we do want a web project, but we aren’t using RIA.

The Service: Server Side

Let’s create a simple calculator service. Sure, it is a simple example, but it will make it easier to focus on the details of Rx rather than puzzling over a more complex web service example.

Create a new service and call it “Calculator.” Just place it in the root of the web application. Create a contract and implement it, so that your service ends up looking like this:

namespace RxWebServices.Web
{
    [ServiceContract(Namespace = "http://csharperimage/")]
    public interface ICalculator
    {
        [OperationContract]
        long Add(int operand1, int operand2);       
    }

    [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
    public class Calculator : ICalculator
    {
        readonly Random _random = new Random();

        public long Add(int operand1, int operand2)
        {
            Thread.Sleep(TimeSpan.FromMilliseconds(_random.Next(1000) + 50));
            return operand1 + operand2;
        }        
    }
}

Notice I built in a delay.… Read more

,