Silverlight's Big Image Problem (and What You Can Do About It)

29 Comments December 17, 2009


Quick: Can you spot the problem with these three lines of code?

BitmapImage bi = new BitmapImage();

bi.SetSource(stream);

TheImage.Source = bi;

These statements create an image from a stream of PNG or JPG image bits and display the image by assigning it to a XAML Image object named TheImage. It's boilerplate code used to display images read from the local file system or obtained from a service. And while there's nothing inherently wrong with the code itself, you'll want to think carefully before including it in any Silverlight application.

I call it "Silverlight's Big Image Problem." Not the kind of image problem a movie star might suffer, but an inherent memory-consumption problem when dealing with large bitmap images in Silverlight.

The problem manifests itself when you handle large images in large numbers. The My Pictures Viewer that I blogged about yesterday is a case in point. When a user running the application selects a folder containing one or more image files, the viewer displays clickable thumbnail versions of the images. The problem is that because Silverlight's BitmapImage class consumes massive amounts of memory (up to 40 or 50 MB per image for a typical 2 to 3 MB digital photo), you simply can't have too many instances extant at once. But to create a thumbnail, you first need a BitmapImage that wraps the entire image. You might create a thumbnail by assigning the BitmapImage to an Image object that measures just 100 by 100 pixels, but if the original image measures 4,000 by 4,000 pixels, it's the latter figure you pay the price for.

To demonstrate, I wrote a simple test harness that you can easily duplicate yourself. I began with an app that pops up an OpenFileDialog and lets the user select an image file from his or her hard disk. Once the image file is selected, the application generates a thumbnail version of the image and adds it to the scene. Then it generates another thumbnail, and then another, and so on and so forth until Silverlight throws an out-of-memory exception. Here is the helper method that I initially used to generate the thumbnails:

private Image CreateThumbnailImage(Stream stream, int width)

{

    BitmapImage bi = new BitmapImage();

    bi.SetSource(stream);

 

    double cx = width;

    double cy = bi.PixelHeight * (cx / bi.PixelWidth);

           

    Image image = new Image();

    image.Width = cx;

    image.Height = cy;

    image.Source = bi;

 

    return image;

}

And here's what happened when I ran the application and selected a 3,648 x 2,736 JPG with a file size of 2.1 MB: 

Out of Memory 

So get this. The application created 26 thumbnails, each measuring a mere 100 x 75 pixels. But attempting to create a 27th thumbnail produced an out-of-memory exception. When the exception occurred, Task Manager showed that the process's working set size had grown from 30 MB to nearly 1.5 GB! It seems crazy on the surface, because a full-color 100 x 75 image should only require about 30K of memory. But it makes a lot more sense when you realize that underlying each thumbnail is a gigantic BitmapImage that retains the full fidelity of the 3,648 x 2,736 original.

The obvious question is what do you do about it? Is there a way to efficiently create thumbnail images from streams of image bits in Silverlight? It's a question that pops up time and again in discussion forums and on message boards. And the short answer is yes, there is a way. But the answer probably isn't the one you expect.

Developers commonly attempt a solution along these lines:

private Image CreateThumbnailImage(Stream stream, int width)

{

    BitmapImage bi = new BitmapImage();

    bi.SetSource(stream);

 

    double cx = width;

    double cy = bi.PixelHeight * (cx / bi.PixelWidth);

 

    Image image = new Image();

    image.Source = bi;

 

    WriteableBitmap wb = new WriteableBitmap((int)cx, (int)cy);

    ScaleTransform transform = new ScaleTransform();

    transform.ScaleX = cx / bi.PixelWidth;

    transform.ScaleY = cy / bi.PixelHeight;

    wb.Render(image, transform);

    wb.Invalidate();

 

    Image thumbnail = new Image();

    thumbnail.Width = cx;

    thumbnail.Height = cy;

    thumbnail.Source = wb;

    return thumbnail;

} 

The basic idea is that instead of creating a thumbnail by assigning a large BitmapImage to a small Image, you use WriteableBitmap.Render with a ScaleTransform to create a thumbnail, and then assign the WriteableBitmap to an Image. Meanwhile, the BitmapImage and the Image it was temporarily assigned to—the one passed to WriteableBitmap.Render—go out of scope and are eventually picked up by the garbage collector.

It works well in theory, but not so well in practice, thanks to an undocumented behavior of WriteableBitmap. In fact, when I plugged the revised CreateThumbnailImage method into my test harness, the application ran out of memory just as quickly as before.

The problem, it turns out, is that when you call WriteableBitmap.Render, WriteableBitmap apparently retains a reference to the XAML object passed in the first parameter. (I was stumped, too, until Jeffrey Richter and I did a little detective work and discovered what was happening under the hood. Jeffrey's my go-to guy for CLR issues, and I'm not sure I would have ever figured this out without him asking the right questions and suggesting solutions.) When CreateThumbnailImage returns an Image holding a reference to a WriteableBitmap, and the WriteableBitmap holds a reference to an Image, and the Image holds a reference to a BitmapImage, none of these objects gets garbage-collected. It seems that WriteableBitmap does nothing to solve the problem, especially given that there's no public method or property you can use to force the WriteableBitmap to release the reference.

But all is not lost. You can make a copy of the WriteableBitmap and assign it to the Image you return. And since you didn't call Render on the copy, it doesn't hold a reference to an Image that prevents the garbage collector from cleaning up the BitmapImage. Here is the fixed and final version of CreateThumbnailImage—this time, one that accomplishes what we set out to do:

private Image CreateThumbnailImage(Stream stream, int width)

{

    BitmapImage bi = new BitmapImage();

    bi.SetSource(stream);

 

    double cx = width;

    double cy = bi.PixelHeight * (cx / bi.PixelWidth);

 

    Image image = new Image();

    image.Source = bi;

 

    WriteableBitmap wb1 = new WriteableBitmap((int)cx, (int)cy);

    ScaleTransform transform = new ScaleTransform();

    transform.ScaleX = cx / bi.PixelWidth;

    transform.ScaleY = cy / bi.PixelHeight;

    wb1.Render(image, transform);

    wb1.Invalidate();

 

    WriteableBitmap wb2 = new WriteableBitmap((int)cx, (int)cy);

    for (int i = 0; i < wb2.Pixels.Length; i++)

        wb2.Pixels[i] = wb1.Pixels[i];

    wb2.Invalidate();

 

    Image thumbnail = new Image();

    thumbnail.Width = cx;

    thumbnail.Height = cy;

    thumbnail.Source = wb2;

    return thumbnail;

} 

When I plugged this implementation into my test harness, it successfully created hundreds of thumbnails (and could have created hundreds, perhaps thousands, more) without significantly increasing the working set size—and without throwing out-of-memory exceptions.

The moral is that you should be very careful about how you use BitmapImage in Silverlight. Even one of them can swell the working set size dramatically, but a couple dozen of them wrapping digital photographs is more than most PCs can handle. With a little care, however, you can scale down the impact of BitmapImage so that memory consumption is proportional to the sizes of the images you're displaying rather than the sizes of the original, unreduced images. And that, in the end, is a handy arrow to have in your arsenal.


29 Comments

  • Gravatar Image
    Raghuraman December 17, 2009 4:00 PM

    Nice.. Thx for writing about this.

    Btw.. Can you please adjust the font of the blogs to be a little bigger by default. It is tough to read.

  • Gravatar Image
    Bertrand Le Roy December 17, 2009 5:24 PM

    Hi,

    I've made some measurements on my own resizing benchmark code, and although the pressure from a single image seem consistent with what you're seeing (40-50MB for a 12 megapixel image), I don't see any memory leak like you are.
    I'm suspecting you're seeing this because you are returning the resized image from your method, and that image still has a reference to the original image (enabling scenarios where you change the transforms I suppose). That cloning the pixels fixes the memory issue seems to confirm it. Another thing that might be worth trying and maybe not as heavy-handed would be to freeze the image before returning it? You might also want to try creating a frame from the image BitmapFrame.Create(image)?

  • Gravatar Image
    uberVU - social comments December 17, 2009 7:56 PM

    This post was mentioned on Twitter by jprosise: Blogged about inefficiencies handling large images in Silverlight and solutions for the same: http://tinyurl.com/ye2ao5q

  • Gravatar Image
    Morten December 17, 2009 11:36 PM

    In WPF there's a much more efficient way of doing this by using the DecodePixelWidth/Height parameter. See here:
    http://weblogs.asp.net/bleroy/archive/2009/12/10/resizing-images-from-the-server-using-wpf-wic-instead-of-gdi.aspx

    Would be nice to have this in Silverlight as well.

  • Gravatar Image
    Nikolay December 18, 2009 4:39 AM

    Hi Jeff! Thanks a lot for this great post. It completely solved my issue.
    Before this I used the following workaround - we can replace BitmapImage with WritableBitmap in the code below:

    private Image CreateThumbnailImage(Stream stream, int width)
    {
    WritableBitmap bi = new WriteableBitmap(0, 0); //BitmapImage bi = new BitmapImage();
    bi.SetSource(stream);
    double cx = width;
    double cy = bi.PixelHeight * (cx / bi.PixelWidth);
    Image image = new Image();
    image.Source = bi;
    WriteableBitmap wb = new WriteableBitmap((int)cx, (int)cy);
    ScaleTransform transform = new ScaleTransform();
    transform.ScaleX = cx / bi.PixelWidth;
    transform.ScaleY = cy / bi.PixelHeight;
    wb.Render(image, transform);
    wb.Invalidate();
    Image thumbnail = new Image();
    thumbnail.Width = cx;
    thumbnail.Height = cy;
    thumbnail.Source = wb;
    return thumbnail;
    }

    In this case garbage collector works fine and there are no memory leaks. The only disadvantage is that WriteableBitmap renders loaded
    image not very smooth (with jagged lines, no antialiasing). And I couldn't find a way to enable it. (BitmapImage does it by default).
    And it seems there are no "RenderOptions.SetEdgeMode(i, EdgeMode.Aliased);" method available as in WPF.

    Could you please advise if render options can be set in Silverlight 3 so we could render WriteableBitmap smoothly?



  • Gravatar Image
    Gabriel December 18, 2009 8:25 AM

    Weird ...

    is that a bug or a feature?????

  • Gravatar Image
    jprosise December 18, 2009 8:34 AM

    Bertrand: Yes, I am returning the resized image from my helper method, which prevents any references held by that image from being garbage collected.

    Silverlight doesn't support freezables as WPF does, and Silverlight doesn't have a BitmapFrame class. As long as WriteableBitmap insists on holding references to objects passed to Render, the only way around the problem that I can find is to copy the pixels into a second WriteableBitmap.

    I can't think of any reason WriteableBitmap should retain that reference, but I've asked the team just to be sure. Maybe I'm missing something obvious. Wouldn't be the first time. :-)

  • Gravatar Image
    MickD December 22, 2009 8:37 PM

    Bookmarked! Thanks for the info

  • Gravatar Image
    hmaprk February 11, 2010 3:55 PM

    Great post. I have run into this issue several times. Thanks for the code! I have linked to this post from my blog.

  • Gravatar Image
    Alex March 10, 2010 3:57 AM

    Great post, this will be a huge time saver :)

  • Gravatar Image
    Brent Huot April 13, 2010 3:21 PM

    Geez... smart guy how on earth did you figure this out.

  • Gravatar Image
    Bastian Kr&#246;ger May 11, 2010 1:14 PM

    Hi Jeff,

    fist thanks for this great example!

    It really made me crazy ;)

    In addition I recommend using GC.Collect(); after the function call!
    You can load a lot of pictures (tried with some 16mb jpegs @ 4GB of ram) then
    without running into OutOfMemoyException.

    If you do not call GC.Collect(); it will be good or bad luck
    whether you have enough ram to load or not ;(
    because maybe some previously loaded pictures did consume it (+ GC did not run before).

    Kind Regards,
    Bastian Kröger

  • Gravatar Image
    sLedgem July 7, 2010 4:47 AM

    Hey

    Thanks for the article, it definitely made my task easier (note how I used the word "easier" and not "solved")

    I'm using the SL drag'nDrop to allow the user to drag images into my app, which I then resize with your code and display. Fine, works like a charm. The problem is that even with the copy operation, the memory is STILL not disposed of till AFTER the whole drag'drop event handler has finished. Which would normally not be a problem, but it does not take into account the possiblity of selecting more than one image and dragging it into the app. it seems that the event handler is keeping the memory alive until it is disposed of. have you got any ideas what I might be doing wrong?

    here is a skeleton of my code:

    void Page_Drop(object sender, DragEventArgs e)
    {

    FileInfo[] files = FileInfo[])e.Data.GetData(System.Windows.DataFormats.FileDrop);
    Canvas Owner = (Canvas)sender;


    foreach (FileInfo file in files)
    {
    CreateImage( file);
    GC.Collect();
    }


    }

    private void CreateImage(FileInfo)
    {
    using (Stream reader = file.OpenRead())
    {


    WriteableBitmap bitmap = Jpeg.GetImageSource(reader, 800, 600);


    }
    }

    I obviously removed alot of code, and I tried reading the file into an array before sending it over to the method and then creating a new memorystream in there, but no Luck, memory Usage just grows and grows...
    (and hitting my head against the keyboard does not solve i, I tried)

  • Gravatar Image
    Sepp August 17, 2010 11:23 AM

    Great post, thanks!

    Is it possible to run this function in a backgroundworker so that the gui doesn't freeze? I tried it, but I get an UnauthorizedAccessException, because BitmapImages can only be used in the gui thread. Is there a solution for this problem?

  • Gravatar Image
    Horst August 17, 2010 11:26 AM

    Great Post!

    Is it possible to run your function in a backgroundworker? I tried it but BitmapImages can only be used in the gui thread. Is there a solution for this problem?

  • Gravatar Image
    Silverlight Performance Blog August 19, 2010 6:04 AM

    A View from the Top So you&rsquo;ve decided that it&rsquo;s time to optimize the memory footprint of

  • Gravatar Image
    Rohti August 31, 2010 6:48 PM

    We are developing an application where every image bytes related to a Document are converted into a Bit Map Image and then converted to a Silver Light Image and added to a Stack Panel (List of all the Images in a "View All" mode
    We have a check box, when checked will display a thumbnail view on the left of the existing stack panel. Un checking it will revert it back to the view all mode.
    If an end user performs this operation multiple times, he/she received a out of memory exception, because each time the user toggles between the views the memory keeps on adding.
    We tried to implement your code, but it doesnt seem to work. We still see the memory raising during toggling.
    Any ideas how to resolve this?

  • Gravatar Image
    jayendra October 23, 2010 2:34 AM

    pls solve mu problem.i have a problem for saving images in the my folder of computer from rumming video.how to save the thumbnail or images ?
    my email id is mistryjayendra96@yahoo.com.my project is in .net 2010 using silverlight 4.0

    i have code below

    writable bitmap.xaml.cs file


    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Net;
    using System.Windows;
    using System.Windows.Controls;
    using System.Windows.Documents;
    using System.Windows.Input;
    using System.Windows.Media;
    using System.Windows.Media.Animation;
    using System.Windows.Media.Imaging;
    using System.Windows.Shapes;
    using System.Windows.Threading;


    namespace DeepZoomSample
    {
    public partial class WritableBitMap1 : UserControl
    {


    public WritableBitMap1()
    {
    InitializeComponent();



    }

    private void me_MouseLeftButtonDown(object sender, MouseButtonEventArgs e)
    {

    // Create a WriteableBitmap and set it to the MediaElement (video).
    // The WriteableBitmap represents a "snapshot" of the video.
    WriteableBitmap wbm = new WriteableBitmap(myMediaElement, null);

    // Create an image of the desired size and set its source to
    // the WriteableBitmap representing a snapshot of the video.
    Image image = new Image();
    image.Height = 64;
    image.Margin = new Thickness(10);
    image.Source = wbm;
    }

    pls solve my problem & reply me on my emil id
    mistryjayendra96@yahoo.com



  • Gravatar Image
    kalle October 26, 2010 6:29 AM

    Great Work! ... but ...

    ... why the hell is it not possible to map a stream directly to a Silverlight Image ?

    Especially in the case you need effects/shaders for the image (which ca be set with the Image.Effect parameter) it is totally strange to need a WriteableBitmap or BitmapImage in between.

    Any hints how to map a MemoryStream directly to an Image without any kind of BitmapXXX in between would be great !!!

  • Gravatar Image
    Mike March 14, 2011 8:04 PM

    Hi,

    I've just tried your code with Silverlight 4 on VS2010 and hit a nasty memory leak. It looks like memory is being allocated when the original bitmap is loaded in bi but it is never garbage collected. After 30-40 decent sized photos memory usage has gone up to 1.5GB and I get an out of memory error. This is repeatable unfortunately, I've tried setting image.source to null, bi to null etc but no joy. Got round it for the mo by using the Component One image C1Bitmap control but this seems to be a lot slower :(

  • Gravatar Image
    Mike March 16, 2011 7:53 AM

    Hi,

    Following my previous post, I've fixed the "memory leak" and described how I did it here:

    http://forums.silverlight.net/forums/p/222517/534854.aspx#534854

    I've used your thumbnailing code and it works great, the only minor change I made was instead of scaling it on the width I used a best fit approach e.g. I calculated the scale for fit to width / fit to height and then used the smallest of the two on the basis that it would fit more of the image into the thumbnail.

    Cheers,

    Mike

  • Gravatar Image
    bob April 29, 2011 6:46 PM

    so when I have to load it via a URL from a different domain, now what?

    I can't get it as a stream due to onerous security in SL.

  • Gravatar Image
    Bob Leonard July 22, 2011 9:33 AM

    Sweet thank you for sharing. Just what I needed.

  • Gravatar Image
    Martin Par&#233; December 1, 2011 10:07 AM

    Hi Jeff,

    3 days working on this issue. Thank you very much. Your article helped me resolve my problem.

    Thank you so much for your very good work !

  • Gravatar Image
    Musya December 27, 2011 5:55 AM

    It's a grate post, thanks a lot. I had related problem and you helped me to figure it out!!

  • Gravatar Image
    Robson February 14, 2012 7:32 AM

    Is there an equivalent solution for WPF? I'm using Framework 4.0 and I can't see Invalidade method and Pixels property.

  • Gravatar Image
    mcrally March 1, 2012 11:48 AM

    It's work very well!!! Thanx
    But i've another problem. I can't thumbnailize a big Image.
    My Image is 10000x10000 JPG format.
    Silverlight raise OutOfMemoryException on BitmapImage.SetSource

    Any Workaround?

  • Gravatar Image
    JR September 4, 2012 3:06 AM

    Thanks for the above post.
    Could you please give some idea, How to store this thumbnail image into file server??

  • Gravatar Image
    Ravi December 17, 2013 5:51 PM

    Thanks for your Article. However when we used this with large images our image gets flipped -90. What would be the reason? Please help

Have a Comment?

Archives

Tags

Blogs