C# Corner

Memory Mapped Files

Patrick Steele shows how you can realize major performance gains when working with large images by using memory-mapped files.

Memory-Mapped Files have been a part of the Win32 API since its inception. Until now, memory-mapped files have required either C++ or a whole bunch of PInvoke code. Not anymore! With .NET 4.0, we get a fully-implemented managed wrapper around memory-mapped files.

So what exactly are memory-mapped files? Memory-mapped files allow you to map a section of your processes memory directly to a file (or part of a file) on disk. Once the mapping is created, operations on the memory are reflected in the file. There's no need to open a file handle, read data in, worry about buffering -- it's all handled by Windows.

Memory-mapped files are also a great IPC (Inter Process Communication) transport for sharing data among applications. Using memory-mapped files for IPC will be covered in a future article.

So why use memory-mapped files? The biggest reason is speed. When you need to process a large file, reading it all into memory is too expensive on resources. You can use Seek operations, but those can be lengthy too. With memory-mapping, you just tell Windows that you want to map a specific area of a file into memory and you work on that memory block.

Processing a BMP File
Here's a quick example showing two different ways to manipulate a Windows BMP image. I picked the Windows Bitmap format since it is uncompressed and traditionally larger, so working with them in memory is taxing on your app's memory footprint. All we're going to do is change every other row in the last 100 rows to be white lines. It's simple and not too exciting, but shows the power of memory-mapped files.

First off, we'll do this using the standard System.Drawing.Bitmap class:

public void WhiteOutRows(Bitmap image)
{
int rowStart = image.Size.Height - 100;
for (var i = 0; i < 100; i += 2)
{
for (int x = 0; x < image.Size.Width; x++)
{
image.SetPixel(x, rowStart + i, Color.White);
}
}
}

Note that the entire Bitmap image was loaded into memory before this call. And this only changes the Bitmap in memory -- the caller will still need to save the image to disk.

Now let's use a memory-mapped file to do the same thing. We'll show this code step-by-step. The full code is available for download at the end of this article.

Manipulation of a MemoryMappedFile is done through "views". A "view" allows you to define what portion of the file you want mapped to memory as well as the type of access you need (Read, Write, Read/Write, etc... ). What we're going to do is read in the BMP header information (using regular file I/O) to determine things like bits-per-pixel, image height and image width. This header data allows us to calculate the position in the file where we want to "white out" the rows.

var headers = ReadHeaders(bmpFilename);
int rowSize = headers.Item2.RowSize; // number of byes in a row of the image

The "ReadHeader" method returns a Tuple<BmpHeader, DibHeader>. Download the source code at the end of this article to see its implementation. The BmpHeader and DibHeader give us the information we need to calculate our memory offsets.

Next, let's create a "white row". This is simply the raw RGB color data that defines a row of white pixels. Since we know the rowSize and we know that an RGB of 255,255,255 is white, we just need a big byte array filled with 255:

var whiteRow = (from b in Enumerable.Range(0, rowSize) select (byte)255).ToArray();

Finally, let's loop through the rows (like we did in the Bitmap example above), but this time we'll create a view to the specific row of data we want to change and then we'll change it:

for (var row = 0; row < 100; row += 2)
{
using (var view = mmf.CreateViewAccessor(headers.Item1.DataOffset + rowSize * row,
rowSize, MemoryMappedFileAccess.Write))
{
view.WriteArray(0, whiteRow, 0, whiteRow.Length);
}
}

The CreateViewAccessor method has a number of overloads. By default (no parameters) it creates a read/write view of the entire file. For this sample, we're using the overload that allows us to define the offset within the file, the views "size" (how much of the file to map) and what kind of access we need (in this case, we need to write data).

The ViewAccessor implements IDisposable so we put it in a using block. The headers.Item1.DataOffset represents the offset inside the file where image data starts. And since BMP's are stored from the bottom up, we start changing data at beginning of the image data. No need to write our changes to disk -- the changes happen automatically as we manipulated our view.

Performance
The memory-mapped file method seems a bit more involved. Granted, it was a little bit more code to write, but the performance gains are immense! On my machine, I was getting results showing the memory-mapped file method to be 32-38 times faster at changing the image than the Bitmap method!

Check it out for yourself by downloading the sample code from this article. Please note that you'll need to download the very large BMP (22MB compressed) file from my Windows Live Gallery site. Use this link to grab the image. Follow the directions in the code sample to line things up. The sample uses the Stopwatch class to calculate the timings so you can see just how fast memory-mapped files can be.

About the Author

Patrick Steele is a senior .NET developer with Billhighway in Troy, Mich. A recognized expert on the Microsoft .NET Framework, he’s a former Microsoft MVP award winner and a presenter at conferences and user group meetings.

comments powered by Disqus

Featured

  • How to Do Machine Learning Evolutionary Optimization Using C#

    Resident data scientist Dr. James McCaffrey of Microsoft Research turns his attention to evolutionary optimization, using a full code download, screenshots and graphics to explain this machine learning technique used to train many types of models by modeling the biological processes of natural selection, evolution, and mutation.

  • Old Stone Wall Graphic

    Visual Studio Code Boosts Java Dependency Viewer

    Easier management of project code dependencies and improvements to extensions for popular Java frameworks and runtimes highlight the February update to Java in Visual Studio Code functionality.

  • Blule Squares

    Visual Studio 2019 for Mac 8.5 Preview Adds ASP.NET Core Authentication

    Microsoft, after shipping Visual Studio 2019 for Mac v8.4 with support for ASP.NET Core Blazor Server applications last month, is now previewing the v8.5 series, adding new authentication templates for ASP.NET Core along with other improvements.

  • Q&A with Brice Wilson: What's New in Angular 9

    We caught up with expert web developer/trainer Brice Wilson to get his take on Angular, which always appears at or near the top of periodic rankings of the most popular JavaScript-based web development frameworks.

  • Entity Framework Core Migrations

    Eric Vogel uses code samples and screenshots to demonstrate how to use Entity Framework Core migrations in a .NET Core application through the command line and in code.

.NET Insight

Sign up for our newsletter.

Terms and Privacy Policy consent

I agree to this site's Privacy Policy.

Upcoming Events