In-Depth
Deep Dive: Task-Based Asynchronous Pattern
This article introduces the TAP and the associated .NET language changes that streamline asynchronous programming and extend the multithreading enhancements in the Microsoft .NET Framework 4.
With each new version of the Microsoft .NET Framework and its associated languages, Microsoft has introduced key new features and capabilities to C# and Visual Basic. In the next versions (vNext) of .NET and C#/Visual Basic, Microsoft selected asynchronous programming as the area of focus. In this article, I delve into the details of the new API and the associated language changes that will accompany it.
Notice that the focus for vNext is on asynchrony and not multithreading. The .NET Framework 4 included a significant series of enhanced multithreading APIs: the Task Parallel Library (TPL) and Parallel LINQ (PLINQ). Although closely associated, multithreading and asynchrony are not synonymous. Asynchrony occurs when transitions of control- flow between the caller and the called become independent; execution of code in the caller is no longer synchronized with when the called code begins or completes. Multithreading enables asynchrony because it allows execution of code within the caller and the called to occur simultaneously and thereby independently. However, there are a myriad of other ways to achieve asynchrony. In fact, these alternatives can often be significantly more efficient than creating a new thread.
To understand and appreciate the distinction between asynchronous programming without multiple threads and asynchronous programming that depends on additional threads, consider two examples. First, consider the case of programming a rich client application such as one using Windows Presentation Foundation (WPF). Imagine that such a program needs to execute another process and capture the output to display it in the UI. To achieve this, you could start the program on the UI thread and then block using one of the synchronous APIs like Process.WaitForExit. The obvious problem with this approach is that the UI thread will be blocked waiting for the launch process to exit, likely generating an unacceptable delay in program responsiveness for the user.
An alternative approach would be to start the process using Process.Start -- an asynchronous call -- after registering for a callback on Process.Exited.
The caveat with this approach is the need to always use the dispatcher to make a context switch back to the UI thread in order to update the display, because the Process.Exited event is unlikely to be executed on the UI thread. (You should never call into the UI from a thread other than the UI thread.) Similarly, any updates to the status during the execution will also need to make the synchronization context switch.
The second scenario takes place on the server, where a request comes in from a client to query an entire table's worth of data from the database. Because querying the data could be time-consuming, a new thread should be created rather than consuming one from the limited number allocated to the thread pool. The problem with this approach is that the work to query from the database is executing entirely on another machine. There's no reason to block an entire thread, because the thread is generally not active anyway.
By understanding these two scenarios, it's clear why multithreading alone was not sufficient to address asynchrony. Although multithreading increases the frequency with which asynchrony occurs, the release of the TPL and PLINQ was not accompanied with any language changes that improved the asynchronous programming experience (multithreaded or otherwise). This means that the cumbersome approach involving callbacks and the potential synchronization complexities that result were not addressed at all. Fortunately, as mentioned earlier, asynchrony is the focus of the next version of the .NET Framework, with the introduction of the Task-based Asynchronous Pattern (TAP). Furthermore, the TAP API will come with important language enhancements to both C# and Visual Basic.
The TAP was created to address these key problems:
- There's a need to allow long-running activities to occur without blocking the UI thread.
- Creating a new thread (or Task) for non-CPU-intensive work is relatively expensive when you consider that all the thread is doing is waiting for the activity to complete.
- When the activity completes (using either a new thread or via a callback) it's frequently necessary to make a synchronization context switch back to the original caller that initiated the activity.
- Provide a new pattern that works for both CPU- and non-CPU- intensive asynchronous invocations -- one that the .NET Framework-based languages support explicitly.
It's important to notice that although the .NET Framework-based languages have been updated to explicitly support the TAP, its key features are not about the language updates themselves, but rather the ability to implicitly make the necessary thread context switches and to capitalize on activities that are already asynchronous, so as to avoid creating a new thread unless there's CPU-intensive work to perform.
By only focusing on the language features themselves, it's difficult to comprehend what the TAP has to offer beyond what was already available via the TPL and PLINQ. However, focusing on the ability to more easily write asynchronous code for non-CPU-intensive asynchronous method invocations -- while still supporting the same pattern for CPU-intensive methods -- is what makes the overall functionality so compelling.
Introducing a TAP Invocation
With this understanding, let's take a look at the details of the TAP. In this article, I'm going to use the Async CTP (SP1 Refresh), which Microsoft made available following the release of Visual Studio 2010 SP1.
Consider a UI event for a button click in WPF as follows:
private void PingButton_Click(object sender, RoutedEventArgs e)
{
StatusLabel.Content = "Pinging...";
Ping ping = new Ping();
PingReply pingReply = ping.Send("www.IntelliTect.com");
StatusLabel.Content = pingReply.Status.ToString();
}
Given that StatusLabel is a WPF System.Windows.Label control and I've updated the Content property twice within the PingButton_Click event subscriber, it would be a reasonable assumption that first "Pinging..." would be displayed until Ping.Send returned, and then the label would be updated with the status of the Send reply.
As those experienced with WPF well know, this is not, in fact, what happens. Rather, a message is posted to the Windows message pump to update the Content with "Pinging..." but, because the UI thread is busy executing the PingButton_Click method, the Windows message pump is not processed. By the time the UI thread frees up to look at the Windows message pump, a second Content property update request has been queued and the only message that the user is able to observe is the final status. To fix this problem using the TAP -- thus without having to worry about synchronization context switching following an asynchronous call to Ping.Send -- I change the code to this:
async private void PingButton_Click(object sender, RoutedEventArgs e)
{
StatusLabel.Content = "Pinging...";
Ping ping = new Ping();
PingReply pingReply = await ping.SendTaskAsync("www.IntelliTect.com");
StatusLabel.Content = pingReply.Status.ToString();
}
In the Async CTP, the SendTaskAsync method is an extension method on Ping (likely to be implemented elsewhere by the time functionality is released) and declared as follows:
Task<PingReply> SendTaskAsync(this Ping ping, string hostNameOrAddress);
This provides an asynchronous method for pinging an address and returning a Task<PingReply>. To access the method and activate the Async CTP (SP1 Refresh) C# 5 compiler, it's necessary to add a reference to the AsyncCtpLibrary assembly. As the highlighted changes demonstrate, the code modifications are relatively minor, yet from a functional perspective there are several features that the code utilizes.
To start, the first line of the method executes on the same thread as the caller. Then, the call to ping.SendTaskAsync is asynchronous and doesn't necessarily involve an additional thread. (Whether it does is entirely dependent on the implementation, but, regardless, it will be asynchronous.) The key is that the asynchronous nature of the call frees up the caller thread to return to the caller's synchronization context and process the update to StatusLabel.Content so that "Pinging..." appears to the user. Third, when ping.SendTask Async returns, it will always execute on the same thread as the caller. (Strictly speaking, the return will always execute in the same synchronization context as the caller. However, given that this is WPF code and the synchronization context is single-threaded, the return will always be to the same thread.)
This is to achieve a key feature of the TAP -- the implicit synchronization context switch back to the calling synchronization context. To achieve this, the TAP switches back to the calling synchronization context following the asynchronous call completion. The UI (calling) thread monitors the message pump (or more generally the synchronization context), and upon picking up the message invokes the code following the await call. This ensures that it's on the same thread as the caller that processed the message pump.
In addition to the functionality of the TAP, there's also a new language syntax with two new contextual keywords -- async and await. The await keyword is what designates that the method could potentially run asynchronously -- specifically that the method invoked with await may complete later than when control-flow returns to the caller. Furthermore, notice that in the signature of SendTaskAsync it returns a Task<PingReply>. However, the variable declaration of pingReply is simply a PingReply. The await keyword automatically takes care of "unwrapping" the Task and returning its result. A similar type of "unwrapping" occurs from an async decorated method that returns one of the Task types (such as Task<T>). See the code here:
async private void PingButton_Click(object sender, RoutedEventArgs e)
{
StatusLabel.Content = "Pinging...";
IPStatus status = await GetPingStatusAsync();
StatusLabel.Content = status.ToString();
}
async private static Task<IPStatus> GetPingStatusAsync()
{
Ping ping = new Ping();
PingReply pingReply = await ping.SendTaskAsync("www.IntelliTect.com");
return pingReply.Status;
}
In the return from GetPingStatusAsync, the signature indicates the return data type is Task<IPStatus>. However, in spite of the declaration, the return statement on the method is on pingReply.Status where Status is of type IPStatus. In summary, decorating a method with the async contextual keyword is what enables the use of an await call within the method implementation, and enables the implicit accessing (unwrapping) of a Task result from the return of the method that the await call invokes.
There's a key code-readability feature built in to the TAP language pattern. Notice in the previous code sample that the call to return pingReply.Status appears to flow naturally after the await, providing a clear indication that it will execute immediately following the previous line.
However, writing what really happens from scratch would be far less understandable for multiple reasons. First, it would be necessary to pass a callback delegate into SendTaskAsync to identify the continuation code or, at a minimum, perhaps continue with via Task.ContinueWith. Second, neither ContinueWith nor a callback would implicitly take care of the synchronization context switch back to the calling thread and instead require it to be written explicitly. Finally, the TAP implicitly uses the current synchronization context (the windows message pump for UI-based applications such as WPF, Windows Forms and Silverlight) to communicate back to the calling thread when the async method completes. This reduces the likelihood of synchronization problems related to deadlocks or race conditions.
There's no limitation to the number of awaits that can be placed into a single method. And, in fact, they're not limited to appearing one after another. Rather, awaits can be placed into loops and processed consecutively one after the other. Consider the example in Listing 1.
Note that regardless of whether the awaits occur within an iteration or as separate entries, they'll execute serially, one after the other and in the same order they were invoked from the calling thread. The underlying implementation is to queue await calls together in the semantic equivalent of Task.ContinueWith, except the code between the awaits will all execute in the caller's synchronization context.
Just as you can wrap awaits into an iteration like the foreach loop highlighted in Listing 1, you can also place them in a try-catch block. Once again, this provides a significant improvement over the error handling in alternate threads. Previously, when an error occurred in a different thread, you needed to capture it out of the worker thread and foist the exception into the caller thread, so it could be displayed to the user, for example. With the TAP, all you need to do is wrap the code (the entire method or just suspicious snippets with embedded await invocation) and then handle any exception that might have occurred in the try-catch block. The highlighting shows the relevant code in Listing 2.
The TAP Syntax Details
The compiler uses the async keyword as a means of identifying that the method returns a supported data type -- always void, Task or Task<T>. Furthermore, although it's possible to decorate a method with async and not use the await modifier within the implementation, doing so will produce a compiler warning indicating that the async keyword isn't achieving anything because the method will run entirely synchronously. The alternative -- using the await keyword in a method not designated as async -- will result in a compile error.
Not surprisingly, async is not only supported on a method, but also on other anonymous function types such as the statement lambda shown in the following code (although expression lambdas are not supported in the Async CTP SP1 Refresh, all anonymous functions will be supported in future releases):
async private void PingButton_Click(object sender, RoutedEventArgs e)
{
Func<Task<IPStatus>> func = async () =>
{
Ping ping = new Ping();
PingReply pingReply =
await ping.SendTaskAsync("www.IntelliTect.com");
return pingReply.Status;
};
StatusLabel.Content = "Pinging...";
IPStatus status = await func();
StatusLabel.Content = status.ToString();
}
Having an async method that returns void has some interesting behavior. When calling such a method, there's no reason for the TAP to synchronize back with the calling synchronization context upon completion, because there's no reason the caller thread needs to know when it finishes, given there are no results.
Invoking CPU-Intensive Activities
GetPingStatusAsync easily supports async because the SendTaskAsync method does. Furthermore, SendTaskAsync isn't CPU-intensive, so it doesn't even create an additional thread in its implementation of asynchronous support. However, if you wished to write an asynchronous method for a CPU-intensive activity, it's relatively easy to do so using the System.Threading.Tasks.TaskEx.Run method (this method will move to System.Threading.Tasks.Task by the time it releases to production). For example, consider you have a synchronous method -- CalculatePi(int numberOfDigits) -- for calculating the value of Pi out to numberOfDigits digits. Presumably this would be a CPU-intensive activity, so the easiest way to invoke it asynchronously would be with TaskEx.Run, as shown here:
async public Task<string> CalculatePiAsync(int digits)
{
return await TaskEx.Run<string>(() => CalculatePi(digits));
}
private string CalculatePi(int digits)
{
...
}
Although TaskEx.Run could be used for non-CPU-intensive method invocations as well, this will result in creating a Task -- a far more expensive operation than the alternative that the TAP enables. The use of an extra miscellaneous Task in a rich client application based on technologies such as Silverlight and WPF is innocuous for the most part. The same cannot be said for a server scenario where the number of requests could be an order of magnitude greater and additional Tasks (especially those coming from the thread pool, as they do with TaskEx.Run) will have a significant impact. Therefore, avoid using TaskEx.Run whenever there are alternatives available.
Implementing a Custom Asynchronous Method
Implementing an asynchronous method by relying on other asynchronous methods (which in turn rely on more asynchronous methods) is relatively easy with the await keyword. However, at some point in the call hierarchy it becomes necessary to write a "leaf" asynchronous Task-returning method. Consider, for example, an asynchronous method for running a command-line program with the eventual goal that the output could be accessed. Such a method would be declared as follows:
static public Task<Process> RunProcessAsync(string filename)
The simplest implementation would, of course, be to rely on TaskEx.Run again and calling both the Start and WaitForExit methods of System.Diagnostics.Process. However, creating an additional thread in the current process is unnecessary when the invoked process itself will have its own collection of one or more threads. To implement the RunProcessAsync method and return to the caller's synchronization context when the invoked process completes, we can rely on a TaskCompletionSource<T> object, as shown in Listing 3.
Ignore the highlighting for the moment and instead focus on the pattern of using an event for notification when the process completes.
Because System.Diagnostics.Process includes a notification upon exit, we register for this notification and use it as a callback from which we can invoke TaskCompletionSource.SetResult. The code in Listing 3 provides a fairly common pattern with which to create an asynchronous method without having to resort to TaskEx.Run.
Another important characteristic that an async method might require is cancellation, and the TAP relies on the same methods as the TPL -- namely System.Threading.CancellationToken. Listing 3 highlights the code necessary to support cancellation. In this example I allow for canceling before the process ever starts, as well as an attempt to close the application main window (if there is one). A more aggressive approach would be to call Process.Kill, but this could cause problems for the program that's executing.
Notice that I don't register for the cancellation event until after the process is started. This avoids any race conditions that might occur if cancellation is triggered before the process is actually started.
One last feature to consider supporting is a progress update. Listing 4 shows the full RunProcessAsync with just such an update.
Advantages
It seems like every release of the .NET Framework includes one additional asynchronous pattern. The TAP is no exception, as it introduces brand new capabilities involving the synchronization context and, therefore, the ability to schedule work back onto a thread implicitly when other asynchronous activities complete. As outlined in this article, the TAP pattern gives guidance not only from the consumer perspective, but also for the producer of an async method. Although examples have centered mainly on client UI processing-type scenarios, the use of such patterns and avoiding multithreading when it isn't required are potentially even more significant in server scenarios.
Consider one particularly compelling example involving database queries. In the past, all database queries on the server have executed by synchronously blocking one of the session threads used to process requests coming in -- again, very limiting and expensive -- or by requesting a thread from the thread pool. By utilizing the TAP on database calls, however, the same thread can be used over and over again even though the work scheduled for the thread hasn't reached completion.
In other words, rather than waiting for a thread's work to complete before returning a thread to the thread pool, the TAP allows the thread to be used to process other requests against the synchronization context every single time (and for the duration) an async method is invoked. This provides a significant increase in potential throughput for each individual thread.
However, the advantages don't stop with performance alone. The language changes that accompany the TAP (making it possible to write code without having to wrestle with the obfuscations to readability introduced by callbacks) are also significant -- even more so, given the prevalence of exclusively asynchronous APIs in Windows Phone 7 and Silverlight.