Practical .NET

Powerful JavaScript With Upshot and Knockout

The Microsoft JavaScript Upshot library provides a simplified API for retrieving data from the server and caching it at the client for reuse. Coupled with Knockout, the two JavaScript libraries form the pillars of the Microsoft client-side programming model.

• By Peter Vogel
• 06/02/2012

One of the components of the new Microsoft ASP.NET MVC 4 beta is the Microsoft Upshot JavaScript library, which is used as part of the architecture of a Single Page Application, or SPA (other JavaScript libraries used to create SPAs are jQuery, Knockout and History). In the default ASP.NET MVC 4 SPA, the user requests a page that initially displays a grid of items. When the user selects an item from the grid to edit, the page hides the grid and displays an editable form showing that item. After the user finishes making changes to the item, the user clicks a button to save his changes. At that point, the single item form is hidden and the grid displayed.

All the data is retrieved from the server and all the updates are sent back to the server through a set of JavaScript libraries communicating with operations in a service on the Web server. That service provides a translation layer between the data model and the page's data management requirements.

The Microsoft Upshot library handles much of the utility code needed to make this work. Upshot makes the requests that retrieve objects from the service's operations, and also handles sending changed objects back to the server for updating. In addition, Upshot manages those objects after they're received at the client.

A second library, Knockout, handles the data binding that gets the data from the objects onto the page for the user to view and update. While Knockout is used by default in ASP.NET MVC 4, Upshot could, presumably, be integrated with other client-side libraries that support data binding (or you could just move the data back and forth yourself).

I'm going to walk through how Upshot is used in the ASP.NET MVC 4 SPA; it should, presumably, work in any Web environment. But this is still beta software, after all, and I don't want to press my luck - in my testing, I never got everything to work at the same time.

In Visual Studio 11, for instance, the dialog box that enables adding new SPA controllers and views would periodically stop appearing; on the client, my JavaScript service calls would suddenly start throwing 404 errors. While you can use ASP.NET MVC 4 in both Visual Studio 2010 and 2011, I was never able to get the grid view to display in Visual Studio 2010 applications. How much of that was my fault and how much the beta's, I don't know.

However, even at the beta stage, the code and design of a SPA in an ASP.NET MVC 4 implementation presumably demonstrates what Microsoft currently considers the best model to follow when implementing applications with Upshot.

If you want to try using Upshot without installing ASP.NET MVC 4, you can use NuGet from within a Visual Studio project (just search for Upshot). The NuGet package will not only add Upshot to your project, but will add the two libraries on which Upshot depends: Knockout 2.0 and jQuery 1.6.

Setting up the Server-Side Resources
After creating an ASP.NET MVC 4 project in Visual Studio 2010 (calling the project NorthwindManagement), I added an Entity Framework model based on the venerable Microsoft Northwind database to my Models folder. After building my project, I added a new Controller (called CustomersController), selecting the Single Page Application template in the Add Controller dialog (see Figure 1). Also, in the dialog, I selected Customer as my model class and my Entity Framework ObjectContext class (called NorthwindEntities) as my data context class.

[Click on image for larger view.]

Figure 1. The dialog for an ASP.NET MVC 4 controller in Visual Studio 2010 and 2011 allows you to select the options to create all the components of a Single Page Dialog (that is, when the dialog appears).

The dialog adds quite a lot to your project: three new files in the Controller folder alone. In addition to the CustomerController, the wizard provides a NorthwindController.Customer.vb and a NorthwindController.vb file. Both files contain partial classes that come together to create a NorthwindController class. The NorthwindController file also contains a second class (called NorthwindAreaRegistration) with a utility method for registering a route that specifies a particular action method (this method is called automatically by ASP.NET MVC).

My NorthwindController.Customer file contains a class that inherits from LinqToEntitiesDataController. The four methods in this class support retrieving all the objects you specified in the Add New Controller dialog and performing inserts, updates and deletes on single instances of those entities. The methods work directly with the entity classes from the Entity Framework model by calling methods built into the LinqToEntitiesDataController (such as InsertEntity).

Notable by its absence is a method to retrieve a single Customer entity. However, the GetCustomers method returns an IQueryable collection, which means you can use the OData syntax to supply criteria in calling the method to control the result.

Listing 1 shows what my NorthwindController.Customer file contains.

In real life you'd want to add code to these methods to do any server-side validation or security checking. Typically, you'd also use this class to work with Data Transfer Objects that combine data from several different entities or limit the data returned from the entity that's driving these methods. (Does the browser really need all the information from the Customer object?)

Building the Client-Side ViewModel
I also have a new file in my Scripts folder: CustomersViewModel. This file contains a Knockout ViewModel in JavaScript. A Knockout ViewModel holds both the code to define the data structure being bound to elements on the page and the code to manipulate that data independently of the UI. The ViewModel generated for you first defines a namespace for the variables used in the ViewModel (to read why this is a good idea, see John Papa's September 2011 column, "Effective JavaScript Tips"):

(function (window, undefined) {
  var MyApp = window.MyApp = window.MyApp || {};

The generated code then defines the ViewModel using a function that accepts a single parameter. That parameter is a data structure that code elsewhere in the application can use to pass application-specific parameters to the code in the ViewModel:

MyApp.CustomersViewModel = function (options) {

A Knockout ViewModel contains two kinds of public items: data items that can be bound to elements on the page, and functions that can be bound to events raised by elements on the page. The ViewModel generated by the dialog contains a function that, when passed a Customer object, defines those data items and sets them to the values on the object passed to the function:

MyApp.Customer = function (data) 
{
  var self = this;
  self.CustomerID = ko.observable(data.CustomerID);
  self.CompanyName = ko.observable(data.CompanyName);
  self.ContactName = ko.observable(data.ContactName);

This code uses the Knockout observable function to implement two-way data binding between the Knockout ViewModel and elements on the page. As these data items are updated, the elements they're bound to on the page are automatically updated; as the user enters data into the bound elements on the page, these data items are updated. (For more on Knockout, see the series of blog posts I wrote for Learning Tree International's "Perspectives on .NET Programming".) If you'd prefer not to write all the assignment statements to move data from the objects returned from the server to your ViewModel, you can use the Upshot map function:

function Customer(data) {   var self = this;
   upshot.map(data, "Customer:#NorthwindManagement", self);

Finally, at the end of the client-side object's definition, the code sets up change tracking and state management by calling the Upshot addEntityProperties, passing a reference to the data structure (self) and the Upshot description of the entity ("Customer:#NorthwindManagement"):

upshot.addEntityProperties(self, "Customer:#NorthwindManagement");
}

In my view folder, I have four new views: _Editor, _Grid, _Paging and Index. The _Editor, _Grid and _Paging views are partial views that the Index copies in and makes visible as needed. Within these views, parts of the Knockout model are bound to elements on the page. This example binds the value property of an input element to the CompanyName property on the Knockout ViewModel:

<input name="CompanyName" data-bind="value: CompanyName" />

The Index view demonstrates a particularly elegant example of using Upshot data bindings. In the Index view, the div elements that contain the _Edit and _Grid views have their visible attributes bound to an item on the model:

<div data-bind="visible: editingCustomer">
  @Html.Partial("_Editor")
</div>

<div data-bind="visible: !editingCustomer()">
  @Html.Partial("_Grid")
</div>

Because of these bindings, code in the ViewModel that controls the editingCustomer value also controls which part of the UI is visible at any time.

The generated code also binds events in the view to functions in the ViewModel. The _Paging view, for instance, contains a set of anchor tags with Knockout databindings that tie to functions in the Knockout ViewModel that handle paging forward and backward through the data. This example binds an anchor tag's click event to a moveFirst function on the ViewModel:

<a href="#" data-bind="click: moveFirst ">First</a>

The Index page also holds the code that initializes the ViewModel (passing any options as a parameter) and binds the page's elements to the Knockout ViewModel:

var viewModel = new MyApp.CustomersViewModel({
  serviceUrl: "@Url.Content("~/api/northwind")"
});

In order for this to work in any environment, you'll need to have all the Upshot client-side dependencies. Here are the client-side libraries that ASP.NET MVC 4 felt I needed in my pages (you can probably omit the nav and native.history.js libraries if you're not going to use History):

• upshot.js
• knockout-2.0.0.js
• nav.js
• native.history.js
• upshot.compat.knockout.js
• upshot.knockout.extensions.js

Configuring Upshot
With that groundwork laid, you can start configuring Upshot. To manage your objects in Upshot, you use the RemoteDataSource function to return a DataSource that will retrieve objects from the server and cache them at the client. When creating a DataSource, you must specify the URL for the service with the operation to call (providerParameters), the client-side description of the data (entityType), whether changes are to be held until you explicitly send them back to the server (bufferChanges), and the function that's to be passed the data object that Upshot retrieves (mapping).

This example specifies the following:

• A relative URL within my NorthwindManagement site
• The server-side operation to call to retrieve objects is GetCustomers
• The entity description is the same as the one used in defining the data structure
• Changes to objects are to be immediately sent to the server for processing (the default is true)
• Retrieved objects are to be passed to the Customer function that defines my ViewModel's data structure:

  		var rmDataSource = new upshot.RemoteDataSource({
  		  providerParameters: { url: "api/Northwind", 
  		                        operationName: "GetCustomers" },
  		  entityType: "Customer:#NorthwindManagement",
  		  bufferChanges: false,
  		  mapping: MyApp.Customer
  		  });

The URL in this case will (thanks to the ASP.NET MVC routing mechanism) direct all requests to my NorthwindController. In an ASP.NET application, when calling a Windows Communication Foundation (WCF) service, I'd use the name of the .svc file containing my service (such as "Northwind.svc").

In addition to the providerParameters shown here, you can also specify the provider that Upshot will use. Upshot comes with a riaDataProvider that allows you to interact with services designed for Silverlight RIA applications and an ODataProvider that lets you use the OData syntax to issue REST-like queries against operations on the service (which must return an IQueryable result to integrate with the queries). This example specifies the ODataProvider:

var rmDataSource = new upshot.RemoteDataSource({
  provider: upshot.ODataDataProvider,
  providerParameters: { url: "api/Northwind", ...

The entity description I referred to earlier is a data structure that describes to Upshot the objects being retrieved from the service. Three Upshot functions must be passed that structure, so that Upshot knows what it's manipulating: RemoteDataSource, metadata and addEntityProperties.

The entity description that Upshot requires begins with the name you'll use to refer to the entity's description when working with Upshot (ASP.NET MVC 4 uses the entity name, followed by ":#" and followed by the server-side project's namespace). The entity's client-side definition has four components:

• The entity's key field: The property on the object that Upshot can use to determine if two objects represent the same entity.
• The entity's fields: The properties from the server-side entity with the data type.
• Property rules: Data constraints including maximum length for strings and whether a field is required.
• Messages: This isn't required, so I'm going to ignore it for this article.

Listing 2 defines metadata for an object called Customer from a project with the namespace NorthwindManagement. It specifies that the key field for the object is CustomerID, and the object has two properties in addition to that field, both of which are of type string. In the rules section, the maximum length for all of the fields is specified, and two of the three fields are specified as required.

While Listing 2 only defines a single entity, you can specify multiple entities in the metadata you pass to Upshot.

It's not clear that creating this structure is a job for mere mortals. Things get more complicated if, for instance, the entity has an association with another entity (for instance, Customers have an association with Orders). In addition to describing the two entities, you must also specify the association between the classes both from the parent side and the child side (in this case, both from the Customer and Orders side).

In ASP.NET MVC 4, the HtmlHelper class includes a method whose sole purpose is to generate this metadata when passed a reference to the service class that Upshot will be working with. This example generates the metadata for the output from my NorthwindController class, for instance, as an IHtmlString:

@Code
  Dim mdata As IHtmlString
  mdata = Html.Metadata(Of 
    NorthwindManagement.NorthwindController)()
End Code

The HtmlHelper method also generates the code that passes the resulting string to the Upshot metadata function. In the absence of HtmlHelper (outside of ASP.NET MVC 4), you'll need to add the code that passes the entity description's string to the Upshot metadata function before instantiating the ViewModel.

The code that makes up your ViewModel belongs in a separate file because it's independent of any page. This also supports testing all of your business logic using a test-driven development process without involving any of your Web pages. In the absence of a tool that will generate the entity description for you, once you do correctly generate the description for any object, you'll want to hang onto it so that you can update it as your service evolves (and share it with other developers using your service).

It would be a good idea to place your entity definition in the same JavaScript file that holds your ViewModel. All you need to put in your page is the code that passes the entity description to the Upshot metadata function and the code (shown earlier) that instantiates your Knockout ViewModel and initiates data binding.

Working with Data
Now you're ready to retrieve objects from the server. After doing all that work, actually working with data had better be easy - and it is.

Before retrieving any entities, you should set a filter to limit your request (unless, of course, you're limiting those entries in your service class or really do want to download all the entities to the browser). The Upshot setFilter function on the DataSource object lets you do that by passing a property name and the criteria you want to use. This example sets the filter to retrieve only those customers where the City property is set to "Berlin" (I could omit the operator setting because it defaults to "=="):

rmSource.setFilter({
   property: "City",
   value: "Berlin",      
   operator: "=="   });

With the filter set you can safely retrieve your entities using the RemoteDataSource asynchronous refresh function. This function calls the server-side methods you set up earlier and moves the objects asynchronously from the server to a client-side cache:

rmDataSource.refresh();

You can provide a callback function to the refresh method to process the results, if you want. This example displays the number of items retrieved, for instance:

rmDataSource.refresh(
  function(res) {alert res.length;});

To retrieve all the objects managed by Upshot, you call the getEntities method or getFirstEntity method. This example stores the results of calling the getEntities function in a variable called custs that, presumably, is part of my Knockout ViewModel and is bound to elements on the page:

self.custs = rmDataSource.getEntities();

Assuming that you've correctly set up your integration between Upshot and your Knockout ViewModel (and bound your page's elements to your Knockout ViewModel), the user can now view and update the data displayed on the page. If you set bufferChanges to true when creating the DataSource, you'll need to call the RemoteDataSource CommitChanges function to move any client-side changes back to the server. With bufferChange set to true, you can also use the DataSource revertChanges method to throw away any changes on the client without sending them to the server. Upshot has an IsUpdated property that you can check to determine if there are any updates pending.

To delete an entity, call the RemoteDataSource DeleteEntity method, passing your client-side object. Typical code might look like this:

rmDataSource.deleteEntity(customer);
rmDataSource.commitChanges(function () {
  alert("Customer deleted";});

You can add new client-side instances of your Customer to the array using the JavaScript push function to put the new item at the end of the array (or unshift to add the new object at the start of the array). A call to CommitChanges will move the object back to the server to be inserted into the database.

Managing Local Data
Up until now, I've been working with the Upshot RemoteDataSource. However, Upshot also includes a LocalDataSource that interacts with HTML5 Local Storage (see Mark Michaelis' November 2011 UI Code Expert column, "Caching in on HTML5 Local Storage," at VisualStudioMagazine.com/Michaelis1111). Rather than go back to the server every time you want to filter or sort the data already available at the client, the LocalDataSource works with the local objects. LocalDataSource functions as a wrapper for a Remote-DataSource (or an array of objects returned by the RemoteDataSource getEntities). For whatever reason, the code generated by the ASP.NET MVC 4 dialog doesn't use LocalDataSources, but that doesn't mean you can't use it.This example creates a LocalDataSource based on the RemoteDataSource I showed you earlier:

var lcDataSource = upshot.LocalDataSource({ 
  source: editorDataSource, 
  autoRefresh: false } );

The autoRefresh parameter specifies whether the LocalDataSource automatically calls the RemoteDataSource it wraps to asynchronously fetch objects from the server. By setting autoRefresh to false, you can control when your page goes back to the server. The LocalDataSource refreshNeeded property will be set to true when a refresh is required.

The LocalDataSource has all the functions that the RemoteDataSource does, and delegates some of those functions to the RemoteDataSource it wraps (the refresh function, for instance). However, the setFilter function on the RemoteDataSource is applied against local objects, allowing you to manipulate objects at the client without going back to the service as the RemoteDataSource does.

An ideal solution would include the ability to persist data to the user's hard disk for non-volatile data that you're willing to leave on the client. Ideally, I'd be able to use the LocalDataSource to page through local data until I run out of objects, at which point the RemoteDataSource would go back to the server to get more objects. However, that probably gives up too much control to the library (not to mention control over how data is stored at the client).

I haven't discussed everything about Upshot. You can, for instance, bind functions to events that DataSources fire (the ServerError event, for instance); to handle paging through data, you call the DataSource setPaging function, passing the number of objects to retrieve and the number to skip; the setSort function allows you to specify the properties to sort data on; and the setFilter function accepts multiple parameters. It's a very functional API.

Overall, I have to admit to liking the relative simplicity of Upshot. I was working with one of my clients recently who was mapping out a strategy for building AJAX applications. We were assuming that we'd be making extensive use of client-side data caching and weren't looking forward to creating the functions to manage that. Upshot neatly encapsulates what we wanted. I was already committed to Knockout, so a tool already integrated with Knockout is also something I value.

I'll admit to being dismayed at creating the metadata structure Upshot requires - I'll be looking for a tool that will generate that for me outside of the HtmlHelper method. You'll also need to get into the habit of thinking in the "Knockout-plus-Upshot paradigm." But, keeping in mind those concerns, Upshot delivers a powerful client-side infrastructure, freeing you to concentrate on the business code that matters to your clients.