Tech Brief

Inside Sync

A new framework for data synchronization.

• By Bill Wolff
• 02/01/2009

The Microsoft Sync Framework is a data-synchronization platform that can be extended to any device on any data store using any network topology. The framework enables developers to build collaboration and offline scenarios for applications, services and devices. It's provider based and includes pre-created providers and a Provider Framework for creating new custom providers. These tools and technologies provide roaming, sharing and flexible storage of any data type over any protocol on any network configuration.

Microsoft has a long history of data-synchronization efforts. Active Directory is a good example of a multiple master replication technology. SQL Server does a great job of replicating data between servers using log shipping and database mirroring. Groove brings this capability to the Office family, allowing disconnected collaboration and document workspaces. Outlook can synchronize with SharePoint lists and libraries. Both Outlook and Internet Explorer consume and store RSS data feeds using shared settings. On the consumer side, Xbox Live synchronizes gamer profile data with the Live cloud, and Zune Marketplace tracks media usage and purchases. In the cloud, Live Mesh enables file synchronization among various types of connected devices. The newly announced Windows Azure Services include SQL Data Services for robust data storage and distribution.

The success of these various projects and products has culminated in the development and standardization of a set of synchronization libraries. Version 1 of the Sync Framework was released last August. Version 2 was announced at the Microsoft Professional Developers Conference in October, and the community technology preview is now available for download.

Sync Services
The main components of the Sync Framework include Sync Services for ADO.NET, File Systems and FeedSync, which is the basis for Web 2.0 collaboration. These pre-created and optimized Sync providers handle database sources, files and folders, and RSS and ATOM feeds.

Sync Services for ADO.NET version 2 works with any database for which there is an ADO.NET Provider. Version 2 uses a hub-and-spoke model for remote workers who need offline database synchronization; this remote model is similar to SQL Merge replication. Version 2 can also be used for peer-to-peer synchronization of SQL Server databases, although SQL Replication is preferred in corporate data centers. The recent release of SQL Server 2008 includes integrated Change Tracking as a declarative database and table-configuration option. This provides robust change tracking with no schema changes required. Sync Services for ADO.NET version 2 works over services like Windows Communication Foundation (WCF) and fits nicely in service-oriented architecture environments.

Sync Services for File Systems manages changes to files and folders. Basic file replication dates back to Windows NT, but Sync Services provides standard programming interfaces that extend beyond the Windows FAT and NTFS file systems to enterprise integration and cloud services.

Custom Providers
Developers can extend the architecture by creating custom providers. They use the Sync runtime to write Sync applications that synchronize stores and custom providers for stores with unique requirements. Sync applications can run on a desktop or server as standalone executables or services. They can also live in the cloud as a Live Mesh or Windows Azure application.

Sync providers exist on the source and target destinations as endpoints. Any data source that provides unique identifying properties per item can be tracked. The provider must manage the metadata associated with change tracking. In some cases, this metadata is part of the data to be synced, and in other cases it's stored in a companion database using a simple local store like SQL Compact Edition 3.5. The state of an item is recorded as a version with a replica ID and tick count. Deletions are recorded as tombstones. This item knowledge is communicated to another sync target via a Sync application. The application orchestrates the dialogue between two providers. The provider must enumerate changes to the Sync app on demand and apply inserts, updates and deletes as needed.

The source and target providers don't need to match because both are based on the same provider framework with a standard set of programming interfaces. This allows one database product (Access) to sync with another (SQL), or one file system (NTFS) to sync with another (Live Mesh).

Custom Sync providers are built in Visual Studio 2008 Service Pack 1 (SP1) and include Designer support for Device Projects. The project templates are N-tier only and generate download-only sample code. They are easily extensible to bidirectional operations. The template creates two projects: a device client application and a WCF Service Library. The Designer support in Visual Studio 2008 SP1 makes implementation quite easy.

What's New in Version 2
Sync Framework version 2 adds built-in provider support for Live Mesh and SQL Data Services. It's generally faster and introduces the Provider Framework, which simplifies custom provider coding chores. Full providers can enumerate objects and tell them apart by properties, determine if they've changed and apply appropriate changes. Anchor-based providers can track changes from a point in the past. Filtering is enhanced with negotiated interfaces. Selective column handling without data loss is accomplished with change units. Constraint conflict resolution is more robust with extensive logging support.

With this more powerful Provider Framework, it's likely that Microsoft and other vendors will quickly introduce compatible providers. This will grow the Microsoft Sync ecosystem exponentially.