In-Depth

Re-Architecting Azure

A year after the first technical previews of Azure, Microsoft is launching a less-ambitious platform with forklift revisions based on developer feedback.

• By Kathleen Richards
• 11/01/2009

Microsoft will reach a promised milestone when it commercializes the Windows Azure Platform at its Professional Developers Conference 2009 (PDC09) in Los Angeles later this month.

The work that has gone into the Windows Azure Platform over the past year is impressive by all accounts. At PDC 2008, the company said it would rely on developer feedback to help shape the roadmap. A year after the first technical previews of Azure, Microsoft is launching a platform with major technology revisions. The good news is that a lot of the forklift changes are based on developer feedback.

"It's very encouraging that they're taking feedback in mid-flight and changing it," says Ranjith Ramakrishnan, co-founder and chief technology officer of Cumulux. An early adopter of Azure, the startup company ported its MobileAxis application, built on the .NET Framework 3.0 and Microsoft Office SharePoint Server 2007, to Azure. The service provides a unified gateway to enterprise applications for smartphone users. It's expected to debut in the PDC09 time frame.

New Business Portal
PDC09 marks the unveiling of the gateway to the consumption-based pricing model that's driving Microsoft's cloud-computing platform. With the new "business" portal, managers can determine how to purchase the Windows Azure Platform, agree to terms of use, provide payment authorization such as credit-card information and then assign the platform to a developer or development team by entering their respective Windows Live IDs.

Developers then have access to the entire Windows Azure Platform through a newly unified (common) interface to the Azure Services developer portals. This allows them to use whatever is needed from the Platform as a Service (PaaS), which at launch will consist of the Windows Azure cloud operating system, SQL Azure relational data storage and .NET Services connectivity.

The meter starts running once a developer deploys an application or storage account to the Azure staging environment, made up of virtual machines (VMs). Usage and billing metrics are pushed to the business portal in near-real time, according to Microsoft, and developers are largely shielded from the financial and legal aspects of Azure.

"The business model drives the technology," asserts George Moore, a Microsoft software architect, who gave an overview of the new business portal in a Channel 9 interview in September. "To a large extent, the shape of the final product-like how the fabric actually works, how it's deployed, how they count things, how the service-level agreements [SLAs] work-all that stuff is driven by the business team."

Maybe that explains some of the head-spinning changes since PDC 2008. Windows Azure and the Azure Services Platform are now referred to as the Windows Azure Platform. The Windows Azure operating system, hosted on Windows Server 2008 in

Microsoft's data centers, brandishes some of the technology improvements floated at PDC 2008. The higher-level developer services-.NET Services, SQL Data Services (now SQL Azure), Live Services, SharePoint Services and Dynamic CRM Services-have been whittled down to two at launch: .NET Services and the revamped SQL Azure.

Unlocking New Capabilities
By most accounts, the Windows Azure OS, which consists of VMs (compute), fabric controller (resource management) and unstructured data storage (Blobs, tables and queues), is stable. Developers can expect announcements around the VMs, an Azure version of Microsoft's Hyper-V, at PDC09 later this month.

An early criticism of the Azure technology by Tom Bittman, chief of research, infrastructure and operations at Gartner Inc., was that the community technology previews (CTPs) did not support scaling up "fixed-size" VMs. Automatic provisioning of elasticity, essentially the ability to spin up virtual servers as demand increases and decrease their number as it subsides, is expected in the first commercial release.

"It's always been planned to have scaling out, they just haven't implemented it," says Roger Jennings, a developer and author of the OakLeaf Systems blog and "Cloud Computing with the Windows Azure Platform" (Wrox, 2009). "That's supposed to be implemented in the Web release at PDC. It's a manual process now."

When the commercial version is launched, Jennings also expects to see final provisioning APIs so that developers can provision their VM instances with code, as well as minimum and maximum instance limits.

As advertised at PDC 2008, Microsoft has expanded Azure beyond managed code to support native code and PHP. Windows Azure Storage has also been improved in the last year with support for limited transactions, among other features. The support for field streams, caches and locks talked about at PDC 2008 have yet to appear.

"When we went out with the CTP last October, it was .NET code only and it was partial trust, and so you couldn't really run anything you wanted to. You had a little bit of a sandbox," says Steve Marx, a technical strategist for the Windows Azure team. "And we opened that up around our MIX conference and said: 'You can run native code if you want; you can run PHP and you can run in full trust.'"

The Windows Azure for PHP SDK on CodePlex targets Windows Azure Storage and provides programming interfaces for Windows Azure's compute and management services. With the Simple Cloud API, a Zend Technologies Ltd.-sponsored open source project launched in September, PHP developers can use three reference interfaces to target multiple cloud vendors' file storage.

Shaped by Developer Feedback
Microsoft did listen to developers and changed course dramatically on SQL Data Services (SDS)-formerly SQL Server Data Services (SSDS)-evolving it from a schema-less data source to a fully relational data storage called SQL Azure. "That was a very good move, mostly because SSDS and SDS really duplicated many of the features of conventional Azure tables," says Jennings.

Jennings hopes to see support for encryption in SQL Azure, particularly transparent data encryption for commercial and regulatory privacy. SQL Azure is on target for release quality at PDC, he says.

At press time, SQL Azure had a storage limit of 10GB, and Azure Storage Blobs could scale to 50GB. "When you talk about scaling beyond 10GB, where your app owns the logic, I think it would be really nice to be able to get some of that partitioning baked into the platform so that I don't have to own that as a developer," says Vishwas Lele, chief technology officer at Applied Information Sciences Inc. (AIS) and a Microsoft regional director.

AIS is working with several customers on Azure prototypes, including a client who has to do a lot of number crunching. "They can spin up the worker role when they have to do all these calculations," says Lele. Unlike Google App Engine, the worker roles and queues in Azure can run 24x7 for background processing.

.NET Services, which originally consisted of a service bus, access control and workflow, has had a tumultuous year. Workflow was taken offline in the summer because developers indicated that they wanted the workflow engine to be consistent with Windows Workflow Foundation 4, expected in .NET 4.

In the October CTP, access control was re-jiggered, with support for WS-* protocols "temporarily" suspended. The team indicated a decision to focus on secure access to Web REST services as more enterprises move to REST. The idea was floated at MIX and developer feedback indicated that it was the right move, according to the .NET Services Team blog. The October CTP, which closely mirrors what will appear at launch, according to Microsoft, includes support for simple Web trust; two token exchange endpoints, including SAML extensions and Active Directory Federation Services (AD FS) 2.0; claims transformation; and Security Token Service.

According to the Sept. 18 blog posting by the .NET Services Team that detailed planned October CTP changes: "Once this infrastructure is in place, we will work on future version features of .NET Services, like Web single sign-on and rich WS-* support. In future releases, we will reinstate full support for the WS-* protocols, Web single sign-on, and round out the .NET Access Control Service offering in a way that spans the REST/SOAP spectrum."

More information on how Windows Identity Foundation and AD FS fit with Azure is likely at PDC09. In August, Microsoft released Project Riviera on CodePlex, the first Azure line-of-business reference application, developed in collaboration with Cumulux.

Designed to support a Customer Loyalty Management program, the sample code uses multi-tenant storage via the Windows Azure Table and SQL Azure, Silverlight 3 and several other technologies. Of particular interest is Project Riviera's use of AD FS, formerly Geneva Framework, and the Security Token Service.

The October CTP also saw drastic changes to the service bus, which is designed to securely connect on-premises and cloud apps through WCF endpoints. Routers have temporarily been dropped. Queues were changed to simpler message buffers. WSHttpRelay Binding was discontinued, with WS2007Relay Binding suggested as a workaround. External Endpoint Registration is no longer available, but Microsoft expects to reinstate it at some point in the future.

The reshuffling of Live Services-which included the Live Framework and Live Mesh synchronization beta-out of the Azure Platform occurred in late August. David Treadwell, Microsoft corporate vice president of Live Services, announced in the Live Services blog that the Live Framework CTP would end on Sept. 8. The Windows Live Web Role template for Azure won't be supported in future tooling.

"Largely, what this is, is that Live Framework and the team building it all sort of merged with Windows Live and just became part of that broader team," says Windows Azure technical strategist Marx.

Developer Experience
A major selling point of Azure for Microsoft developers is the ability to use their existing skill sets and familiar tooling to program cloud services: Windows Azure Tools for Visual Studio 2008 and Visual Studio 2010 beta 1 CTP, which includes the Windows Azure SDK. Windows Azure currently supports .NET 3.5 even if you're using VS 2010.

To run code in the cloud, developers use the Azure Services Developer Portal to launch application code and a service model (worker and Web roles). The service model during the CTPs was quite limited, with multiple worker roles as the back-end and Web roles as the front-end just introduced in the July CTP. "The fundamental communication model and the kinds of things you could run in those [roles] didn't really expand, and that's what you can look for coming to this PDC09 and even beyond, is to just open up that box and enable more and more complex services," says Microsoft's Marx.

The Azure Tools provide a developer fabric and storage services for creating, debugging and packaging cloud services on your local desktop using C# and VB project templates. "I've found that the developer fabric has gotten better," says Lele, who has been working with the Azure technologies since PDC 2008. Developers can run their code and storage locally using a mixed mode, where they run their code locally and use SQL Azure, Azure Storage or host everything in the Azure fabric.

A Service Management API and upgrade option released to the Web in September enables programmatic improvements. "So now people can do things like automate their build process and deploy to the cloud and run some tests, or something like that," says Marx.

Early on, some developers complained about the need to create a Visual Studio solution for the cloud, particularly if they wanted to use some existing code. "We actually cleaned some of that up in the past year, so now it's a little easier to share code between something that you're deploying to Windows Azure with something that you're not, or make that decision later in the development cycle," says Marx.

As early adopters like Kevin Marshall, director of product management for Clarity Consulting Inc., have noted, there are also slight differences between what works locally on your desktop in Window Vista or Windows 7 and the cloud because it's a Windows Server environment.

"My goal is that, if it works on your laptop, it's going to work in the cloud, and we're some 80 percent to 90 percent of the way there, but I'd like see us just ratchet it up as we go," says Marx. "In the storage service, in particular, there are some features that we've released to the cloud that we haven't put in the developer version of those services yet."

Deployment to the staging environment in the previews was "brutally slow," says Clarity Consulting's Marshall, who has hosted apps on Azure and Google App Engine. Making changes to even one line of code, once an app or service has been deployed, is tedious. Azure requires that you recompile and deploy the whole app instead of making changes on the fly and patching it.

During the previews, developers had to go through an arduous process of copying their logs locally just to view them. In early October, Microsoft announced on the Windows Azure blog that it was disabling the ability to retrieve logs through the Azure portal because an improved logging system is going to be introduced at launch.

Not all that much has changed since the Windows Azure tools were first introduced at PDC 2008. Microsoft is primarily working on performance improvements and "fit and finish" of the tooling, says Marx.

For those concerned about security issues, Microsoft is working on a whitepaper for the PDC09 time frame that explains what is happening "under the hood," says Marx. The cloud isn't introducing new security concepts, he asserts, especially if users understand the virtualization model.

Hello Real World
The first external Azure app from Microsoft appeared in beta in late June. Called Microsoft Hohm, it's designed to help consumers monitor their home energy consumption and ultimately save money.

Numerous Azure prototypes and enterprise apps are in the works. TriSys Business Software Ltd. is using SQL Azure for its TriSys Recruitment Software as a Service (SaaS). Epicor Software Corp. is migrating the enterprise search component of its ERP to Azure.

"I really want to see what they're going to do around the small and [midsize] space," says Bill Wagner, founder of SRT Solutions and a Visual Studio Magazine contributor. "I think the current pricing model works well for a large enterprise, but I really don't think it's competitive against PHP and LAMP stack hosting for smaller business. I think they need an offering to get into that space."

Marx expects many ISVs to look at Azure for building and hosting SaaS. "I think there's a lot of benefit to these small development teams that don't have big data centers-and don't want to have big data centers-but want to be able to develop and deliver software over the Internet," he explains.

One of those ISVs, Cumulux, would like to see integrated billing. "I want to give one bill to my customer," says Ramakrishnan.

Jennings hopes to see pricing that's competitive with Google App Engine, but he doesn't think that's going to happen. "Microsoft thinks [it's] competing with Amazon," he says.

However, like many developers, Jennings is impressed with all that has happened during the year since the first Azure previews were released. "It's very encouraging," he says. "I think they've kept most of their commitments. They've listened to their developers and they had the courage to make the changes when they needed to."

Later this month, if everything goes as planned, the Windows Azure Platform will become real. As one Fortune 100 developer remarked at the Azure introduction a year ago: "We haven't seen a lot of the application lifecycle, so there's a promise and it's very easy to get in-but what happens then?" We're about to find out.