News

Open Source at the Heart of Microsoft's Intelligent Cloud Development Efforts

At the company's Azure OpenDev online presentation, Microsoft and its open source partners showcased a number of tools and resources meant to provide developers a leg up on developing for the "intelligent cloud & an intelligent edge."

• By Kurt Mackie
• 06/26/2017

John Gossman, lead architect for Microsoft Azure, kicked off an online presentation last Wednesday saying that "Microsoft developers love open source" because they can debug code in a way that they could not do with proprietary source code. His presentation kicked off 3.5 hours of demonstrations of open source tools that can be used to tap Microsoft Azure resources and perhaps was aimed at convincing developers to use technologies such as containers for their regular dev-test work.

Gossman said that there are currently more than 15,000 Microsoft employees that have GitHub accounts. Microsoft Technical Fellow Anders Hejlsberg uses GitHub to maintain the open source TypeScript language, which adds static checking and code refactoring tools to JavaScript applications. Microsoft's Visual Studio Code lightweight editor and integrated development environment supports TypeScript, as well as Go and Node. Visual Studio Code is based on the Atom source editor and is an open source project on GitHub, Gossman explained. He added that John Howard, a senior program manager on the Windows team, is the leading contributor to Docker, a maker of container technologies for Linux and Windows.

The Cloud Foundry Foundation was part of the presentation. Microsoft announced earlier this month that it had joined that organization. The foundation is a collaborative project of the Linux Foundation, according to Abby Kearns, executive director of the Cloud Foundry Foundation. She added that the collaboration with Microsoft offered a tremendous potential for developers to use Cloud Foundry on Azure.

Joshua McKenty, head of global ecosystem engineering at Pivotal, argued during the presentation that applications should be able to run in the public cloud of the developer's choice. He said that Pivotal is working with Microsoft to get its "patterns" to work in .NET. Pivotal offers its own Cloud Foundry implementation for developers.

Open Source Announcements
The open source partners profiled during the presentation included representatives from Docker, Canonical, Pivotal, Red Hat and Chef. There were a couple of announcements made during the event.

First, Docker announced that it will be bringing the Docker Community Edition to the Microsoft Azure Container Service. It will be scalable and secure by default using Swarm, Docker's native clustering solution with native load balancing, according to Michael Frissm, a Docker product manager. During the event, Frissm demonstrated using this solution to build an application on the Azure Container Service in a couple of minutes. He mentioned that using containers is a great way to package an application and share it with colleagues. With containers, only the operating system is virtualized. The processors and file system get sandboxed, so there's a low hardware overhead. Docker provides container images that developers can use to "containerize" their apps, he added. Currently, Docker offers two product editions, Community and Enterprise, which are based on open standards.

The second announcement during the presentation concerned Chef's work with Habitat, which is an open source Apache project on GitHub for building, deploying and managing applications in any environment, according to Nell Shamrell-Harrington, a senior software development engineer at Chef. She's also the core maintainer of Habitat at GitHub.

Habitat packages apps and works with Docker and Kubernetes. It assumes failures and is self-healing. Habitat works without containers but it "shines when using them," Shamrell-Harrington said. She explained that containers can sometimes be obscure and painful to use, but when a container image is created with Habitat, "it's not a black box." While the Bash shell for Linux is currently used to create Habitat packages, Shamrell-Harrington announced the news that "soon you'll be able to use it with Windows." Habitat for Windows is still in development, but it'll be designed to leverage PowerShell to create packages.

Open Source Demos
Other open source technologies were demonstrated and discussed during the event.

On the Docker side, Scott Johnston, chief operating officer, made the claim that the use of microservices will revolutionize application development workflows. Existing applications can be containerized, which adds security because of the isolation and adds greater efficiency because half the resource gets used, he claimed. The Docker Enterprise Edition can be used to "modernize" existing apps by putting them in a container. Docker has partnered with Microsoft and Avanade on a proof-of-concept project to modernize applications, he added.

One of the Docker tools that can be used to more easily move traditional applications to containers is the Image to Docker tool. It inspects a virtual machine image to determine the components that can be "Dockerized," according to Frissm.

Mark Shuttleworth, founder of Ubuntu Linux and Canonical, described using "conjure-up" with Kubernetes as way to harness Microsoft Azure's compute capabilities. Ubuntu conjure-up works with Juju, MAAS and LXD to package solutions in cloud infrastructures, according to Ubuntu's documentation. Shuttleworth said that conjure-up goes out to Azure and requests the resources to build virtual machines, and allows the individual components to scale.

Shuttleworth also described using Helm to deploy applications on top of Kubernetes. Helm is an open source Microsoft tool added with the acquisition of Deis in April. It's a package manager for Kubernetes to keep track of resources, according to Michelle Noorali, a senior software engineer for Microsoft Azure and the core maintainer of the Kubernetes Helm project. Noorali explained that a package in Helm is called a "Chart," which consists of metadata, Kubernetes resource definitions, configurations and documentation. It's a tool to ease developers into Kubernetes, which is "still really hard" to master, she said.

Microsoft also has an experimental open source project to streamline Kubernetes development on the Azure Container Service called "Draft." It will detect the language used in the source tree and containerize an app, according to Gabe Monroy, a lead program manager for Microsoft Azure and former chief technology officer at Deis. Draft will take a Helm Chart and get it running. It looks for file changes and pushes them up to the cluster so that developers can see their changes live. Using Draft, developers don't need to know anything about Docker or Kubernetes, he added.

Red Hat demonstrated Red Hat OpenShift on Azure, which is used for Kubernetes container orchestration and management. It can deploy on any infrastructure and can even be installed on a laptop, according to Nicholas Gerasimatos, a technical cloud evangelist at Red Hat. He argued that while containers are easier to build and consume than virtual machines, they also add complexity. Organizations need a way to manage scale as they grow. OpenShift, built on Red Hat Enterprise Linux, can be used to automate scale and developers also can use it to create reproducible images that can run on any infrastructure, making it easy to deploy containers.

Skype on Linux Migration
Last but not least, Microsoft discussed its experience running Skype, the voice and videoconferencing service it acquired back in 2011. Kaspars Mickevics, a senior engineering manager at Skype, explained that Skype has been running on Linux for the past 10 years. Microsoft expanded Skype into its datacenters and it's still running on Linux on that infrastructure.

Microsoft was able to support Skype on the new infrastructure using "relays," which Mickevics described as C++ apps. The relays ran on bare metal, but Microsoft decided to move them to Azure infrastructure as a service. The move of the relays from Skype datacenters to Azure infrastructure was supported through the use of a "health check" service, which made it easy to do load balancing.

Microsoft used about 1,000 virtual machines for the move. Mickevics said that Microsoft used the "classic" Azure portal to make the move because the Azure Resource Manager portal still lacked some features at the time. The Skype servers were kept running during the migration. Microsoft had a hybrid setup in existence for three to four months before it switched the relays over to Azure exclusively.

Microsoft has been running the Skype relays on Azure infrastructure for about the last three years and is happy with the performance and stability, Mickevics added. He now advocates using Azure Resource Manager as the way to go. The Skype team is also fans of Azure CLI 2.0 because it's written in Python. For orchestration, the Skype team uses the open source Ansible automation engine, Mickevics said.

The Azure OpenDev presentation is available on demand here.