In-Depth
The Making of Visual Studio 2010
With the ambitious nature of the Visual Studio 2010 upgrade, it's hardly surprising Microsoft faced challenges on its way to RTM. We take a look behind the development of Microsoft's game-changing IDE.
Ask Dave Mendlen which day was toughest in the years-long development effort behind Visual Studio 2010, and the senior director of developer marketing at Microsoft is blunt: "The day we decided to add an RC [release candidate] and push our schedule out was the darkest day in our launch."
It's hard to argue that point. Visual Studio 2010 had, despite some of the usual pre-release misgivings about performance and functionality, requited itself well throughout the extensive beta testing process. Visual Studio 2010 beta 1, released in May 2009, offered a robust trial version of the integrated development environment (IDE). And beta 2, released at the Microsoft Professional Developers Conference (PDC) in November, boasted vast gains in both performance and stability. Everything was on track for the Visual Studio March 22 launch, also announced at PDC.
"We were ready to declare victory," recalls Mendlen. "We shipped beta 2 and were feeling good about our product and the functionality. We were feeling great. And then we started getting the feedback around performance."
The news was not good. Beta testers were reporting significant issues with performance and stability across a number of hard-to-predict scenarios.
"That was a dark day," Mendlen continues. "That day had a massive impact on everybody. We were doing a global launch. Imagine having the 30 launch events, and the five major launch events, and how do we fix that problem?"
Back to the Beginning
Microsoft's answer, of course, was to delay the launch and insert into the development schedule a public RC, which ended up shipping on Feb. 8.
Visual Studio 2010 is now slated to ship on April 12, and by most accounts, the critical performance and stability issues are well behind it.
Still, given the ambitious nature of the Visual Studio 2010 upgrade, it's hardly surprising that Microsoft faced challenges on its way to release to manufacturing (RTM). From newly supported platforms like Windows Azure, SharePoint and Silverlight, to the integration of advanced Windows Presentation Foundation (WPF) and Managed Extensibility Framework (MEF) technologies, Microsoft undertook a multi-front development effort that taxed the resources of the Visual Studio team.
Rob Sanfilippo, analyst at Directions on Microsoft, lauds Redmond for its ambitious platform support effort. "I think this is a worthwhile risk for Microsoft to take, rather than leaving out technologies for this major Visual Studio release. There will be some growing pains, but Visual Studio 2010 will enable developers to create some great new, high-quality applications," Sanfilippo says.
The effort started several years earlier, well before the November 2007 RTM of Visual Studio 2008, which was code-named "Orcas." Visual Studio 2008 added support for the Microsoft .NET Framework 3.5 and tooling for WPF, Windows Communication Foundation (WCF) and Windows Workflow Foundation (WF), as well as AJAX development. But Visual Studio 2010 aimed to make key changes to the core IDE, including the designer and code editor, as well as introduce a customizable interface for rich tool integration and purpose-built developer experiences.
"What was unique for Visual Studio 2010 was a renewed focus in the Visual Studio platform itself," says Chris Dias, group program manager for Visual Studio at Microsoft. He explains that, after Visual Studio 2008, the IDE was big and getting bigger, and something had to be done to ensure the "long-term health and well being" of the franchise.
"Visual Studio is a very large native COM and managed application. It has been in existence for more than a decade and has been developed by hundreds of different developers over the years," Dias says. "Our ability to innovate on top of Visual Studio was being hampered by the complexity of the underlying code base."
Look and Feel
Much of the work on the underlying code base occurred in the areas of extensibility and presentation. Visual Studio 2010 incorporates WPF 4, the latest version of Microsoft's advanced, XAML-based graphics framework, to enable a much more dynamic and precise visual development environment. The IDE also adds support for MEF, a technology that enables software to discover and interface with software components. For Visual Studio, the incorporation of MEF enables opportunities to customize the IDE to mission-specific roles and seamlessly integrate extended functionality.
"MEF and WPF clearly are the two technologies that Microsoft is betting on for extensibility and presentation, so it made sense for Visual Studio to take bets on these technologies," says Mendlen, who adds: "Our bet could only drive up the capabilities and quality of these technologies."
MEF itself actually got its start several years ago as part of an incubation project that included the new Visual Studio 2010 Editor. Originally called the Asset System, Dias says the technology was later taken up to serve as the general extensibility system for .NET.
"While Visual Studio has long offered a robust extension architecture, it was optimized for building large and deeply integrated toolsets that complement existing Visual Studio functionality," Dias says. "With MEF in mind, we've been able to focus on scenarios that allow a developer to replace or enhance existing functionality at a micro level."
The Visual Studio team faced challenges as it created MEF extension points in the IDE. Dias says the lack of diagnostic tools turned early MEF implementations into a "trial and error" process. He says teams often defined their own logic for populating catalogs, building and tearing down composition containers and establishing binding conventions. It fell to the Visual Studio group to untangle the knot.
"As we started integrating features together, we had to reconcile these differences, and were able to adopt best practices from each team's implementation approach," recalls Dias, noting that the simplest designs and conventions were most successful.
The early work on MEF has paid off in enabling the Visual Studio team to extend functionality to new platforms, says Tony Goodhew, product manager at Microsoft. "Going forward, technologies like MEF can really help us support more platforms easier, because we can define platform-agnostic abstractions and build specific implementations for each platform," Goodhew explains.
Visually, Visual Studio 2010 shows significant gains over its predecessor. Dias says the Visual Studio team made a concerted effort to improve the visual fit and finish of the IDE, from effective use of contrast to emphasize the code editor window to softened edges on windows and toolbars. The goal: To help developers focus more on core tasks.
The effort to implement WPF 4 in Visual Studio was aided by the fact that the Visual Studio and WPF teams are neighbors, says Dias.
"We're lucky that both the Visual Studio team and the WPF team are part of the Developer Division, because our business objectives and product schedules are closely aligned," Dias explains. "We realized early on in the 2010 product cycle that we had mutually beneficial goals. The WPF team wanted an enterprise-class application to showcase their framework and Visual Studio wanted a technology that could support a new, separated presentation architecture."
Continues Dias: "The WPF team even moved into offices adjacent to Building 41 so that we could work closer together."
A team of seven developers and five testers worked full time on WPF integration, collaborating with Visual Studio feature teams to validate the new design and verify component behavior in the environment. To mitigate risk, the group took a staged approach, first inserting a model/view separation before working to build view models specific to the new WPF skin. WPF code paths were turned off by default in the early daily builds, with only the WPF shell team working with the new presentation layer.
Once the WPF shell was ready, it was enabled for the entire team. "At this point we had two fully functional window managers and command-bar systems in the product, and we could switch between them to identify behavior differences and regressions," Dias says. Once this work was validated, the old code was simply removed.
"The 'point of no return' decision was made around December 2008," says Dias. "By that time, I'd say we had a very clear idea of what it would take to finish. We had cost projections from all teams, we had identified the high-risk components, and we had acknowledged our available safety valves, just in case."
Visual Studio wasn't the first Microsoft product to add a WPF layer over a native, Win32-based application. The Expression Suite, based on WPF 3.5, earned that honor. And that gave Dias and his crew a leg up on WPF integration. "If we found a bug or a critical feature missing in WPF 3.5, we could work with the WPF team to add it to 4.0."
Still, issues cropped up. Visual Studio, Dias says, makes heavy use of Handle to a Window (HWND) interoperability, with WPF chrome often hosting a Win32 window. Says Dias: "At the minimum, as long as we can get an HWND, we can host the design surface or tool window. With a mixture of WPF and HWNDs in the same UI, some tricky problems surface."
Dias singles out the "airspace" issue, where an HWND clips a WPF surface, as well as issues around the handling of focus and activation and how it flows between WPF and HWNDs. The challenge was particularly thorny in the way focus is restored when moving between window content and administrative UI elements like menus and toolbars. New features were baked into WPF 4 to address these issues.
Platform Plays
As it turned out, the heavy lifting on WPF 4 and MEF paid big dividends as Microsoft worked toward another goal of Visual Studio 2010: extending the reach of the IDE to new platforms, including Windows Azure, SharePoint Server and Silverlight 4.
Microsoft Product Manager Goodhew says the closely matched, XAML-based architectures of WPF 4 and Silverlight 4 enabled the Visual Studio team to quickly integrate new Silverlight 4 tooling and functionality.
Of course, Silverlight 4 support isn't present in the April 12 drop of Visual Studio 2010. The beta tooling found in Visual Studio 2010 beta 2 was excised when the rest of the IDE was packaged in RC form. Goodhew says developers will be able to add the Silverlight 4 tooling when Silverlight 4 ships, which could be in the first half of 2010.
That decision reflects the reality of complex software development.
"Sometimes -- as in this case -- things just don't align, and rather than doing a rushed release, it's better for everyone to focus on the end game," Goodhew says.
In the meantime, the Visual Studio group worked to update the Silverlight Designer in Visual Studio 2010 to better support data-centric application development -- an area where Silverlight tooling has fallen short. Goodhew singles out the new data-binding builder and improved IntelliSense, which reduce the need for hand-coding in data-centric Silverlight apps. Next up is a drag-and-drop binding interface in the Data Sources Windows, making it easier to build data scaffolding for service-driven Silverlight apps.
"Data-centric Silverlight applications typically rest on a services layer," Goodhew says. "The multitargeting feature of Visual Studio 2010 makes it easier to migrate the service layer of your Silverlight application to .NET 4 on your own schedule, but still work with the latest Silverlight release for your UI layer."
The Silverlight effort came with a host of challenges. Project system integration, designer features and rendering of Silverlight surfaces all vexed the Visual Studio development team. The hybrid, Web/client nature of Silverlight projects also created a bit of cognitive dissonance, says Goodhew. "The Web team had to teach the Web project system how to host and manipulate a client project, how to create XAP files, deploy them to the server, etc."
Bigger challenges cropped up in the WPF-based designer, which in Visual Studio 2008 treated Silverlight elements like WPF and created direct links to WPF dependencies. Rather than rewrite the designer or cross-compile it for Silverlight, the Visual Studio team created a platform-neutral abstraction layer with implementations for WPF and Silverlight.
On the rendering side, developers had to find a way to deal with the fact that Silverlight uses a subset of the .NET Framework. "Our designer runs against the larger desktop .NET runtime, and while we could talk to the smaller runtime inside a Silverlight instance, the communication was extremely slow and hard to program against," Goodhew explains.
Their solution? Work with the Blend and Silverlight teams to make it possible to host Silverlight on the full desktop version of the .NET runtime.
"We also had to invent an efficient way to compose Silverlight content with WPF content, because while the actual rendered view of the designer is Silverlight, the rest of the designer -- and Visual Studio itself -- is built on WPF," says Goodhew. He credits the Blend team for creating an abstraction layer that lets a single code base work with both WPF and Silverlight frameworks. It was an effort that took months.
SharePoint Adventures
A different set of challenges awaited the Visual Studio team as it worked to extend support to SharePoint. As was the case with Silverlight, the team benefited from work on earlier tooling -- in this case, the Visual Studio 2008 extensions for Windows SharePoint Services. The extensions were incomplete and lacked full integration with the Visual Studio IDE, but they produced a stream of feedback that helped guide development of the integrated tooling in Visual Studio 2010.
Jay Schmelzer, principal group program manager for Visual Studio Business Applications at Microsoft, says developers wanted a streamlined F5 build/deploy/debug experience, more flexible solutions packaging, and a flexible project system within Visual Studio, including the ability to create a single project that contains multiple types of SharePoint customizations within it.
Schmelzer describes SharePoint as "an ASP.NET application that doesn't look like a traditional ASP.NET application." The challenge for his team was to create a Visual Studio experience that would be consistent with ASP.NET development, while at the same time handling SharePoint-specific wire-up and deployment details behind the scenes.
Among the key challenges was the effort to get the 32-bit Visual Studio development environment to work against the 64-bit SharePoint platform, says Schmelzer. "We implemented an out-of-process Visual Studio host that handled the calls between Visual Studio and SharePoint," he explains.
Schmelzer credits a strong collaborative effort between the Visual Studio and SharePoint teams, which formed a series of cross-departmental virtual teams focusing on specific components of the dev effort. "It was a great partnership from the beginning," he says.
Goodhew agrees. "Extending Visual Studio to support new targets requires participation from many teams within Visual Studio, and it always reaches into more nooks and crannies than you first realize," he says. "It also requires participation from the folks working on the target platform."
The End of the Beginning
Mark Driver, research vice president at Gartner Inc., calls Visual Studio 2010 "probably the biggest change since .NET first came out." But he says the upcoming version of the IDE is only the beginning.
"I think a lot more is on the table. I think this is by no means an exhaustive delivery -- it's the tip of the iceberg," Driver says. "Don't get comfortable."
According to Microsoft's Dias, work on the rest of the iceberg is already underway. "We have a lot of block-and-tackle work we can do here to enable better experiences that target specific tasks," he says. "For example, I'd love to be able to open an instance of the shell on a second monitor and set it up as my debug experience."
He looks forward to Deep Zoom and overlays enabling developers to zoom out of a Solution to see the structure of the application and then overlay data about performance and unit test coverage. He also expects more work on immersive data visualization, especially in debugging scenarios.
For the moment, work on Visual Studio 2012 or 2015 or whatever it may end up being called can take a backseat to the actual final delivery of Visual Studio 2010 on April 12. As Mendlen notes, the Visual Studio development team engaged in a sustained, years-long effort to deliver one of the largest upgrades ever to its flagship IDE, only to run a last-minute sprint as it worked to iron out late performance and stability issues in the beta code.
"I'll tell you that we all held our breath probably from that day in early December to probably about two days after the RC drop," recalls Mendlen, who says he's going "someplace tropical" for vacation after the launch. "We got our first wave of RC feedback and we said, 'OK, that doesn't tell us much.' When we were two or three days into the twitter feeds, post-drop, we said, 'Ah, we did it. We're going to be OK.'"