News

How Visual Studio IntelliCode Helped Shape GitHub Copilot and Modern AI

• By David Ramel
• 10/16/2024

Microsoft has baked advanced GitHub Copilot tech into the latest edition of Visual Studio, but the aging IntelliCode IDE feature actually helped shape the modern experience.

That shaping comes in the form of UX paradigms that originated specifically with IntelliCode -- introduced in 2018 -- and later became AI standards. In other words, the way a user interacts with Copilot and other AI coding assistants is based on the way IntelliCode worked.

This little side story is just one part of a fascinating new read about the history of Visual Studio, IntelliCode and GitHub Copilot, an Oct. 16 post titled "How we build GitHub Copilot into Visual Studio."

"IntelliCode was built before open standards for leveraging AI models had gained traction so much of the integration is bespoke," the team said. "However, IntelliCode helped inform UX paradigms that would be generally adopted for future AI interactions such as 'ghost text' for completions and 'tab to accept' for repeated edits." Indeed, those UX mechanisms for AI functionality are now being used in other IDEs and editors also.

Otherwise, today's post showcases the behind-the-scenes issues and decisions that need to be made in complex integration scenarios like adding Copilot AI to Visual Studio and its existing IntelliCode functionality that pioneered the field of automatic code-completion and other developer help. Since they work differently, the team had to decide which way to approach the project.

For example, Copilot functionality comes in two arbitrary flavors, completions and chat, with the latter being part of a follow-on Copilot X project that facilitates much more sophisticated developer interaction including code generation, explanations, and more.

The Visual Studio team decided to focus on chat, which led to many more benefits that continue to improve the developer experience via AI.

"The Visual Studio team set out to deeply integrate 'chat' functionality into Visual Studio," the team said. "This started with basic features like a chat tool window and inline chat; however, many other features have subsequently been added which leverage this 'chat' functionality, such as rename suggestions, generating commit and PR messages, helping to diagnose exceptions, etc."

That required some engineering work, basically creating .NET 8 processes instead of some Node or older .NET Framework processes used by IntelliCode. Those Node processes -- outside of the devenv.exe process used by Visual Studio and also IntelliCode -- were created to help facilitate development across various Copilot clients like JetBrains, but don't work well in the Visual Studio architecture.

That choice, also, paid off in other dividends.

"This architecture has allowed us to leverage/influence many .NET improvements, for example: server-sent events, System.Text.Json to JSON schema mapping, tokenizers, various aspects of semantic kernel, and others. This includes the recently announced Microsoft.Extensions.AI.Preview."

Another benefit is that the out-of-process .NET components can help the team leverage and inform new framework types in .NET 8/9 that help with the creation of "intelligent" AI-powered apps.

Considering all that, the Visual Studio integration with Copilot seems to be working much better than the integration with Copilot and Visual Studio's IntelliSense, the non-machine-learning precursor to IntelliCode. Instead of machine learning tech leveraged by IntelliCode and Copilot, IntelliSense used static code analysis and language semantics, which caused issues as detailed in the articles:

• "Devs Cite Problems with GitHub Copilot and IntelliSense Working Together"
• "GitHub Copilot Makes Nice with Visual Studio IntelliSense"

Anyway, the team is promising more exciting goodies coming up.

"This is a bit of the history and high-level architecture of GitHub Copilot integration in Visual Studio," the team said at the end of the lengthy, detailed post. "AI is progressing exceedingly rapidly and building on .NET is one way that the VS team can keep pace. There will be many enhancements, changes, benefits, and surprises in upcoming VS and GitHub Copilot releases, so keep an eye out!"