News

Visual Studio IntelliCode Still Among Top AI Code Assistants

• By David Ramel
• 06/26/2024

In the age of GitHub Copilot, ChatGPT, Google Gemini and all the rest, one of the most-used AI coding assistants is still the venerable IntelliCode feature of Microsoft's Visual Studio IDE, whose six-year-old tech now seems positively ancient.

A recent survey from Stack Overflow, probably the go-to help resource for devs, revealed this surprising finding when examining code assistants as part of its Stack Overflow Knows series, this one focusing on "Developers and AI at work."

While the report is general in nature, addressing several aspects of the usage of CodeGen tools (its TLDR was "Developers are highly satisfied with AI tools that are easy to use and make them feel productive, but inaccuracy and understanding complexity remain challenges"), the surprising popularity of IntelliCode is of particular interest to Visual Studio Magazine readers.

"We found that most of those using code assistant tools report that these assistants are satisfying and easy to use and a majority (but not all) are on teams where half or more of their coworkers are using them, too," SO said. "These tools may not always be answering queries accurately or solving contextual or overly specific problems, but for those that are adopting these tools into their workflow, code assistants offer a way to increase the quality of time spent working."

When asked what coding assistants they used, professional developers overwhelmingly reported ChatGTP (84 percent) followed by GitHub Copilot (49 percent), with IntelliCode unexpectedly taking the bronze medal (albeit with a big drop off in respondents at 11 percent).

SO discussed how cost and experience influence the choice of code assistants, including the baked-in IntelliCode.

"There are more code assistant tools being released everyday, though two are dominating the space so far: ChatGPT and GitHub Copilot," SO said. "Professional developers and those learning to code are equally likely to be using ChatGPT, but those learning are less likely to be using GitHub Copilot (29 percent vs. 49 percent). Given that ChatGPT offers a popular free option and GitHub Copilot offers a time-limited free trial, this makes sense. Visual Studio IntelliCode is more popular with those learning (16 percent vs. 11 percent), as there is a free version of this IDE for those not using an enterprise license."

Still, the numbers are quite similar for pros and learners.

While IntelliCode, introduced in 2018 (see "Microsoft Brings AI to Visual Studio with IntelliCode"), is showing its age after OpenAI changed everything with ChatGPT, Microsoft has been keeping the tool updated even as it courted OpenAI and leveraged its tech for GitHub Copilot (GitHub is owned by Microsoft) and the subsequent generation of Copilots of all kinds that is changing the very DNA of Microsoft's software products and services.

For example, in April 2023 Microsoft infused IntelliCode with deep learning tech (see "Visual Studio IntelliCode AI Assistant Gets Deep Learning Upgrade"). Later that year, that effort paid off for Python programmers (see "IntelliCode Advances with First Deep Learning Model for Python in VS Code").

Even before those updates, IntelliCode impressed developers with its pre-Gen AI capabilities (see "Exploring the 'Almost Creepy' AI Engine in Visual Studio 2022").

Even so, it seems logical that tech based on the large language model (LLM) Gen AI approach will become baked in completely to the IDE in some form or another, with some name or another. The same might happen for VS Code, whose IntelliCode extension has been installed some 43.5 million times.

ChatGPT compared and contrasted IntelliCode with the new wave of Gen AI tools:

Microsoft's IntelliCode is an AI-assisted coding tool integrated into Visual Studio and Visual Studio Code. It leverages machine learning models trained on thousands of high-quality open-source GitHub repositories to provide context-aware code suggestions, such as auto-completions, refactoring recommendations, and coding style adherence.

Relationship to Generative AI (e.g., ChatGPT)

• Generative AI (ChatGPT): Uses LLMs, specifically GPT (Generative Pre-trained Transformer), to generate human-like text based on massive datasets.
• IntelliCode: Employs machine learning models trained on code-specific datasets. It doesn't generate text in the same way as GPT but instead uses predictive models to suggest code completions and improvements.

Differences in Backing Technologies

ChatGPT and AI Coding Assistants:

• LLMs: These models, such as GPT-4, are trained on vast amounts of textual data across various domains. They generate coherent text by predicting the next word in a sequence.
• Transformers Architecture: The architecture allows for the handling of long-range dependencies in text, making it suitable for complex text generation tasks.

Microsoft IntelliCode:

• Custom Machine Learning Models: IntelliCode uses models trained specifically on high-quality code repositories, focusing on coding patterns and best practices.
• Training Data: Primarily code-centric datasets, which help the models learn coding conventions, common patterns, and context-specific recommendations.

In essence, while both technologies utilize AI to enhance productivity, their core methodologies and applications differ significantly. ChatGPT's generative capabilities are broader and more text-focused, whereas IntelliCode's predictive models are specialized for code enhancement and developer assistance.

Meanwhile, in examining the current state of AI coding assistants, SO's report sports data points including:

• The majority of respondents (76 percent) let us know they are using or are planning to use AI code assistants.
• Some roles use these tools more than others amongst professional developers: Academic researchers (87 percent), AI developers (76 percent), frontend developers (75 percent), mobile developers (60 percent), and data scientists (67 percent) currently use code assistants the most.
• Other roles indicated they are using code assistants (or planning to) much less than average: data/business analysts (29 percent), desktop developers (39 percent), data engineers (39 percent), and embedded developers (42 percent).
• The tools that are more satisfying to use also rank high for being easy to use: the top-box scores for easiest to use are Codeium (84 percent), GitHub Copilot (76 percent) and ChatGPT (61 percent) respectively and the same three in order were ranked for top-box satisfaction scores (86 percent, 72 percent, 65 percent respectively).

  

• 38 percent of developers report code assistants provide inaccurate information half of the time or more.
• With the time saved by using coding assistants, the largest plurality of developers (27-29 percent dependent upon adoption rate) reported they do "More high-level strategic work projects."

"The nature of working as a developer is complex and dynamic," SO concluded. "Even if productivity isn't yet clearly articulated as a business KPI, developers are highly satisfied with AI tools that are easy to use and make them feel productive, and teams are slowly beginning to incorporate these tools into their workflows.

"Complexity and inaccuracy remain challenges when it comes to widespread adoption and use of code assistants, but perhaps developers on teams with lower adoption rates may actually be using the time they gain from coding assistants to engage in more creative work. This also could be a turning point where the long-term investment of an elevated experience at work becomes more important than near-term improvements on productivity metrics."

The report, published May 29, is based on a survey of more than 1,700 people in the SO community.