Keyboardless Coding? GitHub Copilot 'AI Pair Programmer' Gets Voice Commands -- Visual Studio Magazine

Keyboardless Coding? GitHub Copilot 'AI Pair Programmer' Gets Voice Commands

By David Ramel
11/15/2022

Got a good microphone? As predicted here many months ago, the GitHub Copilot "AI pair programmer" coding assistant has introduced voice commands, experimental at first.

" 'Hey, GitHub!' enables voice-based interaction with GitHub Copilot, enabling the benefits of an AI pair programmer while reducing the need for a keyboard," Microsoft-owned GitHub said last week in a recap of the company's GitHub Universe 2022 event.

This was predicted here in May during Microsoft's Build developer conference in the Visual Studio Magazine article, "Microsoft Build Conference Heralds Era of AI-Assisted Software Development." That article said: "Heads are still exploding as Copilot, starting with relatively simple IntelliSense-like code completion -- even whole-line code completion -- has been improved to the point where normal-language typed commands (voice is surely coming) can create entire projects such as simple games."

**[Click on image for larger view.]** Turning Words into Code *(source: OpenAI).*

GitHub Copilot shook up the software development space last year when it debuted with cutting-edge AI natural language processing tech that goes way beyond IntelliSense -- and even Microsoft's improvement on that, IntelliCode -- and which can be leveraged in the Visual Studio IDE and Visual Studio Code editor. The new voice functionality only works in the latter at this early stage.

GitHub Copilot's capabilities are possible because of Codex, a machine learning model that can translate natural language commands into code in more than a dozen programming languages, created by AI research/development company and Microsoft partner OpenAI. A May 25 video shows a programmer using natural language commands with Codex to easily and quickly build 3D scenes by translating the commands into Babylon.js code (a 3D renderer that runs in the browser), with commands like "add teal spheres above and below the cube" and then "make the cube spin."

**[Click on image for larger view.]** Using Codex Natural Language to Build 3D Scenes *(source: Microsoft).*

GitHub has continually improved the product through a long preview and many updates, with the voice command experiment the latest.

**[Click on image for larger view.]** 'Hey GitHub' in Action *(source: Microsoft).*

In addition to generating code for line completion, voice commands can also be used for navigation ("go to line 34"), controlling an IDE ("run the program") and code summarization, wherein a developer can ask GitHub Copilot to explain lines 3-10 and get a summary of what the code does, for example.

"If GitHub Copilot is our pair programmer, why can't we talk to it?" GitHub said last week. "That's exactly what the GitHub Next team is working towards. 'Hey, GitHub!' enables voice-based interaction with GitHub Copilot and more. With the power of your voice, we're excited about the potential to bring the benefits of GitHub Copilot to even more developers, including developers who have difficulty typing using their hands. 'Hey, GitHub!' only reduces the need for a keyboard when coding within VS Code for now, but we hope to expand its capabilities through further research and testing."

To take part in the experiment, developers can apply to join the waitlist.

About the Author

David Ramel is an editor and writer at Converge 360.

Printable Format

comments powered by Disqus

Featured

Compare New GitHub Copilot Free Plan for Visual Studio/VS Code to Paid Plans

The free plan restricts the number of completions, chat requests and access to AI models, being suitable for occasional users and small projects.
Diving Deep into .NET MAUI

Ever since someone figured out that fiddling bits results in source code, developers have sought one codebase for all types of apps on all platforms, with Microsoft's latest attempt to further that effort being .NET MAUI.
Copilot AI Boosts Abound in New VS Code v1.96

Microsoft improved on its new "Copilot Edit" functionality in the latest release of Visual Studio Code, v1.96, its open-source based code editor that has become the most popular in the world according to many surveys.
AdaBoost Regression Using C#

Dr. James McCaffrey from Microsoft Research presents a complete end-to-end demonstration of the AdaBoost.R2 algorithm for regression problems (where the goal is to predict a single numeric value). The implementation follows the original source research paper closely, so you can use it as a guide for customization for specific scenarios.
Versioning and Documenting ASP.NET Core Services

Building an API with ASP.NET Core is only half the job. If your API is going to live more than one release cycle, you're going to need to version it. If you have other people building clients for it, you're going to need to document it.