Azure DevOps Team Speeds Up Git with New Scalar Project

Microsoft's Azure DevOps team announced Scalar, a new project to speed up the operations of Git, the popular, open source, distributed version control system commonly used with source code repository platforms like GitHub.

The team created the project to improve on its efforts associated with VFS for Git, its tool used to deal with the special problems encountered with huge projects, like Windows itself, described years ago as "The largest Git repo on the planet." VFS for Git uses a virtualized filesystem "to bypass many assumptions about repository size, enabling the Windows developers to use Git at a scale previously thought impossible."

In working with VFS for Git, Microsoft developers found performance bottlenecks in the Git client itself, and contributed to the development of the client to address those issues.

"Building on these contributions and many other recent improvements to Git, we began a project to support very large repositories without needing a virtualized filesystem," said Derrick Stolee, principal software engineer, Azure DevOps, in a Feb. 12 blog post. "Today we are excited to announce the result of those efforts – Scalar. Scalar accelerates your Git workflow, no matter the size or shape of your repository. And it does it in ways we believe can all make their way into Git, with Scalar doing less and Git doing much more over time."

The core of the performance problems with gigantic repos results from Git being a distributed version control system that keeps track of all files in the entire history. That can add hundreds of megabytes of data to even mid-sized repos, never mind Windows-sized projects for which core commands like git status, git add and git checkout can slow to a crawl, causing developers to switch contexts, which reportedly lowers productivity, Stolee said. And back in 2017, the Windows repo's code base was about 3.5 million files, creating a repo of about 300GB.

Scalar, developed on GitHub, is a .NET Core application coded in C# that improves the performance of Git commands via the setting of recommended config values and background maintenance operations.

"Scalar accelerates Git commands in your existing repositories, no matter what service you use to host those repositories," Stolee said. "All you need to do is register your biggest repositories with Scalar and then see how much faster your Git experience becomes."

This week's post delves deeply into the work behind the project and points to future planned Git contributions by Microsoft, as the company's plan is to infuse Scalar features into the Git client itself, obviating the need for a separate Scalar tool.

One planned work item is to replace Scalar's Watchman filesystem watcher -- which isn't aware of Git -- with a custom filesystem watcher in Git itself so the team can optimize that interface to fit Microsoft's needs.

Documentation for Scalar, including getting started guidance, a FAQ and more is available here.

About the Author

David Ramel is an editor and writer for Converge360.

comments powered by Disqus


  • AI for GitHub Collaboration? Maybe Not So Much

    No doubt GitHub Copilot has been a boon for developers, but AI might not be the best tool for collaboration, according to developers weighing in on a recent social media post from the GitHub team.

  • Visual Studio 2022 Getting VS Code 'Command Palette' Equivalent

    As any Visual Studio Code user knows, the editor's command palette is a powerful tool for getting things done quickly, without having to navigate through menus and dialogs. Now, we learn how an equivalent is coming for Microsoft's flagship Visual Studio IDE, invoked by the same familiar Ctrl+Shift+P keyboard shortcut.

  • .NET 9 Preview 3: 'I've Been Waiting 9 Years for This API!'

    Microsoft's third preview of .NET 9 sees a lot of minor tweaks and fixes with no earth-shaking new functionality, but little things can be important to individual developers.

  • Data Anomaly Detection Using a Neural Autoencoder with C#

    Dr. James McCaffrey of Microsoft Research tackles the process of examining a set of source data to find data items that are different in some way from the majority of the source items.

  • What's New for Python, Java in Visual Studio Code

    Microsoft announced March 2024 updates to its Python and Java extensions for Visual Studio Code, the open source-based, cross-platform code editor that has repeatedly been named the No. 1 tool in major development surveys.

Subscribe on YouTube