News
Azure DevOps Team Speeds Up Git with New Scalar Project
Microsoft's Azure DevOps team announced Scalar, a new project to speed up the operations of Git, the popular, open source, distributed version control system commonly used with source code repository platforms like GitHub.
The team created the project to improve on its efforts associated with VFS for Git, its tool used to deal with the special problems encountered with huge projects, like Windows itself, described years ago as "The largest Git repo on the planet." VFS for Git uses a virtualized filesystem "to bypass many assumptions about repository size, enabling the Windows developers to use Git at a scale previously thought impossible."
In working with VFS for Git, Microsoft developers found performance bottlenecks in the Git client itself, and contributed to the development of the client to address those issues.
"Building on these contributions and many other recent improvements to Git, we began a project to support very large repositories without needing a virtualized filesystem," said Derrick Stolee, principal software engineer, Azure DevOps, in a Feb. 12 blog post. "Today we are excited to announce the result of those efforts – Scalar. Scalar accelerates your Git workflow, no matter the size or shape of your repository. And it does it in ways we believe can all make their way into Git, with Scalar doing less and Git doing much more over time."
The core of the performance problems with gigantic repos results from Git being a distributed version control system that keeps track of all files in the entire history. That can add hundreds of megabytes of data to even mid-sized repos, never mind Windows-sized projects for which core commands like git status, git add and git checkout can slow to a crawl, causing developers to switch contexts, which reportedly lowers productivity, Stolee said. And back in 2017, the Windows repo's code base was about 3.5 million files, creating a repo of about 300GB.
Scalar, developed on GitHub, is a .NET Core application coded in C# that improves the performance of Git commands via the setting of recommended config values and background maintenance operations.
"Scalar accelerates Git commands in your existing repositories, no matter what service you use to host those repositories," Stolee said. "All you need to do is register your biggest repositories with Scalar and then see how much faster your Git experience becomes."
This week's post delves deeply into the work behind the project and points to future planned Git contributions by Microsoft, as the company's plan is to infuse Scalar features into the Git client itself, obviating the need for a separate Scalar tool.
One planned work item is to replace Scalar's Watchman filesystem watcher -- which isn't aware of Git -- with a custom filesystem watcher in Git itself so the team can optimize that interface to fit Microsoft's needs.
Documentation for Scalar, including getting started guidance, a FAQ and more is available here.
About the Author
David Ramel is an editor and writer at Converge 360.