.NET infrastructure team invests in consistency to accelerate open-source innovation
Watch how shared tooling and a single, shared CI system across GitHub repositories deliver big gains in developer productivity.
The challenge: Streamlining collaboration for open-source contributors
As an independent organization created to foster innovation through open development and collaboration, the .NET Foundation supports a large and complex set of open-source projects, including .NET Core. The Microsoft .NET Engineering Services team contributes to the project and is responsible for much of the infrastructure that enables contributors from around the world to work together.
With dozens of different git repositories on GitHub, a broad range of tools, and several different continuous integration (CI) systems in use across the project, there was a lot of confusion that inhibited productivity. The sheer scale of the project added to the complexity. For example, the Roslyn (C# compiler) repo alone runs more than 600,000 automated tests for one iteration of a pull request. With more than 50 pull requests per week, each with multiple iterations, the number of CI tests is in the billions. For CoreFx (libraries), the numbers are even higher. As the .NET Engineering Services team planned for the .NET Core 3.0 release, they decided to make big changes to establish greater consistency and enable contributors to realize the full potential of the project.
The challenge: Adapting to a cloud operating model
"A distributed repo structure and tooling makes it a lot harder for developers to move from one repo to another … And if it's this hard on Microsoft developers, how can we expect most individual contributors to figure it out?"
Matt Mitchell, Principal Software Engineer, .NET Engineering Services
Improving productivity through shared tooling and a single CI system
To make it easier for developers to move between different parts of the software stack, the .NET Engineering Services team worked to bring all repos under a common directory structure, set of commands, and build-and-test logic. The team eliminated further barriers to productivity by moving all existing workflows from the different CI systems into a single system in Azure Pipelines. Because the system is fully managed, they no longer have the burden of managing the CI server infrastructure. The team is now practicing configuration as code through YAML files that are stored in GitHub, and YAML pipelines are used everywhere. Whereas some repos had previously done CI serially—requiring developers to sometimes wait two or three hours between iterations on their code—all CI pipelines are now parallelized to enable faster iterations.
The challenge: Adapting to a cloud operating model
"Now that we longer need to worry about the operational aspects of CI, we're free to focus on further improvements that will be felt by all—including Microsoft developers and individual contributors alike."
Matt Mitchell, Principal Software Engineer, .NET Engineering Services
Better collaboration makes a better product
Today, developers can more easily move from repo to repo, track down the root cause of a bug, and propose and test a fix—all without nearly as much specialized knowledge as they needed in the past. This streamlined open-source collaboration in GitHub enables developers to innovate more quickly as they work together to write, test, build, and ship high-quality software. And because the .NET Engineering Services team can make infrastructure changes across the full stack, they're able to deliver tooling improvements that developers can take advantage of across all repos much more quickly and efficiently.
