At JetBrains, we build TeamCity, a powerful continuous integration server for Java, .NET and many other platforms. A build server is a typical example of variable workload: on any given day, having 2 build agents running may be sufficient. Other days, for example near release dates, we would need 10 build agents to be able to cope with the load of compiling, testing and deploying our applications. That’s why we have built the TeamCity Azure plugin: it lets TeamCity provision and de-provision build agents automatically, reducing the running cost of our build farm while having optimal capacity for fast developer feedback.
After installing the plugin on our TeamCity server, we can add a new Microsoft Azure cloud profile and pass in a management certificate and the subscription ID in which we want to have TeamCity create build agents. Next, we can provide some options, like selecting the virtual machine image we want TeamCity to provision as well as the virtual machine size. We can have it launch any virtual machine size supported by the subscription: from an Extra Small machine with few resources to a high-end D-series machine where we can run builds of SSD’s. Build agents can be either Windows or Linux.
Once saved, TeamCity will first try starting builds on a non-cloud agent. If there are no regular agents available, TeamCity finds a virtual machine or virtual machine image with a compatible agent and starts it on Azure. TeamCity ensures that the configured maximum number of instances is never exceeded and the virtual machines are shutdown properly to reduce cost.
Since TeamCity is built in Java, we had to pick an approach of communicating with the Microsoft Azure Management API. We could either write raw REST calls against it, or build the TeamCity Azure plugin using the Microsoft Azure SDK for Java, a ready-to-go client library written in Java. We decided to go with the Microsoft Azure SDK for Java, as it can easily be consumed from Maven and encapsulates all the REST internals. It’s also completely open source, which makes it easy to look under the hood or extend it with additional functionality.
Building on top of the Microsoft Azure SDK for Java was a nice experience. Every action that we can do from the Microsoft Azure management portals can be done using the SDK. We use it to start and stop existing virtual machines, as well as create and start new virtual machines from a virtual machine image making it possible to create as many build agents as needed. As an added bonus, the Microsoft Azure SDK for Java provides asynchronous methods for long-running operations, never blocking any calls to the management API.
We’ve open sourced the plugin on GitHub, making it possible to contribute or simply look at how we have implemented things. Since we’re using the Microsoft Azure SDK for Java the plugin can be extended with many more features like virtual networking support and so on. If you want to give the TeamCity Azure plugin a try, we have a free edition of TeamCity available for download. It’s recommended to have both the TeamCity server as well as the agents hosted on Microsoft Azure.