How Halo 5 implemented cloud scale telemetry using Azure Event Hubs

Publicado el 19 enero, 2016

Principal Program Manager, Hyper-scale Compute

Halo 5 Guardians

With over 65 million units sold, Halo is one of the most popular game franchises of all time. Halo 5: Guardians is a first-person shooting game for Xbox, which supports both single and multi-player gaming. Gaming Trend called Halo 5 the best Halo game so far and the reception has been strong. In the first week players collectively played 21 million hours of Halo 5, nine million in multiplayer mode.

Sophisticated telemetry is critical to delivering a quality gaming experience and the Halo team has a long track record of using telemetry and analytics to improve every aspect of the gaming experience from smooth in-game runtime to compelling story, characters and art. Halo performs both real-time and long-term analytics over the volumes of telemetry the game creates.

Among other uses, analytics are used to gain operational intelligence like match making quality, as well as input on how the game can be tuned better to delight its players. This telemetry powers the Halo website and APIs. Gathering and processing this much telemetry is a massive task. Halo ingests hundreds of billions of events on a given day with sustained peaks above one million requests per second.

Why Azure Event Hubs

Halo produces telemetry from within the game experience and from its runtime services. The challenge was to collect and use this telemetry without negatively impacting either the user experience or the Halo platform. The Halo team needed a highly scalable telemetry ingress capability that was easy to use, reliable to run, and could scale for the massive traffic and elastic throughput that a world-class multiplayer game requires. With a long history of working with the Azure Service Bus team, Halo chose to use Azure Event Hubs in their telemetry pipeline.

The Halo team considered using various distributed logging software including Apache Kafka, but decided on Azure Event Hubs in part because as a PaaS (Platform as a Service) offering, there is no runtime management or maintenance to be performed. This provided a smaller ramp up requirement because there was no need to train a team to setup and maintain the logging subsystem.

Critical to the Halo team, was the ability to process game events with sub-second latency, even when millions of players are engaging, and to never lose events. The distributed logging mechanism offer by Event Hubs was a great match for the scale and speed required by Halo.

Event Hubs offers ingress of millions of events per second with sub-second latency into a durable buffer that can be read many times by multiple consumers. This allows Halo to perform both fast and slow processing on their telemetry and derive maximum value from their data.

Next steps

If you’re dealing with telemetry for games, apps, or IoT – whether at small scale or large – try Azure Event Hubs using your free trial. And don't forget to get your copy of Halo 5: Guardians to for yourself what results deep telematics can provide.