{"id":54429,"date":"2026-10-12T06:00:00","date_gmt":"2026-10-12T13:00:00","guid":{"rendered":"https:\/\/azure.microsoft.com\/en-us\/blog\/?p=54429"},"modified":"2026-10-12T06:43:44","modified_gmt":"2026-10-12T13:43:44","slug":"deep-dive-into-azure-maia-300-architecture","status":"publish","type":"post","link":"https:\/\/azure.microsoft.com\/en-us\/blog\/deep-dive-into-azure-maia-300-architecture\/","title":{"rendered":"Deep dive into Azure Maia 300 architecture"},"content":{"rendered":"\n

From its inception, Azure Maia has been built around a single architectural philosophy: optimize the entire system for LLM inference so that every watt and every dollar delivers more intelligence. Now in its third generation<\/a>, Maia 300 is the fullest expression of that philosophy. Maia 300 is built as a complete system, with software, silicon, physical rack infrastructure, networking, and datacenters co-designed to operate as a single platform to meet the demands of cloud-scale agentic inference.<\/p>\n\n\n\n

\n
Learn more about Maia 300 here<\/a><\/div>\n<\/div>\n\n\n\n

That system-level approach is increasingly necessary as agentic workloads, including multi-step execution, long-context decoding, and tool use, change the computational profile and arithmetic intensity of inference. Meeting these demands requires sustaining locality, supplying compute with enough memory capacity and bandwidth, minimizing data movement, scaling communication efficiently, and fitting the most capability within a datacenter\u2019s fixed power and cooling envelope.<\/p>\n\n\n\n

At Microsoft, we see this transformation firsthand through the products and platforms our customers use every day, from Microsoft Copilot<\/a> and GitHub Copilot<\/a> to Microsoft Foundry. Operating these workloads at scale gives us a unique view into how AI is being adopted across industries and the infrastructure needed to support the next generation of AI applications and agents. Maia 300 is designed to maximize useful tokens per watt and useful tokens per dollar, bringing more capacity, capability, and performance to these products and Microsoft\u2019s broader AI portfolio for our customers.<\/p>\n\n\n\n

Energy-efficient inference through software-defined dataflow<\/h2>\n\n\n\n

Maia 200 introduced a new class of accelerator architecture called Software Defined Locally Accessed Dataflow Architecture<\/a>, or SDLA. It begins with the observation that efficient data management, including storing, moving and converting data, is one of the central challenges in modern AI systems. The Maia programmable dataflow architecture separates control path programming from data path execution and allows software to explicitly coordinate data placement, movement, conversion and synchronization across specialized local memories, DMA engines, compute engines, and the network.<\/p>\n\n\n

\"Diagram\n\t\t\t\n\t\t\t\t\n\t\t\t<\/svg>\n\t\t<\/button>
Maia 300 system-on-chip architecture<\/figcaption><\/figure>\n\n\n\n

Maia 300 evolves this architecture for long-context, reasoning-intensive, and decode-heavy inference. First, memory moves from the uniform memory access (UMA) organization of the SoC to a locality-aware non-uniform (NUMA)<\/a> organization across chiplets, so software can place data near the compute that uses it. Second, new narrow-precision formats with hardware-accelerated data type conversion reduce the bytes moved for each token. Third, the rack doubles to 96 accelerators and the scale-up domain grows to a superpod of 12,288 accelerators, enabling more accelerators to work effectively as a single system.<\/p>\n\n\n\n

Rather than scaling performance through increased power and silicon area alone, Maia 300 improves performance per watt through a set of architectural innovations that optimize data movement, locality, precision, and memory efficiency across the system:<\/p>\n\n\n\n