Unleashing innovation: The new era of compute powering Azure AI solutions

As AI continues to transform industries, Microsoft is expanding its global cloud infrastructure to meet the needs of developers and customers everywhere. At Microsoft Build 2024, we’re unveiling our latest progress in developing tools and services optimized for powering your AI solutions. Microsoft’s cloud infrastructure is unique in how it provides choice and flexibility in performance and power for customers’ unique AI needs, whether that’s doubling deployment speeds or lowering operating costs.

That’s why we’ve enhanced our adaptive, powerful, and trusted platform with the performance and resilience you’ll need to build intelligent AI applications. We’re delivering on our promise to support our customers by providing them with exceptional cost-performance in compute and advanced generative AI capabilities.

Powerful compute for general purpose and AI workloads

Microsoft has the expertise and scale to run the AI supercomputers that power some of the world’s biggest AI services, such as Microsoft Azure OpenAI Service, ChatGPT, Bing, and more. Our focus as we continue to expand our AI infrastructure is on optimizing performance, scalability, and power efficiency.

Microsoft takes a systems approach to cloud infrastructure, optimizing both hardware and software to efficiently handle workloads at scale. In November 2023, Microsoft introduced its first in-house designed cloud compute processor, Azure Cobalt 100, which enables general-purpose workloads on the Microsoft Cloud. We are announcing the preview of Azure virtual machines built to run on Cobalt 100 processors. Cobalt 100-based virtual machines (VMs) are Azure’s most power efficient compute offering, and deliver up to 40% better performance than our previous generation of Arm-based VMs. And we’re delivering that same Arm-based performance and efficiency to customers like Elastic, MongoDB, Siemens, Snowflake, and Teradata. The new Cobalt 100-based VMS are expected to enhance efficiency and performance for both Azure customers and Microsoft products. Additionally, IC3, the platform that powers billions of customer conversations in Microsoft Teams, is adopting Cobalt 100 to serve its growing customer base more efficiently, achieving up to 45% better performance on Cobalt 100 VMs.

We’re combining the best of industry and the best of Microsoft in our AI infrastructure. Alongside our custom Azure Cobalt 100 and Maia series and silicon industry partnerships, we’re also announcing the general availability of the ND MI300X VM series, where Microsoft is the first cloud provider to bring AMD’s most powerful Instinct MI300X Accelerator to Azure. With the addition of the ND MI300X VM combining eight AMD MI300X Instinct accelerators, Azure is delivering customers unprecedented cost-performance for inferencing scenarios of frontier models like GPT-4. Our infrastructure supports different scenarios for AI supercomputing, such as building large models from scratch, running inference on pre-trained models, using model as a service providers, and fine-tuning models for specific domains.

One of Microsoft’s advantages in AI is our ability to combine thousands of virtual machines with tens of thousands of GPUs with the best of InfiniBand and Ethernet based networking topologies for supercomputers in the cloud that can run large scale AI workloads to lower costs. With a diversity of silicon across AMD, NVIDIA, and Microsoft’s Maia AI accelerators, Azure’s AI infrastructure delivers the most complete compute platform for AI workloads. It is this combination of advanced AI accelerators, datacenter designs, and optimized compute and networking topology that drive cost efficiency per workload. That means whether you use Microsoft Copilot or build your own copilot apps, the Azure platform ensures you get the best AI performance with optimized cost.

Microsoft is further extending our cloud infrastructure with the Azure Compute Fleet, a new service that simplifies provisioning of Azure compute capacity across different VM types, availability zones, and pricing models to more easily achieve desired scale, performance, and cost by enabling users to control VM group behaviors automatically and programmatically. As a result, Compute Fleet has the potential to greatly optimize your operational efficiency and increase your core compute flexibility and reliability for both AI and general-purpose workloads together at scale.

AI-enhanced central management and security

As businesses continue to expand their computing estate, managing and governing the entire infrastructure can become overwhelming. We keep hearing from developers and customers that they spend more time searching for information and are less productive. Microsoft is focused on simplifying this process through AI-enhanced central management and security. Our adaptive cloud approach takes innovation to the next level with a single, intelligent control plane that spans from cloud to edge, making it easier for customers to manage their entire computing estate in a consistent way. We’re also aiming to improve your experience with managing these distributed environments through Microsoft Copilot in Azure.

We created Microsoft Copilot in Azure to act as an AI companion, helping your teams manage operations seamlessly across both cloud and edge environments. By using natural language, you can ask Copilot questions and receive personalized recommendations related to Azure services. Simply ask, “Why is my app slow?” or “How do I fix this error?” and Copilot will navigate a customer through potential causes and fixes.

Starting today, we will be opening the preview of Copilot in Azure to all customers over the next couple of weeks. With this update, customers can choose to have all their users access Copilot or grant access to specific users or groups within a tenant. With this flexibility to manage Copilot, you can tailor your approach and control which groups of users or departments within your organization have access to it. You can feel secure knowing you can deploy and use the tool in a controlled manner, ensuring it aligns with your organization’s operational standards and security policies.

We’re continually enhancing Copilot and making the product better with every release to help developers be more productive. One of the ways we’ve simplified the developer’s experience is by making databases and analytics services easier to configure, manage, and optimize through AI-enhanced management. Several new skills are available for Azure Kubernetes Service (AKS) in Copilot for Azure that simplify common management tasks, including the ability to configure AKS backups, change tiers, locate YAML files for editing, and construct kubectl commands.

We’ve also added natural language to SQL conversion and self-help for database administration to support your Azure SQL database-driven applications. Developers can ask questions about their data in plain text, and Copilot generates the corresponding T-SQL query. Database administrators can independently manage databases, resolve issues, and learn more about performance and capabilities. Developers benefit from detailed explanations of the generated queries, helping them write code faster.

Lastly, you’ll notice a few new security enhancements to the tool. Copilot now includes Microsoft Defender for Cloud prompting capabilities to streamline risk exploration, remediation, and code fixes. Defender External Attack Surface Management (EASM) leverages Copilot to help surface risk-related insights and convert natural language to corresponding inventory queries across data discovered by Defender EASM. These features make database queries more user-friendly, enabling our customers to use natural language for any related queries. We’ll continue to expand Copilot capabilities in Azure so you can be more productive and focused on writing code.

Cloud infrastructure built for limitless innovation

Microsoft is committed to helping you stay ahead in this new era by giving you the power, flexibility, and performance you need to achieve your AI ambitions. Our unique approach to cloud and AI infrastructure helps us and developers like you meet the challenges of the ever-changing technological landscape head-on so you can continue working efficiently while innovating at scale.

Discover new ways to transform with AI

• Learn how Azure helps build AI experiences
• Read more about AI-powered analytics

Key Microsoft Build sessions

• BRK126: Adaptive cloud approach: Build and scale apps from cloud to edge
• BRK124: Building AI applications that leverage your data in object storage
• BRK 129: Building applications at hyper scale with the latest Azure innovations
• BRK133: Unlock potential on Azure with Microsoft Copilot
• BRK127: Azure Monitor: Observability from code to cloud