{"id":36784,"date":"2024-10-02T09:00:00","date_gmt":"2024-10-02T16:00:00","guid":{"rendered":"https:\/\/azure.microsoft.com\/en-us\/blog\/?p=36784"},"modified":"2024-10-02T10:31:49","modified_gmt":"2024-10-02T17:31:49","slug":"microsoft-launches-latest-azure-virtual-machines-optimized-for-ai-supercomputing-the-nd-h200-v5-series","status":"publish","type":"post","link":"https:\/\/azure.microsoft.com\/en-us\/blog\/microsoft-launches-latest-azure-virtual-machines-optimized-for-ai-supercomputing-the-nd-h200-v5-series\/","title":{"rendered":"Microsoft launches latest Azure virtual machines optimized for AI supercomputing, the ND H200 v5 series\u00a0"},"content":{"rendered":"\n

The need for scalable and high-performance infrastructure continues to grow exponentially as the AI landscape advances. Our customers rely on Azure AI infrastructure to develop innovative AI-driven solutions, which is why we are delivering new cloud-based AI-supercomputing clusters built with Azure ND H200 v5 series virtual machines (VMs) today. These VMs are now generally available and have been tailored to handle the growing complexity of advanced AI workloads, from foundational model training to generative inferencing. The scale, efficiency and enhanced performance of our ND H200 v5 VMs are already driving adoption from customers and Microsoft AI services such as Azure Machine Learning and Azure OpenAI Service.<\/p>\n\n\n\n

\n

“We’re excited to adopt Azure’s new H200 VMs. We’ve seen that H200 offers improved performance with minimal porting effort, we are looking forward to using these VMs to accelerate our research, improve the ChatGPT experience, and further our mission.\u201d<\/em> \u2014Trevor Cai, head of infrastructure, OpenAI.<\/p>\n<\/blockquote>\n\n\n\n

The Azure ND H200 v5 VMs are architected with Microsoft’s systems approach to enhance efficiency and performance, and feature eight NVIDIA H200 Tensor Core GPUs. Specifically, they address the gap due to GPUs growing in raw computational capability at a much faster rate than the attached memory and memory bandwidth. The Azure ND H200 v5 series VMs deliver a 76% increase in High Bandwidth Memory (HBM) to 141GB and a 43% increase in HBM Bandwidth to 4.8 TB\/s over the previous generation of Azure ND H100 v5 VMs. This increase in HBM bandwidth enables GPUs to access model parameters faster, helping reduce overall application latency, which is a critical metric for real-time applications such as interactive agents. The ND H200 V5 VMs can also accommodate more complex Large Language Models (LLMs) within the memory of a single VM, improving performance by helping users avoid the overhead of running distributed jobs over multiple VMs. <\/p>\n\n\n\n