Today we’re previewing the Azure HPC Cache service, a new Azure offering that empowers organizations to more easily run large, complex high-performance computing (HPC) workloads in Azure. Azure HPC Cache reduces latency for applications where data may be tethered to existing data center infrastructure because of dataset sizes and operational scale.
Scale your HPC pipeline using data stored on-premises or in Azure. Azure HPC Cache delivers the performant data access you need to be able to run your most demanding, file-based HPC workloads in Azure, without moving petabytes of data, writing new code, or modifying existing applications.
For users familiar with the Avere vFXT for Azure application available through the Microsoft Azure Marketplace, Azure HPC Cache offers similar functionality in a more seamless experience—meaning even easier data access and simpler management via the Azure Portal and API tools. The service can be driven with Azure APIs and is proactively monitored on the back end by the Azure HPC Cache support team and maintained by Azure service engineers. What is the net benefit? The Azure HPC Cache service delivers all the performance benefits of the Avere vFXT caching technology at an even lower total cost of ownership.
Azure HPC Cache works by automatically caching active data in Azure that is located both on-premises and in Azure, effectively hiding latency to on-premises network-attached storage (NAS), Azure-based NAS environments using Azure NetApp Files or Azure Blob Storage. The cache delivers high-performance seamless network file system (NFSv3) access to files in the Portable Operating System Interface (POSIX) compliant directory structures. The cache can also aggregate multiple data sources into an aggregated name space to present a single directory structure to clients. Azure compute clients can then access data as though it all originated on a single NAS filer.
Ideal for cloud-bursting applications or hybrid NAS environments, Azure HPC Cache lets you keep your data on existing datacenter-resident Azure NetApp or Dell EMC Isilon arrays. Whether you need to store data on premises while you develop your cloud strategy for security and compliance reasons, or because you simply have so much data on-premises that you don’t want to move it, you can still take full advantage of Azure compute services and do it sooner, rather than later. Once you are ready or able to shift data to Azure Storage resources, you can still run file-based workloads with ease. Azure HPC Cache provides the performance you need to lift and shift your pipeline.
To the cloud in days, not months
Combined with other Azure services such as the Azure HB- and HC-series virtual machines (VMs) for HPC and the Azure CycleCloud HPC workload manager, Azure HPC Cache lets you quickly reproduce your on-premise environment in the cloud and access on-premise data without committing to a large-scale migration. You can also expect to run your HPC workloads in Azure at performance levels similar to your on-premises infrastructure.
Azure HPC Cache service is easy to initiate and manage from the Azure Portal. Once your network has been set up and your on-premises environment has IP connectivity to Azure, you can typically turn on Azure HPC Cache service in about ten minutes. Imagine being able to do HPC jobs in days rather than waiting for months while your IT team fine-tunes data migration strategies and completes all required data moves and synchronization processes.
From burst to all-in: Your choice, your pace
The high-performance Azure HPC Cache delivers the scale-out file access required by HPC applications across an array of industries, from finance to government, life sciences, manufacturing, media, and oil and gas. The service is ideally suited for read-heavy workloads running on 1,000 to 50,000 compute cores. Because Azure HPC Cache is a metered service with usage charges included on your Azure bill, you can turn it off—and stop the meter—when you’re done.
Azure HPC Cache helps HPC users access Azure resources more simply and economically. You can deliver exactly the performance needed for computationally intensive workloads, in time to meet demand. Start by using Azure capacity for short-term demand, and enabling a hybrid NAS environment, or go all-cloud and make Azure your permanent IT infrastructure. Azure HPC Cache provides the seamless data access you need to leverage cloud resources in a manner and at a pace that suits your unique business needs and use cases.
Proven technology maintained by Azure experts
Azure HPC Cache service is the latest innovation in a continuum of high-performance caching solutions built on Avere Systems FXT Edge Filer foundational technology. Who uses this technology? A diverse, global community that includes post-production studio artists in the UK, weather researchers in Poland, animators in Toronto, investment bankers in New York City, bioinformaticists in Cambridge and Switzerland, and many, many more of the world’s most demanding HPC users. Azure HPC Cache combines this most sought-after technology with the technical expertise and deep-bench support of the Microsoft Azure team.
Can’t wait to try it?
Ready to get off the sidelines and start running your HPC workloads in Azure? We have a few opportunities for customers to preview Azure HPC Cache. Just complete a short survey, and we’ll review your submission for suitability.
The Azure HPC Cache team is committed to helping deliver on Microsoft’s “Cloud for all” mission and will work with you to design a cloud that you can use to quickly turn your ideas into solutions. Have questions? Email them to AzureHPCCache@microsoft.com.