Hybrid HPC in Azure with HPC Pack
Microsoft HPC Pack is a free high performance computing (HPC) solution built on Microsoft Azure and Windows Server technologies. HPC Pack combines a comprehensive set of deployment, administration, job scheduling, and monitoring tools for your Windows and Linux HPC cluster environment, providing a flexible platform for developing and running HPC applications on premises and in Azure.
This solution shows the process for using HPC Pack to create a hybrid (on-premises and Azure) HPC environment.
Log into on-premises head node
Add Azure compute nodes to the cluster
Start the compute nodes
Submit jobs to the cluster
HPC Pack sends jobs to on-premises and Azure nodes based upon the node group selected
Monitor job progress
Stop the compute nodes or configure auto-scaling
- 1 Log into on-premises head node
- 2 Add Azure compute nodes to the cluster
- 3 Start the compute nodes
- 4 Submit jobs to the cluster
- 5 HPC Pack sends jobs to on-premises and Azure nodes based upon the node group selected
- 6 Monitor job progress
- 7 Stop the compute nodes or configure auto-scaling
Related solution architectures
Big compute and high performance computing (HPC) workloads are normally compute intensive and can be run in parallel, taking advantage of the scale and flexibility of the cloud. The workloads are often run asynchronously using batch processing, with compute resources required to run the work and job scheduling required to specify the work. Examples of Big Compute and HPC workloads include financial risk Monte Carlo simulations, image rendering, media transcoding, file processing, and engineering or scientific simulations.
High performance computing (HPC) applications can scale to thousands of compute cores, extend on-premises big compute, or run as a 100% cloud native solution. This HPC solution including the head node, compute nodes, and storage nodes, runs in Azure with no hardware infrastructure to maintain.