Big compute with Azure Batch

Big compute and high performance computing (HPC) workloads are normally compute intensive and can be run in parallel, taking advantage of the scale and flexibility of the cloud. The workloads are often run asynchronously using batch processing, with compute resources required to run the work and job scheduling required to specify the work. Examples of Big Compute and HPC workloads include financial risk Monte Carlo simulations, image rendering, media transcoding, file processing, and engineering or scientific simulations.

This solution implements a cloud-native application with Azure Batch, which provides compute resource allocation and management, application installation, resource auto-scaling, and job scheduling as a platform service. Batch also offers higher level workload accelerators specifically for running R in parallel, AI training, and rendering workloads.

This solution is built on the Azure managed services—Virtual Machines, Storage, and Batch. These services run in a high-availability environment, patched and supported, allowing you to focus on your solution.

Macrocomputação com o Azure BatchAs cargas de trabalho de macrocomputação e de computação de alto desempenho (HPC) são, normalmente, de computação intensiva e podem ser executadas em paralelo, ao tirar partido das capacidades de dimensionamento e da flexibilidade da cloud. As cargas de trabalho são, muitas vezes, executadas de forma assíncrona com processamento em lotes, com os recursos de computação necessários para executar o trabalho e o agendamento de tarefas necessário para especificar o trabalho. Exemplos de cargas de trabalho de Macrocomputação e de HPC incluem simulações Monte Carlo de risco financeiro, composição de imagens, transcodificação de multimédia, processamento de ficheiros e simulações científicas ou de engenharia.123456

Upload input files and the applications to your Azure Storage account.

Create a Batch pool of compute nodes, a job to run the workload on the pool, and the tasks in the job.

Batch downloads input files and applications.

Batch monitors task execution.

Batch uploads task output.

Download output files.

  1. 1 Upload input files and the applications to your Azure Storage account.
  2. 2 Create a Batch pool of compute nodes, a job to run the workload on the pool, and the tasks in the job.
  3. 3 Batch downloads input files and applications.
  1. 4 Batch monitors task execution.
  2. 5 Batch uploads task output.
  3. 6 Download output files.

Implementation guidance

Products/Description Documentation

Storage Accounts

Massively scalable object storage for unstructured data.

Batch

Cloud-scale job scheduling and compute management.

Product information

The links to the right provide documentation on deploying and managing the Azure products listed in the solution architecture above.

Related solution architectures

HPC híbrido no Azure com o HPC PackO Microsoft HPC Pack é uma solução de computação de alto desempenho (HPC) gratuita concebida com as tecnologias do Microsoft Azure e Windows Server. O HPC Pack combina um conjunto completo de ferramentas de implementação, administração, agendamento de tarefas e monitorização para o seu ambiente do cluster HPC Windows e Linux, o que lhe dá uma plataforma flexível para desenvolver e executar aplicações HPC no local e no Azure.1234567

Hybrid HPC in Azure with HPC Pack

Microsoft HPC Pack is a free high performance computing (HPC) solution built on Microsoft Azure and Windows Server technologies. HPC Pack combines a comprehensive set of deployment, administration, job scheduling, and monitoring tools for your Windows and Linux HPC cluster environment, providing a flexible platform for developing and running HPC applications on premises and in Azure.