We are happy to announce the release of MPI support for Linux on Azure Batch.

Azure Batch makes it easy to run large-scale jobs in parallel. You tell Batch what kind of VM you need, how to configure them, the jobs and tasks to run, and the service takes care of the rest. It’s great for Azure developers who need to scale-out work like engineering simulations, transcoding or financial risk models.

We previously released MPI support for Windows and recently introduced Linux support on Azure Batch. We are now excited to annouce MPI support for Linux on Batch. Linux Support on Azure Batch is currently in preview and will be generally available soon. Additionally, we also provided password-less SSH on communication-enabled Linux pools for a specific user.

By creating a pool of A8 or A9 compute nodes, Batch MPI tasks can fully leverage the high-speed, low-latency RDMA network for those Azure VMs. We also present an end-to-end example of running an OpenFOAM MPI solution on Azure Batch and added a sample on GitHub with a detailed explanation in this post.

OpenFOAM (“Open source Field Operation And Manipulation”, wiki) is a C++ toolbox for the development of customized numerical solvers, and pre-/post-processing utilities for the solution of continuum mechanics problems, including computational fluid dynamics (CFD). It is a highly compute intensive application suitable for MPI.

The CentOS-HPC image from the Azure Virtual Machines Marketplace has Intel MPI in-built. We use this image to create a Batch pool of Standard_A9 VMs. We use the Azure Batch-exposed Start Task facility to prepare the pool of VMs with prerequisites. We then add a MPI task to run OpenFOAM. The multi-instance task is configured to run a coordination command on all participating nodes. The coordination command sets up and prepares a NFS share with input data required to run OpenFOAM. After running the coordination command, the Batch service runs the application command on the head (or “primary”) node. The application command executes mpirun on OpenFOAM. The primary or head node compresses the result and could upload it to Azure storage. We render/generate video using Paraview software. The sample on GitHub automate end-to-end workflow for this.

The same can also be run with no code using Azure portal. Create an Azure Batch account. Select the Batch account that was created. Click Pools tile followed by Add. Now add a new Job as Azure portal –> Batch account –> Jobs –> Add Job. Then Submit a MPI task for OpenFoam (or any other application) as Azure portal –> Batch account –> Jobs –> Job –> Add Task. Once above task is submitted, Batch will run OpenFOAM on the input data. When the task completes, result file can be downloaded by navigating to the task in the Azure portal as Azure portal –> Batch account –> Jobs –> yourJobId –> Tasks -> yourTaskId –> Task Files -> Files on node.

For more information about Batch Linux MPI support and details on OpenFoam end-to-end solution, please check out this post.

If you have questions about MPI on Linux in Azure Batch, or about the examples in this blog post, please visit the Azure Batch forum and post your questions.