MPI Apps with Singularity and Docker

The Message Passing Interface (MPI) is one of the frameworks used in high-performance computing (HPC) for a wide variety of parallel computing architectures. It can be used for running applications on multicore systems and on multicore/multinode systems, where you map a “process” to a processing unit, typically a CPU core. MPI has been around for many years and is used by a great number of applications to scale, in terms of performance.

Classically, the way to run an MPI application is something like:

$ mpirun -np 16 --hostfile <list_of_hosts> ./<application> [<application options>]

Note that this command does not really correspond to a specific MPI implementation; rather, it is meant to illustrate the basics of most MPI implementations.

The mpirun command launches and coordinates the application (<application>), the -np option is the number of processes to be used, and --hostfile names a simple text file (<list_of_hosts>)that has a list of hostnames (servers) to be used in running the application.

With the development of containers and their use in HPC and artificial intelligence (AI), the challenge becomes how to run MPI applications that are in the container. In this article, I show an approach to accomplishing this goal.

Test Software

I'm sure a number of people have come up with several ways to run MPI applications in containers. The methods presented in this article are the best ways I’ve found.

Although these tests are not performance tests that require detailed specifications about the hardware and software, I offer a brief description. The system ran Linux Mint 19.3 and used the latest PGI Community Edition compilers, version 19.10. Open MPI 3.1.3, which came prebuilt with the PGI compilers, was used in the tests.

The Docker-CE (Community Edition) v19.03.8 build afacb8b7f0, Ubuntu version (on which Linux Mint is based), was downloaded. For Singularity, the latest version as of the writing of this article, 3.5.3, was built and installed. It uses Go 1.13, which was installed from binaries on the golang website.

The simple MPI used is the classic 2D Poisson equation solver, poisson_mpi, which I procured from an online set of Fortran 90 examples.

HPCCM

HPC Container Maker (HPCCM) was used to create the specification files for creating the container images. It was installed in the Anaconda distribution (Python plus R programming language), which included conda (a package and environment manager) version 4.8.2 and Python 3.7. HPCCM version 20.2.0 was installed (the latest as of the writing of this article).

HPCCM was discussed in a previous HPC article. The tool, written in Python, is very easy to use and allows you to specify a basic description of how you want your container built. This HPCCM “recipe” can then create a Dockerfile for Docker or a singularity description file for Singularity, which you can modify. These files are then used to create the container images. For this example, the same HPCCM recipe is used for both Docker and Singularity.