MPI Apps with Singularity and Docker

Docker

Docker was not originally designed for HPC, but over time, it has adopted HPC and AI capabilities. The method for running MPI applications requires running as a privileged user (root):

$ sudo docker run --rm -it poisson mpirun --allow-run-as-root -n 2 /usr/local/bin/poisson_mpi

Note that although you can run an MPI application in a Docker container other ways, here, the container is run as a privileged user (root), because in my opinion, this is the simplest method I’ve found. You don’t treat the container as an application, but it does get the job of running the MPI application done.

Summary

When containers broke into the mainstream (I'm not counting chroot), HPC was ignored. However, some container ideals are very useful in the HPC environment, and one key HPC capability is running MPI-based applications.

In this article, I explained how to run MPI applications in Singularity and Docker containers. The best way to get started is to use HPCCM to write a simple recipe that can then be used to create the appropriate specification file for Singularity or Docker. Then, you can keep the HPCCM recipe as the “source” for the container spec files, making container life just a bit easier.

I also illustrated how to execute the containers so that the MPI application can run. Although the application was very simple, the examples did show how the process works and how straightforward and, dare I say, simple, the process can be.