A Container for HPC

Singularity Is Not Just for HPC

Although the motivation for Singularity was based in the HPC world, it can be used for virtually any application in which mobility of compute, application appliances, or workflow distribution is important. Nothing limits the type of application that can be in a Singularity container. In fact, Singularity goes a step beyond other container systems and provides a very clever way of running virtually all applications.

Summary

Containers are all the rage today for a very good reason: Users and developers want to be able to develop on one distribution and execute on another. This capability divorces the developers from the production environment, allowing them to develop an application whenever they want (or need) and insert that application into the production environment.

Containers also allow applications and services to be isolated from one another on a system for a variety of reasons, including security and stability. I've heard of people running each of their applications in a separate container isolated from each other. They might run Firefox in its own container, gcc in another container, and even a code editor in another container. Granted this is not the most space-efficient way of running systems, but it does allow you to isolate applications from one another.

The most popular container system today is arguably Docker. People have experimented with it for HPC applications but have not found it to be the most appropriate container because – among other reasons – it's possible for users to gain root access. Solutions for getting around this usually involve running a cluster environment on an existing cluster.

Singularity is a new container paradigm that focuses specifically on one of the core missions of containers: mobility of compute. Using a specfile, very much like a specfile for RPMs, Singularity creates a container (.sapp file), examines the targeted binary for dependencies, and packages the application, the dependencies, and a couple of small tools into an executable package. These Singularity containers can interact with files local to the container in the directory where the container is executed.

Singularity containers can be created for any application and are not just limited to HPC applications. However, one of the absolute coolest things that Singularity can do is create simple containers for MPI applications. These have been very difficult for other container systems to handle, but Singularity treats the applications like an executable, so the mpirun command treats it like any other executable.

Singularity is still developing, but it can already be used to create and run containers for large MPI applications. I tested several existing applications, and they ran without a problem. I highly recommend giving Singularity a try, especially if you are running HPC applications.

Is Singularity the future of mobility of compute using containers? I sure think so.