Building Containers with HPC Container Maker

Summary

Containers are being used more and more in the HPC and scientific worlds, including artificial intelligence. They have a number of attractive features, such as reproducibility and portability, but building containers for complex applications is not easy, especially for workflows that involve multiple applications.

HPC Container Maker was created to abstract the building of container spec files. Using Python and HPCCM, you can quickly and easily describe the container you want without having to worry about the details. HPCCM then creates the container spec file and you can build and test the container. Using HPCCM, you can easily change the container definition, including such things as the base OS, the compilers, the libraries used, how dependent libraries are built, and if the container uses GPUs. Doing this by hand is very complex and prone to error, but using HPCCM, the process is simple, easy to modify, and reproducible.

Thanks!

Appreciation for helping with this article goes out to Scott McMillan from Nvidia, who helped answer questions, some on the silly side, as well as review the article for accuracy.