New Release of Lmod Environment Modules System

Module Usage

In an article a couple of years ago, I presented a way to gather logs about TCL/C environment usage. It was a bit of kludge, but it did allow me to gather data about module usage. In version 6.x of Lmod, this ability was brought to the forefront.

Tracking module usage is conceptually fairly easy, but a number of steps are involved. Having this information can be amazingly important, because it allows you to track which tools are used the most. (I associate one tool with one module.) If you have various versions of a specific tool, it allows you to track the usage of each so that you can either deprecate an older version or justify keeping it around and maintaining it. You can also see which modules are used as a function of time, which helps you understand when people run their jobs.

The topic is important enough to warrant its own article, so stay tuned for an upcoming article on gathering module usage information.

Summary

Although I've written about Lmod before, I continue to come back to it because it is so useful. It greatly helps users sort out their environment so that they don't accidentally load conflicting modules. The first time you have to debug a user's code when they have mixed MPI implementations, you will be thankful for Lmod.

Environment modules in general, and Lmod specifically, allow you to keep multiple versions of the same package on a system to service applications that have been built with older versions of a compiler, MPI, or library. I even saw a recent posting to the Open MPI mailing list asking about LAM-MPI, even though it basically has been dead for a decade. You would be surprised how long applications stick around and bring their dependencies with them.

Because Lmod can read TCL module files in addition to Lua (the preferred language), you can move easily from TCL/C Environment Modules to Lmod. As you can see from the Lua module file examples I used here, the syntax is very clean and simple, making them very easy to read.

Finally, Lmod is developing tools to audit module usage. This information is amazingly useful, as pointed out in two articles from Harvard (Scientific Software as a Service Sprawl, parts one and two). The author gives a very good explanation about how to set up Lmod for collecting module usage, putting it into a database, and mining that database – which is very cool stuff indeed.

Look for an upcoming article that goes into more depth on this topic.