Going Parallel

Matlab-like tools are extremely useful in HPC, even though they are serial applications. As I mentioned previously in this article, Matlab and Matlab-like tools can be used for tasks such as parameter sweeps by running something like 25,000 simultaneous instances of the application. However, in other situations, you might want to run the underlying functions in parallel.

For example, you might want to perform a large FFT or a large SVD (single-value decomposition) as quickly as possible by running the application using all of the cores in the node, or even by running the computations across several distributed nodes.

Several parallel processing options for Scilab are summarized in the Scilab parallel computing documentation. The first option is to use the inherent multicore capabilities in the functions used in Scilab.

For example, certain libraries perform the linear algebra computations in Scilab, and these libraries could perform the computations using all of the cores in the system. Intel's MKL library can use all of the cores for performing matrix multiplications or other functions. Typically, this is done using OpenMP but not necessarily. However, these computations are limited to intrinsic functions, so you can't parallelize Scilab code such as a for loop.

Scilab also has the capability of running more explicit parallel applications on multicore systems (i.e., cores on the same node). A function called parallel_run allows parallel calls to a function. This allows you to parallelize function calls on the system – but remember that the execution is on a single node (but with four-socket AMD systems, you can get 64 cores on a single system).

For parallel distributed applications on Scilab, you can also use PVM (Parallel Virtual Machine). PVM is a rather old approach to parallel programming and has given way to MPI (Message Passing Interface) for the most part, but it is still used in some areas. A good blog post discusses how to use PVM within Scilab (but it is two years old by now). A Git repository holds some early code developed by Scilab Enterprises to create MPI capability for Scilab.

In a manner similar to Scilab, Octave can also use numerical libraries that have been parallelized to run on a single node, such as Intel's MKL or something similar, perhaps using OpenMP. You just have to build Octave yourself and use the appropriate libraries.

Octave also has a parallel toolbox to use for running applications on a cluster or a distributed system, and with the parcellfun command, you can execute parallel function calls on the same node. This is very similar to Scilab's parallel_run command.

The openmpi_ext toolbox uses MPI to allow Octave instances on different nodes to communicate and share data. It requires the use of Open MPI [15], but if you have experience in HPC, it isn't difficult to build and install.

Parallel coding in FreeMat is a little more difficult. Evidently, early versions of FreeMat could use MPI for parallel coding; however, it appears this work has not been continued in the current versions of FreeMat.

One interesting FreeMat feature is the use of threads within the language. FreeMat threads can communicate with each other through the use of global variables. Although I have not tested this feature, it appears to be in the current versions.

Summary

In this article, I briefly reviewed three Matlab-like tools: Scilab, Octave, and FreeMat. All three have pluses and minuses that can be debated, but the one you choose ultimately depends on your requirements. For further comparison of these tools, check out the technical report from the University of Maryland [16].

If you need a general-purpose numerical tool for HPC, any one of these tools is a good candidate. If you are willing to stray further from Matlab compatibility, other candidates could work as well, but that is the subject of another article and likely another series of debates. In the meantime, give one of these applications a whirl  – I think you'll like what you see.