SSHFS – Installation and Performance
Sharing data saves space, reduces data skew, and improves data management. We look at the SSHFS shared filesystem, put it through some performance tests, and show you how to tune it.
A Library for Many Jobs
The Joblib Python Library handles frequent problems – like parallelization, memorization, and saving and loading objects – in almost no time, giving programmers more freedom to push on with their core tasks.
Pandas: Data Analysis with Python
The Python Data Analysis Library, or Pandas, is built on top of the fast math library NumPy and makes analysis of large volumes of data an easy and efficient experience.
Monitoring HPC Systems: Process, Network, and Disk Metrics
In the continuing story of monitoring HPC systems, we look at code that measures process, network, and disk metrics.
Monitoring HPC Systems: Processor and Memory Metrics
One goal of HPC administration is effective monitoring of clusters. In this article, we talk about writing code that measures processor and memory metrics on each node.
Monitoring HPC Systems: What Should You Monitor?
In rapidly growing HPC installations, you need to understand what is happening within the system to make improvements or simply to justify the purchase.
Understanding I/O Patterns with strace, Part III
In the third article of this three-part series, we look at simple write examples in Python and track the output with strace to see how it affects I/O patterns and performance.
Understanding I/O Patterns with strace, Part II
In the second article of this three-part series, we look at simple write examples in Fortran 90 and track the output with strace to see how it affects I/O patterns and performance.
S3QL Filesystem for HPC Storage
Many HPC sites with petabytes of data need some sort of backup solution. Among the many candidates, cloud storage is a serious contender. In this article, we look at one solution with some serious advantages: S3QL.