Lead Image © Wong Yu Liang, 123RF.com

Tuning ZFS for Speed on Linux

ZFS Tuning for HPC

Article from ADMIN 57/2020

By Petros Koutoupis

The ZFS filesystem and volume manager simplifies data storage management and offers advanced features that allow it to perform in mission-critical or high-performance environments.

If you manage storage servers, chances are you are already aware of ZFS and some of the features and functions it boasts. In short, ZFS is a combined all-purpose filesystem and volume manager that simplifies data storage management while offering some advanced features, including drive pooling with software RAID support, file snapshots, in-line data compression, data deduplication, built-in data integrity, advanced caching (to DRAM and SSD), and more.

ZFS is licensed under the Common Development and Distribution License (CDDL), a weak copyleft license based on the Mozilla Public License (MPL). Although open source, ZFS and anything else under the CDDL was, and supposedly still is, incompatible with the GNU General Public License (GPL). This hasn't stopped ZFS enthusiasts from porting it over to the Linux kernel, where it remains a side project under the dominion of the ZFS on Linux [1] (ZoL) project.

The ZoL project not only helped introduce the advanced filesystem to Linux users, it garnered its fair share of users, some developers, and an entire community to support it. That aside, with a significant user base and the filesystem's use for a wide variety of applications (HPC included), it often becomes necessary to know how to tune the filesystem and understand which knobs to turn.

You should understand that when you decide to apply the methods exercised in this article, you must do so with caution or after dry runs before rolling it out into production.

Creating the Test Environment

To begin, you need a server (or virtual machine) with one or more spare drives. I advise more than one because when it comes to performance, spreading I/O load across more disk drives instead of bottlenecking a single drive helps significantly. Therefore, I use four local drives – sdc, sdd, sde, and

...

Use Express-Checkout link below to read the full article (PDF).