Stopwatch

Solid State Drives (SSDs) have grown beyond their status as a niche product and now increasingly compete with traditional hard disk drives (HDDs) as an attractive option for server systems in terms of capacity and price. Whether SSDs really do boost performance depends on the particular application, because the requirements placed on the storage systems can differ fundamentally. Users must first decide on SSDs, HDDs, or a combination of the two technologies. Once you've decided which medium to use, you still need to choose an appropriate model.

Test for Yourself

To begin, you will want to make sure that the speed of the selected hardware meets your current and future performance requirements. Meaningful and transparent performance tests considerably facilitate your search for the right SSD or HDD. From the perspective of the end user, the more information a test delivers, the more valuable it is. Often the manufacturer's performance specifications are not authoritative, because neither the software used nor the test method is specified.

The Storage Networking Industry Association (SNIA) has released the "Solid State Storage Performance Test Specification," a document that provides a specific test description for SSDs in the enterprise space [1]. In its tests, the specification specifically targets the unique properties of SSDs and describes the approach to achieving accurate and reproducible results. TKperf, an open source tool developed by Thomas-Krenn.AG, implements this specification and prepares the results in a test report [2]. In addition to testing for SSDs, some initial basic tests for hard disks are also available; they will be expanded in future versions of TKperf.

Fio and Python

In the background, TKperf uses the Flexible I/O Tester (Fio) [3] developed by Jens Axboe, the maintainer of the Linux block layer. Fio offers many options and the flexibility to implement the SNIA performance tests: Features include run-time limits, direct I/O, number of parallel jobs, I/O depth, or the use of non-compressible test data.

TKperf requires Fio 2.0.3 or higher. In older versions of Fio, some terse output information is missing; this is minimal output that can be easily processed by other programs [4]. Distributions like Ubuntu 12.04 or Debian squeeze have older Fio versions in their repositories (v1.59 or v1.38), but Fio can be compiled easily from the source code in the Git repository  [5].

Other components of TKperf rely on the hdparm utility and Python. Hdparm uses hdparm -I to provide automated information about a device under test. The tool can also perform a secure erase on SATA drives. For SSDs, this means that all flash chips are erased, thus restoring the SSD's as-received condition performance. Secure erase, which is a key part of the initialization phase of the tests, is also the reason why some manual attention is still needed when testing SAS or PCIe SSDs.

For SAS devices, you need to modify the sg_format secure erase method in sg3_utils (the tools for executing SCSI commands). Secure erase is even more complicated for PCIe SSDs because they require vendor-specific, command-line tools. For Intel's PCIe SSD 910, for example, you need to launch secure erase via the isdct Datacenter tool.

The Python scripts in TKperf start the Fio jobs, prepare the results, and generate charts. All Fio calls are recorded in a logfile. At the end of a test, the test results are also logged. The logged performance values are stored in XML format and used to create the charts. With the help of the Python Matplotlib library, TKperf generates the graphical representations of the results required by the specification  [6].

TKperf Test Scenarios

A bandwidth test, an input/output operations per second (IOPS) test, a latency test, and a saturation test are all available for SSDs. Listing 1 shows the output of TKperf during testing of an Intel DC S3700 200GB SSD.

$ sudo tkperf ssd intelDCS3700 /dev/sdb -nj 2 -iod 16 -rfb
!!!Attention!!!
All data on /dev/sdf will be lost!
Are you sure you want to continue? (In case you really know what you are doing.)
Press 'y' to continue, any key to stop:
y
Starting SSD mode...
Model Number:       INTEL SSDSC2BA200G3
Serial Number:      BTTV241303MG200GGN
Firmware Revision:  5DV10206
Media Serial Num:
Media Manufacturer:
device size with M = 1000*1000:      200049 MBytes (200 GB)
Starting test: lat
Starting test: iops
Starting test: writesat
Starting test: tp

The bandwidth, IOPS, and latency tests show some similarities that are reflected in the graphics. All of these tests are performed in rounds, with measurements taken in each round. Taking into account a dependent variable, the test calculates whether the SSD is in a stable state. This approach avoids influencing the measurements in a "fresh out of the box" state or undefined transitions between different workloads. Only if the values of the dependent variable are stable over the last several rounds the values of this measurement window are incorporated into the test results.