IOPS Measured

Using an IOPS measurement as an example, the following sections illustrate the test sequence from secure erase, through assessment of the steady state, to representation of the results. First, a definition of terms significant for the rounds-based tests:

• Dependent Variable: This is necessary to check whether a stable state has been reached. This variable is defined by the specification and consists of a combination of workload and block sizes. For the IOPS test, this variable is equal to the IOPS achieved in a random write with a block size of 4KB.
• Steady State: This is the state in which the last measured values do not differ significantly from one other. The parameters for deciding whether or not the steady state has been reached are the gradient of the straight line predicting the next value and the difference between the largest and smallest measured value. The measured values of the previous five rounds are used as reference variables for the steady state. The steady state verification plot shows the evolution of the dependent variable and proves that the SSD is in a stable state (Figure 1). The pseudo-code of the IOPS test illustrates which operations are carried out in a round (Listing 2).

Figure 1: The steady state verification plot demonstrates the stable state of the SSD, which must be achieved in a test setup so that the results meet the specification.

01 Make Secure Erase
02 Workload Ind. Preconditioning
03 While not steady state
04     For workloads [100, 95, 65, 50, 35, 5, 0]
05         For block sizes ['1024k', '128k', '64k', '32k', '16k', '8k', '4k', '512']
06             Random Workload for 1 Minute

In detail, the steps mean:

1. A secure erase guarantees that the SSD is in a defined state.

2. For preconditioning, the device is overwritten twice (sequential access with a 128KB block size).

3. Until steady state is reached, various combinations of block sizes and mixed workloads are started in each round. A workload of 100 means 100 percent reads, 95 stands for 95 percent read and 5 percent write access, and so on. Random access (random I/O) is always used for the IOPS test. As the dependent variable, IOPS with 4KB block size are used for writing. The steady state convergence plot shows the evolution of all block sizes (including the dependent variable) for random writing (Figure 2).

Figure 2: The steady state convergence plot shows the evolution of the dependent variable (black line) and the other block sizes for random writing.

If the device is in a stable condition, the test is terminated and the measured values are processed for creating charts. To do this, TKperf computes the mean value in the measurement window for the measured variables (workload and block sizes). The measurement plot represents the collated computed values (Figure 3).

Figure 3: The measurement plot shows the computed mean values in the measurement window.

The previous example of the IOPS tests showed the basic sequence in which the results are determined. For the bandwidth and the latency tests, other workloads, block sizes, and dependent variables apply (see Table 1).

A Special Case: Write Saturation

The write-saturation test is not aimed at achieving a stable state, but it needs a minimum amount of data written to the device. The test shows how an SSD behaves in continuous writing with random access. In terms of the volume of data written, care is taken to ensure that it is a multiple of the device size.

Although this test is also rounds-based, a dependent variable to elicit a steady state does not exist. In each round, TKperf randomly writes to the device with a 4KB block size. It keeps track of how many bytes were written to the device in a minute (one round), and the tool adds the number of bytes per round. The end of the test is reached when this number becomes greater than four times the size of the device. A second termination condition is a time limit: After 24 hours running time the test also ends. Figure 4 illustrates the aim of this test, which reveals the stability of the SSD during extended writing with random access.

Figure 4: The write-saturation test reveals how stable an SSD is when it is written to for an extended period. The Enterprise SSD DC S3700 (top) clearly demonstrates its benefits against an Intel 520 Series 240GB SSD (bottom).

For Hard Disk Drives

TKperf adapts the rounds-based system for hard drives; however, stable state detection is not meaningful for HDDs. Instead, the software takes into account the difference in performance at the beginning and end of a HDD. Because HDDs are written to from the outside in, shorter access paths of the read/write head lead to better performance, and faster track speeds lead to improved performance. For the hard drive test, TKperf therefore divides the device into 128 equal parts, which are tested separately. For determining throughput, it reads or writes to the entire area once. To measure IOPS, it randomly accesses data in a given section for one minute. This procedure illustrates the performance development of the hard disk when running through the sectors from the outside to the inside.