Deduplication and Delta Snapshots

Enterprise-class deduplication, thin provisioning, and high-density NAND flash combine to deliver greater energy efficiency thanks to lower power, cooling, and space requirements. In contrast to this technological convergence, however, manufacturer-specific platforms that differ (in terms of software, hardware, integrated stacks, etc.) will still be on the market for quite some time. That said, increasing competitive pressure is a catalyst for accelerating IT infrastructure and application consolidation in the form of hybrid (multi)cloud models and standards.

Because storage areas are provided as LUNs, the storage space must be allocated up front (overprovisioning). With array virtualization, storage systems can mask the provisioning of each storage block in a LUN until it is actually written to. The storage space required for the user is reserved, but nothing more. Therefore, more storage can be provided than is actually available. This thin provisioning effect can potentially save half the energy consumption per terabyte of storage required because you can plan for smaller sized systems at the time of purchase. The fact that less storage is required has a direct effect on floor space required, air conditioning requirements, power consumption, and capital expenditures.

A delta snapshot, in turn, is a kind of point-in-time copy that preserves the state of the data at a specific time. To do so, it only stores blocks that differ from an existing complete copy of the data. In contrast, clones and snapshots are complete copies of the data. Delta snapshots are based on various forms of copy-on-write technology. Blocks are only written if new data is written to them. Delta snapshots typically save 80-98 percent of the raw capacity per snapshot required to store the target data set. Depending on the usage profile, the potential for significant energy savings is great.

Especially in backup scenarios, the same data is often written to a device multiple times. Data deduplication replaces these multiple copies with a single source copy and multiple references to it, which can result in space savings of up to 99 percent. However, the deduplication rate for production environments and their primary storage is significantly lower than for backups. Dedupe rates from 3:1 to 5:1 are achievable if you mix workloads such as databases and virtualized infrastructures. Server and desktop virtualization benefit because of the similarity of the VM images. Deduplication in database applications does not offer significant capacity gains; again, everything depends on the use case.

Compression: It Depends

Data compression is based on an algorithm that allows more data to be stored in less space, which also saves time and bandwidth during data replication. Data compression can be used on primary, secondary, and tertiary storage environments. In contrast to tape, optimization in hard disk and flash-based storage systems by inline compression is technically more complex, affects performance, and is less effective depending on the type of data because 4 or 8KB blocks are usually stored.

Nevertheless, compression offers advantages in terms of energy efficiency and sustainability, especially for flash systems, and includes parameters such as increased logical capacity (by a factor of 10:1) and lower write amplification (i.e., longer life by a factor of five of the NAND storage medium). However, the controller's design becomes more complex and can cause the overall cost of the system to rise and affect performance, depending on the vendor's implementation.

Tape for Cold Data

The increasing amount of inactive data requires a coordinated IT strategy in terms of the required and appropriate data management tools and IT infrastructure components. In addition to hard disk developments such as heat- and microwave-assisted magnetic recording (HAMR and MAMR) for greater HDD capacity and the advantages of flash, linear tape-open (LTO) magnetic tape as a currentless medium for archive environments up to the exabyte range is moving into the center of attention on the storage medium side. New developments in durability and capacity play a role in CO2 reduction and low storage costs. Additionally, management tools for file formats and Amazon simple storage service (S3) object data formats are key to energy-efficient handling of very large data volumes.

Tape libraries only consume power when they are online (i.e., when data is written to them or retrieved for specific purposes). Companies can significantly reduce their CO2 emissions by migrating infrequently accessed (cold) data to tape. One report estimated that by moving cold data from disk to tape storage, an 87% reduction in carbon emission and an 86% reduction in total cost of ownership could be achieved over 10 years [8].