Evolutionary Theory

Future of Block-Level Storage

At first glance, questions relating to the future of block storage may sound strange in the context of what has been said so far, but they are justified in the longer term considering the innovation dynamics mentioned at the beginning. The reason is simple: If you analyze the extremely rapid growth of semi-structured and unstructured data, you find that Internet of Things (IoT), artificial intelligence, and machine learning data; video files; images; audio files; and the like are growing disproportionately.

This development is just the beginning of an almost explosive trend. According to IDC, more than 80 percent of all data stored worldwide could be in archives by 2024 [1]. The growth of file and object storage systems significantly faster than iSCSI, Fibre Channel, and NVMe-based block-level storage would not be without consequences for the further development and potential market growth of such systems.

On the other hand, experience to date shows that, with few exceptions, complementary technologies coexist on the market for a long or even very long time, not least to avoid radical breaks in the critical IT infrastructure. In the meantime, the probability that NVMeOF will displace the iSCSI protocol for high-performance block storage access to flash media is relatively high, whereas the increasing importance of file and object storage will negatively affect both block storage-based FC SANs and iSCSI storage networks. This trend is already visible today.

Plus and Minus of NAS and Object Storage

File-based systems configured as a DAS solution or as NAS are easy to implement and operate, which explains their great popularity for decades. However, file storage (filers) can only scale arbitrarily by adding more systems, not by simply adding more capacity. The inherent disadvantage of NAS approaches is the almost linear increase in complexity and cost as unstructured data volumes increase sharply.

One reason is that, with common NAS systems, data is organized within a fixed folder hierarchy whose paths quickly become complex and long. The architecture is therefore not a very good match for rapidly growing unstructured data volumes in the double-digit multi-petabyte or exabyte range. For this reason, development in the direction of clustered scale-out filesystems has been going on for some time. In general, high-performance filesystems are the platform for scalable storage infrastructures, whether as software-defined storage such as Ceph on an open source basis or as a manufacturer-specific implementation, of which many are on the market today.

In contrast to NAS, object-based storage has a flat structure for data management. Files are divided into individual areas and distributed by server systems (nodes). These objects are not stored as files in folders or even blocks on servers, but in a repository, and are linked with the associated metadata (i.e., Global Namespace architecture), which allows scaling to very large data volumes and is ideal for storing unstructured data formats. However, object storage is not suitable for classic database environments because writing takes far too long compared with block storage. Native programming of a cloud-based application in conjunction with Amazon Simple Storage Service (S3) as the object storage API can also be far more complex than using file storage.

Unified Storage

Global filesystem implementations go one step further and are designed as hybrid cloud storage per se. Although the cloud is used as a central data repository, the system is logically presented as if it were an on-premises NAS system. This approach offers cost advantages over traditional on-premises solutions, as well as compared with popular public cloud storage. Today, powerful global filesystems are already capable of storing file data in the cloud as objects. This approach allows users to access files as they would on a standard NAS while the majority of the data resides on a cost-optimized, highly scalable object backend.

The limitations of traditional NAS and SAN systems for the aforementioned workloads are increasingly driving enterprises today to look for object-based storage solutions that support (global) filesystem capabilities. An object-based storage solution with integrated native file management capabilities therefore makes the transition from NAS to object storage interesting and opens up a wealth of new application possibilities, from backup and disaster recovery to compliance-compliant archiving and highly secure, centrally consolidated cloud services for global file access and file synchronization. Compared with the file services offered by large cloud providers, they are then a genuine economic alternative and ideally lead to greater IT acceptance in the company.