Rook Storage Orchestration

Rook [3] is a cloud-native storage orchestrator for Kubernetes. It is based on a Ceph cluster (Luminous or higher, Kubernetes 1.6 or higher) and runs a distributed filesystem directly on the storage cluster. Rook provides interfaces for scheduling, life cycle and resource management, security, monitoring, and the cloud user experience. The software is built on top of Ceph's node structure and has additional components that monitor and control the installation and operation of Ceph pods. For example, a Rook agent is installed on each node. It provides part of the storage driver for Kubernetes. The Rook operator, mainly developed by CoreOS, monitors and controls the individual agents and other parts of the Ceph cluster within a Kubernetes environment.

Rook offers the following functions and others: storage management, even in hyperscaled or hyperconverged storage clusters; effective data distribution and replication; and provisioning of file, block, or object storage for various providers. Rook can also be used to optimize workloads on commodity hardware.

Swift Object Storage in OpenStack

Swift [4] is another important project used in OpenStack to implement object storage with ring structures. The rings are used for mapping between the names of entities stored in the cluster and their physical equivalents on disk. Within the ring are zones, devices, partitions, and replicas. Each partition is replicated at least three times in the cluster. The locations of the copies are stored in the ring mappings.

In case of failure, the ring takes over the switching to intact resources. Data can be isolated within one zone of the ring; replicas are kept in different zones (data center, cabinets, servers, or even switches). Partitions are distributed across all the devices in a ring. The ring does not manage itself, but is managed externally.

The replication mechanism continuously checks for the desired three copies by reading the hash files of the partitions. Zones (racks, servers, one or more drives) are designed to isolate errors. Partitions, on the other hand, are collections of stored data (e.g., account or container databases). Partitions form the core of the replication system.

Each ring is accessed by proxies and their APIs. Proxies are also responsible for coordinating responses and timestamps and handling failures. They have a share-nothing architecture and can scale as needed. At least two need to be present for redundancy. Containers are represented as individual SQLite databases distributed across the cluster. The same is true for accounts. Here, an account database contains all containers that belong to the account. A container database stores all objects in the container.

Other Open Source Projects

The Soda Foundation's Soda Open Data Autonomy (SODA) project [5] is interesting. A uniform API layer is planned, through which applications can access data independent of the underlying storage or logical structures. However, the respective platforms need a SODA plugin.

SODA consists of an infrastructure manager for the entire storage infrastructure. The SODA API acts as a central external interface that seamlessly connects to heterogeneous storage back ends, unifying the usual heterogeneous data and storage management APIs. One controller handles all metadata and state management. The drivers of the different storage back ends are connected to what is known as a SODA dock. There is also a component for multicloud management.

The following is a quick look at some of CNCF's other container-related open source projects:

• Linstor is a Kubernetes-integrated block storage management tool for large Linux clusters and implements persistent block storage for OpenStack, OpenNebula, and OpenShift.
• Longhorn is useful for building distributed block storage in Kubernetes environments.
• OpenEBS implements open container-attached storage in Kubernetes environments, which enables stateful applications to access dynamic local or replicated PVs more easily. Users include Arista, Orange, Comcast, and CNCF.
• Stash backs up stateful applications in Kubernetes environments. The project is based on the Restic open backup application. Stash uses a declarative interface and custom resource definition (CRD) to control backup behavior.
• Velero also backs up Kubernetes resources, but it is also useful for migrations and disaster recovery of persistent volumes between Kubernetes cluster resources.
• MinIO implements an S3 object store for Kubernetes environments without interacting directly with Kubernetes. The solution gets by with a single software layer. Features include erasure coding, encryption, immutable storage, identity management, continuous backup, global data aggregation, and a universal cloud interface. MinIO runs on bare metal and all private clouds but can connect to NAS storage.