Lead Image © kateleigh, 123RF.com

Distributed storage with Sheepdog

Data Tender

Article from ADMIN 23/2014

By Udo Seidel

The Sheepdog distributed storage tool exhibits pronounced doggedness and deserves appropriate attention, benefiting projects such as QEMU and OpenStack.

Kazutaka Morita launched the Sheepdog project [1] in 2009. The primary driver was the lack of an open source implementation of a cloud-enabled storage solution – as a kind of counterpart to Amazon's S3. Scalability, (high) availability, and manageability were and still are the essential characteristics that Sheepdog seeks to provide. For Morita, setting up Ceph, which already existed at that time, was just too complicated. GlusterFS in turn was largely unknown. What these tools had in common was that they then tended to address the NAS market.

Sheepdog was intended from the beginning as a storage infrastructure for KVM/QEMU [2] (see also the box "Virtual Images with Sheep"). Because Morita was working at NTT (Nippon Telegraph and Telephone Corporation), there are corresponding copyright notices in the source code, but the code itself is released under the GPLv2.

Virtual Images with Sheep

Sheepdog started life with the objective of being the storage system for QEMU. In 2009, this meant a data store for the hard disk images of the virtual servers. This paradigm is still anchored very deeply in the project. The management of the stored data is based on virtual disk images (VDIs).

The command-line tool uses the vdi keyword to access the storage objects. Even for access via the POSIX layer, sheepfs relies on VDIs behind the scenes. "Normal" files cannot be stored without a few tricks. The user first needs to create a VDI, provide it with a filesystem, and finally mount the VDI.

Sheepdog manages four types of information for each storage object: the actual data, the metadata (name, size, timestamp, ID), the status of the associated virtual server, and the VDI attributes. In principle, you

...

Use Express-Checkout link below to read the full article (PDF).