Obstacles to Practical Deployment

The amount of storage space you can save by deduplication depends to a great extent on the data you store. Although the developers promise savings of up to 95 percent, this also assumes that SDFS finds 95 percent identical data blocks. Additionally, the risk of data loss increases with the percentage; if a data block fails, you cannot restore any of the files belonging to it.

In contrast to other deduplicating filesystems, SDFS does offer the option of storing data blocks redundantly in a cluster. However, you need to enable this mirroring explicitly. Doing so means that the data again exists in multiple instances and achieves precisely the opposite of what deduplication promises.

Additionally, you need to back up the metadata of the individual volumes yourself. The SDFS developers suggest using DRDB [8] for this. Multicast packages not only increase the network load, they also force administrators to drill holes in their firewalls. Finally, the communication between the individual components in the cluster is unencrypted by default – although you can at least enable SSL encryption.

The developers have not done the best job of documenting their tools. On the project website, you will find a Quick Start Guide and a description of the architecture, which the developers refer to as an "Administration Guide" [9]. The tools do output a long reference of all parameters when started with the --help option.

Conclusions

SDFS sets up quickly; volumes can also be created easily across multiple computer nodes, and there is support for virtual machines. However, administrators will do well to bear in mind the risks of a deduplicating filesystem. The use of SDFS thus mainly makes sense if you have many similar or identical files that are not particularly important – such as temporary files or logfiles.