Config Battle

The hype surrounding Docker has raised the awareness of many administrators regarding a category of programs that were almost unknown before now: fleet and configuration managers. Several representatives of this species are vying for the attention of users: Etcd [1], Consul [2], and ZooKeeper [3] are just a few.

What are these configuration managers actually about? What are their preferred fields of application, and why should administrators employ this innovation? In this article, I explain the ideas behind the tools and then let three candidates do battle.

Was Everything Better in the Past?

Administrators often reminisce about how managing IT environments used to be easier: Each host had a defined role that did not change over the course of the server's life. When people started looking into clusters, it was usually a matter of small, high-availability setups, such as Pacemaker and DRBD synchronizing data between hosts. However, administrators were fighting a problem that still exists today: The configuration of services needs to be identical on two hosts so that if one system fails the other can take over without any problems; therefore, it is essential to synchronize configuration files in some way. Home-made solutions based on Rsync were often used in these cases, and although they might not have been pretty, they served the purpose.

In recent years, however, IT setups have continued to grow and, especially, to become more dynamic. The self-made solutions that administrators used to keep the configuration files at the same level between hosts became more and more complex. Keeping the state of services consistent across multiple hosts also proved to be more and more difficult. Moreover, with scale-out setups, it is important to pay attention to the status of all instances. Databases provide a perfect example: A writing process in a MySQL database clustered with Galera involves events on multiple cluster hosts – a complex overall structure is formed.

Each service with support for scaling out faces the need to explain each of several versions of the data in the current and valid data set within the network. It is important that all the components that are active in a cluster agree on the state of the cluster and all its services in the end.

The programs presented here aim to help and go beyond the scope of simple configuration management: They aim to be a reliable source in scale-out setups that can provide information about the state of the entire system. However, although it might sound simple, it turns out to be technically demanding in practice.

Puppet and Others

The growing complexity of IT environments dictated centralized management tools for the configuration of files. The use of automation tools like Puppet or Chef reduces the admin work involved to importing a template and configuration parameters so that the right data ends up in /etc. Nevertheless, this approach only partially solves the problem, because the configuration of individual services is not nearly as static in scale-out setups as in conventional setups.

Cluster services can be scaled to any size by starting more and more instances as required. Starting Puppet or Chef to provide configuration data for each of these instances would be impractical.

Typical automation tools such as Puppet are therefore barely used inside cloud VMs. Clouds need completely different approaches to manage the configuration of services sensibly. The appropriate tools were barely known until the cloud hype surrounding Docker and OpenStack.

What Is Configuration?

The lack of useful tools is strange, particularly because the problem seems rather simple. Historically, programs on Linux systems, as on its predecessor Unix, always assumed the relevant configuration was in files in the /etc directory. Cluster configuration files follow the same principle in many cases: Pairs of parameters and values obeying a fixed syntax reside in a text file. Therefore, configuration files are just rudimentary key-value stores that can only be managed – poorly – in text files across multiple machines.

However, more than enough meaningful solutions exist for key-value stores in the example of databases. What would make more sense than transferring the database principle to the management of configuration files? This is where the gate opens to configuration databases.