Smart Monitoring with Consul

Consul helps users who want to monitor cloud environments independent of the tools available in a specific installation. Consul can also be used on bare metal for comprehensive monitoring, making it a kind of miracle weapon for modern, flexible environments.

Some admins might recall Consul from HashiCorp. The program written in Go was presented by the manufacturer as early as 2014 and was initially intended primarily for operation in clouds. That Consul is also very useful for various tasks on physical servers only gradually became apparent over the years.

HashiCorp itself markets Consul as a tool for three central tasks in distributed environments:

• Service Discovery: Consul can act as a central interface between servers and clients. Servers register as services with Consul, which it keeps in a list. Clients request exactly that list if required and find out, for example, at what address their server can currently be reached.
• Health Checking: Consul automatically checks whether the servers that have logged on to it are still working. In this way, Consul avoids telling a client the address of a MySQL database that is currently inactive.
• Key Value (KV) Store: Consul also maintains a configuration database according to the key-value principle to which clients can add arbitrary entries. Because Consul is a cluster service, of which an instance will ideally be running on every system in the setup, all information of the KV store is available at any time on all systems of the platform.

Consul offers several advantages with regard to its architecture: Consul contains a cluster consensus algorithm that automatically implements a quorum and thus prevents split-brain situations from arising in the first place, which is particularly important for the KV store.

Consul is also extremely lightweight. The program enters the system as a Go binary, has no dependencies, and only needs a configuration file. Therefore, Consul is easy to install on older systems where current versions of Python or Ruby are not available. Moreover, Consul is very frugal when it comes to resource requirements.

Consul's biggest advantage is undoubtedly that it is designed to work with other programs out of the box. A huge amount of information can be retrieved from Consul via a RESTful interface. As a hook back to the old world, Consul also offers the option of generating configuration files from information in its own service directory.

In fact, Consul makes it possible to implement the new type of monitoring that is necessary in dynamic environments. Two examples illustrate this: monitoring hardware and monitoring dynamic environments.

Conventional Monitoring

When working with real hardware, one of the most interesting questions is how the admin integrates a greater number of new servers into the existing monitoring system as fast as possible. Because clouds are extremely dynamic, 500 new servers wanting to be integrated into a setup in a week is not unusual.

Classical approaches like Icinga or Nagios have a difficult time keeping up. For example, the administrator has to help with an automation solution such as Ansible or Puppet. Consul does this work for the admin by cleverly combining the functions of the software.

Consul on All Systems

To use Consul in physical setups, you need to roll out the service on all participating systems in the first step, which is best done with the help of an automation solution such as Ansible. Consul has two different operating modes: server and agent. In everyday life, the two operating modes differ primarily in that the servers decide on the quorum and execute the consensus algorithm, whereas the agents primarily take care of themselves. Clouds almost always contain a category of servers that operate many cloud services as controllers – they are also ideal for the role as servers in Consul. The normal compute nodes, on the other hand, only work as agents.

The result of simply rolling out Consul on all systems results in a comprehensive database of all available cluster nodes. Where to go from there is also a question of the admin's personal preferences. The simplest option is to have Consul check whether or not certain services are available.