More than a Thousand Words

Amazon CloudWatch

Amazon offers its own metrics service – also a time series database – for its Amazon Web Services cloud. A plugin for Grafana can read data from CloudWatch and present it visually. Grafana thus demonstrates that it is also suitable for hybrid workloads: With a single click, you can visualize the current load data for a private cloud or your part of a public cloud.

Prometheus

A prime example of the full integration of a MAT solution in Grafana is Prometheus [6], a monitoring solution that is a time series database at its core (see also the article on Prometheus in this issue). The aim of the original SoundCloud project was to monitor large and fast-growing computer farms, including the ability to detect failures very quickly and to plan setup and scale-out in the long term.

In contrast to the solutions already presented in this article (i.e., Graphite, OpenTSDB, InfluxDB), one Prometheus developer focused on performance: The Prometheus server thus uses a proprietary format to store data on disk and index it in the LevelDB format. Of the back ends for Grafana referred to here, Prometheus is the only complete monitoring solution: In all other cases, the time series database is only the back end used by another solution (e.g., Sensu) to store its data.

In terms of functionality, Prometheus is state of the art and owes great flexibility to its highly modular architecture. The Node Exporter is a good example of this. It runs and performs tests on the individual hosts and records the results. Subsequently, the central Prometheus server retrieves the Node Exporter results on all hosts and saves the information accordingly.

Prometheus shows evidence that it is designed for large and highly elastic environments. For example, it has a direct interface to Etcd, a daemon that keeps records for all the nodes in a cluster. As soon as you add a new server to a cluster, Prometheus automatically learns about it via a detour to Etcd. If the automation works and automatically rolls out a Node Exporter on each server, you only need to install a new server to integrate it into your existing monitoring setup.

When the first Prometheus article [6] went to press, the developers still offered a graphical interface. PromDash (Figure 4) was in many ways a competitor of Grafana, but far inferior in almost all aspects. In the end, Grafana released a plugin to access Prometheus data (Figure 5), and shortly afterward, the Prometheus developers announced they had stopped working on PromDash.

Figure 4: SoundCloud originally developed its own GUI for Prometheus, PromDash, but it is now officially dead …

Figure 5: … and the Prometheus developers recommend using Grafana.

Because the Grafana developers wrote the storage back end for their metrics from scratch, they took the opportunity to develop their own query language: PromQL. Grafana makes good use of the query language: Once you have set up a Prometheus server in Grafana as a data source, you can configure arbitrary PromQL queries to generate graphics in Grafana. All told, the team of Grafana and Prometheus turns out to be an extremely potent combination: If you are looking for real MAT, you definitely should take a closer look at this combination.

Gnocchi

Grafana also offers genuine added value for OpenStack admins with Gnocchi [7]. This database was developed by the OpenStack project as its metering service and is configurable as a data source in Grafana. You might remember that Ceilometer was designed to take care of metering in OpenStack, but for many years, it has led the life of a wallflower, largely because the first versions of Ceilometer expected MySQL as the storage back end. Metering data from OpenStack includes values such as the number of virtual machines running per customer, the number of virtual networks created, or the entire memory consumed by all of a customer's volumes. Such a workload is not perfectly suited for MySQL, so Ceilometer turned out to be resource-hungry and thus unpopular with admins in the OpenStack environment.

Gnocchi is an attempt to fix these problems. This time series database was specially designed for data in a metering and billing context. Gnocchi assumes the role of the storage engine, and Ceilometer populates it with the acquired data. The Ceilometer back end in Grafana lets you to read several values from Gnocchi and display them in a visually appealing way.

Grafana does not create a complete billing solution with Ceilometer and Gnocchi. If you want to use the data from Gnocchi for billing purposes, Gnocchi still needs to be integrated into the billing system. The strength of the Grafana-Gnocchi combination is that it visualizes the workloads of individual projects, on which you can base a forecast as to when the next expansion stage will be needed. If relevant data is available from all the projects – even better!

In the context of OpenStack, Grafana also offers the only good approach to visualizing the Gnocchi data in a way that lets you draw conclusions at a glance. OpenStack cloud admins could thus do worse than taking a closer look at Grafana.