Stand and Deliver

Visualizing data trends and issues when they arise is clearly of significant benefit, whether for applications or the systems on which they run. Having access to sophisticated tools that can provide highly customizable options to create bespoke visual dashboards is key and an important facet in being able to maintain high levels of uptime. A welcome bonus is if such tools are compatible with multiple types of data sources, so you can centralize your single-pane-of-glass viewing platform.

Step forward, Grafana [1]. It has the capability to gather metrics from all types of applications and treat each as an individual data source. From there, it is possible to query and filter results precisely to your needs. Coupled with another exceptional bedmate, Prometheus [2], it is possible to incorporate monitoring and alerting into a uber-capable, bespoke visualization toolset. Whereas Grafana is open source visualization software, Prometheus is an open source event monitoring and alerting tool. The two applications do have some crossover [3], though, and the dashboards from Grafana sit well with Prometheus' sophisticated time series handling [4].

In this article, I look at Grafana and Prometheus and how to get started with dashboarding.

Spirographs

For the example in this article, I'm using Linux Mint 20 (Ulyana), which has the long-term support (LTS) version of Ubuntu 20.04 running under the bonnet. In true DevOps style, I will not install Grafana locally, because in the past, I have seen some compatibility issues when running with Prometheus if it was also installed locally. Instead, I will use Docker containers for both Grafana and Prometheus.

For Grafana, I am using the clear instructions on the Grafana website [5]; remember to get the latest versions and decide whether to use the Alpine or Ubuntu version, depending on your needs. The simple command to fire up the latest version is:

$ docker run -d -p 3000:3000 grafana/grafana
3fa2f72de6bc4fd61a0cecdb1d18a8bf1aaf7b71b49f731fe4ae6eaab841d3fd

If you want to run a specific version, which some applications might prefer for compatibility, you can use a command that explicitly states a version as the image tag:

$ docker run -d -p 3000:3000 --name grafana grafana/grafana:6.5.0

After running the non-specific version of that command, you should see the output in Listing 1 (abbreviated here) to see which containers are running.

$ docker ps
CONTAINER ID   IMAGE               COMMAND      CREATED            STATUS
3fa2f72de6bc   grafana/grafana     "/run.sh"    25 seconds ago     Up 24 seconds

Gooey

If you now check TCP port 3000 in a browser by visiting http://localhost:3000/login , you will see the aesthetically pleasing Grafana interface (Figure 1). The user and password are admin , which you are immediately prompted to change at login.

Figure 1: The login page of the open source Grafana user interface.

On the right side of the Welcome page, click Create your first dashboard (Figure 2). Next, choose the Add new panel button to continue (Figure 3). You are then offered the Random Walk dashboard (Figure 4). If you select the small blue question mark icon next to default and choose the – Grafana – option in the pull-down menu under the Query box on the left side, you can read the description for that data source.

Figure 2: Creating your first dashboard in Grafana.

Figure 3: Add a new panel to get started.

Figure 4: Random walk example dashboard.

Here, you can flick through the different data source types and read the descriptions, which I'll look at in more detail in a moment. The default data source (Grafana Fake Data Datasource – Native Plugin) demonstrates nicely what you can expect from a simple panel; you should try zooming into it by selecting specific time periods to become familiar with this functionality.

Options allow you to adjust your query alongside transformations and alert settings. Under Transform , the following narrative is provided: "Transformations allow you to join, calculate, re-order, hide and rename your query results before being visualized." You can find more information on the Grafana website [6], which explains that "Transformations process the result set of a query before it's passed on for visualization." Apparently, when you have multiple dashboards with a high volume of queries, "the ability to reuse the query result from one panel in another panel can be a huge performance gain." As a result, it's well worth reading up more about transforming your data if you use dashboarding extensively.

If you click the Save button at top right and name your dashboard something like Random Walk , you can come back to it later. Next, click the Grafana logo at top left and go back to the Welcome screen.

Golden Source

As mentioned, Grafana supports a number of data sources that you might want to explore (Table 1). The documentation notes that, as well as setting up custom data sources, three other sources are treated a little differently, as you saw when you clicked the small blue question mark icon a moment ago to read their descriptions.

First, the – Grafana – test source helps you test graphing and dashboards' visual aspects. The – Mixed – data source allows you to query several sources in the same panel, so that every time you create a new query, you can choose a different data source. Finally, the – Dashboard – source lets you "use a result set from another panel" within the same dashboard. All in all, it's possible to use these special data sources to add extra flexibility to your graphing.

Many applications can push data to Grafana in a sane and compatible format. If you look at the documentation for Grafana plugins [7], you might be pleasantly surprised at the multitudinous array of community and officially provided plugins that help you get data into Grafana. Of the tens of plugins available, a number of well-known names include the monitoring solution Zabbix, the container orchestrator Kubernetes, the analytics engine Elasticsearch, the code repository GitHub, the databases MySQL and PostgreSQL, and the popular alerting tool Datadog. Incidentally, Grafana helpfully provides a GitHub repository [8] to help you get started creating your own plugins.