Causal Chain

If you want to take home a message from the coronavirus pandemic, one would be that it is acting as a catalyst for digitization in all areas of life. However, it is precisely this phenomenon that presents IT administrators with new challenges because environments are not only becoming more complex but also more diverse, which goes hand in hand with an increase in administrative overhead. Gone are the days when infrastructures could be managed manually. Modern infrastructure management environments open an opportunity for avoiding error-prone manual adjustments.

Insiders have bandied about Infrastructure as Code (IaC) as the key to tomorrow's infrastructure management for some time now. IaC is to be understood as an abstraction solution for managing the hardware and software components of an IT infrastructure. Machine-readable definition files are used instead of a physical hardware configuration or special configuration tools. In this article, I describe the basic structure and practical use of StackStorm, "a platform for integration and automation across services and tools [that] ties together your existing infrastructure and application environment so you can more easily automate that environment" [1].

StackStorm at a Glance

StackStorm [2] is often mentioned in the same context as SaltStack, which was acquired by VMware, and Ansible; however, the comparison is misleading because StackStorm focuses on running management tasks or workflows on an event-driven basis. In particular, the tool defines triggers and events, which it then reacts to when those triggers or status changes occur. StackStorm supports automatic correction of system settings, security reactions, rules-based troubleshooting, and deployment. The tool also has a rules engine and a workflow manager.

In the major leagues, StackStorm is still a fairly unknown player that targets the integration and automation of services and tools. The goal is to capture an existing infrastructure and application environment and react automatically across the infrastructure when certain events occur.

A few examples illustrate the potential of using StackStorm. For example, assume you rely on Nagios for infrastructure monitoring; you could use StackStorm to trigger further diagnostic checks and make the results available to third-party applications. That's not all, though: You can also use the tool for automated remediation by monitoring critical systems and initiating follow-up actions when errors are identified. Finally, StackStorm supports you during deployment. For example, you can use the tool to deploy a new AWS cluster or activate a load balancer for essential load distribution in the event of an imminent system overload.

From Event to Action

To initiate various actions, the system needs to know about the corresponding states or events. To do so, StackStorm draws on various sensors, which are Python plugins for the integration of the various infrastructure components (Figure  1). When a sensor registers a defined event, it issues a StackStorm trigger. The management environment distinguishes between generic and integration triggers. You can define your own trigger types in the form of a sensor plugin, should StackStorm itself not provide the desired trigger type.

Figure 1: In StackStorm, events from the sensors are aggregated then matched with triggers; if necessary, actions are then triggered. Workflows are optional.

Another important element of the solution is actions, which subsume all actions the management environment can perform on infrastructure components. Most actions are executed automatically, but it is also possible to run commands from the integrated command-line interface (CLI) or with an application programming interface (API). The most common use case, however, is the use of actions in rules and by triggers.

StackStorm also has workflows that bundle different actions into what it dubs "uber-actions"; they define the execution order and the transition conditions and take care of the data transfer. Because most automation tasks are a sequence of two or more actions, workflows should be considered the defining element in the StackStorm environment. To simplify the execution of recurring tasks, the tool has a workflow library.

In the form of "packs," the management environment provides another function for bundling tasks that supports grouping of integration functions (triggers, actions) and automation mechanisms (rules, workflows). A growing number of integration modules for Active Directory, AWS, Icinga, Jenkins, Exchange, MySQL, Nagios, and OpenStack are available from the StackStorm Exchange platform [3].

StackStorm also has an auditing mechanism that records all relevant details for manual or automatic execution. The core function of the tool can be described by the cycle of trigger, rule, workflow, action, results. According to the StackStorm website, several well-known companies (e.g., Cisco, NASA, and Netflix) are already using these environments in their IT infrastructures.

Getting Started Quickly

StackStorm was developed for Linux-based operating systems and cooperates especially well with Ubuntu, Red Hat Enterprise Linux, and CentOS. The installation is particularly easy on a new Linux installation. Make sure that Curl is present; then, working as an administrator, run the command to install:

curl -sSL https://stackstorm.com/packages/install.sh | bash -s -- --user=st2admin --password='<secret>'

This command installs a full StackStorm version. The developers explicitly point out that problems are bound to occur on Linux systems with enterprise applications already installed. If you are running the installation behind a proxy server, export the proxy environment variables http_proxy, https_proxy, and no_proxy before running the script:

export http_proxy = http://proxy.server.com:port
export https_proxy = http://proxy.server.com:port
export no_proxy = localhost, 127.0.0.1

Firewall settings might need to be adjusted to access the web GUI.

The core function of StackStorm is provided by the st2 service, which you will find in /opt/stackstorm/st2. This service is configured by the associated configuration file /etc/st2/st2.conf. StackStorm has its own web GUI, which can be found in the directory /opt/stackstorm/static/webui and is configured by the JavaScript-based configuration file webui/config.js.

The developers prefer to use the CLI with StackStorm. Some basic commands will help you familiarize yourself with the environment. For example, to output the version in use and view the available triggers, actions, and rules, use the commands:

st2 -version
st2 action list --pack=core
st2 trigger list
st2 rule list

StackStorm not only has some default triggers and rules but also various predefined actions. You can retrieve these with the same scheme. To retrieve the list of all actions in the library, get the metadata, view the details and available parameters, and initiate an action from the CLI, use the respective commands:

st2 action list
st2 action get core.http
st2 run core.http -help
st2 run key=value '<arguments>'

To execute a Linux command on multiple hosts over SSH, you can use the core.remote action. All that is required is that passwordless SSH access is configured on the various hosts. Execution is according to the scheme:

st2 run core.remote hosts='<www.examplehost1.com>, <www.examplehost2.com>' username='<SSH user>' -- ls -l

You can view the action history and execution details and list executions with:

st2 execution
st2 execution list

To limit the output to the last 10 executions, use

st2 execution list -n 10

Rules are an essential tool of the StackStorm concept. The tool uses rules to execute actions or workflows when specific events have occurred in the IT infrastructure. Events are usually registered by sensors. When a sensor detects an event, it fires a trigger, which itself triggers the execution of a rule again. The conditions of such a rule determine which actions take place.

By default, a StackStorm installation has a sample pack that includes various sample rules. One of them is the Sample rule with webhook (Listing 1). The rule definition is a YAML file that includes three sections: trigger, criteria, and action. This sample is designed to respond to a webhook trigger and apply filter criteria to the contents of the trigger.

name: "sample_rule_with_webhook"
pack: "examples"
description: "Sample rule dumping webhook payload to a file."
enabled: true
trigger:
      type: "core.st2.webhook"
      parameters:
           url: "sample"
criteria:
      trigger.body.name:
           pattern: "st2"
           type: "equals"
action:
      ref: "core.local"
      parameters:
           cmd: "echo \"{{trigger.body}}\" > ~/st2.webhook_sample.out ; sync"