Robot Admin

Ansible and SaltStack, with their ability to create entire OpenStack and Ceph setups on their own, have become hot topics in automation and are the focus of major distributors. Just a few years ago, neither were on the radar; instead, Puppet and Chef were vying for the admin's favor.

However, they are far from history: Puppet and Chef continue to enjoy a large fan base, and both tools are still under active development. The new 2.0 version of Chef Automate, for example, officially celebrated its premiere in front of an audience at the Chef Conference in May 2018.

As usual, the manufacturer is not stingy with its promises: Chef Automate 2.0 vows to make everything better, faster, more convenient, and, of course, more automatic than its predecessors. In this article, I take a close look at the innovations in version 2.0 of Chef Automate.

What Is Chef Automate?

Many admins may have already met Chef in one form or another, and most think intuitively of Chef as the tool responsible for automation. In fact, Chef is still the core of the solution – but Chef Automate includes significantly more components.

First introduced in mid-2016, Chef Automate was intended to add various automation functions, with the manufacturer thus promoting the platform as a continuous automation tool; the obvious implication of continuous integration and continuous delivery (CI/CD) is no coincidence.

Automate now comprises three tools. All are open source software and, in principle, handle their tasks independently of each other. InSpec was added to Chef Automate to check compliance measures on systems. Automate also includes Habitat, a CI/CD framework that allows developers to develop, build, and distribute their applications fully automatically. Along with Chef, these components result in a complete automation toolchain.

Version 2.0 adds another component, Chef Workstation, which is intended to bring the many functions of the major Chef solution to the local computer and make life easier for both developers and admins.

Additional components include various dashboards and analysis tools that glue everything together, graphically display the state of the entire environment, and show what is going on at a glance. Remarkably, it is this glue that sets Automate apart from the simple combination of the three components described above, and it is precisely this part that is undergoing fundamental restructuring in version 2.0.

Chef developers noticed that DevOps practitioners have a vested interest in shifting new features and functions into production as quickly as possible, this being a central premise of DevOps. Instead of developing a new function behind closed doors for weeks and months and rolling it out as a large release with troubles to match, DevOps follows the principle of taking small steps and rolling out features to production as soon as they are available and tested. Many companies that follow the DevOps paradigm prioritize this goal.

The scope for interpretation between theory and practice becomes apparent every day in IT departments. In many cases in DevOps practice, very little remains of the noble goal of fast releases. The end result is an operating mode that has certain agile components, but to a large extent does not offer the flexibility that the stakeholders hoped for during implementation. Chef supports this claim with concrete figures, showing that many companies that have introduced DevOps principles have had only partial success [1].

From System to App

Chef attributes some of this failure to thrive to the toolchains companies use for their projects. Especially when it comes to Chef, you need to clean up at home first. Chef has always assumed the system to be the smallest unit to be maintained. The dashboards available in Chef Automate, for example, provided information on the status of individual systems. However, it was not possible to monitor the roll-out of individual applications.

Chef Automate was also not particularly well interwoven with other solutions from the CI/CD environment. Historically, when Chief development began several years ago, the maintainers focused on finding automation solutions for those tedious day-to-day system administration tasks that, up to that point, had been done manually in batches or pieced together in scripts.

The idea that automation could play a role in the CI/CD context did not develop until the advent of Docker and the like, when containers became the focus of interest. In such scenarios, you no longer need to roll out the application using an automation tool. Instead, you can use the automation tool to provision the application as a container. However, Chef offered very few benefits in these cases, although in version 2.0 of the software, this is expressly intended to change.

Complete Redesign

Chef Automate 2.0, say its developers, has been completely rewritten and now has a Go-based architecture. Also, version 2.0 now relies on a microarchitecture in which several components interact, which means that the individual components are now – in the best cloud style – RESTful API interfaces that can be controlled externally through standardized protocols. At the same time, the administrator will be able to manage all the processes of the various Automate components in a uniform interface.

The numbers Chef handles in this context are quite impressive: Chef Automate 2.0 is said to be able to control and process several tens of thousands of nodes, and it will be completely irrelevant where they run in the future. Whether on bare metal or as virtual machines (VMs) in a public cloud, Chef Automate 2.0 knows how to handle all the layouts.

The completely redesigned GUI that most users will use for their daily work with Automate 2.0 is a web interface (Figure 1). A kind of news ticker informs quickly and clearly which actions are taking place on the platform. If something goes wrong, a clear warning appears. Trending graphs and a separate query language are also included in Automate 2.0.

Figure 1: The Chef Automate 2.0 GUI has been completely redesigned, but the changes under the hood are just as massive.

Chef Automate 2.0 also includes various compliance functions, which I discuss later in this article. In the future, Chef Automate will help find compliance violations as early as possible through a "detect, correct, automate" process under a single plane of control.