Rollback

Puppet and Chef also take quite different approaches when it comes to rolling back actions. The procedure is clear with Chef: Because a cookbook is essentially a collection of commands, you can logically define a counter-function for each function. If a cookbook contains a rollback parameter, you can call it to revoke the changes already implemented and restore a previous state. Many Chef cookbooks use rollback parameters already.

In Puppet, the issue is more complex. Because of the previously mentioned declarative nature of the host configuration, a Puppet setup only defines a desired state; Puppet doesn't have a clue what the previous state on a system looked like following a call, although it does know what the current state should look like.

A useful rollback is basically impossible with Puppet. To undo configuration changes, the admin needs to store the desired previous state in the form of a manifesto in Puppet and then apply it to the affected hosts. This task can be tricky in extreme cases: When Puppet overwrites files during a call, it does not create a backup.

Admins thus need to create a backup manually and before Puppet deployment. Moreover, best practices dictate sending any changes through a staging installation before rolling them out into production.

Scary Updates

The last and biggest stumbling block I will describe relates to updating components that belong to an automated installation. Chef has left a lasting impression with admins in this regard, and not a good one.

Time and time again, changes to the Chef server in the scope of Chef updates have affected compatibility with previous versions. At worst, the admin then sits sweating in front of a dead server, wondering what to do next. Puppet has largely been spared problems with server and client updates, at least so far, but that does not mean you can always expect things to go well when system components are updated.

It is typically not the Puppet components themselves that cause trouble, but Puppet modules off the Internet. The modules are typically only quality-controlled by their own authors. Whatever plugins you install yourself from PuppetForge should therefore be regarded as untested (Figure 2).

Figure 2: Quality checks for Puppet modules, like this one for OpenStack Neutron, are mostly left entirely to the developer, which can lead to problems in the real world.

Module updates can also be a problem, especially when they are extensive: The Puppet modules for OpenStack bear witness to the disaster that awaits admins in a worst-case scenario.

The update of OpenStack Havana to Icehouse, for example, introduced so many new features and incompatibilities that manifests for the predecessors were practically useless. When combined with the lack of rollback mechanisms in Puppet, this kind of setup necessitates an appropriate staging system that gives admins the ability to test new modules or configurations customized for the new situation.

Quality of External Modules

For both Puppet and Chef, the quality of many external modules leaves much to be desired. Puppet users experience this problem over and over when they need to integrate a module off the web with one of the External Node Classifiers; I'm referring to front ends such as Foreman [7], the Puppet Dashboard, or, more frequently of late, Hiera.

The UIs work similarly: Using classes, admins can assign individual functions to their hosts. For this principle to work, however, all essential functions of a Puppet module must be available as a class.

Many Puppet modules assume that the host configuration is defined in a site manifest (site.pp) and that functions for hosts can be called without any detours – but this is a function the graphical front ends usually lack (Figure 3).

Figure 3: In some cases, whole classes or groups disappear from Puppet modules or Chef cookbooks – much to the dismay of the admins during upgrades.

The excitement of discovering precisely the Puppet module you need can quickly turn to frustration if the module comes without wrapper classes for its functions. With Chef, the quality scatter is just as bad as with Puppet (Figure 4), although many US corporations prefer to use Chef with cookbooks rather than Puppet with modules.

Figure 4: Chef comes with its own pitfalls, although they are different from the problems you'll face with Puppet.