If you have tried a manual OpenStack installation before, you will have noticed that some configuration steps are identical among the various OpenStack components; that is, you perform the same operations several times. If this makes you think of automation, you are right on track, and Kickstack provides an approach to solving this problem: Based on Puppet, it offers meaningful and efficient administration of OpenStack clouds.

Cloud computing installations in particular must be able to scale massively horizontally – the perfect use case for classic system automation. So, if you want to know how to set up your own OpenStack cloud within 20 minutes, you have come to the right place.

OpenStack Automation

Regular ADMIN readers might recall reading an article a few months ago that explained how OpenStack components work [1]. The article demonstrated one thing: Installing OpenStack is no trivial matter if you want to do it manually. Various services need to be distributed to their own servers if you do not want an all-in-one setup. The problem is compounded by the many commands in the scope of the OpenStack setup that are not very intuitive and make little sense at first glance. Those who have not worked with technologies such as Open vSwitch will find it difficult to understand many of the commands in the installation of OpenStack Neutron.

Kickstack is an additional layer that resides between the admin and OpenStack, using Puppet modules that have existed for some time for all core components of OpenStack, while Kickstack takes on the automation tasks. Although normal Puppet modules can also install the OpenStack services in an automated process, it requires painstaking configuration of various Puppet parameters, so time benefits are mostly non-existent, especially compared with a manual installation.

Kickstack tidies things up radically. It automatically takes care of creating data, such as Keystone users, ensures that all services of the OpenStack environment and communications with their databases use password protection, and ensures that the OpenStack services run on the hosts previously allocated by the admin. For a variety of parameters, from which the administrator can choose in the Puppet modules, Kickstack filters out the most important ones and then does the rest automatically.

A specially introduced role system assigns specific roles to the various machines in the OpenStack installation. To understand the motivation behind this, I’ll review the components that make up OpenStack.

Service Components

OpenStack is not a massive program but a collection of components. This fact is of fundamental importance to understanding the way Kickstack or Packstack work (see the “Kickstack vs. Packstack” box). Table 1 lists the nine core components of OpenStack.

The crux of the matter is that even the OpenStack components are not individual programs but, in turn, break down into several modules. Virtually every service in OpenStack has its own API that exposes a RESTful API and thus ensures that the service is controllable via the HTTP protocol. From a technical perspective, it can be quite useful to expose these APIs on the Internet, as it does for Rackspace in their own OpenStack cloud. However, this approach involves a setup in which the individual component modules are distributed across different systems, because the other components do not necessarily, or should not, run where the API components do.

Further atomization takes place when individual parts of an OpenStack component must be distributed across several systems. The cinder-volume service, for example, is the interface that connects VMs on hypervisor systems with the assigned persistent storage. The service therefore runs on hosts where space is available on disks. cinder-scheduler usually runs on a different host – the component coordinates access to multiple storage back ends, if they are configured. Then you have the services that need to be available multiple times within an OpenStack installation: nova-compute, which starts virtual machines on behalf of Nova, is a good example.

Roles Not Programs

To deploy OpenStack meaningfully, it is necessary to think in terms of roles and not individual components. Obviously, it would not make much sense to distribute the services that belong to OpenStack as separate groups on different computers.

Kickstack therefore uses a different approach and assumes that a variety of different roles can be defined within an OpenStack installation more or less as factory defaults. Each role is characterized by a specific combination of services. You need to understand the group schema before starting to work with Kickstack, because the role system is central to Kickstack. Kickstack uses the roles in Table 2 by default.

From the designated node roles, you can create a mixture of nodes in Kickstack; arbitrary role combinations are possible – even all-in-one installations. In this case, all roles would be assigned to a single node at the same time. That would not exactly be the typical production system, however. The trend is toward defining different node types in OpenStack, which then perform specific tasks.

Moreover, different roles can be assigned to more than one node at the same time; it would not make sense to have just a hypervisor node, for example. The Compute node can thus be assigned to an arbitrary number of nodes.

Kickstack Installation

The basic approach of Kickstack fundamentally relies on these defined roles. Working in the web interface of the Puppet Master, the admin adds the Kickstack nodes to be managed to the appropriate groups and then assigns the desired roles to each node, again in a web interface. The node itself then just needs the agent for Puppet, which implements the defined node configuration (Figure 1). For this process to work, you first need a working Kickstack installation, and this in turn inevitably means having Puppet in place.

Figure 1: Running the Puppet agent on the servers applies the predefined rules.

A Suitable Lab Environment

If you want to build the sample setup described in this article, you do not need tons of hardware: The proposed installation will easily install on four Ubuntu LTS version 12.04 virtual machines. One of these machines is the Puppet Master, simply known as puppet, and the three other computers, Alice, Bob, and Charlie, run the OpenStack installation. Specifically, the VMs, provided in this example by VirtualBox, do not have less than 1GB of RAM. As usual, the more RAM, the merrier.

If you have bare metal at hand, a setup of this kind can be built on real hardware, allowing you to test OpenStack under realistic conditions. The only restriction in such a scenario is that Charlie needs three network ports, whereas Bob and Alice need at least two each. This requirement is determined by the OpenStack network service, Neutron, and not by Kickstack.

Step 1: Puppet

To begin, you need to set up the Puppet infrastructure. The example is based on a classic Puppet architecture comprising a master server and clients that obtain their configuration from that master. For this setup, you need to ensure that the hosts can reach each other by means of a working DNS and that the hostnames of the machines are registered correctly in DNS. After the installation, it is advisable to edit the Puppet installation that exists below /etc/puppet/puppet.conf on Ubuntu. You will want to enable Puppet reports and storage of the Puppet information in a database. Enabling exported resources is also important in a Kickstack context. The configuration parameter required for this is storeconfigs=true. However, the Puppet developers suggest that new users might prefer to use PuppetDB, which produces the same effect and also supports exported resources.

After preparing the Puppet Master, the nodes follow, which also need a puppet.conf instructing them to connect with the Puppet Master. Next, almost all the OpenStack modules required for Kickstack can be found on PuppetForge (Figure 2). The exceptions are the modules for Neutron and Open vSwitch. The module for Glance also had to be installed manually, because the version available on PuppetForge had an annoying bug.

Figure 2: The module list for the Puppet Master makes it clear that Kickstack relies on the OpenStack Puppet modules.

Step 2: Kickstack

The current version of Kickstack is available directly from GitHub or PuppetForge. Once the module and its dependencies are installed, you can begin.

The previously described node roles need to be distributed in a meaningful way to the available nodes. The example here follows a conservative approached to the distribution: Charlie becomes the network node that runs all Neutron components except the Neutron API, Bob represents the computing node and owns the appropriate role, and all other roles are assigned to Alice, who handles the lion’s share of the work.

The following example assumes that the individual nodes have already been made part of the Puppet installation after installing Kickstack on the Puppet Master. In this case, they appear in the Puppet Dashboard under Nodes after the admin logs in.

Pressing Groups in the menubar at the top takes you to the only group that currently exists, named kickstack – nodes that take over tasks in the cloud in the context of Kickstack should be members of this group (Figure 3). Pressing this kickstack link followed by another click on Edit allows you to add nodes to the group.

Figure 3: The Puppet Dashboard with an overview of the important configuration parameters of the kickstack group and the participating nodes.

The editing window for the group also provides access to the main settings related to the OpenStack Cloud; for example, if you want to use full virtualization within OpenStack (i.e., Qemu without KVM), then the value in kickstack_nova_compute_libvirt_type is correct. Also of great importance are the entries that specify which network types reside on which network interfaces. OpenStack distinguishes between

• kickstack_nic_management, the management network, which directly connects the individual services on the nodes (eth1 in this example),

• kickstack_nic_data, the data network, which routes the traffic between VMs running in the cloud (eth2 here), and

• kickstack_nic_external, the external network, which connects the virtual machines with the outside world (i.e., eth3).

If you are trying out Kickstack and OpenStack on virtual machines, it is a good idea to create the VM network interfaces so they match this scheme; if you are installing on bare metal, you might need to modify the parameters of the kickstack group. If the values cannot be modified uniformly for all nodes (e.g., because the network node uses a different NIC for the management network than the API node), you can also customize the parameters separately for each node in the next step.

The kickstack_cinder_lvm_pv parameter for the node that owns the Storage role (i.e., Alice in this example) is particularly important: Kickstack automatically converts the device stated here (/dev/sdb) into a logical volume in LVM to make it usable for Cinder. Once the configuration of the Kickstack group is correct, you can continue with the roles.

Step 3: Assigning Roles

Pressing Nodes in the menubar of the Puppet Dashboard takes you directly to the individual entries of nodes that Puppet knows. Clicking on a node name and then on Edit takes you to the configuration dialog for that node.

The penultimate box at the bottom marks the classes to which a node belongs: You can use autocompletion to assign the appropriate roles here. For Charlie, this is kickstack::node::network, for Bob kickstack::node::compute; all the other roles are assigned to Alice.

The end result is a series of calls to the Puppet agent on Alice, Bob, and Charlie. If the Puppet agents there are not running continuously in daemon mode, you might have to perform several Puppet runs consecutively to apply all the roles (remember, Puppet understands dependencies; in other words, it only performs those tasks in a Puppet run that are not prevented by an unfulfilled dependency).

After the final Puppet runs, you will see in the dashboard on the left-hand side that each role has been assigned once within the Kickstack class (Figure 4). OpenStack is already running: The OpenStack Dashboard (Figure 5) is accessible at http://<IP-of-dashboard-node>/horizon (e.g., on http://192.168.122.111/horizon for Alice). The credentials required for logging in can be found in the openstackrc file, which resides in /root on the node with the auth role. Kickstack leaves one task to the admin: creating networks in Neutron, which can be done manually with scripts, or retroactively via the Dashboard. The same is true for importing an image – CirrOS is recommended for testing.

Figure 4: After assigning all node roles, you can view the results in a pane on the left in the Puppet Dashboard’s Class overview.

Figure 5: After installing OpenStack with Kickstack, you need to create a network for the admin tenant.

Conclusions

Kickstack is a welcome change for those who want try OpenStack but are wary of the time-consuming and complex installation. With Kickstack, you simply install Puppet and the required Kickstack modules. In fact, a basic Puppet setup with one master and several Puppet clients can be established relatively quickly. Additionally, configuring the individual nodes to create OpenStack machines is much more convenient in the built-in Puppet Dashboard than setting up the services manually (Figure 6). If you want to extend your OpenStack installation later to add more computers, you can do so at the push of a button in the GUI.

Figure 6: You can easily see the OpenStack services that originated with Kickstack.

However, this is not the end of the road for Kickstack in terms of desirable features: In particular, the solution still falls short when it comes to high availability. First, the OpenStack project has had to develop a concept of how it intended to implement high availability; second, typical HA tools such as Pacemaker are not very well integrated into Puppet right now.

Info

[1] “OpenStack” by Martin Loschwitz, ADMIN, 2013, issue 17, pg. 28

The Author

Martin Gerhard Loschwitz is Principal Consultant at hastexo, where he is intensively involved with high-availability solutions. In his spare time, he maintains the Linux cluster stack for Debian GNU/Linux.