In the Stacks

OpenStack has marginalized its rivals with an impressive pervasiveness. Although Eucalyptus and CloudStack were technically on par with OpenStack, today they play only a minor role. The technical body of OpenStack hardly justified the attention it got in the early days, but the solution has now matured significantly, especially in the last two years: Many problems that used to make life difficult for admins have been eliminated, and the biannual OpenStack conferences show that some major corporations now rely on OpenStack.

The greatest practical challenge is its complexity. When you build a cloud environment with several hundred or several thousand nodes, most concepts that are fine for conventional data centers no longer apply, both when planning the physical environment (e.g., structural conditions, racks, servers, hardware, network, and power) and the software that provides the core functionality.

The complexity of the construct means that getting started with OpenStack requires much education and practice, and the path to the first OpenStack installation running on virtual machines (VMs) as a test setup can be long and fraught with pitfalls.

This first part of this series revolves around introducing the most important OpenStack components and methods; these are the current defaults for deploying OpenStack. Part two takes a hands-on approach and describes the OpenStack installation on VMs based on Metal as a Service (MaaS) and Canonical's own automation solution, Juju. Finally, the third part deals with transferring what you have learned into practice: What can be observed in large OpenStack installations in terms of daily life at the data center? How can high availability be achieved? What options are available for software-defined networking (SDN) and software-defined storage (SDS)?

An Entire Cosmos of Components

When talk turns to OpenStack, you might intuitively think of just one big lump of software. However, OpenStack is a collection of many programs (Figure 1) that share communication channels in the background and have one important element in common: The entire OpenStack source code is written in Python.

Figure 1: Although OpenStack comprises many more services than those shown here, six components is enough for a basic cloud: Keystone (Identity), Glance (Image), Nova (Compute), Neutron (Network), Cinder (Block Storage), and Horizon (Dashboard) [1].

What exactly is a part of OpenStack and what is not is no longer clear: In the early days of the project, the term was reserved as a trademark for the services officially referred to as the core services. A single component only became part of the core after spending a while in incubator status, because it needed to be backed by a large community. The OpenStack Technical Committee officially decides when to include a component in the core services.

The Big Tent initiative put an end to that setup; now, a component only needs to promise to adhere to some basic rules to be considered a part of OpenStack. The original idea behind the Big Tent was to increase innovation within the project, and the developers have achieved this goal, with more than 30 components now. Fears that the Big Tent would dilute the OpenStack brand with immature services have not materialized – or at least not yet.

However, distinctions are still made between the collection of core services and ornamental accessories. To provide useful services with an OpenStack cloud, you need six components: Keystone, Glance, Nova, Neutron, Cinder, and Horizon. The six names are code names, by the way; each OpenStack service also has an official name that is hardly ever used, even though it would be easier to guess the function of a service. Glance, for example, is officially the Image service. By the way: individual OpenStack components are typically not single programs but are in turn divided into several services (e.g., Glance has an API and a registry service).

Support Services

Before I look into the OpenStack components, I'll look at two services that can be found in practically every OpenStack production cloud: the message broker RabbitMQ and MySQL. The OpenStack components use MySQL wherever they need persistent metadata (i.e., run-time data caused by user interaction or administrator intervention, such as the created users or the active VMs.

The need for RabbitMQ is slightly more complicated. Why does cloud software need a remote procedure call (RPC) service in the background? According to OpenStack developers, the individual service components need to communicate with one another across node boundaries. A good example of this is Nova, the virtualization service in OpenStack: A command to create a new VM reaches its API component and creates a new VM on an arbitrary hypervisor node. The communication between Nova's individual services is handled via the Advanced Message Queuing Protocol (AMQP) offered by RabbitMQ.

Alternatives to RabbitMQ and MySQL are possible. Because access to RPC relies on an internal Python library that all services in OpenStack use and that supports the Apache messaging service Qpid, RabbitMQ could be replaced with Qpid without any problems. Similarly, MySQL could be replaced by PostgreSQL without difficulty. In practice, though, MySQL and RabbitMQ dominate the scene.

Keystone: The Bouncer

The Keystone service is responsible in OpenStack for everything that has to do with user authentication and authorization. Keystone joined OpenStack far later than Nova or Glance, but it is essential. Keystone is only a single application: The service is designed as a RESTful API, which all OpenStack services use, that can be addressed using HTTP commands. APIs are OpenStack's standard approach for responding to requests from users or other components.

Keystone's main task is to issue tokens to clients: A token is a temporarily valid string that a user can use to authenticate against other services. When a user asks Keystone for a token with a valid combination of username and password, they can then embed the token as a header in HTTP requests to other services. Keystone can do even more. In the background, the service can connect to LDAP, and it is also used for authentication by the OpenStack components themselves: If a service wants to send a request to another service, it needs a valid token, like any other client (Figure 2).

Figure 2: Keystone governs authentication and authorization in OpenStack using tokens [2].

Keystone is internally organized along the lines of users, clients, and roles. The users belong to a specific client and have a specific role within this client. At their sole discretion, admins determine the roles, whose permissions are set at the level of each individual service.