On the Stack

A fierce battle is going on in the world of IT, and it has nothing to do with mobile operating systems or their hardware incarnations. This battle is for future leadership in cloud computing: the popular buzzword that has fired the imagination of analysts [1], consultants [2], hardware vendors, data center operators, and CIOs of multinational corporations.

This continuing euphoria is based on the dream of needs-driven computational and storage capacity. Amazon popularized this model in the form of the Elastic Cloud (EC2), using cloud services to generate revenue with the online store's unused server capacity outside of the Christmas and peak season. A host of competitors have copied this approach, and even classical hosting service providers are rolling out their own versions of cloud services. All of these offerings are generally referred to as public clouds: a cloud of computers and storage that offers consumption-dependent, Internet-based services to anybody with a credit card.

The need for security and access control soon gave rise to the desire for private cloud technologies. The idea behind a private cloud is to create an in-house cloud solution that makes optimal use of existing resources while offering the same kind of scalability and failover provided with a commercial cloud service. Ideally, the private cloud communicates via the same interface available to a public service, making it possible to design inter-clouds that rely on the private cloud for daily business but still have access to public resources in times of peak usage (cloud burst) or for purposes of load balancing (cloud balancing).

The OpenStack initiative, which was initiated in July 2010 by US hosting service provider Rackspace, NASA, and other partners, is designed to provide provide an IaaS (Infrastructure as Service) layer for the cloud, virtualizing the various hardware components and making them available through a standard API. The first OpenStack release (dubbed "Austin") became available October 22 [3];

The OpenStack project consists of two primary components: OpenStack Compute is a set of tools for provisioning virtual instances and managing the virtual server environment; OpenStack Storage lets you manage the use of a distribution object database (OpenStack Storage). Internally, these two components are still referred to by their codenames, Nova and Swift, both of which have their origins with Rackspace products. NASA contributed its own open source hybrid cloud platform, Nebula [4], to the OpenStack framework. NASA and Ubuntu has both announced that they are leaving the alternative Eucalyptus cloud project in favor of OpenStack [5] [6].

This article shows how to set up your own simple test system to start exploring the OpenStack environment.

Anatomy of the OpenStack Cloud

According to the OpenStack project, OpenStack Compute is "software to provision and manage large groups of virtual private servers." The various components within OpenStack Compute let you "run instances, manage cloud networks, and manage access to the cloud through users and projects." OpenStack does not provide its own virtualization. Instead, the OpenStack platform offers a structure for deploying other virtualization technologies in a cloud-based setting, with easy access to the scalability and failover features commonly associated with the cloud. The OpenStack API can communicate with virtualization environments such as KVM, Xen, HyperV, and QEMU.

If you are familiar with the Amazon web service, you will not be surprised to discover that OpenStack includes many similar features. Virtual machines are assigned to a project. Each user has a key pair (private, public) for access to one or multiple projects. Each project is specifically assigned processor cores, storage and RAM, and so on.

The use of the API is managed by means of Role Based Access Control (RBAC). Figure 1 shows a view of the OpenStack architecture.

Figure 1: The components of the OpenStack architecture provide a versatile environment for cloud computing.

Cloud information and settings are all stored in the database. The REST API gives users the ability to address any cloud component. The current release highlights the command line as a means for reaching the API, but eventually, interaction with the API will happen through fat clients or web applications (web dashboard). The Auth Manager manages role-based access to the API functionality, thus avoiding users starting and shutting down individual instances. The Scheduler distributes access and computational time to individual, virtual components.

Not all of OpenStack's components are fully implemented in the current release. For example, the Object Store currently only supports storage of objects smaller than 5GB, and role-based quotas are also missing. But the current release is certainly complete enough for you to set it up and get started.

Installing a Simple OpenStack Node

Much of OpenStack is well-known, and not much is genuinely new. To create and manage your own virtual machines with OpenStack, all you really need is a recent Linux system. Linux distros such as Debian, Ubuntu, and CentOS all let you install the required packages through the package manager. Of course, the real power of the cloud comes with multi-node, networked systems that provide scalability and failover, but this simple configuration will help you take your first steps.

The following task are required for an initial test with Austin:

1. Set up an OpenStack administrator.

2. Create an OpenStack project.

3. Generate suitable private and public keys.

4. Create a suitable image for running virtual machine instances.

5. Upload the images and start the instance(s).

I will be using Ubuntu as a test operating system in the following installation example. OpenStack relies on the RabbitMQ [8] messaging server and the NoSQL Redis [9] database. To get started, enter the following commands:

aptgget install rabbitmqsserver redissserver
aptgget install pythonnnova
aptgget install novaaapi novaoobjectstore U
novaccompute novasscheduler novannetwork

Finally, you also need the Euca2ool Suite [10], which supports a command line tool for communication with the REST-based OpenStack web service. Similar in scope to the Amazon command line tools, Euca2ool lets you manage images, instances, and volumes.

aptgget install euca2ools unzip

Once you have all the tools in place, you can create an administrative user with Nova-Manage:

novammanage user admin rwartala

This process creates an access key and a security key:

export EC2_ACCESS_KEY=713211a477a154470fUU
ae543346b52e30a0e
export EC2_SECRET_KEY=244de6a188aa344e12UU
9521003ac756abbdf

The next step is to create a new Nova project (OSTEST simply means OpenStack test). The following command

novammanage project create OSTEST rwartala

then creates the certificates, private keys, and public keys required for the project (Listing 1).

01 novammanage project zipfile OSTEST rwartala
02
03 Generating RSA private key, 1024 bit long modulus
04 ........................++++++
05 ................................................++++++
06 e is 65537 (0x10001)
07 Using configuration from ./openssl.cnf
08 Check that the request matches the signature
09 Signature ok
10 The Subject's Distinguished Name is as follows
11 countryName           :PRINTABLE:'US'
12 stateOrProvinceName   :PRINTABLE:'California'
13 localityName          :PRINTABLE:'MountainView'
14 organizationName      :PRINTABLE:'AnsoLabs'
15 organizationalUnitName:PRINTABLE:'NovaDev'
16 commonName            :PRINTABLE:'rwartala22010111220T17:02:57Z'
17 Certificate is to be certified until Nov 20 17:02:57 2011 GMT (365 days)
18
19 Write out database with 1 new entries
20 Data Base Updated

Unpack the data in your home directory:

unzip nova.zip

What's missing now is a suitable image you can use to create the first virtual machine. Rackspace offers a prebuilt test image for running a virtual machine. You can enter the following command:

wget http://c2477062.cdn.cloudfiles.U
rackspacecloud.com/images.tgz

to download the image, and enter:

tar xvzf images.tgz

to unpack it. To create an executable image for OpenStack Compute, you need to run the Euca2ool Suite as follows:

eucabbundleiimage ii U
images/akillucid/image pp kernel U
-kkernel true
Checking image
Tarring image
Encrypting image
Splitting image...
Part: kernel.part.0
Generating manifest /tmp/kernel.manifest.xml

The following commands:

eucabbundleiimage ii images/arillucid/
image pp ramdisk -rramdisk true

then create a matching kernel/rram disk pair from the image and generate a matching image manifest. This describes the image and contains the certificates generated in a previous step. We can load both images into the cloud as follows:

eucauuploadbbundle mm U
/tmp/kernel.manifest.xml bb mybucket
eucauuploadbbundle mm U
/tmp/ramdisk.manifest.xml bb mybucket

What you still need are matching keys to allow SSH-based access to the instance. The following commands

eucaaaddkkeypair rwartala_key > U
rwartala_key.priv chmod 600 rwartala_key.priv

create the keys. You can start the instance as follows:

eucarruniinstances U
amigg06qbntt -kkernelamiffcbj2non U
-rramdisk amioorukptrc kk rwartala_key

Once the virtual machine is running, use the SSH key generated previously to connect with the machine:

ssh ii rwartala_key.priv root@10.0.0.3

That's it!

Behind the Scenes

When new technologies are introduced, people always ask themselves it they are really worth the effort. It is interesting to see who is backing the OpenStack project and who isn't. The spotlight is on the two initiators, Rackspace and NASA, but behind the scenes, major players like AMD, Dell, and above all, Citrix [11] are on board, too.

The RabbitMQ messaging middleware in OpenStack was developed by Springsource, among others. SpringSource has been part of VMware since summer 2009. And the free Redis key-value store used by OpenStack is currently sponsored by VMware.

Thus far, the documentation is fairly rudimentary, and OpenStack lacks management tools. However, the OpenStack community is taking a very professional approach to developing the system. In mid-November 2010, some 250 visitors from more than 90 corporations attended the first Design Summit.

The "Bexar" release of OpenStack appeared in February of 2011, and the "Cactus" release is slated for the Spring. Stay tuned to discover whether OpenStack develops into the Apache of the cloud computing era.