Lead Image © Author, 123RF.com

Your own AWS-compatible cloud with Eucalyptus

Cloud in a Box

Article from ADMIN 17/2013

By Tim Schürmann

If you don't trust the cloud services of Amazon or cannot store customer data there for privacy reasons, you can create your own Eucalyptus cloud that is compatible with the Amazon tools and can be installed in a flash.

Eucalyptus originated as a research project at the University of California [1]. In 2009, the spinoff Eucalyptus Systems Inc. took over further development and commercialization. The software has been available ever since in two flavors: a free open source version, which is completely under the GPLv3 license, and the commercial Eucalyptus Enterprise Edition, which supports the integration of SANs and VMware infrastructure.

Homegrown Amazon

Eucalyptus builds a cloud from several computers in which you can start virtual machines and access storage – much like what Amazon offers with its AWS services. However, with Eucalyptus, you are the operator of the cloud. Administrators not only have complete control, the servers, and thus the data, remain within your own company walls. A Eucalyptus cloud can even be set up quickly on one computer (see the "All on One" box).

All on One

You can install all of the Eucalyptus components on a single physical computer. In fact, the Eucalyptus developers officially support this mode. This "cloud-in-a-box" approach is ideal for small workgroups and administrators who want to experiment with the settings.

The FastStart Medium lets you quickly install the Cloud-in-a-Box. In the boot menu, just select Install CentOS 6 with Eucalyptus cloud-in-a-box . The wizard then asks some questions, as with any normal installation.

To start a cloud with Eucalyptus, you need physical computers that have processor support for the Intel VT or AMD-V virtualization feature. A test installation on VirtualBox or VMware is therefore not possible, but a workaround exists for KVM [2]. The computers should have at least 4GB of RAM and a 250GB hard disk – the more memory, the more virtual machines can be started in the cloud. Additionally, you need an extra set of free IP addresses that Eucalyptus can assign to the systems running in the cloud later on.

Kickstart

In terms of supported operating systems, Eucalyptus is extremely picky. Officially, the developers only support 64-bit versions of two Linux distributions: CentOS 6 and Red Hat Enterprise Linux 6. Briefly, Eucalyptus was also an official part of Ubuntu, but Canonical replaced it in Ubuntu 11.10 (Oneiric) by competitor OpenStack. As of Eucalyptus 3.2, the Eucalyptus developers stopped providing prebuilt Ubuntu packages.

Installing a Eucalyptus cloud on RHEL or CentOS from the provided packages is quite time consuming; the underlying Linux systems need to fulfill many conditions – this is reflected by a massive 96 pages of installation guide. Luckily, the Eucalyptus developers provide special installation media, so administrators can set up a small Eucalyptus cloud in a few mouse clicks.

This FastStart installation method can also serve as a starting point for a larger cloud. Because Eucalyptus wants to have the cloud computers all to itself, admins can avoid the time-consuming setup simply by deleting the hard disks of the computers involved and leaving it to the FastStart image to set up a Eucalyptus system. This approach has the pleasant side effect of giving you a working basic configuration so you can start right away.

Access to and management of the current cloud is handled on client computers with any operating system – either conveniently with the mouse in a web interface or via the official command-line tools known as Euca2ools. The latter can also be found in the repositories of most major distributions. See the "Command-Line Tools" box for more information.

Command-Line Tools

With the Euca2ools toolbox, you can start and stop a virtual machine at the command line. Many Linux distributions have it in their repositories, and they are installed on the front-end computer. To use the Euca2ools, you first need to identify yourself as the rightful owner of the cloud. For this purpose, begin by picking up your credentials packet from the front end:

usr/sbin/euca_conf   --get-credentials admin.zip
unzip admin.zip
source eucarc

The first command in this set must be run as root; the last command sets a few environment variables for the Euca2ools. Then, it generates a key pair by the name of eucatest. The private key in the file is stored in eucatest.private:

euca-add-keypair eucatest >   eucatest.private
chmod 0600 eucatest.private

Now you can view all the available operating system images, which you can start on virtual machines (Figure 1):

euca-describe-images

You need to locate your preferred image from the tangle of text. Each operating system image consists of three files: the kernel, a ramdisk, and the actual image. The latter can be identified by the .img part of the file name. You need the image to start a virtual machine. You will find it most quickly by looking at the description in the second line. The internal identification number here is important; it follows the IMAGE tag. Figure 2 shows the image for a small CentOS 6 system emi-AF4736C9. Try to remember this name. The following command outputs all kinds of virtual computer models:

euca-describe-availability-zones verbose

In the second column, you will see the names of the virtual computer models; their hardware configurations follow on the right. For example, the computer named m1.small only has one CPU and 256MB of RAM. The free/max column is also interesting: The number below free shows how many virtual machines you can still launch with this model, and max shows you the maximum possible number of machines. Once you have decided on a model, you can start the virtual machine:

euca-run-instances -k eucatest   emi-AF4736C9 -t m1.small

Eucalyptus now shows you the same status bar you see in the User Console. If no IP addresses are specified yet, you can try again after a few seconds:

euca-describe-instances

Note the internal identification number next to INSTANCE (it starts with i-); you will need this with specific commands:

euca-terminate-instances i-45C44614

For example, you can use the preceding command to switch off an active virtual machine.

Figure 1: Viewing all the available operating system images with the Euca2ools command euca-describe-images.

Figure 2: Working with the Euca2ools is not as intuitive as with the user interfaces.

Eucalyptus itself consists of several components (Figure 3), each of which plays a specific task. To begin, you need one or more computers on which the virtual machines can later run and which store the data. A node controller (or NC for short) runs on each of these nodes. The NC essentially starts, stops, and manages the virtual machines running on its hardware.

Figure 3: Eucalyptus consists of several parts that build upon one another.

Services

The front end decides which nodes run on which machines. It also provides the web interface and responds to management commands. Under the hood, the front end comprises several individual services discussed in the "Five Is the Key" box. The front end should also run on a separate computer. An initial, small cloud thus consists of two or more node controllers that do the actual work and another computer that acts as the front end (Figure 4). Thanks to the separation of the components, you can gradually grow the cloud, for example, by adding more node controllers.

Figure 4: For a first small cloud, you need three physical computers and a client for access. The node controllers do the actual work; the front end coordinates it.

Five Is the Key

Eucalyptus comprises five components. Each runs as a web service and relies on the functions of the other services (Figure 3).

Node Controller (NC): The note controller runs on the nodes. It starts, stops, and manages the virtual machines running on its own hardware.
Cluster Controller (CC): Several nodes are grouped by the cluster controller to form a cluster. The cluster controller determines which virtual machine starts on which node; it manages the network to which virtual machines connect.
Cloud Controller (CLC): The cloud controller has authority over the individual clusters. It creates the actual cloud from the clusters, makes all superordinate decisions, and accepts requests from users. It also provides the web interfaces.
Storage Controller (SC): The storage controller consolidates space in volumes, which in turn can be mounted by virtual machines or accessed as block devices. The functionality is similar to Amazon's Elastic Block Store (EBC).
Walrus: The Walrus component is the storage service that is compatible with Amazon Simple Storage Service (S3). Like the original, it stores data in buckets in the cloud. Walrus can be accessed using the command-line tools or addressed from the virtual machines.

1 2 3 Next »