Cloud Creator

OpenStack has been constantly improved to meet the requirements of the demanding virtualization and cloud computing market. To challenge the competitive cloud solutions, OpenStack creators have been implementing a number of improvements and features in automation, orchestration, and scalability. Metadata services in cooperation with the cloud-init script deserve special attention.

The administration of large-scale production cloud environments requires the management of dozens of customer's virtual servers on a daily basis. Manual configuration of the multiple newly created instances in the OpenStack cloud would be a real pain for cloud administrators. Of course, you could use popular automation tools (e.g., Ansible, Chef, Puppet) to make a post-installation configuration on the virtual machines, but it still requires additional effort and resources to create the master server, templates, manifests, and playbooks and to build a host inventory or set up communication to virtual hosts. That's where the metadata and cloud-init duo comes in.

Metadata Service

The OpenStack metadata service usually runs on a controller node in a multinode environment and is accessible by the instances running on the compute nodes to let them retrieve instance-specific data (e.g., an IP address or hostname). Instances access the metadata service at http://169.254.169.254 . The HTTP request from an instance either hits the router or DHCP namespace, depending on the route in the instance. The metadata proxy service adds the instance IP address and router ID information to the request. The metadata proxy service then sends this request to the neutron-metadata-agent service, which forwards the request to the Nova metadata API service (nova-api-metadata server daemon) by adding some new headers (i.e., instance ID) to the request. The metadata service supports two sets of APIs: an OpenStack metadata API and an EC2-compatible API.

To retrieve a list of supported versions for the OpenStack metadata API from the running instance, enter:

$ curl http://169.254.169.254/openstack
2012-08-10
2013-04-04
2013-10-17
2015-10-15
2016-06-30
2016-10-06
2017-02-22

To retrieve a list of supported versions for the EC2-compatible metadata API, enter:

$ curl http://169.254.169.254
1.0
2007-01-19
2007-03-01
2007-08-29
2007-10-10
2007-12-15
2008-02-01
2008-09-01
2009-04-04

The cloud-init package installed on an instance contains utilities for its early initialization according to the instance data. Instance data is a collection of configuration data usually provided by the metadata service, or it can be provided by a user data script or configuration drive attached to the instance during its creation.

Getting the Image with cloud-init

Assuming you already have access to the tenant in an OpenStack environment, you can easily download a ready-to-use OpenStack QCOW2 image, including cloud-init from the official image repository website [1].

All the images from the official OpenStack website have a cloud-init package installed and the corresponding service enabled, so it will start on an instance boot and fetch instance data from the metadata service, provided the service is available in the cloud. Once launched and running, instances can be accessed via SSH with their default login, given on the website, and passwordless key-based authentication.

To download a CentOS 7 QCOW2 image to your controller node, enter:

[root@controller ~]# wget http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-1808.qcow2

Because you are going to use the image on a tenant only, you can upload it as a private image available exclusively in your project. First, you need to import your tenant/project credentials by means of sourcing a Keystone file to your environment variables to gain access to the project resources:

[root@controller ~]# source keystonerc_gjuszczak
[root@controller ~(keystone_gjuszczak)]#

Now, you should import your previously downloaded QCOW2 image to Glance so it is available for instances launching within your project (Listing 1).

[root@controller ~(keystone_gjuszczak)]# openstack image create --private --disk-format qcow2 --container-format bare --file CentOS-7-x86_64-GenericCloud-1808.qcow2 CentOS_7_cloud_init

Launching with cloud-init

To create a CentOS_7_CI instance based on a CentOS_7_cloud_init QCOW2 image, enter:

[root@controller ~(keystone_gjuszczak)]# openstack server create --flavor m2.tiny --image CentOS_7_cloud_init --nic net-id=int_net --key-name cloud --security -group default CentOS_7_CI

Next, assign a floating IP to the instance, so you can access it from an external public network:

[root@controller ~(keystone_gjuszczak)]# openstack server add floating ip CentOS_7_CI 192.168.2.235

As mentioned before, the default user for this particular image, centos , has no password set, and root access for the image is restricted, so you need to access the instance with cloud.key from the key pair used on instance creation:

$ ssh -i ~/.ssh/cloud.key centos@192.168.2.235

After accessing the instance, it's vital to check whether the cloud-init service is running:

[centos@centos-7-ci ~]$ sudo systemctl status cloud-init

Then, you should analyze cloud-init.log to verify what modules were run on instance startup:

[centos@centos-7-ci ~]$ sudo cat /var/log/cloud-init.log

Listing 2 shows an example snippet from cloud-init.log that presents some modules executed during boot. From the listing you can find out that cloud-init has adjusted the instance partition size to the storage capacity as defined through the flavor setting (see the OpenStack documentation for more on flavor, which defines the compute, memory, and storage capacity). cloud-init has also resized the filesystem to fit the partition and set the hostname according to the instance name in the cloud.

...
2018-10-09 20:42:56,669 - handlers.py[DEBUG]: finish: init-network/config-growpart: SUCCESS: config-growpart ran successfully
2018-10-09 20:42:57,666 - handlers.py[DEBUG]: finish: init-network/config-resizefs: SUCCESS: config-resizefs ran successfully
2018-10-09 20:43:00,261 - handlers.py[DEBUG]: finish: init-network/config-set_hostname: SUCCESS: config-set_hostname ran successfully
...

There is one thing worth mentioning here: The centos-7-ci hostname visible in the command prompt is a copy of an instance name, CentOS_7_CI , which was set when the instance was created. This is the typical example of how a metadata service works – the hostname is delivered to the instance OS on the basis of the instance name in OpenStack. The typical metadata-based data sources exposed to virtual hosts in OpenStack (hostname, instance ID, display name, etc.) is one of the improvements offered by the metadata service. While creating hundreds of instances in the tenant, you don't need to set the hostname for each separately; OpenStack does it automatically, saving you time.