Steer the Sphere

In addition to supporting hypervisors such as Amazon EC2 and Google GCE, Puppet lends itself to the use of VMware vSphere. A private cloud created with Puppet and vSphere is superior to public clouds in two respects: In addition to far lower total cost of ownership, given 24/7 use, you also benefit from increased data security with local hosting. In this article, I target IT managers with Puppet experience who thus far have not looked into the use of the product's cloud service modules.

Test Infrastructure

Establishing a complex VMware infrastructure requires huge hardware investments, so instead, I used a VMware Workstation as the basis of my setup. To understand the process, though, you need a workstation with at least eight cores, 16GB of RAM, and 100GB of hard disk storage space. The host operating system is Windows 8 for simplicity's sake – a 64-bit version is mandatory – and VMware Workstation should be installed as usual.

Puppet networks can be controlled by a master, which must be a Linux system. The first step is thus to set up a virtual machine with Ubuntu. Be frugal with the hardware resources; ESX just loves burning up the remaining computing power for virtual machines and the vCenter appliance.

In the next step, log in to your Puppet master system and download the tarball with Puppet Enterprise [1]. After starting the installation wizard, you can accept all the defaults, because you are generating a test installation only.

While the installation process is running, the Ubuntu host needs additional resources. Even though only 3GB of RAM and two processor cores are required, the installation takes two to three hours. After the installation, if you see a warning regarding the MCollective package, you will have to install it manually. Because of a known bug, you also must install the STOMP library before the actual deployment can take place:

sudo apt-get install ruby-stomp
sudo apt-get install mcollective

Installing the master does not automatically enable Puppet Enterprise to interact with the cloud: For some time, the necessary packages have not been enabled by the setup process. To resolve this issue, you need to open a command prompt and install the missing components manually:

cd /puppet-enterprise-3.8.1-ubuntu-14.04-amd64/packages/ubuntu-14.04-amd64
sudo dpkg -i pe-cloud-provisioner-libs_0.3.2-1puppet1_amd64.deb
sudo dpkg -i pe-cloud-provisioner_1.2.0-1puppet1_all.deb

vSphere Setup

After installing Ubuntu, it's time to let ESX out of its cage. Download the VMware vSphere image [2] and configure a new virtual machine. The successful use of ESX requires pass-through virtualization, the existence of at least two processor cores, and at least 2GB of RAM. For the vCenter server, required for communication between Puppet and ESX, you need 10GB of RAM; this can be reduced later if necessary.

The details of the ESX installation are not critical; simply follow the instructions on the screen. The only difficulty relates to the strange command schema. Some dialogs can be cleared off the screen just by pressing the F key.

Once the two basic components are on your system, you need to wire the Puppet master to the ESX host that is responsible for management of the virtual machines. Communication between Puppet and ESX runs through a vCenter server. Download the VMware vCenter Server Appliance [3] and mount the ISO file. Because the installation of a standalone vCenter server requires the presence of a Windows Server operating system, you need to move the administration instance to the ESX server instead. To do this, you need the VMware-VCSA-all ISO image [3], which you can mount on a Windows 8.1 host by double clicking.

In the next step, you simply navigate to the VCSA subfolder and install VMware Client Integration Plugin when you get there. Then, open the vcsa-setup.html file in a browser of your choice. I used Chrome, which proved very cooperative in this scenario. If you get security prompts, select the option Launch Application and press Install on the web page that appears.

After agreeing to the EULA, the installation wizard prompts you, among other things, for the IP address and the password of the ESXi server intended for use as the host. Although P@ssw0rd met all the conditions imposed by the validator, the installation wizard was annoyed but accepted it. The Embedded Platform Services Controller serves as the deployment type, the new SSO domain goes by the name vsphere.local. Use P@ssw0rd for the administrator account, too.

Because the installation wizard still requires 8GB of RAM on the ESX host, even if you choose an appliance of the Tiny type, you should stop all the VMs and expand the ESX instance's RAM to 10GB at this point. In the actual deployment, make sure you select the Enable Thin Disk Mode option, too, because the virtual instance of vSphere otherwise allocates all the space right from the outset. Use the ESXi host for time synchronization, and click on Finish to start the deployment.

After successfully processing the deployment, the VM is available from the URL mentioned on the Installation Complete page; in my case, this was https://192.168.121.136/vsphere-client . To log in, use the username administrator@vsphere.local .

Unfortunately, your instance knows nothing about its host; therefore, you need a new data center as the first step. To do this, go to the vCenter Inventory Lists | Datacenters section and create a new entry by clicking the plus icon. In the documentation, the Puppet developers explicitly point out that the data center must reside at the top level of the hierarchy, which means that the vCenter server must be the parent object.

The host can move in during the next step. Click on vCenter Inventory Lists | Hosts to open the dialog box where you can enter a new host by clicking the plus icon. Enter the IP address of the ESXi host and add it to the newly created data center. After a few seconds of computing time, the VM generated by the installation wizard appears in the vSphere web client.

Puppet Configuration

The Puppet Cloud Provisioner module from version 3.8 onward is based on the fog library [4]. Its configuration is located in the root directory of the user responsible for provisioning; to create it, you need to type touch ~/.fog. Fog will accept a variety different parameters. For VMware, it is sufficient to add the following five lines to the .fog file using Gedit:

:default:
  :vsphere_server: 192.168.121.136
  :vsphere_username: administrator@vsphere.local
  :vsphere_password: P@ssw0rd
  :vsphere_expected_pubkey_hash: XXX

The statement starting with :default: stipulates that the subsequent configuration is used as the default. If you want to connect your Puppet Master instance to multiple cloud servers, then you can create multiple configurations. To switch between them, in this case, you need to enter:

FOG_CREDENTIAL=default puppet node_VMware <optional commands>

Hashes are used for identification. Unfortunately, it is possible to talk a vCenter server into divulging its hashes. For this reason, I went for a workaround and set XXX as the hash then ran an idempotent command against the server (Listing 1). Puppet responded with an error message, indicating that the hashes did not match. The hash supplied by the server was copied to the FOG file, thus completing the wiring work for Puppet and ESX. For more information on the Warning message in Listing 1, see the "Deprecated" box.

puppet node_VMware list
Warning: Cloud Provisioner is deprecated in PE 3.8.
         For more information and recommendations,
         see the release notes documentation here:
         https://docs.puppetlabs.com/pe/3.8/release_notes.html
Notice: Connecting ...
Error: The remote system presented a public key with
       hash 31452e1f896 f71542b6b9198188de1b5e59f5af62ffcefdc261df324636c90c7
       but we're expecting a hash of XXX. If you are sure
       the remote system is authentic set
       vsphere_expected_ pubkey_hash: <the hash printed
       in this message> in ~/.fog
Error: Try 'puppet help node_VMware list' for usage