Small Scale

In two previous articles [1][2], I introduced how to bring up lightweight container machines with Footloose and cloud virtual machine stacks on a laptop with Multipass. The Footloose approach didn't require a lot of computing resources or any hardware-based virtualization support. On the other side, Multipass required built-in virtualization capability within the processor and was limited to Ubuntu virtual machines (VMs). Security is a downside of containers because they provide thinner isolation compared with the strong isolation of virtual machines. Can you have the strong isolation of VMs with the lightweight and fast bring up/bring down behavior of containers to create a cloud VM kind of stack on a laptop? Can you also have the tooling that provides a declarative automation experience? The answer is yes!

The tooling provided in this article is for newer machines with virtualization capability provided by the processor. Still, the container machine approach, presented in the first part of this series of articles about Footloose, is an option, in case you don't have such hardware available. I tested the snippets shown or referenced in this article on my 8GB quad CORE i7 laptop running Ubuntu 18.04 LTS.

Firecracker and Ignite

Firecracker [3] is an open source virtual machine monitor created by Amazon Web Services (AWS) to accelerate its serverless offerings (e.g., AWS Lambda and AWS Fargate). It uses a Linux-kernel-based virtual machine to create lightweight VMs known as microVMs. These microVMs eliminate unnecessary devices and guest functionality to reduce resource footprints. The result is enhanced security over traditional VMs, plus the efficiency of containers. MicroVMs, then, provide enhanced isolation, reduced startup time, and greater hardware utilization. Interestingly, the AWS folks demo'd starting 4,000 microVMs on an Elastic Compute Cloud (EC2) bare metal instance showcasing the capabilities of Firecracker [4].

Ignite is an open source virtual machine manager that combines Firecracker with Docker and Open Container Initiative (OCI) images to create a great developer experience out of the box. It can manage microVMs declaratively in a GitOps (Git + continuous deployment tools) fashion.

Ignite Quickstart

To begin with microVMs, you need to install the required dependencies to make use of Ignite. On my Ubuntu laptop (please refer to the install link provided in the Info section [5] for other distributions), I used:

sudo apt-get update && sudo apt-get install -y --no-install-recommends dmsetup openssh-client git binutils which containerd || sudo apt-get install -y --no-install-recommends containerd

The ignite command is provided as a static binary, so just download and make it executable, and it's ready to use. To set up Ignite use:

sudo curl -sSLk "https://github.com/$(curl -sSLk https://github.com/weaveworks/ignite/releases | grep download|grep amd64 | awk -F '"' '{print $2}' | head -1)" -o /usr/local/bin/ignite && sudo chmod +x /usr/local/bin/ignite

Now executing ignite should dump its help screen as shown in Figure 1. Please note that, currently, any Ignite operation shown requires sudo privileges. If you have ever used the Docker CLI (command-line interface), then you will be at home with the ignite commands right away.

Figure 1: Sub-commands and flags for the ignite command.

Ignite requires two kinds of OCI images to create microVMs: The first is for the desired Linux kernel version, and the second is to create the respective userspace filesystem with an init system installed for various GNU/Linux distributions. Currently, multiple versions of the kernel are available: Amazon Linux 2, Alpine, CentOS 7/8, Ubuntu {16,18,20}.04, and other base images are provided by the Ignite GitHub project [5].

To create and run a microVM with an Alpine base image, execute

sudo ignite run weaveworks/ignite-alpine -n myalpinemvm -i --ssh

The run command combines multiple actions: It imports the Alpine image, creates the microVM, runs the microVM, and attaches the TTY device (SSHs into the VM's shell). You'll see prompts to enter the user login/password (enter root /root ); voilá, you're in the running microVM. Now feel free to manipulate your microVM any way you like.

Figure 2 shows the Ignite run sequence on my laptop. The run command takes some defaults for the CPU, memory, disk, kernel image, and so on. To see these defaults and command-line switches, enter:

ignite run -h

Figure 2: Ignite run in action.

You will see for yourself how quickly a microVM comes up compared with a VM.

The correct way to terminate an interactive microVM (-i option) session is with the reboot command (Figure 3). To clean up the stopped microVM, use

sudo ignite rm <microvm name>

Figure 3: Terminating an interactive microVM.

Append the -f flag to clean up a running microVM.

I also passed the --ssh option to the run command, which creates an SSH key pair and imports the public key into the created mircoVM. (As another option, you could pass in your SSH public key pair, instead.) To log in to the running microVM and dump info about all microVMs created by Ignite, use the following commands:

sudo ignite ssh <microvm name>
sudo ignite ps -a

(Drop the -a if you are interested only in the currently running microVMs.) These examples are the main Ignite commands that accomplish most of the general tasks related to microVMs. Feel free to explore more about any Ignite command by suffixing the -h flag to dump the help screen.

Declarative Ignite

Now you can manipulate Ignite in a declarative manner with the use of some prior knowledge gained in the first article of this series with Footloose for automation. To begin, grab the Footloose binary from the GitHub project site [6], if you haven't set it up already, and create the YAML file shown in Listing 1 in the directory from which you will launch Footloose.

cluster:
  name: cluster
  privateKey: cluster-key
machines:
- count: 1
  spec:
    image: weaveworks/ignite-ubuntu:20.04
    name: ubuntu2004%d
    hostname: ubuntu2004%d
    backend: ignite
    ignite:
      cpus: 2
      memory: 2G
      diskSize: 20G
      kernel: "weaveworks/ignite-kernel:5.13.3"
- count: 1
  spec:
    image: weaveworks/ignite-ubuntu:18.04
    name: ubuntu1804%d
    hostname: ubuntu1804%d
    backend: ignite
    ignite:
      cpus: 2
      memory: 2G
      diskSize: 20G
      kernel: "weaveworks/ignite-kernel:5.13.3"
- count: 1
  spec:
    image: weaveworks/ignite-centos:8
    name: centos8%d
    hostname: centos8%d
    backend: ignite
    ignite:
      cpus: 2
      memory: 2G
      diskSize: 20G
      kernel: "weaveworks/ignite-kernel:5.13.3"

As you can observe, the back end is now Ignite, and an ignite section declares CPU, memory, disk size, and kernel properties for the respective microVM(s). Now, executing

sudo footloose create
sudo footloose delete

brings up the declared microVM stack along with a new SSH key pair in the current working directory and cleans up the created stack. Could it be simpler? You could dump the running microVM info with sudo ignite ps because sudo footloose show doesn't dump microVM IP info, in case you are looking forward to using the created SSH private key to log manually into a running microVM.

Ignite itself takes a different form of YAML configuration if you don't want to use Footloose to bring up the microVM stack declaratively. A minimal Ignite API object to create a microVM with default properties would be:

apiVersion: ignite.weave.works/v1alpha4
kind: VM
metadata:
  name: myubuntumvm
spec:
  image:
    oci: weaveworks/ignite-ubuntu
  ssh: true

You could further customize your microVMs with more resources (Listing 2).

apiVersion: ignite.weave.works/v1alpha4
kind: VM
metadata:
  name: myubuntumvm
spec:
  image:
    oci: weaveworks/ignite-ubuntu
  kernel:
    oci: weaveworks/ignite-kernel:5.13.3
  cpus: 2
  diskSize: 2GB
  memory: 20GB
  ssh: <absolute path to ssh public key on your laptop>

The Ignite API object has so many other properties you can use to customize your microVMs. Each of the properties is documented online in the official Ignite documentation [7] with examples. You could bring up your microVM stack by providing the YAML declarative configuration name to the sudo ignite run --config command. Similar is the requirement for the sudo ignite rm -f --config command to clean up your running microVM stack. You also need to provide your local SSH key pair private key path to the sudo ignite ssh command (with the -i flag) if you create the microVM with your public key.