Universal Tool

Administrators often first encounter the phrase "continuous development and continuous integration" (CI/CD) in the context of container-based setups with Kubernetes. Infrastructure as code (IaC) can also be on board as part of the strategy. IaC is exactly what the name suggests: the ability to describe a complete setup with a standardized format. Special software then reads this description and translates it into running resources, such as virtual instances or correctly configured containers.

Numerous corresponding tools are now on the market, some of which can be combined but that compete with each other in many respects, as well. Spacelift [1] looks to help administrators by offering nothing less than the glue for perfect integration of various tools in the form of standardized APIs and the ability to deploy initial production resources within minutes. Unlike other tools, the claim is that administrators don't need to spend weeks reading books and manuals to even test the tool.

Support is available for classics such as Pulumi and Terraform, as well as its fork OpenTofu. The developers have also put some thought into the cloud connection, which is, to all extents and purposes, the software's preferred deployment target. Spacelift can read its descriptions from Git, Azure DevOps, Bitbucket, and others. All kinds of major and minor helpers supplement the construct, supporting, for example, the enforcement of specific security rules, sophisticated monitoring, and monitoring of rolled-out resources. That sounds wonderful, but does it work in the real world? How complex is getting started with Spacelift, and how well does the published integration scope work?

Only for Hyperscalers

To get to the bottom of Spacelift, you first need a description of what Spacelift seeks to be and which features it doesn't even want to take on. After a quick look at the tool's website, you initially get the impression that it can also be used in on-premises installations and with bare metal. However, appearances are deceptive. Although the common definitions of IaC now also include on-premises hardware and bare-metal setups, and you can find tools capable of automatically handling physical installations with defined parameters (e.g., tools such as Canonical's MaaS (metal as a service) or Foreman to handle lifecycle management for physical servers), Spacelift has stated that it does not want to go down these paths.

Native integration is only offered for the large hyperscalers (i.e., AWS, Azure, and GCP). If you don't want to roll out your setups on one of these platforms, you can forget Spacelift. Supported deployment tools such as Pulumi or Ansible do offer the option of rolling out deployments on other platforms, such as OpenStack. In Spacelift, however, OpenStack will cause additional administrative overhead in many cases. Moreover, vendor support for this kind of setup is unlikely and is particularly unfavorable from a European perspective, because not every company in Europe is allowed to or wants to store its data with hyperscalers – the keyword here being digital sovereignty.

However, if you have no reservations about AWS, Azure, or GCP, you can include Spacelift in your short list of management tools for your virtual environment. In this case, the fact that Spacelift is usually a hosted service that runs in the cloud itself is irrelevant. The code and all company-specific configurations required for Spacelift also end up there. Spacelift does offer the option of renting your own instance of the service for larger setups. Again, this runs in AWS and is essentially a personalized version of the generic Spacelift offering that everyone else gets. Spacelift gives you virtually no alternative to hyperscalers and none at all to AWS.

Terminology

If Spacelift presents nothing fundamentally wrong from your point of view, the next step involves checking out the solution's terminology. You will find a whole host of Spacelift-specific terms, although you do not need to know them all (which would contradict the provider's mantra that Spacelift is geared for production use within a few minutes). A few of the terms from the CI/CD context will very likely already be familiar.

Two familiar items will be Terraform and OpenTofu, two deployment tools that abstract the CI/CD services of the major providers (e.g., AWS CloudFormation) and make them controllable with a standardized declaration language of their own. Terraform is undoubtedly the better known solution, and OpenTofu is strictly speaking also Terraform. This fork of the parent project took place shortly after a change in licensing by HashiCorp. You will also have heard of terms such as Open Policy Agent (OPA), a standard for defining configuration parameters on systems as a kind of framework that allows more or less generic rules, and Git.

If you are familiar with typical CI/CD terms, the rest of the terms used by Spacelift should become clear quite quickly. The term "stack," for example, is central and is probably familiar to those having some experience with container deployments in cloud environments. In Spacelift, a stack refers to the complete set of resources and to the configuration and metadata of a rolled-out setup.

The precise purpose of Spacelift is to roll out complete environments, including the software running in them, to one of the hyperscalers in the shortest possible time. Spacelift identifies several components that add up to a stack. The stack can then be managed and maintained as an independent unit by Spacelift. Among the various ways of creating stacks in Spacelift, the developers recommend the built-in Terraform provider to convert configuration settings previously stored in a Git directory into a setup for one of the hyperscalers.

The Spacelift developers explicitly point out that this procedure requires you to write the corresponding configuration and all code required for execution in Terraform syntax. However, because Spacelift also supports other IaC tools such as Ansible, stacks can be created in other ways, such as a blueprint, which generates a stack in the same way that Spacelift's Terraform integration would but also contains instructions on the integration to be used for the supported IaC components.

In addition to Terraform and OpenTofu, components also include Ansible, Terragrunt, Kubernetes, and Pulumi (Figure 1). One thing is quite striking: Puppet and Chef, the first-generation automation tools, are missing from the list. The same applies to Salt, which is now also quite widespread, so anyone who primarily relies on these tools has some migration work ahead of them if they want to use Spacelift.

Figure 1: In addition to its own run engine for Terraflow, Spacelift supports various other tools for infrastructure as a service (IaaS), including Ansible and Pulumi. The popular Puppet and Chef tools are missing. © Spacelift

IaC and Version Management

As an IaC solution, Spacelift needs complete integration into standard CI/CD workflows; the current Spacelift development tool scenario makes things easy. Apart from Git, practically no other relevant solutions are out there in the field, and Spacelift integrates perfectly with both GitHub and local GitLab instances (Figure 2). The service also supports Bitbucket, but the interest of global users in this aspect of the service is likely to be limited.

Figure 2: Spacelift promises an all-inclusive IaC experience and integrates the version control systems of various providers. © Isaac Johnson

From the perspective of the average user, it seems far more important that Spacelift is capable of implementing CI/CD pipelines itself and of being part of such pipelines. Spacelift's internal working methods are also fully in line with the principles of CI/CD. For example, Spacelift regards a fully rolled out stack as a CI/CD pipeline that can have several runs or calls – an excellent illustration of how Spacelift works under the hood.

In the first step and immediately after gaining access to Spacelift, the administrator creates a (still empty) stack in Spacelift. The next step is to link a Git directory containing all the code required to execute all the stack's necessary resources. Note that Spacelift itself cannot create resource definitions for Kubernetes, Ansible, or AWS CloudFormation. Instead, you need to create the required definitions in the form of source code in one of the supported configuration formats.

Once this step is done, however, Spacelift takes over the helm by retrieving and parsing the configuration data from the Git directory, then uses its internal functions to run tests defined by the administrator. These tests can relate to, say, the target platform or the source code. The situation is similar when working with a blueprint, where in addition to Terraform, the other IaC tools can be used, as well. Spacelift then creates the stack and evaluates a list of the tasks you define stored therein.

Once the stack has been configured, you need to create what is known as a run. Depending on the configuration, it can be a development run or a proposed run (Figure 3). If so desired, a proposed run automatically cleans up its resources when done, to avoid expensive surprises caused by test setups left behind in AWS. Any form of the run can also be repeated. If something goes wrong on the first attempt, you need to change the configuration in Git and then restart the run. Spacelift automatically retrieves the latest configuration and tries again.

Figure 3: In Spacelift, a run refers to all the steps required to implement a complete deployment, starting with the plain-vanilla source code. There are different types of runs for development and production. © Spacelift

Proposed runs designed for deployment in production simulate the changes they would make and indicate whether or not the run would be successful. Like the development run, the proposed run creates the required components but does not access production resources. It launches the configured services and resources in a fresh environment on the target platform with the respective IaC tool.

In the usual way for CI/CD systems, Spacelift tracks whether all the desired resources were created. If not (e.g., because the configuration is not perfect), you need to change the configuration in Git and relaunch the development or proposed run. If everything works as desired, the roll-out takes the form of a tracked run (i.e., a monitored instance) in your stack's production environment. Spacelift can either start a new run or update an existing one with a new configuration on the fly from a new tracked run. Spacelift is idempotent.

Even if all the resources configured by the administrator are available on the target platform, Spacelift's work is far from done. The tool's feature set includes all kinds of add-ons and extensions, not only to implement the deployment but to ensure security and monitoring. Spacelift can therefore roll out many components from the open source world (e.g., Prometheus) in addition to your defined setup. Spacelift largely works out the configuration for this itself on the basis of the known configuration, which means that complete monitoring can be set up very quickly for a distributed application in Kubernetes.

On top of this, Spacelift itself has a GraphQL interface, meaning that metrics can be exported to an existing Grafana installation in next to no time.