Construction Guide

When confronted with Kubernetes, many admins immediately think of the major league products by Red Hat and Ubuntu. Red Hat's OpenShift, for example, is a massive Kubernetes distribution comprising multiple components, an integrated app store for container applications, and a great deal of complexity under the hood.

Rancher sets the bar somewhat lower. The product now belongs to SUSE, which has so far largely not interfered with the Rancher developers' work. Because of this, Rancher has retained much of the simplicity that its fans have loved for years. It also means that if you want to get started with Kubernetes today, you'll find Rancher a virtually barrier-free way of doing so.

That said, even a Rancher setup is not exactly a no-brainer. Various factors have to be considered in terms of the hardware, especially the number of machines you require and their dimensioning. Once the hardware is ready to run in your racks, the next step is to install a Kubernetes distribution, because Rancher maps its own infrastructure completely in Kubernetes. Although this sounds complicated, it is not a problem in practice because Rancher has its own Kubernetes core distribution in tow in the form of K3s to provide all the required features.

In this article, I guide you through the process of adding Rancher to your setup. Starting with a proverbial greenfield, my goal is to help you set up a Rancher cluster that can be used in production. To achieve this goal, however, I first need to clarify a few terms from the Rancher world up front with which you can assume you will be regularly confronted.

Rancher Architecture

If you already have some experience with one of the other major Kubernetes distributions, you may be a bit overwhelmed with Rancher's central features at first because Rancher works differently from most competitor products. A comparison quickly makes this clear. OpenShift, for example, consists of a management plane, or control plane, that encompasses all of the environment's core services. Although OpenShift also uses Kubernetes for its management, an OpenShift setup includes precisely one Kubernetes instance. If users launch containers in Kubernetes, the containers run on the existing cluster and use the existing infrastructure.

Rancher deviates massively from this approach. It not only sees itself as a tool for managing workloads in Kubernetes but also as a tool for controlling and managing Kubernetes environments. Therefore, applications do not run in the Kubernetes cluster that Rancher requires for its own Kubernetes components. Instead, Rancher assumes that each end-user setup has its own Kubernetes installation, which Rancher then also runs.

Because Rancher installs agents in these satellite setups, it is able to power workloads in these secondary Kubernetes clusters, too. As a consequence, the programs that a Kubernetes cluster is supposed to run in Rancher never run in the same instance as the Rancher services themselves but always in a separate Kubernetes environment that Rancher controls somewhere downstream.

Some administrators might already be cringing at this point. After all, the Rancher approach seems clumsy at first glance, and above all, it seems to waste resources. It is true that the Kubernetes infrastructure components that belong to each Kubernetes cluster generate some overhead themselves, but because the individual Kubernetes instances in Rancher rarely run thousands of pods – you would run individual K8s instances for these if they're not part of the same setup – the resulting compute overhead is manageable.

On top of this, and on a positive note, you have a genuinely clear-cut separation of individual applications in the Kubernetes cluster, as well as comprehensive monitoring and alerting capabilities for each end-user Kubernetes instance. Additionally, this approach lets Rancher offer an option that other environments do not have. If required, Rancher also manages the commercial Kubernetes offerings found in AWS, Azure, or Google Cloud, for example.

Hardware Procurement

Once an organization has decided to use Rancher, the first thing on the agenda is basic deployment, and you need hardware for this. It is true that Rancher can also be completely virtualized, and, in principle, nothing can be said against this approach. However, if third-party workloads with completely different applications are running on certain nodes in addition to the Kubernetes components for Rancher, you risk resource bottlenecks.

Like any other application, Rancher feels most at home on its own hardware. Because the setup in this example is supposed to reflect a Rancher setup in a production environment in the most realistic way possible, I will be assuming that Rancher uses its own hardware. To achieve high availability, you need two servers, each with 128GB of RAM and at least 32 virtual cores (vCPUs). Unfortunately, what the manufacturer means by vCPUs in this context is not clear from the documentation.

You can safely assume that a recent Xeon with 24 physical cores (i.e., 48 threads) will be fine for most Rancher setups. Rancher itself states that a machine of this dimension can meaningfully run 2,000 clusters and up to 20,000 compute nodes, even with the database belonging to Rancher grabbing two cores and 4GB of RAM for itself in the background. Input/output operations per second (IOPS) values are more important for the database. The systems therefore ideally need to reside on fast flash memory for the servers.

In a normal setup, the worker nodes would also need at least two systems (i.e., the hosts on which the Kubernetes user clusters will run). The documentation alternately refers to these as compute or target nodes, but it means the same servers in all cases. There are virtually no limits to your imagination when it comes to hardware, but make sure the systems are not under-dimensioned. Unlike machines in full and paravirtualized setups, containers do not need CPU time to run their own vCPUs and Linux, but the containers and their applications do need to have enough RAM. If you dimension the target nodes like nodes for operations with KVM, you normally will not put a foot wrong.

Unlike the nodes for the Rancher components, you need to pay attention to local storage media on the systems for the compute nodes; after all, the containers are stored there during operation. A fast but small NVMe-based drive might endanger your setup. If you designate a few gigabytes of drive space, you are on the safe side in most cases.

Key Question

For Rancher services to run on a system at all, a Linux installation is an absolute prerequisite. This requirement forces you to opt for a specific distribution to run with Rancher. You might be tempted to use what you already know from your own experience, but doing this may be wasting a great opportunity because Rancher's components are not legacy software packages: Rancher itself is fully containerized.

The upside is that any distribution suitable for running containers with Docker or Podman is also great for Rancher. From the administrator's point of view, there is no reason to opt for a complete Linux system – a microdistribution such as CoreOS (Figure 1) is all you need. Flatcar Linux is an alternative, and one that I wrote about in the past [1]. Ubuntu Core is another option. Rancher even had its own downscaled Linux named RancherOS in the past, but it is no longer officially supported.

Figure 1: The target systems for Rancher do not have to be kitted out with a full Linux distribution. CoreOS or Flatcar Linux are quite okay for running container workloads. © Red Hat

If you feel unhappy with a microdistribution, the usual suspects will do the job, but make sure you roll out as lean a system as possible right from the outset. On top of this, the Rancher roll-out provides a good opportunity for taking care of automation issues. Kickstart rules for AlmaLinux or Rocket Linux can be created quickly and will save you a huge amount of work, especially when you scale the platform later. Additionally, make sure from the outset that you automate the system configuration on the individual servers to the extent possible.

One thing becomes clear at this point. A sensibly planned Rancher setup requires some work up front before it can go live, but this investment will pay dividends later on because you will be able to grow the cluster quickly and easily.