Load Balancing for Containers

In common among all the load balancers presented so far is that admins often run them on separate hardware and integrate them into an HA setup, which is of great relevance for classic applications, conventional web server environments, and other standard software. However, it does not apply to cloud-native computing, because in this case, everything is software-defined to the extent possible and is managed by the users themselves. The centralized load balancer that handles forwarding for a website or several installations does not exist in this scenario.

Instead, the admin has Kubernetes (or one of the distributions based on it), which rolls out the app in pods that the respective container orchestrator distributes arbitrarily across physical systems. The network is purely virtual and cannot be tied to a specific host, so the typical load balancer architecture comes to nothing. If a setup requires load balancing (e.g., to allow various microcomponents of an application to communicate with each other in a distributed manner), the legacy approach fails.

Don't Do It Yourself

As a developer or administrator, you could create a container with a load balancer yourself, which you then roll out as part of the pod design. Technically, there is nothing to prevent an appropriately preconfigured Nginx from acting reliably as a balancer. However, setups that run in container orchestrations are usually very dynamic. The number of potential source and target servers on both sides changes regularly, for which classic software load balancers do not cope well.

Sidecar Solution

Fortunately, developers have already come up with a solution to this problem: software-based load balancers designed specifically for such container architectures. Istio [5] is something of a shooting star in this software segment. The software initially implements plain vanilla load balancing with integrated termination for SSL connections. If desired, Istio can also create Let's Encrypt SSL certificates with an ACME script.

For this purpose, Istio provides the containers in pods with sidecars, which act as a kind of proxy for the individual containers. In addition to simple balancing, Istio implements security rules and can make load balancing dependent on various dynamic parameters (e.g., on the load to which the individual services on the back end are currently exposed). Moreover, Istio comes preconfigured as a container and can be integrated with Kubernetes without too much trouble.

For applications that follow microarchitecture principles, approaches like Istio that are specifically designed for this purpose offer the better approach. They should be given preference to homegrown, statically rolled out load balancer containers in any case, because they offer a significantly larger feature set and are usually backed by a large community, which takes a lot of work off the admin's shoulders.