Thinking Ahead

In the meantime, several solutions use or combine parts of the concept. The Ethernet virtual private network (EVPN), for example, is also based on BGP but shifts the routing completely to the switch level. The individual hosts no longer need BGP daemons.

Even genuine Layer 3 routing up to the host has ready-made instructions and solutions: The hosts then use the unnumbered BGP extension, meaning not every host needs an internal autonomous system number (ASN). The configuration is largely automatic. Routing on the host, however, requires the SDN solution used by the particular environment to provide support.

Calico expands on this principle. Basically, Calico sees itself as an SDN solution that implements the classic SDN overlay – not in the form of encapsulation, but by BGP. Specifically, Calico docks with the respective virtualization solution on one side (e.g., to OpenStack (Figure 1) or Kubernetes (Figure 2)), and on the other side, it speaks BGP to connect the instances in the virtual environment.

Figure 1: As an SDN solution, Calico supports both OpenStack … © OpenStack, Apache-2.0 [2]

Figure 2: … and Kubernetes, plus several other solutions. © Kubernetes, CC BY 4.0 [3]

Security plays a prominent role in this context. Because traffic at the Layer 2 level cannot be isolated (encapsulation is not used), Calico has to solve this problem differently. The developers strictly adhere to the requirement of using existing technology to the extent possible. Accordingly, they fall back on the filter mechanisms that Linux includes out of the box: network namespaces, groups, and packet filters.

Calico Architecture

Calico now faces a common challenge in the cloud: The configuration is not where it is needed. Most cloud concepts assume the existence of controllers that contain the statuses of all resources in the environment. However, the virtual instances, whether containers or real VMs, run on separate systems, where the configuration stored on the controllers must somehow be put into practice.

Calico is accordingly based on a control plane (on the controllers) and agent (on the target systems) architecture. Several plugins are at hand for external solutions such as OpenStack or Kubernetes.

Central Datastore

The Calico developers require their software, as an application of the cloud age, to follow the common maxims of modern development. Therefore, Calico stores its configuration data in a central database.

Data storage is of great importance in a distributed solution, providing a single source of truth (i.e., an instance in the cluster whose database is considered correct and binding for all cluster-wide actions). The Calico developers could have reinvented the wheel at this point, but that would not have made much sense. The best-known implementations include Consul and Etcd, which is also one of the two variants that the Calico developers have decided to use. If desired, Calico will also store its complete configuration in Etcd.

Etcd has several advantages. First, it has proven to be a robust consensus algorithm for distributed systems that does not care about the failure of individual nodes. Second, Etcd itself is distributed: The service is written in Go, does not require much in terms of resources, and runs on the hosts in the environment.

Because a configuration change in Etcd is automatically propagated to all other Etcd instances, the complete configuration database with all relevant parameters is always available on each host, saving the agent on the target systems a great deal of network traffic for queries to a central database like MySQL to retrieve values. The connection to Etcd is particularly suitable for setups with conventional virtualizers like OpenStack or without a central control mechanism.