In the past, network duties in the data center were strictly separated: admins took care of the Linux systems, storage admins of the network storage, and network admins of the routers and switches. For individual customers, specific configurations were built on the data center's own hardware; everything was set up once, and the job was done.

Recently, however, responsibilities have shifted considerably – largely because of the cloud, which has made IT services far more dynamic. Customers no longer want to wait weeks or months for a custom infrastructure to be set up; they want storage and compute immediately. As a result, IT providers have become platform operators. Instead of customer-specific setups, a generic platform of compute and storage resources greets customers, who then consume bit by bit according to their needs.

Now, the network magic happens on the customer side with software-defined networking (SDN). Calico [1] steps in on the software side to replace the functions of classic network devices in virtual environments. Admittedly, the project is not at the forefront of the SDN movement. Other approaches (e.g., Open vSwitch) claim seniority. However, Calico is fundamentally different from most other solutions: It comes without an encapsulation technique and relies instead on standard Layer 3 protocols.

Calico makes some bold claims. The developers of the solution promise special security and native kernel speed on Linux systems, which is reason enough to put the product through its paces.

SDNs

A short excursion into the terms and functions of SDN helps to understand the central unique selling proposition behind Calico. SDN is no longer a homogeneous approach, and different implementations compete for admins' favor. The core idea of the SDN is to shift network functions from typical network hardware to the software level and thus to individual systems.

One example is the separation of traffic for different systems. Virtual LANs (VLANs) are used for this purpose on switches; they implement the division in Layer 2. SDN is therefore basically making up for a part of the virtualization that was missing until the emergence of the first conceptual networks. RAM and CPUs had long been virtualized when physical networks were still routed into virtual machines over bridge interfaces.

In large environments, such as those operated by platform providers, this approach is not useful. It would simply be impossible to reconfigure all the switches in the system to create a new customer and that customer's virtual networks. Instead, in a completely flat Layer 2 segment, the SDN solution takes care of the problem.

Underlay and Overlay

Conventional SDN implementations are divided into an underlay and an overlay. The underlay is the Layer 2 segment referred to earlier, whereas the overlay describes the virtual environment within which the packages with payload traffic find their way.

Almost all common SDN approaches use encapsulation solutions in the underlay, such as generic routing encapsulation (GRE), virtual extensible LAN (VXLAN), or generic network virtualization encapsulation (Geneve), to separate data traffic in the overlay from that in the underlay. Encapsulation is the process by which a packet finds its way from virtual machine (VM) A on host 1 to VM B on host 2, but it is precisely this encapsulation that regularly causes trouble in the context of SDN. In theory, encapsulation should not have any effect on performance, but in practice, it has a significant effect.

One well-known example is Open vSwitch, earlier versions of which built a kind of packet Multitron and were happy to drown hosts in Address Resolution Protocol (ARP) requests. In the meantime, such weaknesses have been eliminated by the developers, but the separation into overlay and underlay, including the encapsulation that goes along with it, still has many admins' backs up.

On Layer 3

Calico promises to get rid of this separation. Instead, it relies completely on Layer 3, by employing a network concept that has been around for some time but has hardly established itself: routing in Layer 3.

To recap: Layer 2, the Link Layer, has, among other things, the task of switching packets within a physical network. It addresses the communication partners by the Media Access Control (MAC) addresses of their network interface cards. Layer 3, the switching layer, on the other hand, takes care of packet forwarding across the boundaries of local networks. This layer is the home of the Internet Protocol, whose addresses ensure that certain network nodes can be addressed worldwide on completely different networks. The device for forwarding packets in Layer 2 is the simple switch, and in Layer 3 the router.

When this division of labor was introduced into the Open Systems Interconnection (OSI) model, however, huge Layer 2 networks, as found today in clouds with thousands of servers, were not yet conceivable. Inventions that were helpful at the time, such as the Spanning Tree Protocol, are now a problem. One way out is Layer 3 routing, which determines the path of a packet from host to host solely on the basis of IP addresses.

The perpetual motion in such a setup is the Border Gateway Protocol (BGP). Every router that speaks BGP knows the routes to all other hosts in the network. A router no longer needs to be a separate box. Services such as Quagga, BIRD, and free range routing (FRR) can run on any Linux host, and now every standard switch from major vendors also supports BGP.

Every Linux server mutates into a small router; accordingly, every server on the network knows the route to every other host. The relevance of the physical infrastructure is in fact dwindling. Because all traffic is routed in Layer 3, network physical boundaries no longer matter.