Safety Net

MP-BGP EVPN Advantages

To sum up, the underlay network does not need multicasting if you have MP-BGP EVPN because unicast-based MP-BGP peering over TCP port 179 is used instead. This peering already supports multiple hops in internal BGP (iBGP) by default. The VXLAN VTEPs can be located several hops distance from each other, which makes deployment very flexible. External BGP (eBGP) involves some manual work on the part of the administrator in the form of an explicit configuration to make this option is available.

Multi-hop functionality in this case refers to the time to live (TTL) in the IP header, which largely corresponds to an interpretation of the hop count. The TTL for BGP packets in an eBGP session is set to 1 in the basic configuration, which means that the peers must be in the same subnet. If this is not possible or desired, the administrator needs to enable BGP multi-hop on the router to increment the TTL. In the case of iBGP peering, multi-hop is allowed in the basic configuration because the underlying TTLs are higher.

The network layer reachability information is therefore only exchangeable between the fixed MP-BGP peers, which offers benefits from an IT security perspective. Potential attacker VTEPs are therefore unable to establish a neighborhood without further tricks, and you have the possibility of authenticating the VTEPs with more modern methods by MD5 authentication in the BGP session or on the basis of TCP authentication options (TCP-AO). The objective of TCP-AO is to provide protection against blind insertions and replay attacks. In flood-and-learn VXLAN networks, unauthorized VTEPs can inject data into the network.

MP-BGP EVPN also solves the problem of suboptimal data forwarding. In this example, the server system at site B would have to communicate over the default gateway at site A to establish a communication relationship from the VLAN, which is possible with a distributed anycast gateway on each VTEP in a VNI.

Specifically, then, each VTEP can act as a gateway with the same virtual gateway IP address and the same virtual MAC address to route IP packets from locally connected hosts to other subnets. Packets no longer need to flow across the WAN between sites. The ARP suppression feature is also interesting, involving the local VTEP caching the IP and MAC information of the connected VNI and responding to corresponding requests. If a host connected to this VTEP sends an ARP request, the local VTEP first checks whether it has an entry in its cache. If so, it responds directly to the ARP request.

If iBGP is used, some special features need to be considered: iBGP peers normally need a routing relationship with all other iBGP peers because routing information from iBGP neighbors is not forwarded among themselves. This organization, of course, creates massive administrative overhead in large environments. In such a case, a type of hub-and-spoke design – known as a route reflector in iBGP – can be used.

The associated VXLAN header is 8 bytes in total and comprises 8-bit flags, 24- and 8-bit reserved fields, and the 24-bit VNI. On transmission, the I bit for a valid VNI must be set to 1 and the R flags to 0 . All reserved bits must also have a value of 0 when sent. The VNI defines the individual VXLAN overlay network.

VXLAN Challenges

Connections that use the maximum packet size, or maximum transmission unit (MTU) [2], are problematic and also known from other tunneling solutions. The additional header increases the size of the transmitted frames. VTEPs are not allowed to fragment traffic at the source according to RFC7348, and fragmented packets can be "dropped silently" into a black hole on the VTEP. The MTU needs to be increased, if possible, or adapted to the VXLAN overhead. However, expansion of the Layer 2 segment also widens some attack vectors. For Layer 2 trunks, VLAN hopping, among other things, adds vectors.

The attack surface for ARP spoofing, for example, is also larger. On multitenant networks, attacks such as VLAN hopping have catastrophic effects. The built-in security mechanisms of the BGP protocol for MP-BGP EVPN are recommended for flood-and-learn-based VXLAN networks. Additionally, you should note that not all switches and hypervisors support VXLAN. Even if they do so fundamentally, it makes sense to examine the supported control plane methods of the component in question. On the other hand, some other methods for crossing Layer 3 boundaries (e.g., the relatively old Layer 2 Tunneling Protocol, L2TP) allow only point-to-point connections.

Another point of criticism is the lack of native integration of encryption methods, which would have to be implemented between the VTEPs by additional encryption procedures, further increasing complexity.

Additionally, VXLAN also does not have permanently assigned protocol identifiers. Therefore, a VXLAN tunnel cannot transport multiple payload types. Moreover, all fields are already permanently assigned in the VXLAN header, which makes extensions difficult. The lack of extensibility problem is addressed by the still fairly new Generic Network Virtualization Encapsulation (GENEVE) protocol, which is described in RFC8926. VMware already uses this in its NSX-T network virtualization. However, GENEVE currently lacks hardware implementations. Cisco Systems is now offering the first hardware switches with GENEVE support, but it remains to be seen whether other manufacturers will follow suit.

Conclusions

Classic VLAN stretching for Layer 2 site coupling is a quick solution, but it causes a few issues. VXLAN creates Layer 2 connectivity in a Layer 3 structure and is the obvious choice to meet increasing bandwidth and scalability requirements while maintaining the flexibility of Layer 2 interconnects for virtualized environments. Despite all the possibilities, however, you will always want to make sure at what points on your own network you need VXLAN. Solutions that do not rely on spreading Layer 2 broadcast domains are definitely preferable.