More Dangers

In his blog [2], Scott Hogg, the Chief Technology Officer of network technology company GTRI, describes some common errors related to spanning tree configurations. One of these errors relates to the failure to observe size limitations of a spanning tree (i.e., the maximum number of cascaded switches). Ideally, a star topology completely prevents cascaded switches, but implementing a star topology is often impossible, for example, in a campus with multiple buildings.

Another fundamental weakness of Spanning Tree is that the algorithm assumes no switch is connected in the absence of BPDUs on a port. This conclusion is potentially false, however, and incorrect configurations or software errors can thus cause loops in the network.

A possible cause for lack of BPDUs is a hardware error. For fiber optic connections, one of the two channels might be damaged so that the connection can still receive BPDUs but not send them. Unidirectional Link Detection (UDLD) [3] helps to counter these problems in OSI Layer 1. UDLD is complemented by Loop Guard [4].

The false conclusion that no switch is connected because no BPDUs arrive is also countered by Bridge Assurance . The Bridge Assurance technology should therefore be in place on all switch ports that connect to other switches. The exceptions are various forms of multiple-chassis EtherChannels; for example, Bridge Assurance is not recommended for Cisco virtual PortChannels (vPCs).

Then and Now

On older Ethernet networks, a topology as shown in Figure 6 is conceivable. In Figure 6, the spanning tree has disabled half the links in the Layer 2 area, and the server (bottom) uses only one of two connections. The role of the root bridge is handled by the AGG1 switch. In conjunction with AGG2, the switch forms the boundary between Layer 2 and Layer 3; the two are referred to as Layer 3 switches, although – according to the OSI Reference Model  – a switch resides in Layer 2. The two Layer 3 switches run a First Hop Redundancy Protocol (FHRP), either with the open VRRP standard or using an alternative such as Cisco's proprietary HSRP counterpart. The protocol is only active on AGG2. The First Hop is the IP default gateway of the server.

Figure 6: In this configuration, the original spanning tree algorithm results in a suboptimal path from the server to the gateway.

The problem of the suboptimal route that the IP packets take from the server to the gateway (AGG2) in Figure 6 results from a spanning tree blocking connections for all routed packets through AGG1. To fix the problem, either AGG2 would have to take over as the root bridge, or you would need to enable the FHRP protocol on AGG1. In a more complex topology, the impact of such an unfavorably placed root bridge often causes much more serious problems.

Non-Blocking Architecture

Because of high port prices and performance requirements for datacenter switches, it is generally not a good idea to use a spanning tree to block links. Various manufacturers offer remedies that allow multiple physical links between two switches, or between a switch and a server, to be combined to form a logical link (Figure 7). Some solutions even allow the combination of more than two switches.

Figure 7: Multiple-chassis EtherChannel bundles switches and network connections together to form logical units.

The aggregation switches AGG1 and AGG2 in Figure 7 now create a big AGG switch, and the two access switches ACC1 and ACC2 form the logical ACC switch. Solutions such as Multiple-chassis EtherChannel, for example, support this kind of aggregation. The server operating system, in turn, bundles the two connections to ACC1 and ACC2 to create one logical link (network bonding) so that the full bandwidth is available.