Dynamic Routing in Linux with Quagga

Complex, redundant networks, such as the Internet, require different routing policies from those typically found on a LAN. Ideally, the routers will know all the paths that lead to the target, but configuring them manually can quickly become confusing and lead to mistakes.

Quagga: Zebra for Admins

On the Internet, any attempt to do this manually would simply be impossible. The solution is to distribute dynamically changing route information automatically – including optional redundancies – if multiple paths lead to the goal. The web world has developed special routing protocols for this, usually only found on routers by Cisco or Juniper. However, Quagga gives IT administrators the option of participating in the world’s largest group of routers – with a Linux computer. The Quagga project originated with the Zebra Routing Daemon by Japanese developer Kunihiro Ishiguro. The software is included in all popular Linux distributions and also runs on Unix derivatives like Solaris and Free/Net/OpenBSD.

Quagga doesn’t handle the routing (that is still the domain of the underlying operating system kernel); however, it does provide a number of routing protocols – Routing Information Protocol (RIP), RIPng, Open Shortest Path First (OSPF) for IPv4 and IPv6, Border Gateway Protocol (BGP), and Intermediate System to Intermediate System (IS-IS) – and it modifies the kernel routing table on the routes it learns.

Scouting 10

The Internet consists of many, sometimes multiply, connected subnets between which packets need to find the optimum path. A network of routers managed by a team of system administrators is known as an “autonomous system” (AS). Routing protocols are divided into protocols that distribute the routes within an autonomous system (which can also be subdivided, so that many routers need to manage many routes) and protocols that distribute the routes between the autonomous systems.

The protocols within an AS are known as Interior Gateway Protocols (IGP), and those for outside an AS are known as Exterior Gateway Protocols (EGP). RIP, OSPF, and IS-IS are IGPs; the only EGP in use is BGP. Within an enterprise, an IGP will thus typically be used for routing, whereas a BGP handles routing between providers and, in some cases, between a provider and an enterprise.

RIP is the oldest IGP – the first version is from 1988. The current version is version 2, which also supports IPv6 via RIPng. Despite this, RIP is considered outdated and is thus not used frequently. In contrast, OSPF is one of the most commonly used IGPs. It introduces a hierarchy of areas to divide the network, where one area can contain multiple networks. The single-stage hierarchy starts with area 0, which is the backbone area. Each further area must be connected via a router, even if it does not need to exist physically in a production environment. Customized configurations assign interfaces to individual areas.

The routers then use multicast to exchange data via their LAN interfaces (on the multicast IP address of 224.0.0.5 ); this can optionally be authenticated via MD5. The routers first send HELLO messages to discover their neighbors. These messages are followed by Link State Announcements (LSAs), which contain information about the routers’ own routing table, areas, and the interface bandwidths. This information is then mutually used to update the routing tables.

When two paths lead to a network, but one router is connected to a gigabit network and the other to a 100Mb network, the path through the faster router wins. Where the paths are equivalent, the administrator can manually add a weighting value to the configuration, which is then reflected in the LSA information. If a route fails, the router on the alternative path handles traffic from that point on.

Dynamic Routing

Admins are not typically confronted with BGP until they redundantly connect their own networks to the Internet as an autonomous system or work for an ISP. The Internet would not work without BGP: The routers on the Internet backbone do not know any default routes; they only know routes to the networks of all other autonomous systems. Thus, the routing tables are correspondingly large (some 400,000 entries in April 2012.)

In contrast to OSPF, admins must configure the connection between two routers that exchange routes via BGP explicitly at both ends. The details include the partner’s IP, authentication information, and the number of one’s own AS. If multiple paths lead to the target, the goal is initially determined by the weighting specified by the administrator. Link speeds, as used by OSPF, do not influence the path selection.

BGP does not just attempt to find the shortest path; it also tries to implement routing policies that reflect contractual agreements and thus provider-dependent costs. This is exactly what the Quagga routing daemon on Linux does.

Quagga Architecture

Quagga includes several daemons: One service exists for each routing protocol (in the case of OSPF, one each for OSPF and OSPFv6), and the admin must configure these services individually. The various routing daemons are managed by a master daemon – still known as Zebra for historical reasons. Every daemon has its own configuration file and can be configured on a separate port. The Zebra control daemon controls and coordinates the whole thing.

Configuring the individual daemons at the command line is very reminiscent of Cisco’s IOS. When you connect to one of the services, you first need to authenticate with the enable command. To make changes to the configuration, you then issue the conf term command to switch to configuration mode. After entering parameters (e.g., hierarchical configuration blocks for an interface or router definition) with the help of commands like interface eth0 , you can type exit to quit configuration mode.

Till Reboot Do Us Part

The configuration file, typically /etc/quagga/zebra.conf , stores the authentication data, details of the logfiles for the daemons, the interfaces that Quagga manages (including IP addresses, although this can be handled in the host configuration), and any static routes. Note that if you stop the Zebra service, any IP addresses you configured only survive until the next reboot.

Only the Zebra process itself communicates with the kernel and manipulates the operating system’s routing table. On Linux, you can use configuration parameters to tell the daemon to manipulate a routing table other than the default and thus only use the routes learned by Zebra for policy routing (ip rule …), leaving packets that are not governed by the Linux policies untouched.

Separate daemons exist for OSPF, OSPF6, RIP, RIPng, BGP, and IS-IS; they are identifiable because their names are the same as the respective protocol names with a d appended. Each is addressable and configurable on its own port; Table 1 shows the assignments. If you value security, you should only address these services on the local interface (as localhost), because the plaintext connection is easy to sniff, including the access passwords.

Quagga with OSPF: A Simple Example

Listings 1, 2, and 3 depict the Quagga, OSPF, and BGP configuration files for a Linux system that exchanges routes with Cisco devices. Figure 1 shows the simple network setup. The Dynamips software simulates the virtual Cisco routers.

Listing 1: zebra.conf  for OSPF

01 !
02 hostname linuxrouter
03 password 8 7kdoaul4.iSTg
04 enable password 8 ZDF339a.20a3E
05 log file /var/log/quagga/zebra.log
06 service password-encryption
07 !
08 interface eth0
09  multicast
10  ipv6 nd suppress-ra
11 !
12 interface eth1
13  ip address 10.55.55.1/24
14  ipv6 nd suppress-ra

Figure 1: The sample network consists of two Linux clients, three software-simulated Cisco routers, and the Linux server with Quagga.

In the example network, two paths lead from the 1.2.3.0/24 network on the left to the 172.16.1.0/24 network on the right. Because Dynamips cannot cope with two routers on the same LAN segment of the host computer, I had to configure two different LAN segments. The configuration with just one segment differs only in terms of the network addresses.

Zebra sets the IP address for the Quagga router’s eth1 interface. Otherwise, it is sufficient to enable multicast on the interfaces that speak OSPF; in the example, this is only eth0 . The ipv6 nd supress-ra setting ensures that zebra does not send routing advertisements without the admin explicitly enabling this functionality. The relevant part of the configuration is found in ospfd.conf (Listing 2).

Listing 2: ospfd.conf

01 !
02 hostname ospfd
03 password zebra
04 enable password secret
05 !
06 interface eth0
07  ip ospf message-digest-key 1 md5 hallo123
08 interface eth3
09  ip ospf message-digest-key 1 md5 hallo123
10 !
11 !
12 router ospf
13  redistribute connected
14  network 172.16.1.0/24 area 0.0.0.0
15  network 172.17.0.0/
16 area 0.0.0.016  network 192,168.1.0/24 area 0.0.0.0
17  network 192,168.2.0/24 area 0.0.0.0
18  network 192,168.40.0/24 area 0.0.0.0
19  area 0.0.0.0 authentication message-digest

The instructions below the interfaces set the password that authenticates the multicast packets. The OSPF routing process definition first specifies which of the router’s known routes it should distribute via OSPF. The redistribute connected statement distributes all routes on the computer’s interfaces, including those that do not use OSPF. The redistribute kernel statement distributes routes that the operating system has received manually, and redistribute static distributes routes from the zebra.conf file.

The admin must assign interfaces to OSPF areas. It is sufficient for one router to know the assignments. The syntax here differs from IOS when it comes to stating network blocks: A.B.C.D/<X> instead of the Cisco syntax with an inverted netmask – that is, 192.168.1.0 0.0.0.255 and 0.0.0.0 for the area instead of just 0 . The last statement stipulates that only authenticated information is allowed in area 0.

Entering the sh ip route command at the Zebra prompt shows you the learned and internal routes; a O at the start indicates that the router learned the route via OSPF. At the Linux prompt, proto zebra shows that the route found its way to the kernel from Zebra. However, it does not tell you which routing protocol taught Zebra the route.

If a link fails without the router noticing – for example, because the Ethernet link is down – it then takes three OSPF Hello intervals for the networks to be connected again. Figures 2 and 3 show a ping and the resulting downtime.

Figure 2: The connection breaks down; it takes a while for the Quagga router to notice …

Figure 3: … but after 36 packets, it has already changed the route.

However, if a router can tell the rest of the array that one branch is not working (e.g., a switch has failed or a cable has become unplugged), then the downtime is not measurable. Because the link failure in this case tells the router that an error has occurred, you cannot use ping to measure the downtime (Figure 4).

Figure 4: The downtime is not measurable if the Quagga router notices the link failure and can react immediately.

You can reduce the downtime by increasing the OSPF message frequency, but it is important to find the right balance for each scenario.

BGP Routing for Autonomous Systems

BGP controls the connection between autonomous systems that are identified by their numbers. In the example, all of the routers on the left side belong to AS 64515, and the Linux router belongs to AS 64513. Listing 3 shows the unspectacular configuration in the bgpd.conf file.

Listing 3: bgpd.conf

01 !
02 hostname bgpd
03 password zebra
04 enable password zebra
05 log stdout
06 !
07 router bgp 64513
08  bgp router-id 192.168.1.253
09  redistribute kernel
10  redistribute connected
11  neighbor 192.168.1.200 remote-as 64515
12  neighbor 192.168.1.200 password test123
13  neighbor 192.168.40.253 remote-as 64515
14  neighbor 192.168.40.253 password test123

After setting the hostname and the login and privileged mode passwords, router defines the BGP router and your own AS number. The router ID that follows is one of the router’s interface addresses. The redistribute statements specify which routes the server should distribute to partners via BGP. This is followed by the definitions of the two partners, at least with the AS at the other end, and the password to authenticate this connection.

The routers immediately try to open the connections and exchange routes. Failover is already supported. However, in the event of a router failure, recovery can take a little longer in this simple configuration (Figures 5 and 6).

Figure 5: In the simple BGP example from Listing 3, …

Figure 6: … it takes no fewer than 145 seconds for a failover to occur.

Maps for the Scouts: Routemaps

Routemaps are used to control which protocol a service uses to distribute which routing information. Additionally, administrators can manipulate values, such as the preferred routes. This means that you can use routemaps to implement specifications such as “export all routes you learned via OSPF to BGP, except for those that point to networks below 172.16.0.0/16” or “export all routes learned via BGP to OSPF, but with a lower preference.”

Thus, in the case of overlaps, the routes learned locally via OSPF on the LAN will take priority. The router would not use the slower or more expensive external route unless no other route was possible. The models are extensible: In complex environments, routemaps with many rules can very quickly lead to constructions that are so extensive that cautious admins will not touch them unless absolutely necessary.

Administrators need to enter routemaps in the configuration files for the routing process that they want to influence. If you are importing from BGP to OSPF, this file would be ospfd.conf . For example, if you want to prevent all routes below 172.17.1.0/24 being exported from BGP to OSPF, the lines in Listing 4 will do the trick.

Listing 4: ospfd_routemap.conf

01 !
02 router ospf
03  redistribute connected
04  redistribute bgp route-map bgp-to-ospf
05 [...]
06
07 ip prefix-list into-ospf seq 5 permit 172.17.1.0/24 le 32
08 ip prefix-list into-ospf seq 10 deny 0.0.0.0/0 le 32
09 !
10 route-map bgp-to-ospf deny 5
11  match ip address into-ospf
12 !
13 route-map bgp-to-ospf permit 10

Within the OSPF configuration, the statement that redistributes the BGP routes in route-map bgp-to-ospf creates the reference to the bgp-to-ospf routemap.

Not Always Logical

Because this example filters the routes according to an IP prefix, you first need to define the prefix. To do so, forget everything you know about logic. It is helpful to imagine every instance of permit as being true , and every instance of deny as being false . IP prefix lists also have a name and a sequence; the system processes them top down, just like Netfilter firewall rules, for example. Lines 7 and 8 in Listing 4 state that the IP prefix 172.17.1.0/24 belongs to the list, but all addresses from 0.0.0.0 to 255.255.255.255 do not. The notation le 32 states the maximum bit count for the area, beginning with a starting value (in this case, 0.0.0.0 ).

The definition of the routemap follows. Line 10 prevents any non-specified routes from being distributed (deny ). Line 11 specifies the condition under which this entry is applied and refers to the IP prefix list. Expressed as a model, if the IP address matches the prefix list named into-OSPF , then this rule applies. In this routemap, the last line of the configuration allows all routes because it is not linked to a condition.

This simple example shows how confusing things can be with many prefix lists and dual logic. To find your way around a complex scenario, you need to be a really good scout.

The configuration for IPv6 is not too different from IPv4. Where the configuration contains ip , Quagga simply replaces this with ipv6 . Again, the configuration differs from Cisco, which uses ip6 . The configuration for OSPFv6 uses a file of its own and addresses a separate daemon. In BGP, the daemon is the same as for IPv4. You can even distribute IPv4 routes to an IPv6 peer, and vice versa. In the first case, you simply define an IPv6 address as the neighbor .

The IPv6 peer is enabled by the

address-family ipv6
neighbor w.x.y.z activate

commands. Now the peer receives all the routes (IPv4 and IPv6). If you only want to distribute IPv6 routes, you need to build your own routemap to do so. The first term in the map is a permit that allows all IPv6 addresses, and the second term is a deny for all IPv4 addresses:

ipv6 access-list allv6 permit any
access-list nov4 deny any

The seemingly logical no activate neighbor x.x.x.x in the IPv4 test configuration resulted in Quagga not distributing any routes at all to its partners in the test configuration.

Conclusions

A Linux computer probably can’t replace a high-end router with multiple 10Gb interfaces. On the other hand, the prices for routers with enough memory to process hundreds of thousands of routes for full BGP are so astronomically high that PC hardware at least provides a cheaper alternative.

If you are familiar with the syntax of Cisco systems, you will quickly find your way around Quagga. Nevertheless, it is unusual to have to start a new session for each routing protocol, and Quagga’s behavior is subtly different from IOS, although this can also be true of different IOS versions.

Overall, you can go a long way with Quagga; it supports redundancy and lets you manage larger numbers of Linux routers in a clear-cut way, thanks to the use of dynamic routing protocols. Given the feature scope that Quagga offers to help you transport IP packets from A to B, the examples shown in this article only scrape the surface. Experienced scouts can benefit from advanced options, such as the ability to connect Quagga to a monitoring system via SNMP traps or to analyze the details of the Zebra protocol.

The Author

Konstantin Agouros works for n.runs AG as a consultant for network security. His main focus is on telecommunications providers. His book DNS/DHCP is published by Open Source Press.