Data center networking with OpenSwitch
Switched On
OpenSwitch is an open source, community-based platform for building customized data center networks. OpenSwitch [1] entered the stage in Fall 2015 [2] when Hewlett-Packard (HP) announced the new network operating system in cooperation with Accton Technology Corporation, Arista, Broadcom, Intel, and VMware. The series of Altoline switches [3] by HP Enterprise (HPE), as the group is now called, were supported from the very beginning. Less than a year later, more advocates were found, such as Mellanox, Cavium Extreme Networks, and even LinkedIn and Microsoft. Since the beginning of June 2016, OpenSwitch has been an official project under the auspices of the Linux Foundation [4] [5].
Difficult Beginnings
OpenSwitch can be regarded from two perspectives: the user's or the developer's. To explore OpenSwitch as a user, you need compatible hardware, such as a suitable switch. For the developer, the entry point is the OpenSwitch Gerrit page [6]. For development work, the prospective programmer needs a GitHub account and the Git [7] and Gerrit [8] tools. The project page [9] explains the individual steps. If you have dealt with Yocto [10] in the past, you should be able to find your way around pretty quickly. Yocto is a Linux distribution for small systems and is the foundation for OpenSwitch (Figure 1).
In this article, I approach OpenSwitch from the end-user perspective, because getting started from the development side would be difficult without user experience. However, not everyone has access to the required compatible hardware; fortunately, the project provides an alternative for virtual servers. The image is available in the Open Virtualization Appliance (OVA) format [11-13]. The tarball contains a prebuilt image in virtual machine disk (VMDK) format [14], which describes the associated virtual machine (Listing 1). If the existing infrastructure does not allow automatic integration of OVA images, you will need to take the manual approach.
Listing 1
Installation for Virtual Servers
$ tar tf openswitch-appliance-image-appliance-ops-0.4.0-master+2016080600.ova OpenSwitch.ovf OpenSwitch.vmdk $ tar xf openswitch-appliance-image-appliance-ops-0.4.0-master+2016080600.ova $ ls OpenSwitch.* OpenSwitch.ovf OpenSwitch.vmdk $ qemu-img convert -f vmdk -O raw OpenSwitch.vmdk OpenSwitch.img $ ls OpenSwitch.* OpenSwitch.img OpenSwitch.ovf OpenSwitch.vmdk
KVM was used as the hypervisor for this article. I took the specification of the virtual machine from the OVF file: two CPUs, 768MB RAM, and eight network interfaces. Converting the hard disk image is a matter of taste, because KVM can also deal with VMDK files. By the way, the first network interface is used for managing OpenSwitch or the underlying hardware itself; therefore, this interface must have an IP address that is reachable over the network. The assignment may be static or via DHCP.
Whether manual or automated, the installation of the virtual instance of OpenSwitch is quite painless and unspectacular. A few seconds after starting, the system is available (Figure 2).
Three users are predefined: root , admin , and netop . Only admin and netop are protected by a password, which is the same as the login name. Access to the all-powerful root user is not protected by additional authentication. For interactive sessions, users can use SSH over the default ports as HTTP or HTTPS. A root login is not allowed for access through the browser. Of course, you should change the passwords for protection against unauthorized access.
To install OpenSwitch on physical hardware, the project relies on the support of the Open Network Installation Environment (ONIE) [15]. ONIE provides methods and concepts for installing custom firmware on network devices. The term "firmware" might sound somewhat misleading here, because it refers in most cases to Linux-style systems, which typically refer to themselves as a network operating system (NOS). Interestingly, ONIE is also based on Linux.
The NOS installation is via the usual Linux bootloader mechanisms. The box "An Installer for Everything" provides further information.
An Installer for Everything
The goal of ONIE is to provide a uniform and simple way to equip network devices with the firmware, or NOS, of their choice. In the background, a modern Linux kernel and BusyBox [16] go about their work. The project even goes so far as to state that it would like to support parallel installations in large environments to the extent possible: Puppet [17], Chef [18], or Ansible [19] provide capable and reliable tools. The switch behaves much like a Linux server; adding administration tools is quite simple.
ONIE is an initiative of the Open Compute Project [20] [21]. Its founding members were, among others: Accton, Agema, Big Switch Networks, Broadcom, Cumulus Networks, and Dell. An overlap with the OpenSwitch proponents of the early days is not accidental. The driving force was Cumulus Networks [22], which is also active in the NOS market. ONIE seeks to serve individual end users, managers of large network environments, and switch manufacturers and vendors equally well.
The workflow of the NOS installation is straightforward (Figure 3). The low-level bootloader of the switch initializes the hardware and finally ONIE, (i.e., a Linux kernel with BusyBox on top). ONIE configures the management interface, loads the NOS installer, and executes it. ONIE can use various methods and configuration options [23] for finding and loading the installation program.
The NOS installer location information can be configured directly in ONIE. Alternatively, the data can be provided via DHCP, multicast DNS, and similar zero-conf methods, including the PXE boot mechanism known from server installations. Both TFTP and HTTP are suitable for transporting the binary data, and it is possible to save the installation program on a locally attached USB disk, which is where ONIE looks first.
Digging Deeper into the Rabbit Hole
As mentioned, the first step after you complete the install, should be to set up new passwords for the root, admin, and netop users. Creating your own users with the useradd
command is also a good idea. The normal Linux commands ps
, top
, and ifconfig
give you an overview, and systemd handles system startup.
If you look at the list of standard processes, you will see that many services with names that start with ops
are represented. Each is responsible for a specific OpenSwitch function. For example, VLAN management is handled by ops-vland
, whereas ops-ospfd
manages events relating to the Open Shortest Path First (OSPF) routing protocol. The project page has detailed documentation for each function [24].
Management requires OpenSwitch admins to start a special command interpreter (Listing 2), vtysh
[25], known from the Quagga routing software. Users in privileged mode then have the ability to copy configurations, for example. If you want to change the configuration, you need to switch to configuration mode, which is done by entering configure
or configure terminal
.
Listing 2
Client Configuration for NTP
root@switch:~# vtysh switch# switch# show ntp status NTP is enabled NTP authentication is disabled Uptime: 137 second(s) switch# switch# show ntp associations ---------------------------------------------------------------------- ID NAME REMOTE VER KEYID REF-ID ST T LAST POLL REACH DELAY OFFSET JITTER ---------------------------------------------------------------------- ---------------------------------------------------------------------- switch(config)# ntp server ntp.linux-magazine.com switch(config)# switch(config)# exit switch# switch# show ntp associations ---------------------------------------------------------------------- ID NAME REMOTE VER KEYID REF-ID ST T LAST POLL REACH DELAY OFFSET JITTER ---------------------------------------------------------------------- 1 ntp.linux-magazine.com - 3 - - - - - - - - - - ----------------------------------------------------------------------
Changes made in this way do not survive a reboot. For the experimental stage, this is quite useful, because a restart restores the system to a plain vanilla state. In normal operation, admins will usually want any changes to persist. With a simple command, you can send the run-time configuration to non-volatile memory,
copy running-config startup-config
which is done directly in the vtysh
.
Freeing the Way for Automation
The configuration of a single switch is only one piece of the puzzle, compared with the actual ambitions of the OpenSwitch project. The target is automation in larger configurations. The project prefers Ansible [19] and provides the documentation for getting started [26]. One argument in its favor is that it requires no installation on the target system (i.e., the OpenSwitch device).
The example given earlier for the NTP configuration of a single switch required logging on, changing to the routing shell, switching to configuration mode, and finally issuing commands. No one wants to do this manually on a large number of devices. The critical reader might argue that the NTP configuration is static in most cases and only needed during initial setup of a new unit, but the described process – login, vtysh
, configure
, <a command>
– can be compared with configuring a new VLAN.
This task is part of the admin's everyday life, and you will perform it in most cases on a series of switches. However, manual implementation doesn't scale. With OpenSwitch and Ansible, the new VLAN configuration can be rolled out to many systems in seconds; it is simple, repeatable, and supported on the hardware regardless of the manufacturer. The Ansible module ops_template
also supports the complete reconfiguration of many switches in a single step.
The core area of Ansible [27] included four modules when this issue went to press. Table 1 lists them with a short description. Of course, you can use other Ansible modules – after all, OpenSwitch is based on Linux. For the actual switch commands, you would need to configure the change to the special shell (vtysh
) via constructs such as:
echo 'show version' | vtysh
Table 1
OpenSwitch-Specific Ansible Modules
Module | Description |
---|---|
ops_command
|
Runs an arbitrary command on OpenSwitch |
ops_config
|
Manages the OpenSwitch configuration at the command line |
ops_facts
|
Collects device information for the switch |
ops_template
|
Installs a run-time configuration on OpenSwitch |
HTTPS almost has a double function: It is also the entry point for a REST interface [28]. Given the appropriate infrastructure, the OpenSwitch environment can be managed using GET
, POST
, DELETE
, and PUT
.
Buy this article as PDF
(incl. VAT)
Buy ADMIN Magazine
Subscribe to our ADMIN Newsletters
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Most Popular
Support Our Work
ADMIN content is made possible with support from readers like you. Please consider contributing when you've found an article to be beneficial.