The Supercomputing Conference always causes me to reflect on the tools I use for system administration and the tools I would love to have. The most difficult situations I encounter as an admin occur when I can’t get the display server to work on a remote system, with no easy way to plug in to a crash cart. Usually I can SSH to the system and poke around to see what is happening: I’m definitely going to want to test the node to see whether my corrections work. This combination of searching, debugging, and testing over a simple login with SSH is not always easy. Sometimes I miss graphical tools, particularly, if I’m looking at time histories of the node. (My favorite is users swapping the node to death or invoking the dreaded out-of-memory (OOM) error, which is sometimes difficult to find.)

For situations like these, I reach for text-based user interface (TUI) tools. They give me some reasonable graphics while not requiring a display server on the system. TUIs are extremely useful but underrated admin tools, of which I’ve used many over the years. In this article, I review a few TUI-based admin tools and applications and point out some tools you can use to create your own TUIs for your situation.

User Interface

About everyone in the world uses a graphical user interface (GUI) to access their desktops, laptops, and mobile devices. The icons and visual indicators of the interface, with some text as a secondary form of input, were developed because command-line interfaces (CLIs) were perceived to be slow and inefficient and could not present graphical images and volumetric data easily. Pretty much every operating system on any device uses a GUI: Windows, macOS, Linux, iOS, Android, desktops, laptops, phones, tablets, ATM machines, and so on. They are the number one way for people to interact with computer systems with a mouse, finger, or stylus.

To utilize a GUI, you need to have some sort of display server on the computer that interacts with a window manager, which is the GUI. These display servers can be quite complicated and require a great deal of computational, memory, network, and disk resources. What happens, though, if the display server misbehaves and you have to debug the problem? How do you interact with the system?

One of the first things I learned as a system administrator is always to have a CLI link to systems so I can edit configuration files, monitor the system, restart services, read logs, or accomplish any of a host of tasks. Within one month of becoming a system administrator, I had a server go sideways, requiring me to log in remotely to stop and restart some services. Having a CLI connection allowed me to do this.

CLIs respond to a command entered by the user, whereas GUIs respond to a click on an icon, perhaps with a bit of input typed into a dialog. However, if the GUI display server is not behaving well and you want to monitor multiple system metrics over time, a CLI won’t necessarily give you the desired results. What you really need is a text-based GUI of some sort. In other words, a TUI.

Text-Based User Interfaces

TUIs were named after GUIs. Like GUIs, they display computer graphics, but they do it in text mode, not graphics mode. The more advanced tools allow you to use a mouse or can have multiple, even overlapping, windows.

As a rough approximation, TUIs are sort of in between a CLI and a GUI. They make CLIs more interactive and can present more information. Relative to GUIs, TUIs can be faster, don’t require display servers, and can be used over SSH. They also can display time-varying information, unlike a CLI.

Before GUIs became standard, Microsoft DOS applications used TUIs, including WordPerfect, WordStar, QBasic, GW-BASIC, QuickBASIC, and Turbo Pascal, which used Turbo Vision for the TUI. Each had its own TUI library because operating systems did not have a standard at the time.

Administration tools are the primary users of TUIs.

TUI Tools

Even today you can find many examples of TUI applications. They are very fast – faster than GUIs – and easy to use. They help CLI applications that have a large number of options and often provide visual feedback. Many of these tools are widely used, including on HPC systems, to collect statistics across the cluster and manage nodes; they are ideally suited for debugging and testing a small number of nodes.

TUIs are still popular for general applications, although perhaps not so much as administration tools. Nonetheless, they are still used and new ones are still being developed. Table 1 lists a number of administrative and non-administrative TUIs.

Table 1: Example TUIs

Writing TUIs

Rather than just present a list of TUI applications that can be used for administration or as everyday applications, I will illustrate that it is fairly straightforward to write these applications, particularly for administration tasks.

A few main libraries or toolkits have been helpful; the first, and probably largest, is curses. Originally written in the late 1970s, curses was optimized and expanded in the early 1980s for the video game Rogue. It then began to be used for a number of applications. Development of curses slowed down, and a new library, ncurses, was developed that proved to be more popular than curses. As part of the GNU Project, it is still being developed, although at a slower pace. Figure 1 shows a sample ncurses screenshot.

Figure 1: ncurses-based menuconfig (Wikipedia, CC BY-SA 3.0).

Another option is the Newt library, which focuses on color TUIs (Figure 2) and uses a widget approach, making programming a TUI much easier. With Newt, you can create stacked windows, entry widgets, checkboxes, radio buttons, labels, plain text fields, scrollbars, and so on. In Python, this library is referred to as snack and was originally developed for Red Hat’s TUI installer, Anaconda.

Figure 2: The Partimage tool uses Newt to create its user interface (Wikipedia, GPL).

A number of tutorials on the web can guide you in using Newt with pretty much any language you want. (Sorry, Fortran-ers, I don’t see anything for us yet.) Python is a fairly common language for using Newt (python-snack), and a Jupyter notebook can help you learn Newt in Python.

Another useful TUI library that can be used with Python and Bash is called dialog, which you can use to create pop-up dialog boxes from Bash scripts (Figure 3). Furthermore, dialog has been rolled into a Python library named, aptly enough, pythondialog that allows you to use dialog boxes and other constructs from the dialog library in your Python code.

Figure 3: Sample pop-up dialog from Bash script: dialog --title "Message" --msgbox 'Your job is done' 6 20

If you are interested in writing TUIs for system administration, a great place to start is with the psutil library, which is great for gathering system statistics. Then, you can take a look at the scripts subdirectory on GitHub, where you can find a number of sample scripts for creating a TUI (e.g., the top clone top.py). You can also look at other TUIs to understand how to create plots, strip charts, and other useful features. Don’t forget, these interfaces are all text based and can be easily used over port 22 (SSH).

Be sure to examine the admin tools mentioned in Table 1. The charting capability, even with just text-based graphics, is great. I love using these tools to get a simple view of how a node is behaving. Take a look at the code and even think about a way to make a function for creating these plots (please create Python interfaces to make life easier).

Summary

A system administrator’s job is to keep systems running smoothly and effectively at all times, which means having to triage misbehaving nodes from time to time. In HPC, these nodes often don’t have a display adapter or has one that is having problems. Being able to log in to the node and use quasi-graphical interfaces over port 22 is a real blessing. You can test applications and monitor the system with the use of TUIs. Although often ignored, believe me, they are very useful in the admin world.

Moreover, TUIs are not difficult to write. With tools from various libraries and the help of online examples, you can create your own tool suited to your situation. Python or C is a great way to get started. If you write something or use TUIs on a daily or weekly basis, please give a shout-out on Twitter – I’m sure everyone would love to learn more.