Stretching devices with limited resources
Fewer Memories
Small embedded devices running Linux often face a trade-off between limited resources in CPU and RAM and the abysmal write performance of SD card storage with abundant capacity. The first rule, as I discussed back in 2018 [1] is to make exclusive use of high-quality SD cards for boot devices – a class 10 device or better for starters – with a reputable brand name printed on it. (I recommend SanDisk or Samsung SD cards.) Lower quality SD cards simply cannot satisfy the write pressure of system boot or other peak logging moments of operation, and while suitable for data, they cannot be used as boot devices. These are the basic table stakes, but this being Linux, you can of course optimize further!
Test Bench
I am going to discuss a specific board for this column to try out the theories in practice. I will be using a Marvell ESPRESSObin [2] v7 (Figure 1), a low-cost, low-power board made by Globalscale Technologies. The ESPRESSObin was one of the earliest single-board computers (SBCs) to sport a 64-bit ARM CPU and remains remarkable today for its inclusion of a SATA port and PCI expansion. The full spec of the board is interesting in its own right (Table 1), but here I will focus on the setup for my tests. I use the external 12V power supply provided and the USB interface for serial access.
Table 1
ESPRESSObin Technical Specs
SOC | Marvell Armada 3700LP (88F3720) ARM Cortex A53 processor |
Dual core up to 1.2GHz | |
System memory | 2GB DDR4 (1GB models also ship) |
Storage | 1x SATA interface |
1x micro SD card slot | |
Footprint for optional 4GB eMMC (not populated) | |
Network | 1x Topaz networking switch |
2x GbE LAN | |
1x Ethernet WAN | |
1x MiniPCIe slot | |
USB | 1x USB 3.0 |
1x USB 2.0 | |
1x micro-USB port | |
Expansion | 2x 46-pin GPIO headers for I2C, GPIOs, PWM, UART, SPI, MMC |
Misc | Reset button |
JTAG interface | |
Power supply | 12V DC jack |
5V via micro-USB port | |
Power consumption | Less than 1W thermal design power (TDP) at 1GHz |
On the client side, I employ the handy Serial [3] Mac program, which includes userspace implementations of driver stacks for the two common USB-to-serial chipsets, eliminating the hassle of installing or updating drivers on multiple computers (and the reboots that go with those Mach kernel modules). The connection runs at 115,200 baud, with no flow control and no parity.
The board bootstraps with U-Boot [4], and from there it loads the operating system. Because the board was first released in 2017, the natural places to build current OS images are the Yocto project [5] and Armbian [6], a community-managed distribution that builds board-specific images of Ubuntu and Debian. I chose to use the stable Armbian build of Debian Stretch for the ESPRESSObin, and the results are shown in Figure 2.
The Right Trade-off
On systems with lots of storage and not enough RAM, one is naturally inclined to think about turning some of the permanent storage into swap space [7]. Unfortunately, the low write performance of SD card storage precludes this avenue to most embedded SBCs. The sensible strategy here is to turn some of the RAM into a write-friendly device and to stretch the amount of RAM allocated with the use of inline data compression for that partition.
I chose Armbian partly because it is a distribution that turns to this strategy by default. Figure 3 shows Armbian using a zram [8] device to store /var/log
, offloading the primary source of boot-time writes from the physical device to a 50MB RAM drive.
Zram, introduced by kernel 3.14 and considered stable, creates a block device in RAM and compresses writes to it on the fly. Default zram configurations on Debian and Ubuntu (controlled by the zram-config package) also create one swap device per CPU core, as seen in Figure 3. The increase in CPU load is counterbalanced by the availability of more memory, as existing RAM is stretched with the use of compression.
Starting with kernel 4.14, zram can be configured as a write-back cache committing data to permanent storage in the background. Until then, this was commonly accomplished with zswap [9], a lightweight memory page compression driver that operates exclusively as a writeback cache for swap.
Swap Usage
Some important workloads, most notably Kubernetes, are openly averse to swap, preferring to address resource limits early rather than accept performance degradation and inconsistency. At the edge or in the Internet of Things (IOT), you have to work with the hardware you have and compromises are more likely. Either way, the free
[10] command provides a simple avenue to check swap use (Listing 1). Currently the system described is not using swap.
Listing 1
Checking Swap
root@espressobin:~# free total used free shared buff/cache available Mem: 2046088 82796 1834332 5548 128960 1891416 Swap: 1023040 0 1023040 root@espressobin:~#
Buy this article as PDF
(incl. VAT)