Same but Different

When talk turns to containers, most people almost automatically think of the well-known Docker engine. For some time now, however, Podman [1] has been a serious alternative. The project was developed together with CRI-O, an implementation of the Kubernetes Container Runtime Interface (CRI). Podman is not limited to use within a Kubernetes environment, however; you also can use it to run individual containers, and you can even adopt the concept of pods, independent of Kubernetes.

Look Ma, No Daemon

Although the Podman developers have made sure that the Podman command-line tool is almost identical to Docker's, the two container engines are fundamentally different in terms of architecture. Everything in the Docker world is based on the client-server principle, whereas Podman relies on the fork-exec model. When Docker starts a new container with the docker command, the instructions are sent to a daemon running in the background, which in turn forwards the command to start the container with containerd, another Docker framework daemon. Podman, on the other hand, does completely without daemons, creating all containers as child processes of the Podman process.

UID-Based Activity Assignments

Thanks to the Podman architecture, the Linux kernel's audit subsystem can uniquely associate activities taking place inside a container with the user who started the container. As you know, every user who logs on to a system is assigned a unique user identifier (UID) that can be read from the /proc/self/loginuid file:

cat /proc/self/loginuid
1000

The login UID does not change, even if processes are started under a different ID. For example, the following command calls the file as root, but the user's login UID does not change:

sudo cat /proc/self/loginuid
1000

All processes the user starts now automatically inherit this login UID, including containers and the processes that run in these containers. The following example reads the /proc/self/loginuid file inside a Fedora container:

sudo podman run --rm fedora cat /proc/self/loginuid
1000

As you can see, the UID is still 1000. Now, you can leverage this fact to audit processes inside containers. In the auditd logfiles, the login UID is listed in the auid field:

sudo ausearch -k watch-passwd
time->Tue May 28 19:52:15 2019
type=CONFIG_CHANGE msg=audit (1559065935.923:2447): auid=1000 ses=2 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 op=add_rule key="watch-passwd" list=4 res=1

In this way, it is very easy to see which processes were executed by which user within a container. With Docker, on the other hand, a daemon is responsible for starting containers, so the login UID is always 4294967295, which is the ID inherited from every process started by the init system and means that it is no longer possible to assign activities within a container to a particular user.

Controlling containers is not the only thing that is different between Podman and Docker. The way container images and access to container registries are managed internally also differ considerably between the two engines. Docker again uses the central Docker daemon, whereas Podman uses libraries from the container repository and is therefore independent of a daemon. These differences are completely transparent for the end user, however.

Starting Containers Without Root Privileges

Another interesting feature of Podman is that non-privileged users can start containers. Processes running as root within the container are converted on the host to the UID of the user who started the container. Other IDs need to be mapped in advance to a specific ID range. The whole thing works because Podman uses the Linux kernel's user namespace. Users who want to work with containers need an entry in the /etc/subuid and /etc/subgid files that map the IDs used in the containers to that on the host. Be careful to avoid overwriting existing IDs on the host:

echo tscherf:100000:65536 > /etc/subuid
echo tscherf:100000:65536 > /etc/subgid

In this example, the first UID in a container is mapped to UID 100000 on the host. The last UID in a container is 65536 and, therefore, 165535 on the host. Mapping the group identifiers (GIDs) works in the same way. Identifier 0 (root) is always mapped by default to the UID of the user who started the container. The following Podman call confirms the correct mapping of the UIDs and GIDs:

podman run fedora cat /proc/self/uid_map /proc/self/gid_map
0   1000       1
1   100000   65536
0   1000       1
1   100000   65536

If you now start a new container, the process inside the container is executed as root (Listing 1). On the host system (Listing 2), however, the process runs under the ID of the user who started the container.

$ podman run -d fedora sleep 1000
 **
$ podman exec 291ee2dda43e ps -ef
UID   PID   PPID  C STIME TTY   TIME     CMD
Root  1     0     0 19:05 ?     00:00:00 sleep 1000

$ ps -ef|grep sleep
Cherf 30328 29757 0 20:44 ?     00:00:00 sleep 1000
Cherf 30396 3353  0 20:44 pts/8 00:00:00 grep --color sleep

Users can therefore run software in a container under the root user context without problems and without having to give the software root privileges on the host system at the same time. At this point, it is worth noting that rootless containers can cause some issues (e.g., for all actions that require root privileges on the host system).

Assume you want to map one of the container's network ports to a privileged port (<1024) on the host; this operation does not work if you have started the container with normal user privileges. In this case, you need to run the container with root privileges; otherwise, the required system call to CAP_NET_BIND_SERVICE cannot be executed on the host.