Curb Complexity

The Security Myth

If you take a closer look, however, the myth of secure containers quickly and decidedly crumbles. As with packages, containers ultimately depend on who their originator is and whether it is a trusted entity.

The popular package managers offered by the major distributors have long since solved this problem. Both RPM and dpkg have mechanisms to establish a chain of trust that is documented right down to the package installed on a system. To this end, most distributors sign their package manager's distribution lists with a digital PGP key. With a signed file, you can then check whether the installed package matches the package listed there (e.g., whether the SHA256 checksums of the individual files match). This way of tracking whether any local changes have been made to the package content is both excellent and compliant.

The situation is quite different in the case of container images. The major vendors also offer ready-made base images, but these image do not typically contain any services. You have two possible courses of action:

creating your own environment for continuous integration and continuous deployment (CI/CD), or grabbing ready-made images off the web.

Complex CI/CD Structure

If you choose the first option of creating your own environment for CI/CD, you inevitably get to use tools like Jenkins or Zuul. The idea behind these tools is that, on the basis of distribution images, CI/CD systems automatically build a new image when a new software version is released and prepare it such that you only have to confirm production deployment by pushing a button. The rollout itself is handled by the CI/CD system, so your workload is kept within narrow limits, because the degree of automation in setups of this type is very high.

The bad news is that if you don't want to buy a ready-made CI/CD system for a large amount of money, you will have to spend time building one. Jenkins and the like are complex tools that can hardly be described as self-explanatory. Much time and effort goes into putting together a CI/CD system that ultimately gives you the same conditions that RPM, dpkg, and other package management systems have provided for decades.

Don't forget updates and how they are handled. You justifiably trust that running apt dist-upgrade on your system will apply all the security updates offered by Canonical or Debian. However, if you rely on CI/CD, you need to track security updates yourself and replace any affected containers individually rather than collectively.

More Overhead Detours

CI/CD systems are by no means self-perpetuating, then, and in some respects, they even cause more work than legacy package managers. Quite a few admins try to avoid this work and instead resort to ready-made images off the web. Docker Hub is a popular source for such images, but anyone can upload virtually any image there – and in any form.

Although ready-made images are available directly from the manufacturers for popular services such as MariaDB or MongoDB, images of lesser known software can often only be obtained from the community. In many cases, these images do not give you an opportunity to understand how they were built; consequently, you can only guess what is hidden inside.

In the past, several Docker Hub images for various services turned out to be Bitcoin miners in disguise. Other images are uploaded only once and are not maintained by their original authors after the event, and if a security update is due for the program, you are left out in the cold. The use of these black boxes in production operation can therefore only be strongly and categorically advised against. If you use them despite the warnings, you are turning the security advantages of containers into the complete opposite and will end up with significantly less secure systems than would be the case if you used a legacy package manager.