Correctly integrating containers

Let's Talk

Pitfalls

Some of the pitfalls on the way to creating useful Kubernetes clusters also need to be mentioned here. Docker, for example, needs to launch a pause container to create a network namespace. In this case, a process that does nothing else runs in this container. Bootstrapping with Docker does not always work without trouble. On Red Hat systems with Calico, for example, inconsistencies with the firewalld daemon occurred and caused a number of solvable yet irritating problems.

Conclusions

Kubernetes is a rapidly evolving ecosystem that pursues many very good strategies the project has not yet fully implemented. If you are aware of the three-month release cycle and are ready to join the development work on Kubernetes, you can look forward to a system that makes developing, updating, and customizing distributed applications easier than any other distributed system.

If you need multitenancy (see the "Multitenancy in Kubernetes" box) or do not work with highly security-sensitive data, Kubernetes can be used now for large, distributed production applications. However, if you have other applications in mind, you should wait one or two versions.

Multitenancy in Kubernetes

Apart from safeguarding network policy, it is also important to secure a cluster at the container level, which means the user is not allowed to use privileged containers unless explicitly authorized to do so.

Users prove their identity to Kubernetes with client certificates or – less recommended – passwords. External services can also be connected. Kubernetes automatically creates client certificates as secrets when creating service accounts. Related concepts will be delivered during the alpha to beta stage of production, although it works quite well already.

Privileged containers are the new root privileges. They are very powerful and dangerous if misused. To build Docker images, you would, for example, embed the Docker socket of the surrounding host in these containers. However, in the worst case, the surrounding node can be hijacked.

The path to a secure system is not easy and can only be outlined here. First, kube-admin creates service accounts, then generates roles via role-based access control (RBAC) [20], and binds them to users or groups. The right to administer a security policy, for example, must be severely restricted.

Roles give rights to resources in the form of verbs. For concepts with pods, verbs are actions: REST API calls here. Put simply, they give rights to perform read (get and list) or write (create or update) operations on pods, services, ingresses, or secrets. Doing so often goes deep into resources and can, for example, prevent users from creating pods without a security context – or with the wrong one.

If you withdraw the right to run root processes in containers with spec.securityContext.runAsNonRoot:true, most of the examples from the Internet will no longer launch. You should definitely switch on the feature in secure environments (Listing 9). It is well implemented with different degrees of success in the various implementations of Kubernetes; you are on the safe side with OpenShift and the Kubernetes distribution by Red Hat.

Listing 9

Pod with Security Context

01 [...]
02 apiVersion: v1
03 kind: Pod
04 metadata:
05   name: hello-world
06 spec:
07   containers:
08   # Specification of the Pod's Containers
09   securityContext:
10     readOnlyRootFilesystem: true
11     runAsNonRoot: true
12 [...]

Infos

Kubernetes: https://kubernetes.io
Cluster networking: https://kubernetes.io/docs/concepts/cluster-administration/networking/
Project Calico on GitHub: https://github.com/projectcalico/calico/
According to the Kubernetes documentation: "a pod is a group of one or more containers (such as Docker containers), with shared storage/network, and a specification for how to run the containers. A pod's contents are always co-located and co-scheduled, and run in a shared context": https://kubernetes.io/docs/concepts/workloads/pods/pod/
Kubernetes v1.7: http://blog.kubernetes.io/2017/06/kubernetes-1.7-security-hardening-stateful-application-extensibility-updates.html
Network policies in Kubernetes: https://kubernetes.io/docs/concepts/services-networking/networkpolicies/
Flannel: https://github.com/coreos/flannel
Packet path: https://github.com/coreos/flannel/blob/master/packet-01.png
Plugin specs for the Container Network Interface: https://github.com/containernetworking/cni/blob/master/SPEC.md
Calico networking using Vagrant and the Calico CNI plugin: https://github.com/projectcalico/calico/tree/master/v2.2/getting-started/kubernetes/installation/vagrant
CoreOS cloud config: https://coreos.com/os/docs/latest/cloud-config.html
ConfigMaps in Kubernetes: https://kubernetes.io/docs/tasks/configure-pod-container/configmap/
DaemonSet: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/
Felix on GitHub: https://github.com/projectcalico/felix
Calico network policy: https://github.com/projectcalico/k8s-policy
HAProxy: https://haproxy.org
Nginx reverse proxy: https://www.nginx.com/resources/admin-guide/reverse-proxy/
F5 hardware: https://f5.com/products/deployment-methods/hardware
"Kubernetes and CoreOS" by Thomas Fricke, ADMIN , issue 36, 2017, pg. 36, http://www.admin-magazine.com/Archive/2016/36/Safeguard-and-scale-containers
RBAC: https://kubernetes.io/docs/admin/authorization/rbac/

The Author

Thomas Fricke is CTO of Endocode AG in Berlin, Germany. He is a cloud, database, and software architect involved in system automation and DevOps. Contrary to all preconceptions about Germany's capital, he travels through the city of Berlin and its forests by bike.

« Previous 1 2 3