Provisioners

Provisioners are the interfaces between plugins in Kubernetes. On the one hand, they dock with the Kubernetes API, and on the other hand, they know which plugins to call to create a defined configuration state.

If you use storage as an example, classes in the Kubernetes API ensure the logical connection between resources of different types and a specific provisioner. In other words, a storage class in Kubernetes defines a specific storage type that Kubelet accesses through an appropriate plugin when it is prompted to create a resource of this kind. The terminology can differ in places for other external extensions, but on the whole, the functional principle remains the same.

Custom Resource Definitions

A custom resource (CR) and custom resource definition (CRD) are beasts you will encounter regularly. Although not primarily an external extension, the CRD API itself extends the Kubernetes API by creating a user's custom objects.

Out of the box, Kubernetes comes with a long list of resources that you access in your DSM manifests, but if you use additional external applications – or need to perform special tasks repeatedly – these standard definitions might not be sufficient, despite their large scope. Although the corresponding infrastructure would be available in Kubernetes, the resources would lie idle because the Kubernetes API would not call the resources. In these cases, CRDs enter the scene: They let you define arbitrary resource types in a Kubernetes cluster and associate them with the required functionality in the background.

CRDs also make it easier to create a kind of template for frequently used resource combinations. You could create, for example, a web server CRD, and then create ad hoc the resources needed for a web server (e.g., a virtual network). At the same time, you would select the correct image and add provisions for dynamic firewalling. Later, you would only need to reference the web server resource in your manifest without specifying everything in DSM-speak each time. This process saves time and helps keep YAML templates short for use in K8s.

As the success of CRDs impressively proves, virtually every external solution for use in Kubernetes – whether it needs plugins and provisioners or simply runs in Kubernetes – comes with its own CRD resources. Working with CRDs is business as usual for Kubernetes admins.

To match the custom resources, custom controllers work like the native Kubernetes controllers but let you perform arbitrary additional operations within the Kubernetes API. The combination of custom resources and custom controllers is known as an "operator." A Kubernetes operator provides most of the functionality you need to get specific software up and running quickly. One well-known example is the Prometheus time series database operator [2], which can be used to establish comprehensive monitoring, alerting, and trending (MAT) for an entire Kubernetes cluster (Figure 4).

Figure 4: Prometheus comes as an operator for K8s and contains definitions for both custom resources and custom controllers, so you can set up a complete monitoring system for K8s in next to no time.

Services

The market for software that adds features at the Kubernetes layer itself is at least as lively as the market for external solutions that connect to Kubernetes. These kinds of solutions have been covered in this magazine many times in the past.

One prominent example is Rook [3], which implements a Ceph cluster at the Kubernetes level. Rook creates volumes with custom CRDs that can then be attached directly to containers as native resources. Another very prominent solution is Istio [4], a service mesh that dynamically implements encryption and load balancing between each pod of an application in K8s (Figure 5).

Figure 5: Istio (shown with the Kiali front end) creates dynamic network meshes in Kubernetes, making cloud-ready implementation far easier. © Kiali

Solutions of this type integrate quite easily with Kubernetes, which is precisely why they are particularly popular on the market, and many companies are attempting to enter the world of K8s with such products, especially because they need have no dependence on an external entity. For example, a Ceph cluster built in Rook can be controlled exclusively by Kubernetes on-board tools, whereas attaching an external Ceph cluster to Kubernetes would require plugins, a storage class, and administrative access to multiple systems.