Watchdog

Sidecar

As already described, OPA itself does not initiate any actions if a policy check produces a negative result. It also does not initiate the corresponding checks itself but merely provides the engine that external components can use. Of course, you must therefore store suitable compliance rules for K8s in OPA, which is relatively simple. Also, you must provide a component to K8s that reads the workloads there, initiates the compliance check by OPA, and, if necessary, takes action if it fails.

The kube-mgmt sidecar, the older of the existing solutions, does several things in parallel for this purpose. First, it rolls out OPA automatically into an existing OPA cluster. The sidecar then takes care of getting OPA up and running, including redundancy, so you do not need to worry about that. The only thing you do have to take care of is providing a suitable SSL certificate. For K8s to communicate with OPA, it insists on an encrypted connection. Once this has been established, OPA can be used in K8s (e.g., to avoid creating resources that fail to meet certain specifications).

Consider the example of an enterprise policy that specifies that all services opened to the Internet must be accessible on port 443 in the example.net domain. To do this, you store appropriate policies for K8s in OPA. For this purpose, you would use kubectl to create an entry of the ConfigMap type; the file contains the plain vanilla instruction for OPA in Rego. Rego supports you with functions such as fqdn_matches_any or fqdn_matches, which can be used to check hostnames automatically.

The ConfigMap object contains only the logic that makes the policy decision according to the parameters of specified hosts. You do not store the domains themselves there but rely on an annotation directive in the pod definition of the namespace where the target services will run. Admins regularly create separate namespaces for different compliance directives in their clusters to keep things clear. Finally, you specify that the existing OPA instance act as the admission controller for K8s, meaning that Kubernetes outsources compliance decisions to OPA.

The rest is then simple. When you launch instances such as ingress controllers in your namespace, you define their basic parameters in the usual way with serviceName and servicePort. If these contradict the specifications previously made in OPA, K8s refuses to create the corresponding resource and outputs an appropriate justification. To put it casually, compliance control via kube-mgmt is thus the smaller and somewhat simpler solution.

Full Solution

Gatekeeper takes a slightly different approach. The differences here are really in the details – and those in the later parts of the setup. First, you need a running OPA instance, even in a setup with Gatekeeper. However, once Gatekeeper is active in K8s, it extends it to include custom resource definitions (CRDs) that allow OPA to be started automatically (Figure 4); then, you do not have just one OPA instance but as many instances as you and users initiate.

Figure 4: Gatekeeper uses CRDs and its own policy templates in Kubernetes to act as an admission controller and thus determine the fates of resources. © Gatekeeper

Another detail is fundamentally different. When using Gatekeeper, you do not define the compliance rules yourself in OPA. Instead, Gatekeeper comes with a wrapper that you use to import the policy rules as constraint templates, which the user then applies to create constraints. This method proves to be extremely practical in everyday use because it not only lets admins with access to OPA define compliance rules but also normal users with access to K8s. The practical thing is that users are allowed to create as many templates with constraints for OPA as they want and can dynamically apply them to different resources, which is far more flexible than the kube-mgmt solution.

Better Focused on Compliance

Gatekeeper and kube-mgmt have many similarities and few – but tangible – differences. Both approaches produce logfiles that can be collected with centralized logging tools such as the Elasticsearch, Logstash, and Kibana stack, or Loki. Moreover, because OPA itself is at the heart of both approaches, it can be configured in both cases to send logs of its compliance decisions directly to an external server over HTTP. From that server, for example, you could grab the logs and make them part of an audit log that any certification authorities are likely to want to see (Figure 5).

Figure 5: Gatekeeper is somewhat better prepared out of the box to be audited than is kube-mgmt.

Alternatively, you can use Gatekeeper's audit-safe trail feature that also logs compliance decisions and makes them immediately visible in the constraint entries in K8s.