Avoiding Pitfalls

Certificate or Token

Users can log on with an X.509 certificate signed by the Certificate Authority (CA) in the Kubernetes cluster. Kubernetes takes the username and group memberships from the CommonName of the certificate; the line

CN=hvapour/o=group1/o=group2

logs on a user named hvapour with the group memberships group1 and group2 . In principle, this method also runs internally in Kubernetes, because you have to generate the certificate requests accordingly and sign them with the Kubernetes CA.

Moreover, it is possible to use a token file in the format:

<Token>,<Username>,<UserID>,"Group1,Group2"

To do this, however, you need to reconfigure the Kubernetes API server, completing the call parameters for kube-apiserver with the --token-auth-file=<Path_to_file> option. However, because the API server also runs in the container as part of Kubernetes, adding the option is not trivial.

The call parameters of the Kubernetes components are in the /etc/kubernetes/manifests/ directory. The kube-apiserver.yaml file (Figure 2) configures the corresponding container and contains all the call parameters as a YAML list. Enter the following line:

--token-auth-file=<path to tokenfile>

Figure 2: Sample configuration for kube-apiserver in /etc/kubernetes/manifests/.

and drop the token file into a folder that is accessible through the container.

Instead of token files, you can use a password file in the same way. Its syntax is similar, except that the first line contains the password. Instead of --token-auth-file=<Path_to_file>, you add the --basic-auth-file=<Path_to_file> option. The API server then uses the HTTP basic auth method, and clients have to use it to send their requests.

Both variants come at the price of the cluster admin only being able to change the passwords and tokens in the file. They also require a restart of the kube-apiserver container. You can trigger this either with a kubelet on the master node or by restarting the pod with kubectl.

OpenID Preferred

OpenID is a variant of OAuth 2.0 [3] that Microsoft and Google offer in their public clouds. The service distributes tokens that prove to be more up to date and which Kubernetes accepts, assuming you have done the prep work. If so, cloud providers then handle the authentication.

Another external method is webhooks. Kubernetes sends a JSON request with a token to an external service and waits for a response, which must contain the username, group membership(s), and extra fields, if any, in the JSON block.

Finally, the path between the user and Kubernetes can contain an authentication proxy that extends the web requests to include X-headers, for example:

X-Remote-User: jack
X-Remote-Group: Dev1

Once the function is activated, Kubernetes accepts the data in this way. To prevent an attacker from spoofing this process, the proxy uses a client certificate to identify itself to Kubernetes.

Service Accounts

The simplest variant of an authentication method is to create a service account. For example, the first command in Listing 2 creates a new service account named testserviceaccount in the default namespace.

$ kubectl create serviceaccount testserviceaccount -n default
$ kubectl describe secret testserviceaccount-token-xnsvx
Name:         testserviceaccount-token-xnsvx
Namespace:    default
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: testserviceaccount
              kubernetes.io/service-account.uid: b45a08c2-d385-412b-9505-6bdf87b64a7a
Type:  kubernetes.io/service-account-token
Data
====
ca.crt:    1025 bytes
namespace: 7 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6InNCRjdsTmJHeGVoUTdGR0ZTemsyalpCaGhqclhoQjVBLXdpZnVxdVNDbXcifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6InRlc3RzZXJ2aWNlYWNjb3VudC10b2tlbi1ic214YiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJ0ZXN0c2VydmljZWFjY291bnQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJiNDVhMDhjMi1kMzg1LTQxMmItOTUwNS02YmRmODdiNjRhN2EiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGVmYXVsdDp0ZXN0c2VydmljZWFjY291bnQifQ.SO9XwM3zgiW6sOfEaJx1P6m1lFdsDj9IgBCogkKgg-b8pYm4wErq9a6ZSyR6Z5E3sdG4Djkc4Gnfs4oxBncl3IpDQSNVjL6ahtPMtlrKq3ssbzvlbDSsoyJDp568O2RzuaqSP_Zy8Kmm9ddaBKUQ46DUfvLw-7MVgUf-_IY8vuaAXCCBvgeJCIpJf0_IWuBQ2uWh-0JtEZwu7OCrVO2B51bvTuXbB7lbYZvnpIJT8umBSjvoA0G78OnawY0tHtycAXedCx8UuY7DET3UQio-31vNOQQ0wKBVUlCw-9ASKX112ZKvbkzbp1ZP8MG-BgMoWJoqFdrErv2Nvo1pN85vTA

Although this account has no permissions yet, the command creates a secret in addition to the account. The output of kubectl get secret now contains an entry (Figure 3); the secret for the newly created testserviceaccount is shown.

Figure 3: With kubectl (the Microk8s variant is shown here), you can create a service account and query the secret.

Now take a closer look at the output of the second command in Listing 2. The entry below token: is for authentication. If the Kubernetes dashboard is in use, you can copy this jumble of characters and use it as a token to log in. However, the user will still not see much in the dashboard because all of the permissions are missing. You will want to add the lines

[...]
users:
  - name: testserviceaccount
    user:
      token: eyJh[...]
[...]

to ~/.kube/config in the section below the users: label – with the complete token, of course.