Handmade

The Self-Signed Certificate Palaver

After investigating various online fixes, I was eventually able to connect the proprietary Jenkins plugin I was using to an online service even though a certificate authority didn't recognize its self-signed certificate. The solution came about through distilling some of the instructions found on Stack Overflow [3]. The salient bits are as follows:

• Enter the container as the root user (just ensure the --user root \ entry is on a line near the top of Listing 1) and run a keytool command to add your online service's certificate to the cacerts file. Next, copy that certificate to your persistent volume.
• With the use of an environment variable when the container is spawned, ensure that Jenkins uses the correct cacerts file on your volume to prevent rebuilds and reboots from failing in the future.

The following workflow shows the simple steps needed to add a recognized self-signed certificate to Jenkins.

Step 1

Enter the Jenkins container by running an exec command to access the filesystem of the persistent volume created. Find the hash ID of the Jenkins container – in case you didn't name it with the -name option in Listing 1. Use the hash ID to enter the container to run the desired commands:

$ hash=$(docker ps | grep jenkins | awk '{print $1}')
bf66cc1b2916
$ docker exec -it ${hash} bash
root@bf66cc1b2916:/#

The first command will only work if one instance of Jenkins is running; in in the unlikely event you are running two Jenkins instances, edit the command accordingly.

Now you should be able to enter your Jenkins container. You have a root prompt inside your container because you left the --user root \ line in Listing 1. You need to be the root user to run the keytool command, but you should remove that line later for much more security. The container should run as the jenkins user by default. Line 47 in an online example that shows the Dockerfile used to create the LTS image should convince you [4].

Step 2

Visit the website that you want Jenkins to trust (e.g., with Google Chrome on Linux, Figure 2 – or you can use openssl) and save the certificate to a location by accessing the website and clicking on the address bar padlock to download the certificate file locally. The file you are after usually will end in .pem – or .cer on Windows.

Figure 2: Click the padlock to view certificate information.

Once you have clicked the Certificate field highlighted in Figure 2, click the Details tab at the top of the next dialog box and then Export at the bottom (Figure 3).

Figure 3: Export the certificate.

Once Export has been clicked, you can follow the prompts to save the certificate locally. On Linux Mint, I just pressed Save As ; by default, cloudnativesecurity.cc is the suggested name because that's the website I visited in my browser, so I adjusted the name slightly to cloudnativesecurity.pem. If you open your PEM file, you can see it's a standard certificate file that will look something like:

$ cat cloudnativesecurity.cc
-----BEGIN CERTIFICATE-----
MIIF1zCCBL+gAwIBAgIRAK7AdUDa5C4Y1o6SSOX4aC0wDQYJKoZIhvcNAQEL
...

Step 3

Now that you have the certificate you want to trust saved locally, copy it with a Docker cp command to the /tmp directory in the Jenkins volume, so you can later move it to a more relevant location:

$ docker cp cloudnativesecurity.pem ${hash}:/tmp/cloudnativesecurity.pem
$ docker exec -it ${hash} ls /tmp

Although a relatively simple step, take care with the syntax of this first command, which reuses the hash variable and the filename chosen earlier for the .pem file. The last command checks to make sure the file made it to its new location, as hoped. (On my version of Docker, the cp command does not seem to show errors if it fails to copy.) If you see your file in the listing, it worked and you can proceed.

Step 4

Now that you have copied the file into your container, you can use your trusty keytool command to add it to your trusted certificates by adding the certificate to the sites that Jenkins trusts. This excellent command can be used to import certificates into the cacerts file that Jenkins uses in its keystore, and with a single command you will be asked to confirm whether to trust the self-signed certificate that you have just imported.

Before you do that (as I discovered after much reading), the Java virtual machine paths seem to differ in some Jenkins versions, so you need to know precisely where the file resides. For this step, you need to be inside the container (as per Step 1). Incidentally, as noted, I am entering this container as the root user on purpose, which is required to run keytool. Use the --user root \ option when you are performing these steps, before you complete the process and need to switch from the jenkins user (just delete that --user root \ line in Listing 1 to do so). Now, check where the existing cacerts file is from inside the container:

root@bf66cc1b2916:/# find / -name cacerts
/usr/local/openjdk-8/jre/lib/security/cacerts

Found! The next step is updating the local cacerts file and then copying it into your persistent volume, having made it readable by the jenkins user. The keytool command refers to the /tmp path from earlier (Listing 2).

root@bf66cc1b2916:/# keytool -import -alias CNS-cert -keystore /usr/local/openjdk-8/jre/lib/security/cacerts -file /tmp/cloudnativesecurity.pem
...
#9: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: E2 B9 A7 59 F6 11 B4 00   3B 76 56 1F 29 5D CF 91  ...Y....;vV.)]..
0010: EA AB 17 F6                                        ....
]
]
Trust this certificate? [no]:

After you enter your password (the default appears to be – all lowercase – changeit ), look at the end of the output where it asks whether you want to trust the certificate. At this point, you can type yes to proceed. You are then offered the confirmation Certificate was added to keystore .

Now that you have updated the cacerts file in your container (so that it survives reboots) and the container receiving the docker rm command, copy it to your persistent volume and make sure the jenkins user can read from it correctly:

$ mkdir /var/jenkins_home/keystore
$ cp /usr/local/openjdk-8/jre/lib/security/cacerts /var/jenkins_home/keystore/cacerts
$ chown /var/jenkins_home/keystore/cacerts

Step 5

Almost finished! You just need to ensure at startup that Jenkins is looking in the correct place for the cacerts file to which you have just added your certificate. Add an environment variable that points at your persistent volume path, which the container will use when spawned, by adding the line

--env "JENKINS_OPT=-Djavax.net.ssl.trustStore=/var/jenkins_home/keystore/cacerts"

to Listing 1. This step ensures that Jenkins is using the correct cacerts file.

Step 6

Restart Jenkins and ensure that changes persist. Because the data is now safe on the persistent volume, stop and delete the older Jenkins container with the commands

$ docker stop $(hash}
$ docker rm $(hash}

Having made the appropriate changes to the Jenkins start-up script (removing the --user root line) and ensuring the addition of the environment variable for the path, you can now execute your run_jenkins.sh script again to start up your container once more.

Once you have patiently waited a couple of minutes for the UI to start up, any plugins that interact with https://cloudnativesecurity.cc should do so without generating any errors. Rinse and repeat for other online services to which you need to connect.

The End

Having also tried the Skip Certificate Check plugin [5] for Jenkins updates, I was relieved that adjusting the cacerts file worked. I tested a number of apparent fixes, such as adding a /etc/default/jenkins.xml file.

The benefits of getting this setup working is that, first, your Jenkins instance is able to confirm that it is connecting to the correct online service (and that no imposters are involved). Second, your traffic is transmitted in an encrypted format. I trust this will help you, too, at some point in the future when using Jenkins.