Update your Docker containers safely

Safe Harbor

Local Docker Script

The script to get Docker file IDs on a Docker host is not much of a script, really; just copy and paste the code in Listing 8 in a terminal to produce the ImageId-file output file, which has two lines per image, as shown in Listing 9. The first line holds the repo ID, the second the image ID. Each line contains the image name, the tag name, and either the image ID or repo ID. If the last word of the line is RepoId , the line holds the repo ID, making it easy to grep lines with the right kind of ID.

Listing 9

ImageId-file List

nginx 1.15-alpine nginx@sha256:385fbcf0f04621981df6c6f1abd896101eb61a439746ee2921b26abc78f45571 RepoId
nginx 1.15-alpine 315798907716a51610bb3c270c191e0e61112b19aae9a3bb0c2a60c53d074750
nextcloud latest nextcloud@sha256:78515af937fe6c6d0213103197e09d88bbf9ded117b9877db59e8d70dbdae6b2 RepoId
nextcloud latest 8757ce9de782c2dd746a1dd702178b8309ca6d2feb5e84bad9184441170d4898
mariadb latest mariadb@sha256:12e32f8d1e8958cd076660bc22d19aa74f2da63f286e100fb58d41b740c57006 RepoId
mariadb latest b468922dbbd73bdc874c751778f1ec0ec10817691624976865cb3ec5c70cd4e0
mvance/unbound 1.8.3 mvance/unbound@sha256:d67469fad9cc965f032e4ea736c509df6d009245dac81339e2c6e1caef9b65ac RepoId
mvance/unbound 1.8.3 mvance/unbound latest a88e44773675294dcd10e08a9a2a5ee2a39796e5a832c99606b3c8a54901ea75

Listing 8

Get Docker File IDs

> ImageId-file
for i in `docker images -f dangling=false| egrep -v TAG | awk '{print $3}'` ; do
  echo copying name, tag and image ID digest to file
  ImageId=`docker image inspect $i | jq -r '.[0] | {Id: .Id}' | egrep Id | awk -F":" '{print $3'} | awk -F"\"" '{print $1}'`
  RepoId=`docker image inspect $i | jq -r '.[0] | {RepId: .RepoDigests}' | jq --raw-output  '.RepId | .[]'`
  ImageData=`docker images | egrep $i | awk '{print $1," " , $2}'`
  echo $ImageData $ImageId
  echo $ImageData $RepoId RepoId >> ImageId-file
  echo $ImageData $ImageId >> ImageId-file
done

You should check ImageId-file, because it might contain lines about images you do not use anymore: The script will produce lines on all images used in containers, even if you no longer run those containers or have not yet run them. Make sure to discard any "old" containers.

A Central Docker Host Check

If you run the script in Listing 8 on a Docker host, it will collect ID lines on all containers on that host. If you have multiple Docker hosts, you can run the script on multiple hosts and combine the lines into one big file on a central host; of course, you could automate this process and run checks on a central Docker host for all the containers you are using, run checks for updates for your <image><tag> combinations, and download the images and check them against known vulnerabilities.

With the repo ID lines collected, all images can be downloaded on the central host, because the repo ID can be used to pull images with docker pull <repo ID>; for instance,

docker pull mariadb@sha256:12e32f8d1e8958cd076660bc22d19aa74f2da63f286e100fb58d41b740c57006

will download that specific MariaDB image from Docker Hub. Therefore, you can download many images to a central host without actually running them in a container with,

for image in `cat ImageId-file | egrep RepoId | awk '{print $3}'` ; do
   echo $image ;
   docker pull $image
done

and show them by entering docker images.

Once you have your images on one host, you will find multiple solutions to check your images against known vulnerabilities. Examples are Anchore, Clair, or Dagda, which all run on Docker, so they are relatively easy to set up [4].

In the next section, however, I show you how to check for container updates – without the need to download the images, by the way.

Docker Updates Script

At this point you have the image ID file, which knows what Docker <image><tag> pairs you have, and the IDs of these images as used in production. Now, you need to see whether these images can be updated.

The first part of the update check script downloads a manifest file from Docker Hub for a specified Docker <image><tag> using the Registry API. The second part of the script does two things: Loops over all image lines, downloading a manifest file for each, and checks the local image ID against the ID from the downloaded manifest. If they differ, an update is available.

Listing 10 shows the first part of the script, which fetches a digest from Docker Hub. As it states (if you run it without arguments), you should feed it two arguments:

./get-docker-hub-digest library/nginx 1.15-alpine

Listing 10

get-docker-hub-digest

01 #!/bin/bash
02
03 # Retrieves image digest from public
04 # images in DockerHub
05 # modified from https://gist.github.com/cirocosta/17ea17be7ac11594cb0f290b0a3ac0d1x
06
07 set -o errexit
08
09 main() {
10   check_args "$@"
11
12   local image=$1
13   local tag=$2
14   local token=$(get_token $image)
15   local digest=$(get_digest $image $tag $token)
16
17 echo " $digest"
18
19 }
20
21 get_token() {
22   local image=$1
23
24   echo "Retrieving Docker Hub token.
25     IMAGE: $image
26   " >&2
27
28   curl --silent "https://auth.docker.io/token?scope=repository:$image:pull&service=registry.docker.io" | jq -r '.token'
29 }
30
31 # Retrieve the digest, now specifying in the header
32 # that we have a token (so we can ...
33 get_digest() {
34   local image=$1
35   local tag=$2
36   local token=$3
37
38   echo "Retrieving image digest.
39     IMAGE:  $image
40     TAG:    $tag
41   " >&2
42 #    TOKEN:  $token # add to echo for debug
43
44   curl --silent --header "Accept: application/vnd.docker.distribution.manifest.v2+json" --header "Authorization: Bearer $token" "https://registry-1.docker.io/v2/$image/manifests/$tag"
45 #    | jq -r '.config.digest'
46 }
47
48 check_args() {
49   if (($# != 2)); then
50     echo "Error:
51     Two arguments must be provided - $# provided.
52
53     Usage:
54       ./get-docker-hub-digest.sh <image> <tag>
55      for instance ./get-docker-hub-digest library/nginx 1.15-alpine
56
57 Aborting."
58     exit 1
59   fi
60 }
61 main "$@"

The script spits out a JSON file with a series of digests, of which the first is the image ID (not the repo ID, for some reason).

The second part of the check script (Listing 11) you will actually use. It checks for updates on Docker images and uses the script in Listing 10 to get digests. You can feed it the image ID file produced in Listing 10 (without the RepoId lines) with:

cat ImageId-file | egrep -v "RepoId|none|^$" | ./check-docker-image-updates.sh

Listing 11

check-docker-image-updates.sh

001 #!/bin/bash
002
003 # Checks Docker Hub for updates on <image><tag><ImageId>
004 # ImageId without sha256: part
005 # script accepts <image> <tag> <ImageId> or sdtin
006
007 # reads file from standard in: lines in the form of
008 # nginx 1.14-alpine #315798907716a51610bb3c270c191e0e61112b19aae9a3bb0c2a60c53d074750
009 # mvance/unbound latest 4568745687569875689745689756
010 #
011 # Script calls other script ./getdigest. Can be found at #https://github.com/hanscees/dockerscripts
012
013 set -o errexit
014 > UpdateTheseImages
015 main() {
016
017 REPOS=""
018 TAG=""
019 ID=""
020
021 if [ -t 0 ] ; then
022   echo terminal input; #and no stdin
023   check_args "$@"  #if input not from stdin
024   REPOS=$1
025   TAG=$2
026   ID=$3
027   report_result $REPOS $TAG $ID
028 else
029   echo "not a terminal, so reading stdin";
030   while read line;  do
031   declare -a ImageData=($line) #bash array
032   REPOS=${ImageData[0]}
033   TAG=${ImageData[1]}
034   ID=${ImageData[2]}
035   report_result $REPOS $TAG $ID
036   done
037 fi
038 }  #end main
039
040 report_result () {
041   REPOS=$1
042   TAG=$2
043   ID=$3
044   echo "repo/image and tag are "
045   echo $REPOS $TAG
046
047   check_result=$(check_for_updates $REPOS $TAG $ID)
048   echo check_result is $check_result
049
050   if [ $check_result ]
051   then
052    echo $REPOS $TAG can be updated: a new version is available
053    echo also check file UpdateTheseImages
054   else
055    echo NO update found for $REPOS $TAG
056   fi
057   }
058
059 check_for_updates () {
060   REPOS=$1
061   TAG=$2
062   ID=$3
063   local myresult=0 #default return value
064   timestamp=`date --rfc-3339=seconds`
065   echo $timestamp >> debug
066   echo $REPOS $TAG >> debug
067   blub=`echo $REPOS | egrep "\/"`
068   if [ ! "$blub" ] ; then REPOS=library/$REPOS ;fi  #add library/ before repo if needed
069   DockerIdNew=`./get-docker-hub-image-tag-digest.sh $REPOS $TAG | jq -r '.config.digest' | awk -F':' '{print $2}' `
070   echo DockeridNew is $DockerIdNew >> debug
071   echo DockerIdLocal is $ID >>debug
072   if [ ! $DockerIdNew ] ; then myresult="" ;echo $myresult ; exit ;fi    #if empty image was probably built locally
073
074   if [ "$DockerIdNew" == "$ID" ]
075   then
076     #echo "you r good, no updates for $REPOS:$TAG"
077     myresult=""
078     echo $myresult
079   else
080     #echo "update available for $REPOS:$TAG"
081     echo "update available for $REPOS:$TAG" >> UpdateTheseImages
082     echo "update available for $REPOS:$TAG" >> debug
083     myresult=1
084     echo $myresult
085   fi
086   }
087
088 check_args() {
089   if (($# != 3)); then
090     echo "Error:
091     Three argument must be provided - $# provided.
092
093     Usage:
094       ./check-docker-image-updates.sh <image> <tag> <imageID>
095       for instance ./check-docker-image-updates.sh library/mariadb latest 2345623745234753647
096       imageID is local digest image without sha256: part
097 Aborting."
098     exit 1
099   fi
100 }
101
102 main "$@"

The output will be:

nginx alpine can be updated: a new version is available
also check file UpdateTheseImages

Two files are produced: UpdateTheseImages, which holds lines on images that need updating, and debug, which holds logging on found IDs. Use cat to see the content of UpdateTheseImages:

cat UpdateTheseImages
update available for library/nextcloud:latest
update available for library/mariadb:latest
update available for mvance/unbound:1.8.3

The GitHub repository also has a Python email script, should you want to email update messages to a local SMTP server.

« Previous 1 2 3 4 Next »