Tips and Tricks for Containers

Inspecting a Docker Container

Containers are relatively new in computing, so when you pull or download container images, it might be a good idea to inspect them as best you can before using them to create a container. Moreover, inspecting a container to learn something from it that you can use in your own containers is a great way to move forward.

Docker has a couple of commands that can be useful in inspecting or learning about the container. The first straightforward command is docker inspect <image> (note that the output has been abbreviated):

$ docker inspect nvidia/cuda:10.1-base-ubuntu18.04
[
    {
        "Id": "sha256:3b55548ae91f1928ae7315b9fe43b3ffa097a3da68f4be86d3481e857241acbb",
        "RepoTags": [
            "nvidia/cuda:10.1-base-ubuntu18.04"
        ],
        "RepoDigests": [
            "nvidia/cuda@sha256:3cb86d1437161ef6998c4a681f2ca4150368946cc8e09c5e5178e3598110539f"
        ],
        "Parent": "",
        "Comment": "",
        "Created": "2019-11-27T20:00:08.137590731Z",
        "Container": "f8cdd4d69d0b5123a712b66cd12a46799daff6e23896e73c6bfd247a981daa71",
        "ContainerConfig": {
            "Hostname": "f8cdd4d69d0b",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
 
...
 
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:cc967c529ced563b7746b663d98248bc571afdb3c012019d7f54d6c092793b8b",
                "sha256:2c6ac8e5063e35e91ab79dfb7330c6154b82f3a7e4724fb1b4475c0a95dfdd33",
                "sha256:6c01b5a53aac53c66f02ea711295c7586061cbe083b110d54dafbeb6cf7636bf",
                "sha256:e0b3afb09dc386786d49d6443bdfb20bc74d77dcf68e152db7e5bb36b1cca638",
                "sha256:37b9a4b2218692d028f9f26aa9cb85bf1f56d9abe612ba31304643bdb448484f",
                "sha256:b16af11cbf2977eb52ba4d6cee5b713721cc19812b8c90ea1f22e7e7641301fa",
                "sha256:808fd332a58a1cc1ecda89295c2d9ef8e594674e476bc5eb25e99374515a1c7d"
            ]
        },
        "Metadata": {
            "LastTagTime": "0001-01-01T00:00:00Z"
        }
    }
]

A second option useful in inspecting a docker image is docker history :

$ docker history nvidia/cuda:10.1-base-ubuntu18.04
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
3b55548ae91f        5 months ago        /bin/sh -c #(nop)  ENV NVIDIA_REQUIRE_CUDA=c...   0B
           5 months ago        /bin/sh -c #(nop)  ENV NVIDIA_DRIVER_CAPABIL...   0B
           5 months ago        /bin/sh -c #(nop)  ENV NVIDIA_VISIBLE_DEVICE...   0B
           5 months ago        /bin/sh -c #(nop)  ENV LD_LIBRARY_PATH=/usr/...   0B
           5 months ago        /bin/sh -c #(nop)  ENV PATH=/usr/local/nvidi...   0B
           5 months ago        /bin/sh -c echo "/usr/local/nvidia/lib" >> /...   46B
           5 months ago        /bin/sh -c apt-get update && apt-get install...   25.1MB
           5 months ago        /bin/sh -c #(nop)  ENV CUDA_PKG_VERSION=10-1...   0B
           5 months ago        /bin/sh -c #(nop)  ENV CUDA_VERSION=10.1.243    0B
           5 months ago        /bin/sh -c apt-get update && apt-get install...   16.5MB
           5 months ago        /bin/sh -c #(nop)  LABEL maintainer=NVIDIA C...   0B
           6 months ago        /bin/sh -c #(nop)  CMD ["/bin/bash"]            0B
           6 months ago        /bin/sh -c mkdir -p /run/systemd && echo 'do...   7B
           6 months ago        /bin/sh -c set -xe   && echo '#!/bin/sh' > /...   745B
           6 months ago        /bin/sh -c [ -z "$(apt-get indextargets)" ]     987kB
           6 months ago        /bin/sh -c #(nop) ADD file:a48a5dc1b9dbfc632...   63.2MB

By using the docker history command, you can almost reverse engineer a Dockerfile from an existing container:

$ docker history --format "{{.CreatedBy}}" --no-trunc nvidia/cuda:10.1-base-ubuntu18.04 | tac
/bin/sh -c #(nop) ADD file:a48a5dc1b9dbfc632f6cf86fe27b770b63f07a115c98c4465dc184e303a4efa1 in /
/bin/sh -c [ -z "$(apt-get indextargets)" ]
...
/bin/sh -c #(nop)  ENV CUDA_VERSION=10.1.243
/bin/sh -c #(nop)  ENV CUDA_PKG_VERSION=10-1=10.1.243-1
/bin/sh -c apt-get update && apt-get install -y --no-install-recommends         cuda-cudart-$CUDA_PKG_VERSION cuda-compat-10-1 && ln -s cuda-10.1 /usr/local/cuda &&     rm -rf /var/lib/apt/lists/*
/bin/sh -c echo "/usr/local/nvidia/lib" >> /etc/ld.so.conf.d/nvidia.conf &&     echo "/usr/local/nvidia/lib64" >> /etc/ld.so.conf.d/nvidia.conf
/bin/sh -c #(nop)  ENV PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
/bin/sh -c #(nop)  ENV LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64
/bin/sh -c #(nop)  ENV NVIDIA_VISIBLE_DEVICES=all
/bin/sh -c #(nop)  ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
/bin/sh -c #(nop)  ENV NVIDIA_REQUIRE_CUDA=cuda>=10.1 brand=tesla,driver>=384,driver<385 brand=tesla,driver>=396,driver<397 brand=tesla,driver>=410,driver<411

Inspecting a Singularity Container

As previously mentioned, you can inspect a Singularity image for the definition file. The same command with other options extracts more information. For example:

$ singularity inspect -l -r -d -e -t cuda_10_1-base-ubuntu18_04.simg
WARNING: No SIF metadata partition, searching in container...
bootstrap: docker
from: nvidia/cuda:10.1-base-ubuntu18.04
 
 
#!/bin/sh
OCI_ENTRYPOINT=''
OCI_CMD='"/bin/bash"'
CMDLINE_ARGS=""
# prepare command line arguments for evaluation
for arg in "$@"; do
    CMDLINE_ARGS="${CMDLINE_ARGS} \"$arg\""
done
 
 
# ENTRYPOINT only - run entrypoint plus args
if [ -z "$OCI_CMD" ] && [ -n "$OCI_ENTRYPOINT" ]; then
    if [ $# -gt 0 ]; then
        SINGULARITY_OCI_RUN="${OCI_ENTRYPOINT} ${CMDLINE_ARGS}"
    else
        SINGULARITY_OCI_RUN="${OCI_ENTRYPOINT}"
    fi
fi
 
# CMD only - run CMD or override with args
if [ -n "$OCI_CMD" ] && [ -z "$OCI_ENTRYPOINT" ]; then
    if [ $# -gt 0 ]; then
        SINGULARITY_OCI_RUN="${CMDLINE_ARGS}"
    else
        SINGULARITY_OCI_RUN="${OCI_CMD}"
    fi
fi
 
# ENTRYPOINT and CMD - run ENTRYPOINT with CMD as default args
# override with user provided args
if [ $# -gt 0 ]; then
    SINGULARITY_OCI_RUN="${OCI_ENTRYPOINT} ${CMDLINE_ARGS}"
else
    SINGULARITY_OCI_RUN="${OCI_ENTRYPOINT} ${OCI_CMD}"
fi
 
# Evaluate shell expressions first and set arguments accordingly,
# then execute final command as first container process
eval "set ${SINGULARITY_OCI_RUN}"
exec "$@"
 
 
#!/bin/sh
# Custom environment shell code should follow
 
 
org.label-schema.build-date: Saturday_2_May_2020_10:52:41_EDT
org.label-schema.schema-version: 1.0
org.label-schema.usage.singularity.deffile.bootstrap: docker
org.label-schema.usage.singularity.deffile.from: nvidia/cuda:10.1-base-ubuntu18.04
org.label-schema.usage.singularity.version: 3.5.3

The various options applied to the original Singularity image (the one without Octave) are:

• -d : show the image definition file
• -e : show the environment settings for the image
• -l : show the labels for the image
• -t : show the test script for the image