« Previous 1 2
Verifying your configuration
Checkup
Container Validation with dgoss
In many cases you only have access to the physical or virtual servers at the application level. Containers have largely replaced running native applications. Packing the applications in container images and running them containerized makes everything immutable to upgrade and rollback on the fly, with fine-grained control over server resources. This scheme also greatly improves the packing density of applications to optimize the available server resources.
Containers present different kinds of challenges, though, because their true colors only show at runtime. Container images are always expected to be as complete as possible but have the minimum possible footprint to launch and keep containerized applications running dynamically. Acceptance testing becomes very important with containers because software version control and configuration drift are a problem. I myself have witnessed folks creating containers with images without any standards and then suffering major outages because of an unstable life cycle. The good news is that you can use your knowledge of goss to create Acceptance-as-Code gates of good quality for container workloads.
The dgoss
shell wrapper over goss adds some extra functionality [5]. One benefit is that you can use dgoss to create an Acceptance-as-Code gate in your container image's build and release pipelines. The usual way to run dgoss
is to download it and then launch it with your goss test configuration, but I'll use dgoss through a Docker driver image to run it containerized without downloading anything. To begin, I'll create a dgoss driver image with all the necessary artifacts in the working directory where the Dockerfile (Listing 7) and goss.yaml
file (Listing 8) are located:
docker build -f Dockerfile_DGossDriver . -t dgoss0320driver
Listing 7
Dockerfile_DgossDriver
FROM alpine:3.12 LABEL "com.richnusgeeks.vendor"="richnusgeeks" LABEL version="latest" LABEL description="dgoss test driver docker image" ENV GOSS_VERSION 0.3.20 SHELL ["/bin/ash", "-o", "pipefail", "-c"] WORKDIR /tmp RUN apk add --no-cache --virtual=goss-deps ca-certificates curl && apk add --no-cache bash tini && curl -sSLk -o /tmp/docker.tgz "https://download.docker.com/linux/static/stable/x86_64/$(curl -sSkL https://download.docker.com/linux/static/stable/x86_64/|grep '^ *<a'|grep docker|grep -v rootless|awk -F '\"' '{print $2}'|sort -nr|head -1)" && tar zxf docker.tgz && mv docker/docker /usr/local/bin && rm -rf docker docker.tgz && curl -sSLk "https://github.com/aelsabbahy/goss/releases/download/v${GOSS_VERSION}/goss-linux-amd64" -o /usr/local/bin/goss && curl -sSLk "https://github.com/aelsabbahy/goss/releases/download/v${GOSS_VERSION}/dgoss" -o /usr/local/bin/dgoss && chmod +x /usr/local/bin/*goss && mkdir -p /etc/goss && apk del goss-deps WORKDIR /etc/goss/ ENTRYPOINT ["tini", "--"] CMD ["dgoss", "-h"]
Listing 8
goss.yaml
port: tcp:22: listening: true ip: - 0.0.0.0 tcp6:22: listening: true ip: - '::' user: sshd: exists: true uid: 101 gid: 65534 groups: - nogroup home: /run/sshd shell: /usr/sbin/nologin process: sshd: running: true
Now you can see dgoss in action by running the command:
docker run -it --rm -v ${PWD}/:/etc/goss:ro -v /var/run/docker.sock: /var/run/docker.sock:ro -e GOSS_SLEEP=2 -e GOSS_FILES_STRATEGY=cp -e GOSS_OPTS='--color -format tap 'dgoss0320driver dgoss run dgoss0320driver sleep infinity
The failures flashed in the run (Figure 4) are from a quick testing criterion of sshd
running in containers, which is not true in the case of this image. The image is not a continuously running service, and that's why I supplied a sleep
command, so that the launched service container is there during dgoss operations, such as copying goss and goss.yaml
into it and launching a goss test run in the newly launched service container. The dgoss behavior is configurable by a number of environmental variables.
A more standard example of a popular Docker image is the high-performance Apache Spark analytics engine. To begin, pull the official Spark Docker image:
docker pull bitnami/spark:latest
Next, prepare the spark.yaml
goss test configuration file in your current working directory (Listing 9).
Listing 9
spark.yaml
port: tcp:7077: listening: true ip: - 0.0.0.0 tcp6:7077: listening: true ip: - :: tcp:8080: listening: true ip: - 0.0.0.0 tcp6:8080: listening: true ip: - :: process: java: running: true
Now, run the command
docker run -it --rm -v ${PWD}/:/etc/goss:ro -v /var/run/docker.sock: /var/run/docker.sock:ro -e GOSS_FILE=spark.yaml -e GOSS_FILES_STRATEGY=cp -e GOSS_OPTS='--color -format tap 'dgoss0320driver dgoss run -e SPARK_MODE=master bitnami/spark
to launch a test run against the official Spark image. It looks a bit surprising that, although the Spark Java service is running, the master and user interface port tests are failing (Figure 5).
I'll try once again by executing the command:
docker run -it --rm -v ${PWD}/:/etc/goss:ro -v /var/run/docker.sock:/var/run/docker.sock:ro -e GOSS_SLEEP=2 -e GOSS_FILE=spark.yaml -e GOSS_FILES_STRATEGY=cp -e GOSS_OPTS='--color -format tap' dgoss0320driver dgoss run -e SPARK_MODE=master bitnami/spark
Now things look better. If you have a Dockerized service that has startup delays to run and respond (e.g., Java, Ruby, etc.), then GOSS_SLEEP
helps to delay before launching a goss run in the spawned service container. You should see all the IPv4 port tests passing now (Figure 6).
Another important dgoss feature is useful when you have a set of precondition tests. If you create the goss_wait.yaml
file (Listing 10) in your working directory and execute the docker
command shown in the previous section, it should wait for the tests in the goss_wait.yaml
file to be passed. In this case, I ensured that no SSH functionality was part of the Spark image (Figure 7).
Listing 10
goss_wait.yaml
port: tcp:22: listening: false ip: - 0.0.0.0 user: sshd: exists: false process: sshd: running: false
Try to change a test in the goss_wait.yaml
to true
, and you should see dgoss abort the test run because of the test failure. The dgoss run after setting the sshd
process running state to true
in goss_wait.yaml
is shown in Figure 8.
Container Stack Validation with dcgoss
Docker Compose is a popular tool that lets you define and run multicontainer applications. Another wrapper created over dgoss tests the containers of the entire Compose stack: Put the code in Listing 11 in your current working directory and run the command
docker build -f Dockerfile_DCGossDriver . -t dcgoss0320driver
to build the driver image.
Listing 11
Dockerfile_DCGossDriver
FROM alpine:3.16 LABEL "com.richnusgeeks.vendor"="richnusgeeks" LABEL version="latest" LABEL description="dcgoss test driver docker image" ENV DCPS_VERSION 2.12.2 ENV GOSS_VERSION 0.3.20 SHELL ["/bin/ash", "-o", "pipefail", "-c"] WORKDIR /tmp RUN apk add --no-cache --virtual=goss-deps ca-certificates curl && apk add --no-cache bash tini && curl -sSLk -o /tmp/docker.tgz "https://download.docker.com/linux/static/stable/x86_64/$(curl -sSkL https://download.docker.com/linux/static/stable/x86_64/|grep '^ *<a'|grep docker|grep -v rootless|awk -F '\"' '{print $2}'|sort -nr|head -1)" && tar zxf docker.tgz && mv docker/docker /usr/local/bin && rm -rf docker docker.tgz && curl -sSLk "https://github.com/docker/compose/releases/download/v${DCPS_VERSION}/docker-compose-linux-x86_64" -o /usr/local/bin/docker-compose && chmod +x /usr/local/bin/docker-compose && curl -sSLk "https://github.com/aelsabbahy/goss/releases/download/v${GOSS_VERSION}/goss-linux-amd64" -o /usr/local/bin/goss && curl -sSLk "https://raw.githubusercontent.com/aelsabbahy/goss/master/extras/dcgoss/dcgoss" -o /usr/local/bin/dcgoss && chmod +x /usr/local/bin/*goss && mkdir -p /etc/goss && apk del goss-deps WORKDIR /etc/goss/ ENTRYPOINT ["/sbin/tini", "--"] CMD ["dcgoss", "-h"]
Next, create the docker-compose.yml
file shown in Listing 12 and the kafka.yaml
file shown in Listing 13 to bring up the ZooKeeper plus Kafka container stack over which the goss tests are run.
Listing 12
docker-compose.yml
version: "3" services: zookeeper: image: 'bitnami/zookeeper:latest' ports: - '2181:2181' environment: - ALLOW_ANONYMOUS_LOGIN=yes kafka: image: 'bitnami/kafka:latest' ports: - '9092:9092' environment: - KAFKA_BROKER_ID=1 - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092 - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092 - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181 - ALLOW_PLAINTEXT_LISTENER=yes depends_on: - zookeeper
Listing 13
kafka.yaml
port: tcp:2181: listening: true ip: - 0.0.0.0 tcp:9092: listening: true ip: - 0.0.0.0 process: java: running: true
Now you can fire dcgoss
to acceptance-test each service container,
docker run -it --rm -v ${PWD}/:/etc/goss:ro -v /var/run/docker.sock: /var/run/docker.sock:ro -e GOSS_FILE=kafka.yaml -e GOSS_SLEEP=5 -e GOSS_FILES_STRATEGY=cp -e GOSS_OPTS='--color -format tap' dcgoss0320driver dcgoss run zookeeper
and run all goss tests in the ZooKeeper service container. You should see that all the ZooKeeper tests pass (Figure 9).
To run an acceptance test for the Kafka container, use the command:
docker run -it --rm -v ${PWD}/:/etc/goss:ro -v /var/run/docker.sock:/var/run/docker.sock:ro -e GOSS_FILE=kafka.yaml -e GOSS_SLEEP=5 -e GOSS_FILES_STRATEGY=cp -e GOSS_OPTS='--color-format tap 'dcgoss0320driver dcgoss run kafka
You should see all the Kafka tests passing now (Figure 10).
I have combined the ZooKeeper and Kafka goss tests in a common file for quick testing, showing ok
, not ok
, and skip
test results, but you could separate the tests into their respective gossfiles and mention the corresponding file with the GOSS_FILE
environment variable. Finally, to clean up the running service containers, use the command
docker ps -q|xargs -I % docker rm -f %
to say goodbye to dcgoss.
Last but not the least, Kubernetes has become a de facto platform to run and manage containerized services at scale, so the goss project also provides another wrapper in the form of kgoss
to acceptance-test containers running in Kubernetes pods. You need to run kgoss
on a machine with kubectl
already installed and configured to talk to your local or remote Kubernetes cluster. The kgoss documentation [6] clearly describes the installation and usage.
Conclusion
Server misconfigurations can create a lot of havoc, resulting in big losses. Goss is an elegant modern solution to bake acceptance testing in your servers. The validation tool is highly dynamic, supporting templating and serving reports on an HTTP endpoint. The icing on the cake is its added value when integrated with modern cloud tools like Ansible, Molecule, Kitchen, and Packer. The goss wrappers created for services running in containers are very useful for quality gating in modern build-and-release pipelines.
Infos
- goss: https://github.com/goss-org/goss
- goss documentation: https://github.com/aelsabbahy/goss/blob/master/docs/manual.md
- goss resources and usage: https://github.com/aelsabbahy/goss/blob/master/docs/manual.md#available-tests
- goss add-ons from the community: https://github.com/aelsabbahy/goss#community-contributions
- dgoss documentation: https://github.com/aelsabbahy/goss/tree/master/extras/dgoss
- kgoss documentation: https://github.com/aelsabbahy/goss/tree/master/extras/kgoss @IE"
« Previous 1 2
Buy this article as PDF
(incl. VAT)