FAQ | This is a LIVE service | Changelog

Skip to content
Snippets Groups Projects
Roy Harrington's avatar
Roy Harrington authored
feat: add check for published docker images

Closes #102

See merge request !156
72c41503
History

DevOps docker images

build status

This repository contains configuration to build the UIS DevOps docker images. The images are built nightly/weekly via GitLabCI and pushed to this repository's Registry.

Images

The following image types are built. See the README files in each image's directory for more details.

Configuration

The GitLabCI configuration defines the images to be built. The jobs are triggered by a weekly or nightly scheduled pipeline and images are pushed to the Registry.

Dynamic pipeline configuration

The .gitlab-ci.yml pipeline file contains a dynamic-pipeline-config job which runs dynamic pipeline generation scripts. These scripts dynamically add build jobs to the pipeline for multiple versions of Terraform.

We have chosen to implement some builds using a dynamic pipeline for some images for the following reasons:

  • We can continue to use the same base .build_and_push job from the build-and-push.gitlab-ci.yml template. This ensures that all our images are built consistently, using the same process.
  • Build jobs can run in parallel. This ensures that single jobs can fail without, necessarily, causing the other jobs to fail. It also ensures that the pipeline completes as quickly as possible.
  • Build jobs automatically pick up new minor and patch releases.

For terraform images, we use the Hashicorp Releases API to query all available Terraform versions and then add jobs for each minor version, e.g. logan-terraform-1.6, logan-terraform-1.5 etc. This ensures that we always build new versions of the image for each new minor release of Terraform. It also ensures that each minor version that we build includes the latest patch release available. For example, assuming that the logan-terraform-1.6 image had previously been built using the 1.6.0 release of Terraform, when Hashicorp release version 1.6.1 this script would rebuild our logan-terraform-1.6 image using the newer 1.6.1 version of Terraform the next time the pipeline runs.

Job variables

Each job can be configured using the following variables.

Name Default Value Description
PUSH_DOCKER True Push the resulting images to the uisautomation public Docker Hub registry.
PUSH_GITLAB True Push to the container to this repository's container registry.
PUSH_TEST_IMAGES False Useful to push individual images to this repository's registry for testing.
ENVSUBST_DOCKERFILE False Use the envsubst command to substitute environment variables into the Dockerfile at build time.
IMAGE_ARCHS linux/amd64 Specify the target platforms for the image. This is a comma-separated string, for example linux/amd64,linux/arm64.

Multi-architecture builds

Images are built using the docker buildx build command which enables building images for multiple architectures. However, during testing it was noted that building linux/arm64 images on x86 runners can be flaky or simply take a very long time. With this in mind, care should be taken when enabling multi-architecture builds for a particular image type as Dockerfiles may need to be refactored to overcome installation issues etc. which otherwise work fine when building for linux/amd64.

Creating test images

The GitLabCI configuration only pushes images to the local repository if the branch is master. To create images while testing, manually start a new pipeline, select the branch to test and set PUSH_TEST_IMAGES to True. Images will then be pushed to a test subfolder of the registry.

You can force dynamic CI pipeline jobs to run by setting FORCE_DYNAMIC_JOBS when running the pipeline.

Adding known vulnerabilities to allowlist file

This project has container-scanning job enabled for all images. As a result, this project has detailed Vulnerability report with information about all detected vulnerabilities. Periodically, the Cloud Team triage those findings and some of them could be count as false positive or known issues. To exclude such issues from the final report, the issues must be added to the vulnerability-allowlist.yml file in the root of this project. The syntax is described in GitLab docs.

generalallowlist:
  # https://gitlab.developers.cam.ac.uk/uis/devops/infra/dockerimages/-/issues/80
  CVE-2023-6879: libaom
images:
  registry.gitlab.developers.cam.ac.uk/uis/devops/infra/dockerimages/ucam-mellon-proxy:
    # https://gitlab.developers.cam.ac.uk/uis/devops/infra/dockerimages/-/issues/81
    CVE-2023-45853: zlib1g

On the example above:

  • CVE-2023-6879: allowed for every image where it is detected in libaom package;
  • CVE-2023-6879: allowed for image registry.gitlab.developers.cam.ac.uk/uis/devops/infra/dockerimages/ucam-mellon-proxy, zlib1g package;

The GitLab issue reference must be added as a comment.

This file is also being used for "container-scanning" job in CI Templates project. The job downloads latest version of this file and use it as a source of information about allowed vulnerabilities. Project's own vulnerability-allowlist.yml file has priority over the one in this repository.