CHANGELOG.md

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[4.5.0] - 2024-08-13

Added

• mkdocs-docs.gitlab-ci.yml: a new template added to the common pipeline which builds documentation if it is present in the target repository and publishes it via GitLab pages.

[4.4.0] - 2024-07-23

Added

• terraform-module.yml: previously deleted job "fmt" replaced with "terraform-fmt" from terraform-lint.yml.

[4.3.1] - 2024-07-19

Fixed

• Since fmt job was removed from the upstream template Terraform-Module.gitlab-ci.yml, it is not possible to refer to that job in terraform-module.yml. That bit was removed to fix "jobs fmt config should implement a script: or a trigger: keyword" linter issue.

[4.3.0] - 2024-06-04

Added

• Terraform jobs now have a local files directory which gets copied from plan jobs to apply jobs exposed via the TF_VAR_local_files_dir environment variable. Some terraform configurations generate files as part of the plan which then need to be present in the apply.

[4.2.0] - 2024-05-20

Changed

• Remove allow_failure: true from commitlint and commitlint-hotfix jobs. This behaviour is no longer desirable as we've built our whole release-it process around the conventionalcommits specification. It's now more beneficial for these jobs to fail by default.

[4.1.0] - 2024-05-16

Added

• A generic get-gcp-secrets fragment to retrieve one or more Google Secret Manager secrets in a CI job.

[4.0.0] - 2024-04-05

Changed

• BREAKING CHANGE: removed legacy /auto-devops/terraform-deployment.yml and /auto-devops/terraform-lint.yml templates as these should not be being used any more.

• Refactored /auto-devops/terraform-pipeline.yml template so that the hidden "template" jobs are now in their own /terraform-pipeline-base.yml template file. This allows more flexibility for some projects which do not/cannot follow the standard three-environment deployment defined in the /auto-devops/terraform-pipeline.yml template.

• Refactored the Terraform testing/linting jobs into their own /terraform-lint.yml template. This is useful for projects which are not being deployed via CI/CD but which we do want to test in a pipeline.

[3.8.2] - 2024-04-10

Fixed

• terraform-pipeline.yml: tflint rule terraform_standard_module_structure is now disabled.

[3.8.1] - 2024-04-08

Fixed

• terraform-pipeline.yml: tflint job updated and now compatible with tflint v0.40.0+

[3.8.0] - 2024-03-26

Changed

• terraform-pipeline.yml: tfsec job replaced with trivy as tfsec is now deprecated.

[3.7.1] - 2024-03-01

Fixed

• artifact-registry.yml: added before_script to fix "docker in docker" service startup.

[3.7.0] - 2024-01-31

Added

• artifact-registry.yml: add support for code repositories with multiple apps.

[3.6.1] - 2024-01-31

Fixed

• check_latest_tag_in_changelog now correctly identifies the latest tag in the CHANGELOG, when there isn't a space after the closing square bracket.

[3.6.0] - 2024-01-16

Changed

• pre-commit.yml now allows hooks to use docker images.
• pre-commit.yml defines the pre-commit job as an extension of the .pre-commit template to allow for customisation.

[3.5.0] - 2024-01-16

• python-tox.yml allows specifying a precise version of tox to use.

[3.4.0] - 2024-01-03

Changed

• terraform-module.yml now uses standard Auto DevOps stages so it can be composed with other templates.

[3.3.0] - 2023-11-29

Fixed

• terraform-pipeline.yml: allow plan and apply jobs to run on git tag actions.

Added

• terraform-pipeline.yml: add TF_PLAN_TARGET variable to allow targeted plan actions.

[3.2.0] - 2023-11-07

Added

• Add new release-it.yml template for automated release management.

[3.1.1] - 2023-10-19

Changed

• The license scanning job was deprecated in GitLab 15.9 and removed in GitLab 16.3. Update the common pipeline to disable the job added by AutoDevOps to remove deprecation warnings in pipelines.

[3.1.0] - 2023-10-16

Added

• commitlint.yml template to ensure the Conventional Commits standard is adhered to for projects who wish to use it.

[3.0.0] - 2023-08-09

Changed

• Jobs using the "push and merge request" job-rules templates now will no-longer disable themselves on push pipelines if a merge request is open. Users of the templates should use workflow rules to disable push pipelines on MRs if that is required. (Otherwise, duplicate jobs are created.)
• The common pipeline now configures a reasonable default workflow rule to run only on branch pipelines.

[2.7.0] - 2023-07-31

Added

• terraform-module: Extend the GitLab Terraform Module to ensure the tag being used to version the module exists in the CHANGELOG.

[2.6.2] - 2023-08-01

Fixed

• pre-commit.yml: relax pre-commit job rules to enable users to decide when to include the job via workflow rules instead.

[2.6.1] - 2023-07-24

Fixed

• artifact-registry: fix issue with $DOCKER_CERT_PATH containing the $DOCKER_TLS_CERTDIR variable. We've seen multiple issues with this variable expanding as an empty value. Instead, we're simply hard coding it instead which appears to have resolved the issue.

[2.6.0] - 2023-07-24

Added

• terraform-pipeline: Allow the tfsec job to exclude specific checks. This is useful to exclude a default list of checks that we don't follow in the boilerplate.

Fixed

• artifact-registry: Allow manual push to registry for non-default branches. This is useful for testing container images from feature branches. However, to avoid littering the artifact registry with endless feature branch containers, it is configured as a manual job.
• terraform-pipeline: Allow the development apply job to fail. This is currently the only way to allow the whole pipeline to show as succeeded if the (optional) development apply job has not been triggered. Otherwise, the pipeline shows as blocked, which is confusing. See the following issue for context https://gitlab.com/gitlab-org/gitlab/-/issues/249524.

[2.5.0] - 2023-07-17

Added

• terraform-pipeline: Add resource_group to Terraform jobs to avoid concurrent pipelines for each environment.

Fixed

• terraform-pipeline: Add docker-in-docker service now that the GKE runner does not mount docker.sock from the host (see https://gitlab.developers.cam.ac.uk/uis/devops/devhub/gitlab-runner-infrastructure/-/merge_requests/14).

[2.4.4] - 2023-07-07

Fixed

• artifact-registry: Add docker-in-docker service now that the GKE runner does not mount the docker.sock from the host (see https://gitlab.developers.cam.ac.uk/uis/devops/devhub/gitlab-runner-infrastructure/-/merge_requests/14).

[2.4.3] - 2023-07-03

Fixed

• Auto DevOps is now enabled when you include the common pipeline even if you don't have one of the magic files the Auto DevOps pipeline looks for to enable itself. (It not looking for pyproject.toml files being a notable example.)

[2.4.2] - 2023-06-30

Fixed

• artifact-registry.yml: include latest tag for development environments.

[2.4.1] - 2023-06-29

Fixed

• artifact-registry.yml: remove needs: build to force the job to wait for tests to complete successfully.

[2.4.0] - 2023-06-28

Added

• New auto-devops/common-pipeline.yml intended to be included from the majority of our applications, packages, libraries and tools.
• New job-rules.yml which contains common job rules templates.
• New auto-devops/python-tox.yml intended to provide a slimmed-down version of the existing tox test pipeline which doesn't do things like spin up databases, etc.
• New auto-devops/python-package.yml intended for standalone Python packages. This new pipeline incorporates functionality from pypi-release.yml and can, in time, replace it.

[2.3.0] - 2023-06-28

Added

• artifact-registry.yml template to push Auto-DevOps built images to an Artifact Registry repository.

[2.2.2] - 2023-06-16

Modified

• terraform-pipeline.yml: add needs relationships to speed up pipeline.

[2.2.1] - 2023-06-15

Modified

• terraform-pipeline.yml: remove duplicate jobs between commits and merge requests.

[2.2.0] - 2023-06-08

Added

• terraform-pipeline.yml: jobs may now be disabled using Auto DevOps-style ..._DISABLED variables.

Modified

• terraform-pipeline.yml: modified job names to be more explicit that they run terraform.
• terraform-pipeline.yml: no longer override Auto DevOps workflow and stage configuration.

[2.1.0] - 2023-05-10

Added

• Add terraform-pipeline.yml template for deploying Terraform projects via the DevOps GKE Gitlab Runner.

[2.0.0] - 2023-03-27

Modified

• Update the Docker image version tags to use the latest release, and the auto-build-image image version tag to use v1.31.0. This is a major change as the previous stable tags haven't updated for over a year so the latest versions may include backwards-incompatible changes. This impacts the templates: extra-tags, terraform-deployment, terraform-lint and tox-tests.

[1.7.3] - 2022-06-22

Bugfix

• Fix incompatibility with GitLab 15

[1.7.2] - 2022-02-16

Modified

• Add a template for terraform deployment to staging and production.

[1.7.1] - 2022-01-21

Modified

• PyPi release: ensure that dangling publish jobs are not created on MR pipelines.
• PyPi release: allow distribution location to be updated using variables.

[1.7.0] - 2021-10-27

Modified

• Tox tests: removing the usage of a custom gemnasium image as this is no longer needed.

[1.6.0] - 2021-08-10

Modified

• Make terraform lint use variables and thus allow overriding of version and/or docker image
• Update default lint version to 1.0.4

[1.5.0] - 2021-03-24

Modified

• Tox tests: allow a custom image to be used in tox test job through TOX_IMAGE_BUILD_COMMAND.

[1.4.0] - 2021-03-18

Modified

• Deployment: perform Django migrations before deploying to cloud run.

[1.3.0] - 2021-02-08

Modified

• Terraform lint: Fixed the version of Terraform used for linting to 0.14.6.

[1.2.0] - 2020-12-17

Added

• Deployment: A new template added to allow multiple cloud run services to be deployed as part of a single deployment project.

[1.1.5] - 2020-12-11

Added

• Terraform lint: Added -diff flag

[1.1.4] - 2020-12-04

Modified

• Terraform lint: Fixed the version of Terraform used for linting to 0.13.5.

[1.1.3] - 2020-12-02

Added

• Deployment: allow gitlab deploy environment to be overridden by downstream jobs in deployment CI template job

[1.1.2] - 2020-11-24

Fixed

• Deployment: fixed variable interpolation so that SERVICE_PREFIX is used for all variables which are required to be set downstream in deployment CI template job

[1.1.1] - 2020-09-21

Added

• Deployment: Added a deployment CI template job that provides jobs for deploying a previously built Docker image to Cloud Run on Google Cloud Platform (GCP).
• Terraform lint: Added a Terraform lint CI template job to run terraform fmt on Terraform configuration files in the local repository.

[1.1.0] - 2020-08-26

Modified

• PEP8: Modified the PEP8 CI template job to be uninterruptible, and to only run for pipelines other than merge request pipelines.
• Tox tests: Modified the Tox tests template to make the documentation job run for pipelines other than merge request pipelines (previously the job was never running due to a bug in the job rules).

[1.0.1] - 2020-08-06

Added

• pypi-release: Added generic stand along jobs designed to allow easy upload of Python packages to PyPI.

[1.0.0] - 2020-07-16

Added

• PEP8: Added a CI template job to check for PEP8 violations in Pyhon files. The CI template generates junit.xml reports that feed back to GitLab CI for interpretation and storage as artifacts. The PEP8 CI template job uses the python tool flake8.