diff --git a/fragments/get-gcp-secrets.md b/fragments/get-gcp-secrets.md deleted file mode 100644 index d0f7dcf0060f2cfdf7f908d94b91977406b4c680..0000000000000000000000000000000000000000 --- a/fragments/get-gcp-secrets.md +++ /dev/null @@ -1,145 +0,0 @@ -# GCP Secrets Helper - -Accessing secrets securely in CI/CD pipelines is notoriously tricky. This -fragment provides some reusable code to make the process of retrieving one -or more Google Secret Manager secrets easier. - -## Prerequisites - -To use this fragment in your CI jobs you will need to ensure that the following -prerequisites are met. - -- You'll need to ensure that `bash`, `curl`, and `jq` are installed in your - jobs' image and that your job is running in a `bash` shell. -- The CI runner executing your job must be running in the context of a Google - Cloud identity. The identity must either have the `secrets.secretAccessor` - role granted for all secrets to be retrieved, or have the ability to - impersonate a service account which does. - -## How-to - -The following steps will retrieve the `latest` version of the Google Secret -Manager secret named `my-secret` from the Google Cloud project named -`my-project` and export the secret value as an environment variable named -`MY_SECRET`. - -1. First, in your `.gitlab-ci.yml` file include the `get-gcp-secrets.yml` - template. - - ```yaml - include: - - project: uis/devops/continuous-delivery/ci-templates - file: /fragments/get-gcp-secrets.yml - ref: <specify version here> - ``` - -1. Next, define a job with the variable `GET_GCP_SECRET_MY_SECRET: - projects/my-project/secrets/my-secret/versions/latest`. - - ```yaml - my-job: - stage: build - image: ubuntu:24.04 - variables: - GET_GCP_SECRET_MY_SECRET: projects/my-project/secrets/my-secret/versions/latest - ``` - -1. Specify the `$GKE_RUNNER_TAG` job tag. When working on UIS DevOps projects, - you'll usually need to run your CI jobs on one of our Google Kubernetes - Engine hosted runners to be able to use this helper. To do that, add the - magic `$GKE_RUNNER_TAG` variable to the list of tags for your job. - - ```yaml - my-job: - stage: build - image: ubuntu:24.04 - variables: - GET_GCP_SECRET_MY_SECRET: projects/my-project/secrets/my-secret/versions/latest - tags: - - $GKE_RUNNER_TAG - ``` - -1. Add a reference to the included hidden `.get-gcp-secrets` job's - `before_script` block in your job's `before_script`. - - ```yaml - my-job: - stage: build - image: ubuntu:24.04 - variables: - GET_GCP_SECRET_MY_SECRET: projects/my-project/secrets/my-secret/versions/latest - before_script: - - apt-get update && apt-get install -y curl jq # Required as this example uses the ubuntu:24.04 base image. - - !reference [.get-gcp-secrets, before_script] - ``` - -1. The `before_script` referenced in `.get-gcp-secrets` will retrieve the value - of the secret in the `projects/my-project/secrets/my-secret/versions/latest` - secret object and export it as an environment variable named `MY_SECRET`. - This variable can then be referenced in the `script` block of your job in the - usual way. - - ```yaml - my-job: - stage: build - image: ubuntu:24.04 - variables: - GET_GCP_SECRET_MY_SECRET: projects/my-project/secrets/my-secret/versions/latest - before_script: - - apt-get update && apt-get install -y curl jq # Required as this example uses the ubuntu:24.04 base image. - - !reference [.get-gcp-secrets, before_script] - script: do-something --secret $MY_SECRET - ``` - -## Specifying secrets - -The `before_script` looks for all variables available to the job which begin -with `GET_GCP_SECRET_`. It expects that each of these variables contains the -full id of the secret version to be retrieved e.g. -`projects/my-project/secrets/my-secret/versions/latest` - -Each secret version is retrieved and exported to an environment variable named -with the variable name minus the `GET_GCP_SECRET_` prefix. For example, if you -use the variable name `GET_GCP_SECRET_API_TOKEN` the secret value will be -exported to the `API_TOKEN` environment variable. - -Providing your runner has the appropriate permissions, it is possible to define -multiple secrets in a single job. For example: - -```yaml - my-job: - stage: build - image: ubuntu:24.04 - variables: - GET_GCP_SECRET_SUPER_SECRET_TOKEN: projects/some-project-1/secrets/some-secret-1/versions/latest - GET_GCP_SECRET_DATABASE_PASSWORD: projects/some-project-2/secrets/some-secret-2/versions/56 - before_script: - - apt-get update && apt-get install -y curl jq # Required as this example uses the ubuntu:24.04 base image. - - !reference [.get-gcp-secrets, before_script] - script: | - do-something --secret $SUPER_SECRET_TOKEN - do-something-else --database-password $DATABASE_PASSWORD -``` - -## Google Service Account impersonation - -In some situations, the identity of your CI runner may not have permission to -access the required secrets itself but it may be able to impersonate a service -account which does. To specify the email address of a service account to be -impersonated you can use the `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` variable. - -```yaml - my-job: - stage: build - image: ubuntu:24.04 - variables: - GOOGLE_IMPERSONATE_SERVICE_ACCOUNT: my-service-account@project-a.iam.gserviceaccount.com - GET_GCP_SECRET_SUPER_SECRET_TOKEN: projects/some-project-1/secrets/some-secret-1/versions/latest - GET_GCP_SECRET_DATABASE_PASSWORD: projects/some-project-2/secrets/some-secret-2/versions/56 - before_script: - - apt-get update && apt-get install -y curl jq # Required as this example uses the ubuntu:24.04 base image. - - !reference [.get-gcp-secrets, before_script] - script: | - do-something --secret $SUPER_SECRET_TOKEN - do-something-else --database-password $DATABASE_PASSWORD -``` diff --git a/fragments/get-gcp-secrets.yml b/fragments/get-gcp-secrets.yml index cdf5d827c2f9a2953d90b56849db28072b6dcb00..a9aae3a201ccfc29bf17d8cf523887a29f960cf4 100644 --- a/fragments/get-gcp-secrets.yml +++ b/fragments/get-gcp-secrets.yml @@ -1,4 +1,5 @@ -# This is a helper script to access GCP secrets in pipeline jobs. For more information see ./get-gcp-secrets.md. +# This is a helper script to access GCP secrets in pipeline jobs. For more information see +# https://guidebook.devops.uis.cam.ac.uk/howtos/get-gcp-secrets/ .get-gcp-secrets: before_script: