FAQ | This is a LIVE service | Changelog

Skip to content
Snippets Groups Projects
Commit ae609a53 authored by Dr Rich Wareham's avatar Dr Rich Wareham
Browse files

Merge branch 'remove-gcp-fragment-docs' into 'master'

chore: move GCP secrets documentation to guidebook

See merge request !108
parents fadf166e 07203690
No related branches found
Tags v6.1.1
1 merge request!108chore: move GCP secrets documentation to guidebook
# GCP Secrets Helper
Accessing secrets securely in CI/CD pipelines is notoriously tricky. This
fragment provides some reusable code to make the process of retrieving one
or more Google Secret Manager secrets easier.
## Prerequisites
To use this fragment in your CI jobs you will need to ensure that the following
prerequisites are met.
- You'll need to ensure that `bash`, `curl`, and `jq` are installed in your
jobs' image and that your job is running in a `bash` shell.
- The CI runner executing your job must be running in the context of a Google
Cloud identity. The identity must either have the `secrets.secretAccessor`
role granted for all secrets to be retrieved, or have the ability to
impersonate a service account which does.
## How-to
The following steps will retrieve the `latest` version of the Google Secret
Manager secret named `my-secret` from the Google Cloud project named
`my-project` and export the secret value as an environment variable named
`MY_SECRET`.
1. First, in your `.gitlab-ci.yml` file include the `get-gcp-secrets.yml`
template.
```yaml
include:
- project: uis/devops/continuous-delivery/ci-templates
file: /fragments/get-gcp-secrets.yml
ref: <specify version here>
```
1. Next, define a job with the variable `GET_GCP_SECRET_MY_SECRET:
projects/my-project/secrets/my-secret/versions/latest`.
```yaml
my-job:
stage: build
image: ubuntu:24.04
variables:
GET_GCP_SECRET_MY_SECRET: projects/my-project/secrets/my-secret/versions/latest
```
1. Specify the `$GKE_RUNNER_TAG` job tag. When working on UIS DevOps projects,
you'll usually need to run your CI jobs on one of our Google Kubernetes
Engine hosted runners to be able to use this helper. To do that, add the
magic `$GKE_RUNNER_TAG` variable to the list of tags for your job.
```yaml
my-job:
stage: build
image: ubuntu:24.04
variables:
GET_GCP_SECRET_MY_SECRET: projects/my-project/secrets/my-secret/versions/latest
tags:
- $GKE_RUNNER_TAG
```
1. Add a reference to the included hidden `.get-gcp-secrets` job's
`before_script` block in your job's `before_script`.
```yaml
my-job:
stage: build
image: ubuntu:24.04
variables:
GET_GCP_SECRET_MY_SECRET: projects/my-project/secrets/my-secret/versions/latest
before_script:
- apt-get update && apt-get install -y curl jq # Required as this example uses the ubuntu:24.04 base image.
- !reference [.get-gcp-secrets, before_script]
```
1. The `before_script` referenced in `.get-gcp-secrets` will retrieve the value
of the secret in the `projects/my-project/secrets/my-secret/versions/latest`
secret object and export it as an environment variable named `MY_SECRET`.
This variable can then be referenced in the `script` block of your job in the
usual way.
```yaml
my-job:
stage: build
image: ubuntu:24.04
variables:
GET_GCP_SECRET_MY_SECRET: projects/my-project/secrets/my-secret/versions/latest
before_script:
- apt-get update && apt-get install -y curl jq # Required as this example uses the ubuntu:24.04 base image.
- !reference [.get-gcp-secrets, before_script]
script: do-something --secret $MY_SECRET
```
## Specifying secrets
The `before_script` looks for all variables available to the job which begin
with `GET_GCP_SECRET_`. It expects that each of these variables contains the
full id of the secret version to be retrieved e.g.
`projects/my-project/secrets/my-secret/versions/latest`
Each secret version is retrieved and exported to an environment variable named
with the variable name minus the `GET_GCP_SECRET_` prefix. For example, if you
use the variable name `GET_GCP_SECRET_API_TOKEN` the secret value will be
exported to the `API_TOKEN` environment variable.
Providing your runner has the appropriate permissions, it is possible to define
multiple secrets in a single job. For example:
```yaml
my-job:
stage: build
image: ubuntu:24.04
variables:
GET_GCP_SECRET_SUPER_SECRET_TOKEN: projects/some-project-1/secrets/some-secret-1/versions/latest
GET_GCP_SECRET_DATABASE_PASSWORD: projects/some-project-2/secrets/some-secret-2/versions/56
before_script:
- apt-get update && apt-get install -y curl jq # Required as this example uses the ubuntu:24.04 base image.
- !reference [.get-gcp-secrets, before_script]
script: |
do-something --secret $SUPER_SECRET_TOKEN
do-something-else --database-password $DATABASE_PASSWORD
```
## Google Service Account impersonation
In some situations, the identity of your CI runner may not have permission to
access the required secrets itself but it may be able to impersonate a service
account which does. To specify the email address of a service account to be
impersonated you can use the `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` variable.
```yaml
my-job:
stage: build
image: ubuntu:24.04
variables:
GOOGLE_IMPERSONATE_SERVICE_ACCOUNT: my-service-account@project-a.iam.gserviceaccount.com
GET_GCP_SECRET_SUPER_SECRET_TOKEN: projects/some-project-1/secrets/some-secret-1/versions/latest
GET_GCP_SECRET_DATABASE_PASSWORD: projects/some-project-2/secrets/some-secret-2/versions/56
before_script:
- apt-get update && apt-get install -y curl jq # Required as this example uses the ubuntu:24.04 base image.
- !reference [.get-gcp-secrets, before_script]
script: |
do-something --secret $SUPER_SECRET_TOKEN
do-something-else --database-password $DATABASE_PASSWORD
```
# This is a helper script to access GCP secrets in pipeline jobs. For more information see ./get-gcp-secrets.md.
# This is a helper script to access GCP secrets in pipeline jobs. For more information see
# https://guidebook.devops.uis.cam.ac.uk/howtos/get-gcp-secrets/
.get-gcp-secrets:
before_script:
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment