diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 923edf46356dd63453f1ae9afc8658ede696682c..841f7d888c789b20d395a5c218c333661a1fa7b9 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -49,3 +49,9 @@ repos: pass_filenames: false entry: aquasec/trivy:latest args: ["--cache-dir", "/tmp/.trivy-cache", "--skip-dirs", "tests", "config", ".", "--exit-code", "1"] + - id: terraform-docs + name: terraform-docs + language: docker_image + pass_filenames: false + entry: quay.io/terraform-docs/terraform-docs:0.17.0 + args: ["."] diff --git a/.terraform-docs.yml b/.terraform-docs.yml new file mode 100644 index 0000000000000000000000000000000000000000..de042c53024afb6d4c0fca5e00ffb121f378ae51 --- /dev/null +++ b/.terraform-docs.yml @@ -0,0 +1,13 @@ +formatter: markdown table +version: "~> 0.17.0" +header-from: docs/templates/header.md +footer-from: docs/templates/footer.md +output: + file: README.md +sections: + show: + - header + - requirements + - inputs + - outputs + - footer diff --git a/README.md b/README.md index c7d9d423eb9e6cafc0c6b7249d8081f8a44d2df9..d7773a5a4db9a62050fd8d93b5e03f8f81ead19d 100644 --- a/README.md +++ b/README.md @@ -1,224 +1,129 @@ -# GCP Cloud Run manager terraform module - -This module manages a Cloud Run-hosted container. It takes care of making sure -the container is connected to a Cloud SQL instance and sets environment -variables on the application. - -Specify the project to deploy into on the command line. So, for example, to -deploy to the project `my-project`: - -```console -terraform init -terraform apply -var project=my-project -``` - -In this example, terraform is configured to use default application credentials. -For Google APIs and these credentials should correspond to a user with owner or -editor access to the target project. You can use the `gcloud` command line tool -to set your personal credentials as application default credentials. See the -`gcloud auth application-default` command output for more information. - -## Versioning - -The `master` branch contains the tip of development and corresponds to the `v8` -branch. The `v1`, `v2`, `v3` etc. branches will maintain source compatibility -with the initial release. - -## Ingress style - -There are two supported ingress styles depending on `var.ingress_style` -variable. - -`var.ingress_style` can be: - -- `domain-mapping` (default): passing DNS domains as `var.dns_names` or - `var.dns_name`, which takes precedence over `var.dns_names`, will create - domain mappings to the Cloud Run service. Before setting this, you _must_ have - verified ownership of the provided domains with Google. [Instructions on how - to do - this](https://guidebook.devops.uis.cam.ac.uk/en/latest/notes/google-domain-verification/) - can be found in the DevOps division guidebook. - -- `load-balancer`: a load balancer will be configured instead of a domain - mapping. The DNS domains in `var.dns_names` or `var.dns_name`, which takes - precedence over `var.dns_names`, will get Google-managed or custom TLS - certificates depending on `var.use_ssl_certificates` and - `var.ssl_certificates`. An IPv6 address can also be allocated to the load - balancer if `var.enable_ipv6` is `true`. - -## Pre-deploy Cloud Run job - -The `v8` release introduced an `enable_pre_deploy_job` variable. When set to -`true` a [Cloud Run job](https://cloud.google.com/run/docs/create-jobs) is -created to execute a configurable command _before_ the main Cloud Run service is -deployed. The initial use case for this is to run database migrations, however -in the future we're sure there'll be more. - -The pre-deploy job uses the image specified in `var.pre_deploy_job_image_name` -if set, otherwise it falls back to the same `var.image_name` that the main -service uses. The command and arguments that the job executes are configurable -via the `pre_deploy_job_command` and `pre_deploy_job_args` variables. - -A `null_resource` is also configured to execute the pre-deploy job whenever it -detects that the value of `var.image_name` has changed (or at every `apply` if -`var.force_pre_deploy_job` is set to `true`). This uses the `gcloud run jobs -execute` command and is run in the context of the `terraform-deploy` service -account via an access token. Using `null_resource` is never ideal. However, in -this situation it provides a very handy way to trigger this simple job so it has -been accepted. - -To ensure that the pre-deploy job always runs _before_ a new revision of the -Cloud Run webapp service is deployed, the resources in question are explicitly -configured with `depends_on` relationships, as follows. - -1. The `google_cloud_run_v2_job.pre_deploy` Cloud Run job has no `depends_on` - relationships defined and is therefore deployed first. -1. The `null_resource.pre_deploy_job_trigger` resource depends on - `google_cloud_run_v2_job.pre_deploy` and therefore won't be deployed until - the Cloud Run job is deployed successfully. -1. Finally, the `google_cloud_run_service.webapp` Cloud Run service depends on - `null_resource.pre_deploy_job_trigger`, meaning it is only deployed once the - `null_resource.pre_deploy_job_trigger` has executed successfully. - -## Monitoring and Alerting - -If the variable [alerting_email_address](variables.tf) is set, the module adds -basic uptime _alerting_ via email for failing http polling. - -If the variable [disable_monitoring](variables.tf) is true, the module will -disable _monitoring_. This is different from disabling alerting; if no alerting -email addresses are provided, the uptime checks will still be configured, there -just won't be any alerts sent if they fail. Disabling monitoring will also -disable alerting as without any monitoring there is nothing to alert(!) - -See [variables.tf](variables.tf) for how to configure alerting and monitoring. - -Note that the project containing resources to be monitored must be in a -Stackdriver monitoring workspace and this must be configured manually. At the -time of writing there is no terraform support for this. This module will error -when applying if this is not so. - -Stackdriver distinguishes between workspaces and projects within those -workspaces. Each workspace must have a host project and that project _must_ be -the default project of the `google.stackdriver` provider used by this module. -The `google.stackdriver` must be configured with credentials allowing monitoring -resources to be created in the _host_ project. - -If the workspace host project differs from the project which contains the -resources to be monitored, you can use a provider alias: - -```tf -provider "google" { - project = "my-project" - - # ... some credentials for the *project* admin ... -} - -provider "google" { - project = "stackdriver-host-project" - alias = "host" - - # ... some credentials for the *product* admin ... -} - -module "cloud_run_service" { - # ... other parameters ... - - providers = { - google.stackdriver = google.host - } -} -``` - -### Monitoring instances which require service account authentication - -If `allow_unauthenticated_invocations` is not true, a Cloud Function will be -created which authenticates via a service account, allowing the StackDriver -monitoring to call the Cloud Function, with the Cloud Function authentication -and proxying the request to the Cloud Run instance. - -Because this requires a Cloud Function to be created, the -`cloudfunctions.googleapis.com` service should be enabled on the project that -houses the Cloud Run instance. - -## Static Egress IP - -A static egress IP can be allocated for the cloud run instance, using the -variable `enable_static_egress_ip`. This will configure the necessary resources -to route outbound traffic from the cloud run through a static ip. - -**Important!** - -The static ip is configured with `prevent_destroy = true`, meaning that it -cannot be destroyed without removing it from terraform state using `terraform -state rm` and then manually destroying the resource within the GCP console. This -is to prevent accidental destruction of an IP which is likely to be whitelisted -within firewall configuration that lives outside of our deployments. - -## Secrets as Volumes and Env Vars - -Secret Manager secrets can be as environment variables or volume mounts (files) -in the running container. - -The service account that Cloud Run runs as needs access to the secrets for this -feature to work. Thus, this module gives `secretAccessor` role to that service -account for the secrets passed on `secrets_volume` and `secrets_envars`. - -Any number of items in the list is supported and not defining these variables -when calling this module is acceptable. The path of the mounted file will be -based on `path/name`. - -For the example configuration below the files will be `/secrets-1/foobarfile1` -and `/secrets-2/foobarfile2`. A common `path` for multiple secrets is not -supported, they must be unique. - -> Note: `name` should only have alphanumeric characters, hyphens and -> underscores. - -Setting `version = ""` is equivalent to `version = "latest"` but the variable is -not optional. - -```tf -module "webapp" { - source = "git::https://gitlab.developers.cam.ac.uk/uis/devops/infra/terraform/gcp-cloud-run-app.git?ref=v3" - -... - - secrets_volume = [ - { - name = "foobarfile1" - path = "/secret-1" - id = google_secret_manager_secret.secret-a.secret_id - version = "latest" - }, - { - name = "foobarfile2" - path = "/secret-2" - id = google_secret_manager_secret.secret-b.secret_id - version = "2" - } - ] - - secrets_envars = [ - { - name = "FOOBAR1" - id = google_secret_manager_secret.secret-c.secret_id - version = "latest" - }, - { - name = "FOOBAR2" - id = google_secret_manager_secret.secret-d.secret_id - version = "" - } - ] - -... -``` - -## Passing Image Names to the Module - -Originally, the module did not deploy images except on the very first use (using -`gcr.io/cloudrun/hello:latest`). - -Currently, the module deploys the image from the mandatory variable -`image_name`. +<!-- BEGIN_TF_DOCS --> +# GCP Cloud Run Terraform Module + +This module manages the deployment of containerised applications on Cloud Run. +It includes the following functionality: + +- Creation of the main Cloud Run service. +- Creation of a dedicated service account with required IAM bindings. +- Deployment of an optional load balancer configuration. +- Deployment of an optional "pre-deployment" Cloud Run job to handle tasks such + as database migrations. +- Deployment of an optional static egress IP address for the Cloud Run service. +- Configuration of simple uptime checks and SSL certificate expiry alerts. +- Convenience variables to configure certain aspects of the Cloud Run service + which would otherwise be complex, such as mounting Cloud SQL instances. + +## Cloud Run v2 API + +Following release `9.0.0`, this module has been refactored to use the Cloud Run +`v2` API resources exclusively (i.e. `google_cloud_run_v2_service`). This means +that many input variables are now different and updating from a previous version +will require some changes to your module definition. + +## Examples + +See the [docs/examples.md](docs/examples.md) page for a full list of detailed +usage examples. + +## Pre-deploy Cloud Run Job + +The `8.0.0` release introduced the `enable_pre_deploy_job` variable which, when +set to `true`, creates a Cloud Run job to execute a configurable command before +new Cloud Run service revisions are deployed. This is a useful way to run +database migrations and other commands which are tightly coupled to the release +cadence of the main Cloud Run service. + +The pre-deploy job is configured via the `pre_deploy_job_*` variables which can +be found in `variables.tf`. + +For more information on how the pre-deploy Cloud Run job works see the +[pre-deploy-job.md](../docs/pre-deploy-job.md) page. + +## Requirements + +| Name | Version | +|------|---------| +| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | ~> 1.3 | +| <a name="requirement_google"></a> [google](#requirement\_google) | >= 4.0 | +| <a name="requirement_null"></a> [null](#requirement\_null) | ~> 3.0 | + +## Inputs + +| Name | Description | Type | Default | Required | +|------|-------------|------|---------|:--------:| +| <a name="input_alerting_notification_channels"></a> [alerting\_notification\_channels](#input\_alerting\_notification\_channels) | A list of notification channel IDs to send uptime alerts to. The format for<br>the channel IDs should be<br>"projects/[PROJECT\_ID]/notificationChannels/[CHANNEL\_ID]". | `list(string)` | `[]` | no | +| <a name="input_alerting_success_threshold_percent"></a> [alerting\_success\_threshold\_percent](#input\_alerting\_success\_threshold\_percent) | If the percentage of successful uptime checks within the given uptime period<br>falls below this, an alert will be triggered. Set to 100 to trigger an alert<br>if any uptime check fails, set to a lower number to tolerate failures without<br>alerting.<br><br>Experience has taught us that uptime checks can fail semi-regularly due to<br>transient problems outside our control, therefore we allow some leeway before<br>triggering an alert. | `number` | `75` | no | +| <a name="input_alerting_uptime_period"></a> [alerting\_uptime\_period](#input\_alerting\_uptime\_period) | Frequency of uptime checks | `string` | `"300s"` | no | +| <a name="input_alerting_uptime_timeout"></a> [alerting\_uptime\_timeout](#input\_alerting\_uptime\_timeout) | Timeout for http polling. | `string` | `"30s"` | no | +| <a name="input_allow_unauthenticated_invocations"></a> [allow\_unauthenticated\_invocations](#input\_allow\_unauthenticated\_invocations) | If true, the webapp will allow unauthenticated invocations. If false, the<br>webapp requires authentication as a Google user with the Cloud Run invoker<br>permission on the deployment. | `bool` | `true` | no | +| <a name="input_containers"></a> [containers](#input\_containers) | Configure one or more container instances for the service. See<br>https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/cloud_run_v2_service#nested_containers<br>for information on the available arguments. | <pre>map(object({<br> name = optional(string)<br> image = string<br> command = optional(list(string))<br> args = optional(list(string))<br> env = optional(list(object({<br> name = string<br> value = optional(string)<br> value_source = optional(object({<br> secret_key_ref = optional(object({<br> secret = string<br> version = optional(string, "latest")<br> }))<br> }))<br> })), [])<br> resources = optional(object({<br> limits = optional(map(string))<br> cpu_idle = optional(bool)<br> startup_cpu_boost = optional(bool)<br> }))<br> ports = optional(list(object({<br> name = optional(string)<br> container_port = optional(number)<br> })), [])<br> volume_mounts = optional(list(object({<br> name = string<br> mount_path = string<br> })), [])<br> working_dir = optional(string)<br> liveness_probe = optional(object({<br> initial_delay_seconds = optional(number)<br> timeout_seconds = optional(number)<br> period_seconds = optional(number)<br> failure_threshold = optional(number)<br> http_get = optional(object({<br> path = optional(string)<br> port = optional(number)<br> http_headers = optional(list(object({<br> name = string<br> value = optional(string)<br> })), [])<br> }))<br> grpc = optional(object({<br> port = optional(number)<br> service = optional(string)<br> }))<br> }))<br> startup_probe = optional(object({<br> initial_delay_seconds = optional(number)<br> timeout_seconds = optional(number)<br> period_seconds = optional(number)<br> failure_threshold = optional(number)<br> http_get = optional(object({<br> path = optional(string)<br> port = optional(number)<br> http_headers = optional(list(object({<br> name = string<br> value = optional(string)<br> })), [])<br> }))<br> tcp_socket = optional(object({<br> port = number<br> }))<br> grpc = optional(object({<br> port = optional(number)<br> service = optional(string)<br> }))<br> }))<br> }))</pre> | n/a | yes | +| <a name="input_create_ipv6_address"></a> [create\_ipv6\_address](#input\_create\_ipv6\_address) | Allocate an IPv6 address to the load balancer if var.enable\_ipv6 is true. | `bool` | `false` | no | +| <a name="input_description"></a> [description](#input\_description) | A description for the Cloud Run service. | `string` | `null` | no | +| <a name="input_dns_names"></a> [dns\_names](#input\_dns\_names) | DNS names to configure for the web application. Note that DNS records are<br>\_NOT\_ created, they are used in the load balancer module to ensure the SSL<br>certificate is generated with the required SANs. The map's keys are arbitrary<br>and are only required to avoid errors when the DNS name is a value which<br>Terraform does not know until after the apply operation.<br><br>For example:<br><br>{ my\_awesome\_dns\_name = "awesome.example.com" } | `map(string)` | `{}` | no | +| <a name="input_enable_alerting"></a> [enable\_alerting](#input\_enable\_alerting) | Enable alerting policies. | `bool` | `true` | no | +| <a name="input_enable_ipv6"></a> [enable\_ipv6](#input\_enable\_ipv6) | Whether to enable IPv6 address on the CDN load-balancer. | `bool` | `false` | no | +| <a name="input_enable_load_balancer"></a> [enable\_load\_balancer](#input\_enable\_load\_balancer) | Whether to configure a load balancer or use the default run.app generated<br>hostname. | `bool` | `false` | no | +| <a name="input_enable_monitoring"></a> [enable\_monitoring](#input\_enable\_monitoring) | Optional. If true, create uptime and SSL expiry checks.<br><br>Note that this is different from not specifying an alerting email address. If<br>no alerting email address is specified the uptime checks are still created,<br>they just don't alert if they fail. | `bool` | `false` | no | +| <a name="input_enable_pre_deploy_job"></a> [enable\_pre\_deploy\_job](#input\_enable\_pre\_deploy\_job) | Configure a Cloud Run Job to be executed *before* the main Cloud Run service<br>is deployed. This is useful for running database migrations for example. | `bool` | `false` | no | +| <a name="input_enable_static_egress_ip"></a> [enable\_static\_egress\_ip](#input\_enable\_static\_egress\_ip) | Whether to assign a static ip for egress from this cloud run instance. If<br>enabled, the "vpcaccess.googleapis.com" API must also be enabled on the<br>project. | `bool` | `false` | no | +| <a name="input_encryption_key"></a> [encryption\_key](#input\_encryption\_key) | The ID of a customer managed encryption key (CMEK) to use to encrypt this<br>container image. | `string` | `null` | no | +| <a name="input_execution_environment"></a> [execution\_environment](#input\_execution\_environment) | The sandbox environment to host this revision. Possible values are<br>EXECUTION\_ENVIRONMENT\_GEN1, and EXECUTION\_ENVIRONMENT\_GEN2. | `string` | `"EXECUTION_ENVIRONMENT_GEN1"` | no | +| <a name="input_grant_sql_client_role_to_webapp_sa"></a> [grant\_sql\_client\_role\_to\_webapp\_sa](#input\_grant\_sql\_client\_role\_to\_webapp\_sa) | When set to true the roles/cloudsql.client role will be granted to the webapp<br>service account at the project level to allow it to connect to Cloud SQL. | `bool` | `false` | no | +| <a name="input_ingress"></a> [ingress](#input\_ingress) | The ingress setting for the Cloud Run service. Possible values are<br>INGRESS\_TRAFFIC\_ALL, INGRESS\_TRAFFIC\_INTERNAL\_ONLY, and<br>INGRESS\_TRAFFIC\_INTERNAL\_LOAD\_BALANCER.<br><br>If var.use\_load\_balancer == true, the provided var.ingress will be ignored and<br>the ingress will be set automatically to<br>"INGRESS\_TRAFFIC\_INTERNAL\_LOAD\_BALANCER". | `string` | `"INGRESS_TRAFFIC_ALL"` | no | +| <a name="input_launch_stage"></a> [launch\_stage](#input\_launch\_stage) | The launch stage for the Cloud Run service. Possible values are UNIMPLEMENTED,<br>PRELAUNCH, EARLY\_ACCESS, ALPHA, BETA, GA, and DEPRECATED. | `string` | `"GA"` | no | +| <a name="input_max_instance_request_concurrency"></a> [max\_instance\_request\_concurrency](#input\_max\_instance\_request\_concurrency) | Sets the maximum number of requests that each serving instance can receive. | `number` | `null` | no | +| <a name="input_min_ports_per_vm"></a> [min\_ports\_per\_vm](#input\_min\_ports\_per\_vm) | When using Cloud NAT to provide an egress route, Cloud NAT's minimum ports per<br>VM can be configured to determine how many concurrent connections can be<br>established to the same destination IP address and port. | `number` | `64` | no | +| <a name="input_monitoring_path"></a> [monitoring\_path](#input\_monitoring\_path) | Path component of url to be monitored. | `string` | `"/"` | no | +| <a name="input_monitoring_scoping_project"></a> [monitoring\_scoping\_project](#input\_monitoring\_scoping\_project) | The ID of a Cloud Monitoring scoping project to create monitoring resources<br>in. If omitted, var.project will be used instead. | `string` | `null` | no | +| <a name="input_mount_cloudsql_instance"></a> [mount\_cloudsql\_instance](#input\_mount\_cloudsql\_instance) | Mount a single CloudSQL instance in each container for the service. This value<br>should be the Cloud SQL instance connection name, for example<br>"example-devel-e662dd2b:europe-west2:sql-6e1dd60b". This is a convenience<br>variable to simplify mounting a single Cloud SQL instance. If you require more<br>control you can define one or more Cloud SQL mounts directly using<br>var.containers and var.volumes instead. | `string` | `null` | no | +| <a name="input_name"></a> [name](#input\_name) | Prefix used to form resource names. | `string` | `"webapp"` | no | +| <a name="input_pre_deploy_job_annotations"></a> [pre\_deploy\_job\_annotations](#input\_pre\_deploy\_job\_annotations) | Map of key/value pairs containing annotations to assign to the pre-deploy Cloud<br>Run job. | `map(string)` | `null` | no | +| <a name="input_pre_deploy_job_container"></a> [pre\_deploy\_job\_container](#input\_pre\_deploy\_job\_container) | Configure the container instance for the pre-deploy job. See<br>https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/cloud_run_v2_job#nested_containers<br>for more information on these options. | <pre>object({<br> name = optional(string)<br> image = optional(string)<br> command = optional(list(string))<br> args = optional(list(string))<br> env = optional(list(object({<br> name = string<br> value = optional(string)<br> value_source = optional(object({<br> secret_key_ref = optional(object({<br> secret = string<br> version = optional(string, "latest")<br> }))<br> }))<br> })), [])<br> resources = optional(object({<br> limits = optional(map(string))<br> }))<br> ports = optional(list(object({<br> name = optional(string)<br> container_port = optional(number)<br> })), [])<br> volume_mounts = optional(list(object({<br> name = string<br> mount_path = string<br> })), [])<br> working_dir = optional(string)<br> })</pre> | `null` | no | +| <a name="input_pre_deploy_job_encryption_key"></a> [pre\_deploy\_job\_encryption\_key](#input\_pre\_deploy\_job\_encryption\_key) | The ID of a customer managed encryption key (CMEK) to use to encrypt this<br>container image. | `string` | `null` | no | +| <a name="input_pre_deploy_job_execution_environment"></a> [pre\_deploy\_job\_execution\_environment](#input\_pre\_deploy\_job\_execution\_environment) | The execution environment to host this task. Possible values are<br>EXECUTION\_ENVIRONMENT\_GEN1, and EXECUTION\_ENVIRONMENT\_GEN2 | `string` | `"EXECUTION_ENVIRONMENT_GEN2"` | no | +| <a name="input_pre_deploy_job_force"></a> [pre\_deploy\_job\_force](#input\_pre\_deploy\_job\_force) | When true, and only when used in addition to var.pre\_deploy\_job\_trigger, the<br>pre-deploy Cloud Run job is executed at every terraform apply, regardless of<br># the status of var.pre\_deploy\_job\_container.image. | `bool` | `false` | no | +| <a name="input_pre_deploy_job_labels"></a> [pre\_deploy\_job\_labels](#input\_pre\_deploy\_job\_labels) | Map of key/value pairs containing labels to assign to the pre-deploy Cloud Run<br>job. | `map(string)` | `null` | no | +| <a name="input_pre_deploy_job_launch_stage"></a> [pre\_deploy\_job\_launch\_stage](#input\_pre\_deploy\_job\_launch\_stage) | The launch stage for the pre-deploy Cloud Run job. Possible values are UNIMPLEMENTED,<br>PRELAUNCH, EARLY\_ACCESS, ALPHA, BETA, GA, and DEPRECATED. | `string` | `"GA"` | no | +| <a name="input_pre_deploy_job_max_retries"></a> [pre\_deploy\_job\_max\_retries](#input\_pre\_deploy\_job\_max\_retries) | Configure the maximum number of retries for the pre-deploy job. | `number` | `null` | no | +| <a name="input_pre_deploy_job_mount_cloudsql_instance"></a> [pre\_deploy\_job\_mount\_cloudsql\_instance](#input\_pre\_deploy\_job\_mount\_cloudsql\_instance) | Mount a CloudSQL instance in the pre-deploy job container. This is a<br>convenience variable to simplify mounting a Cloud SQL instance. However, if<br>you require more control over this you should define it directly in<br>var.pre\_deploy\_job\_container instead. | `string` | `null` | no | +| <a name="input_pre_deploy_job_parallelism"></a> [pre\_deploy\_job\_parallelism](#input\_pre\_deploy\_job\_parallelism) | Specifies the maximum desired number of tasks the execution should run at<br>given time. | `number` | `null` | no | +| <a name="input_pre_deploy_job_task_count"></a> [pre\_deploy\_job\_task\_count](#input\_pre\_deploy\_job\_task\_count) | Specifies the desired number of tasks the execution should run. | `number` | `null` | no | +| <a name="input_pre_deploy_job_timeout"></a> [pre\_deploy\_job\_timeout](#input\_pre\_deploy\_job\_timeout) | Configure a timeout, in seconds, for the pre-deploy job. | `string` | `null` | no | +| <a name="input_pre_deploy_job_trigger"></a> [pre\_deploy\_job\_trigger](#input\_pre\_deploy\_job\_trigger) | When true, the pre-deploy Cloud Run job is executed via a<br>null\_resource-triggered gcloud command whenever Terraform detects that<br>var.pre\_deploy\_job\_container.image has changed. | `bool` | `true` | no | +| <a name="input_pre_deploy_job_volumes"></a> [pre\_deploy\_job\_volumes](#input\_pre\_deploy\_job\_volumes) | Configure one or more volumes for the pre-deploy job. See<br>https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/cloud_run_v2_job#nested_volumes<br>for more information on these options. | <pre>list(object({<br> name = string<br> secret = optional(object({<br> secret = string<br> default_mode = optional(number)<br> items = optional(list(object({<br> path = string<br> version = optional(string)<br> mode = optional(number)<br> })), [])<br> }))<br> cloud_sql_instance = optional(object({<br> instances = optional(list(string))<br> }))<br> }))</pre> | `[]` | no | +| <a name="input_pre_deploy_job_vpc_access"></a> [pre\_deploy\_job\_vpc\_access](#input\_pre\_deploy\_job\_vpc\_access) | Configure VPC access for the pre-deploy job. See<br>https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/cloud_run_v2_job#nested_vpc_access<br>for more information on these options. | <pre>object({<br> connector = optional(string)<br> egress = optional(string)<br> network_interfaces = optional(object({<br> network = optional(string)<br> subnetwork = optional(string)<br> tags = optional(string)<br> }))<br> })</pre> | `null` | no | +| <a name="input_project"></a> [project](#input\_project) | Project containing the webapp. | `string` | n/a | yes | +| <a name="input_region"></a> [region](#input\_region) | Location used to create Cloud Run service and other resources. | `string` | n/a | yes | +| <a name="input_revision"></a> [revision](#input\_revision) | The unique name for the revision. If this field is omitted, it will be<br>automatically generated based on the Service name. | `string` | `null` | no | +| <a name="input_scaling"></a> [scaling](#input\_scaling) | The minimum number of auto-scaled instances defaults to 0, thus, the container<br>will stop if it doesn't receive requests for a period of time and the<br>following request will make the container start from cold. This should be<br>carefully considered for containers that take a significant amount of time<br>starting from cold.<br><br>For a container with N-workers, the maximum number of auto-scaled instances<br>should be less than 1/N of the maximum connection count for the Cloud SQL<br>instance. | <pre>object({<br> min_instance_count = optional(number)<br> max_instance_count = optional(number)<br> })</pre> | `null` | no | +| <a name="input_service_account_display_name"></a> [service\_account\_display\_name](#input\_service\_account\_display\_name) | If non-empty, override the default display name of the webapp service account. | `string` | `""` | no | +| <a name="input_service_account_id"></a> [service\_account\_id](#input\_service\_account\_id) | A service account is always created for the web application. If non-empty this<br>variable overrides the default service account id. The default id is formed<br>from the "name" variable value with "-run" appended. | `string` | `""` | no | +| <a name="input_service_annotations"></a> [service\_annotations](#input\_service\_annotations) | Map containing additional annotations to be added to the Cloud Run service<br>itself. | `map(string)` | `{}` | no | +| <a name="input_service_labels"></a> [service\_labels](#input\_service\_labels) | A set of key/value label pairs to assign to the Cloud Run service. | `map(string)` | `{}` | no | +| <a name="input_session_affinity"></a> [session\_affinity](#input\_session\_affinity) | Enables session affinity. For more information, go to<br>https://cloud.google.com/run/docs/configuring/session-affinity. | `bool` | `null` | no | +| <a name="input_sql_instance_project"></a> [sql\_instance\_project](#input\_sql\_instance\_project) | Project containing SQL instance. Defaults to var.project. | `string` | `null` | no | +| <a name="input_ssl_certificates"></a> [ssl\_certificates](#input\_ssl\_certificates) | A list of self-links to any custom TLS certificates to add to the load<br>balancer. Requires that var.use\_load\_balancer be "true". The self-link is<br>available as the "self\_link" attribute of "google\_compute\_ssl\_certificate"<br>resources. | `list(any)` | `[]` | no | +| <a name="input_ssl_policy"></a> [ssl\_policy](#input\_ssl\_policy) | By default, the google\_compute\_ssl\_policy.default SSL policy is applied to the<br>load balancer in load\_balancer.tf. This sets the SSL profile to MODERN and<br>restricts TLS to >= 1.2. If a different SSL policy is required, it should be<br>created outside of this module and its ID passed through using this variable. | `string` | `null` | no | +| <a name="input_static_egress_ip_cidr_range"></a> [static\_egress\_ip\_cidr\_range](#input\_static\_egress\_ip\_cidr\_range) | The cidr range used to create a subnet that this cloud run will use if assigned<br>a static ip | `string` | `"10.124.0.0/28"` | no | +| <a name="input_static_egress_ip_subnetwork_id"></a> [static\_egress\_ip\_subnetwork\_id](#input\_static\_egress\_ip\_subnetwork\_id) | When using an existing VPC Access Connector with the static egress IP<br>configuration an existing subnetwork must be provided. | `string` | `null` | no | +| <a name="input_template_annotations"></a> [template\_annotations](#input\_template\_annotations) | Map containing additional annotations to be added to the Cloud Run service<br>template. | `map(string)` | `{}` | no | +| <a name="input_template_labels"></a> [template\_labels](#input\_template\_labels) | A set of key/value label pairs to assign to the Cloud Run service revision. | `map(string)` | `{}` | no | +| <a name="input_timeout_seconds"></a> [timeout\_seconds](#input\_timeout\_seconds) | The maximum duration, in seconds, the instance is allowed for responding to a<br>request. Maximum is 900s. | `string` | `"300s"` | no | +| <a name="input_traffic"></a> [traffic](#input\_traffic) | Configure traffic allocation between one or more service revisions. | <pre>list(object({<br> type = optional(string)<br> revision = optional(string)<br> percent = optional(number)<br> tag = optional(string)<br> }))</pre> | `[]` | no | +| <a name="input_use_ssl_certificates"></a> [use\_ssl\_certificates](#input\_use\_ssl\_certificates) | Whether to use the custom TLS certs in var.ssl\_certificates for the load<br>balancer or the Google-managed certs for the specified var.dns\_names. | `bool` | `false` | no | +| <a name="input_volumes"></a> [volumes](#input\_volumes) | Configure one or more volumes for the service. See<br>https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/cloud_run_v2_service#nested_volumes<br>for more information on these options. | <pre>list(object({<br> name = string<br> secret = optional(object({<br> secret = string<br> default_mode = optional(number)<br> items = optional(list(object({<br> path = string<br> version = optional(string)<br> mode = optional(number)<br> })), [])<br> }))<br> cloud_sql_instance = optional(object({<br> instances = optional(list(string))<br> }))<br> }))</pre> | `[]` | no | +| <a name="input_vpc_access"></a> [vpc\_access](#input\_vpc\_access) | Configure VPC access for the Cloud Run service. For more information on these<br>options see<br>https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/cloud_run_v2_service#nested_vpc_access | <pre>object({<br> connector = optional(string)<br> egress = optional(string)<br> network_interfaces = optional(object({<br> network = optional(string)<br> subnetwork = optional(string)<br> tags = optional(string)<br> }))<br> })</pre> | `null` | no | +| <a name="input_vpc_access_connector_max_throughput"></a> [vpc\_access\_connector\_max\_throughput](#input\_vpc\_access\_connector\_max\_throughput) | Optional. The maximum throughput of the connector in megabytes per second.<br>Defaults to 300. | `number` | `300` | no | + +## Outputs + +| Name | Description | +|------|-------------| +| <a name="output_load_balancer"></a> [load\_balancer](#output\_load\_balancer) | Load balancer for the webapp | +| <a name="output_network_endpoint_group"></a> [network\_endpoint\_group](#output\_network\_endpoint\_group) | Network endpoint group for the load balancer. | +| <a name="output_service"></a> [service](#output\_service) | Webapp Cloud Run service resource | +| <a name="output_service_account"></a> [service\_account](#output\_service\_account) | Service account which service runs as | +| <a name="output_ssl_policy"></a> [ssl\_policy](#output\_ssl\_policy) | The ssl\_policy object, if one is being created. | +| <a name="output_static_egress_ip"></a> [static\_egress\_ip](#output\_static\_egress\_ip) | The static egress IP assigned to this cloud run instance. Only populated<br>if the variable `enable_static_egress_ip` is true. | +<!-- END_TF_DOCS --> diff --git a/docker-compose.yml b/docker-compose.yml index 03e5e1d71bd20e52b66524709d2b4d65ba2814fd..8aed5c3f558e9d96173ffb4e0cdecd87250c8a9b 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -11,3 +11,9 @@ services: volumes: - .:/workdir:rw - ~/.config/gcloud/application_default_credentials.json:/root/.config/gcloud/application_default_credentials.json:ro + terraform-docs: + image: quay.io/terraform-docs/terraform-docs:0.17.0 + entrypoint: ["."] + working_dir: /workdir + volumes: + - .:/workdir:rw diff --git a/docs/examples.md b/docs/examples.md new file mode 100644 index 0000000000000000000000000000000000000000..60e75de4afe38456870fbccbc7fb60e9fee3dcf9 --- /dev/null +++ b/docs/examples.md @@ -0,0 +1,345 @@ +# Example Usage + +This page contains some examples of the different ways this module can be +configured. + +## Basic + +A basic Cloud Run service with a single container definition. + +```hcl +module "webapp" { + source = "gitlab.developers.cam.ac.uk/uis/gcp-cloud-run-app/devops" + version = "~> 9.0" + + region = "europe-west2" + project = "example-project-id-1234" + + containers = { + webapp = { + image = "us-docker.pkg.dev/cloudrun/container/hello" + } + } +} +``` + +## Load balancer + +A basic Cloud Run service configured to use a load balancer for ingress. + +```hcl +module "webapp" { + source = "gitlab.developers.cam.ac.uk/uis/gcp-cloud-run-app/devops" + version = "~> 9.0" + + region = "europe-west2" + project = "example-project-id-1234" + + containers = { + webapp = { + image = "us-docker.pkg.dev/cloudrun/container/hello" + } + } + + enable_load_balancer = true + + dns_names = { + webapp = "webapp.test.example.com" + } +} + +resource "google_dns_record_set" "load_balancer_webapp" { + name = "webapp.test.example.com." + type = "A" + ttl = 300 + managed_zone = "example-zone" + project = "example-project-id-1234" + + rrdatas = [ + module.webapp.load_balancer.external_ip + ] +} +``` + +## Secret environment variables and volumes + +A Cloud Run service configured to load environment variables and mount volumes +via Google Secret Manager secret objects. + +Note that you need to grant the created service account identity the +ability to access the secret objects _outside_ of this module call. + +```hcl +resource "google_secret_manager_secret" "main" { + secret_id = "my-secret" + project = "example-project-id-1234" + + replication { + auto {} + } +} + +resource "google_secret_manager_secret_version" "main" { + secret = google_secret_manager_secret.main.id + secret_data = "my-secret-data" +} + +resource "google_secret_manager_secret_iam_member" "main" { + project = "example-project-id-1234" + secret_id = google_secret_manager_secret.main.id + role = "roles/secretmanager.secretAccessor" + member = "serviceAccount:${module.webapp.service_account.email}" +} + +module "webapp" { + source = "gitlab.developers.cam.ac.uk/uis/gcp-cloud-run-app/devops" + version = "~> 9.0" + + region = "europe-west2" + project = "example-project-id-1234" + + containers = { + webapp = { + image = "us-docker.pkg.dev/cloudrun/container/hello" + env = [ + { + name = "SECRET", + value_source = { + secret_key_ref = { + secret = google_secret_manager_secret.main.id + version = "latest" + } + } + } + ] + volume_mounts = [ + { + name = "secret-volume", + mount_path = "/secrets" + } + ] + } + } + volumes = [ + { + name = "secret-volume", + secret = { + secret = google_secret_manager_secret.main.id + items = [ + { + version = "latest", + path = "my-secret" + } + ] + } + } + ] +} +``` + +## Mounting CloudSQL instances + +A Cloud Run service which mounts an existing CloudSQL instance using the +`mount_cloudsql_instance` helper variable. + +```hcl +module "webapp" { + source = "gitlab.developers.cam.ac.uk/uis/gcp-cloud-run-app/devops" + version = "~> 9.0" + + region = "europe-west2" + project = "example-project-id-1234" + + containers = { + webapp = { + image = "us-docker.pkg.dev/cloudrun/container/hello" + } + } + + mount_cloudsql_instance = module.sql.instance_connection_name +} + +module "sql" { + source = "GoogleCloudPlatform/sql-db/google//modules/postgresql" + version = "~> 17.0" + + database_version = "POSTGRES_15" + name = "test-sql-1234" + project_id = "example-project-id-1234" + tier = "db-f1-micro" + availability_type = "ZONAL" + region = "europe-west2" + zone = "europe-west2-a" + deletion_protection = false + deletion_protection_enabled = false +} +``` + +## Pre-deploy job + +A Cloud Run service with a corresponding "pre-deploy" Cloud Run job. See the +[Pre-deploy Cloud Run Job](../README.md#pre-deploy-cloud-run-job) section in the +README.md for more information. + +```hcl +module "webapp" { + source = "gitlab.developers.cam.ac.uk/uis/gcp-cloud-run-app/devops" + version = "~> 9.0" + + region = "europe-west2" + project = "example-project-id-1234" + + containers = { + webapp = { + image = "registry.gitlab.developers.cam.ac.uk/uis/devops/infra/dockerimages/django:5.0-py3.12" + } + } + + mount_cloudsql_instance = module.sql.instance_connection_name + + enable_pre_deploy_job = true + + pre_deploy_job_container = { + image = "registry.gitlab.developers.cam.ac.uk/uis/devops/infra/dockerimages/django:5.0-py3.12" + command = ["python3"] + args = ["/usr/src/app/manage.py", "migrate"] + } + + pre_deploy_job_mount_cloudsql_instance = module.sql.instance_connection_name +} + +module "sql" { + source = "GoogleCloudPlatform/sql-db/google//modules/postgresql" + version = "~> 17.0" + + database_version = "POSTGRES_15" + name = "test-sql-1234" + project_id = "example-project-id-1234" + tier = "db-f1-micro" + availability_type = "ZONAL" + region = "europe-west2" + zone = "europe-west2-a" + deletion_protection = false + deletion_protection_enabled = false +} +``` + +## Multi-container deployment + +A Cloud Run service which defines multiple containers (sidecars). For more +information see the [Cloud Run +documentation](https://cloud.google.com/run/docs/deploying#sidecars). + +```hcl +module "webapp" { + source = "gitlab.developers.cam.ac.uk/uis/gcp-cloud-run-app/devops" + version = "~> 9.0" + + region = "europe-west2" + project = "example-test-b99f7ad6" + + containers = { + webapp1 = { + name = "webapp-1" + image = "us-docker.pkg.dev/cloudrun/container/hello" + ports = [ + { + container_port = 8080 + } + ] + } + webapp2 = { + name = "webapp-2" + image = "us-docker.pkg.dev/cloudrun/container/hello" + } + } +} +``` + +## Canary release traffic distribution + +A Cloud Run service which allocates incoming traffic equally between two +revisions. + +This example uses the `revision` variable to deploy named revisions of the Cloud +Run service. This allows you to target these named revisions specifically to +split traffic between one or more revisions via the `traffic` variable. + +```hcl +module "webapp" { + source = "gitlab.developers.cam.ac.uk/uis/gcp-cloud-run-app/devops" + version = "~> 9.0" + + region = "europe-west2" + project = "example-project-id-1234" + + revision = "v1-1-0" + + containers = { + webapp = { + image = "us-docker.pkg.dev/cloudrun/container/hello" + } + } + + traffic = [ + { + type = "TRAFFIC_TARGET_ALLOCATION_TYPE_REVISION" + revision = "v1-0-0" + percent = 50 + }, + { + type = "TRAFFIC_TARGET_ALLOCATION_TYPE_REVISION" + revision = "v1-1-0" + percent = 50 + } + ] +} +``` + +## Static egress IP configuration + +A Cloud Run service configured with a static IP address for egress. See the +[Static Outbound IP +Address](https://cloud.google.com/run/docs/configuring/static-outbound-ip) page +in the Cloud Run documentation for details of this implementation. + +The address is available in the `static_egress_ip` output of this module. + +```hcl +module "webapp" { + source = "gitlab.developers.cam.ac.uk/uis/gcp-cloud-run-app/devops" + version = "~> 9.0" + + enable_static_egress_ip = true + region = "europe-west2" + project = "example-test-b99f7ad6" + + containers = { + webapp = { + image = "us-docker.pkg.dev/cloudrun/container/hello" + } + } +} +``` + +## Uptime and SSL Monitoring + +A basic Cloud Run service with default monitoring enabled. + +```hcl +module "webapp" { + source = "gitlab.developers.cam.ac.uk/uis/gcp-cloud-run-app/devops" + version = "~> 9.0" + + region = "europe-west2" + project = "example-project-id-1234" + + enable_monitoring = true + + containers = { + webapp = { + image = "us-docker.pkg.dev/cloudrun/container/hello" + } + } +} +``` diff --git a/docs/pre-deploy-job.md b/docs/pre-deploy-job.md new file mode 100644 index 0000000000000000000000000000000000000000..9e618883c6b7c359fc974d46430457a509d2f32e --- /dev/null +++ b/docs/pre-deploy-job.md @@ -0,0 +1,32 @@ +# Pre-deploy Cloud Run job + +The `8.0.0` release introduced the `enable_pre_deploy_job` variable which, when +set to `true`, creates a Cloud Run job to execute a configurable command before +new Cloud Run service revisions are deployed. This is a useful way to run +database migrations and other commands which are tightly coupled to the release +cadence of the main Cloud Run service. + +The pre-deploy job is configured via the `pre_deploy_job_*` variables which can +be found in `variables.tf`. + +## Triggering the Cloud Run job + +The Cloud Run job is executed by a `null_resource` resource which simply runs +the `gcloud run jobs execute` command. The `null_resource` is triggered each +time the `var.pre_deploy_job_container.image` value changes by default, although +you can force it to run via the `pre_deploy_job_force` variable. + +## Order of operations + +To ensure that the pre-deploy job runs before a new revision of the Cloud Run +service is deployed, the resources in question are explicitly configured with +`depends_on` relationships as follows. + +- The `google_cloud_run_v2_job.pre_deploy` Cloud Run job has no `depends_on` + relationships defined and is therefore deployed first. +- The `null_resource.pre_deploy_job_trigger` resource depends on + `google_cloud_run_v2_job.pre_deploy` and therefore won't be deployed until the + Cloud Run job is deployed successfully. +- Finally, the `google_cloud_run_service.webapp` Cloud Run service depends on + `null_resource.pre_deploy_job_trigger`, meaning it is only deployed once the + `null_resource.pre_deploy_job_trigger` has executed successfully. diff --git a/docs/templates/footer.md b/docs/templates/footer.md new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/docs/templates/header.md b/docs/templates/header.md new file mode 100644 index 0000000000000000000000000000000000000000..13f31184f5ff33fb4f39baae522865ff1fab1d4c --- /dev/null +++ b/docs/templates/header.md @@ -0,0 +1,40 @@ +# GCP Cloud Run Terraform Module + +This module manages the deployment of containerised applications on Cloud Run. +It includes the following functionality: + +- Creation of the main Cloud Run service. +- Creation of a dedicated service account with required IAM bindings. +- Deployment of an optional load balancer configuration. +- Deployment of an optional "pre-deployment" Cloud Run job to handle tasks such + as database migrations. +- Deployment of an optional static egress IP address for the Cloud Run service. +- Configuration of simple uptime checks and SSL certificate expiry alerts. +- Convenience variables to configure certain aspects of the Cloud Run service + which would otherwise be complex, such as mounting Cloud SQL instances. + +## Cloud Run v2 API + +Following release `9.0.0`, this module has been refactored to use the Cloud Run +`v2` API resources exclusively (i.e. `google_cloud_run_v2_service`). This means +that many input variables are now different and updating from a previous version +will require some changes to your module definition. + +## Examples + +See the [docs/examples.md](docs/examples.md) page for a full list of detailed +usage examples. + +## Pre-deploy Cloud Run Job + +The `8.0.0` release introduced the `enable_pre_deploy_job` variable which, when +set to `true`, creates a Cloud Run job to execute a configurable command before +new Cloud Run service revisions are deployed. This is a useful way to run +database migrations and other commands which are tightly coupled to the release +cadence of the main Cloud Run service. + +The pre-deploy job is configured via the `pre_deploy_job_*` variables which can +be found in `variables.tf`. + +For more information on how the pre-deploy Cloud Run job works see the +[pre-deploy-job.md](../docs/pre-deploy-job.md) page. diff --git a/docs/testing.md b/docs/testing.md new file mode 100644 index 0000000000000000000000000000000000000000..a0b865dbd096b3851490e189978d215f46fbb9c0 --- /dev/null +++ b/docs/testing.md @@ -0,0 +1,117 @@ +# Testing + +This project makes use of Terraform's built-in `test` command to run integration +tests. The tests are configured in the `tests` directory and deploy resources to +our dedicated Infra Testing GCP project. + +## Running tests locally + +To run tests locally you can use the `run_tests.sh` helper script. By default +the script will execute _all_ tests, however, it is often useful to target a +specific test file using the `-t` option. For example, to run the +`cloud_run_service.tftest.hcl` test you would use the following command. + +```bash +# You must have authenticated and set the application-default credentials. +gcloud auth application-default login + +./run_tests.sh -t tests/cloud_run_service.tftest.hcl +``` + +Note that the `test` service defined in `docker-compose.yml` sets the +`GOOGLE_IMPERSONATE_SERVICE_ACCOUNT` variable. You must have permission to +impersonate this service account to be able to run these tests. + +## GitLab CI/CD test jobs + +The `tests` job in the `.gitlab-ci.yml` file is configured to run all of the +test files in the `tests` directory. The tests are run in parallel using a +`matrix` job. Variables are used to allow us to specify version constraints to +test against multiple supported major versions of the Google Terraform provider. + +The `tests` job is configured to require a manual trigger. This is due to the +number of resources that the jobs will deploy and the length of time the jobs +take to complete. With this in mind, you should generally only need to run the +tests job at the point you open a merge request, ensuring the job is passing +before requesting a review. + +## Resource teardown and cleanup + +If a test job fails Terraform attempts to teardown any resources it has already +created. This seems to work well the majority of the time. However, to protect +against resources not being destroyed, and potentially costing £££, there are +two `cleanup` jobs configured in the `.gitlab-ci.yml` file, `pre-cleanup` and +`post-cleanup`. These jobs both run the `tests/cleanup.sh` script which is +configured to check for any resources that _could_ have been created and delete +any that it finds. We run the `pre-cleanup` job to ensure that there is a clean +environment prior to the current test run, avoiding any subnet clashes etc. + +It's also fine to run the `tests/cleanup.sh` script from your local machine to +perform an ad-hoc cleanup. First authenticate your `gcloud` session and then +simply run the script, for example: + +```bash +gcloud auth login + +./tests/cleanup.sh +``` + +## Troubleshooting + +### Google's eventually consistent APIs + +Many of Google's APIs are eventually consistent. This often causes issues as IAM +bindings and API enablement can be delayed causing our Terraform to fail. +Unfortunately, this is simply unavoidable and the only workaround is to rerun +the failed job. + +#### Error 403: Permission 'iam.serviceaccounts.actAs' denied on service account + +The following error is an example of the eventual consistency issue. If you're +unlucky enough to see this failure you should simply retry the job as often it +just works the second time. + +```bash +tests/cloud_run_service.tftest.hcl... in progress + run "setup"... pass + run "test_service_with_default_variable_values"... fail +╷ +│ Error: Error creating Service: googleapi: Error 403: Permission +│ 'iam.serviceaccounts.actAs' denied on service account +│ test-fab59940-run@infra-testing-int-e2395220.iam.gserviceaccount.com +│ (or it may not exist). +│ +│ with google_cloud_run_v2_service.webapp, +│ on main.tf line 23, in resource "google_cloud_run_v2_service" "webapp": +│ 23: resource "google_cloud_run_v2_service" "webapp" { +│ +╵ +``` + +### Invalid IPCidrRange: 10.124.0.0/28 conflicts with existing subnetwork + +This error usually means a previous test run failed to tear down its resources +correctly so the subnet range is already in use. You should investigate the +previous test runs and destroy all orphaned resources before rerunning the +failed job. + +```bash +tests/monitoring.tftest.hcl... in progress + run "setup"... pass + run "test_monitoring_with_alert_policies_created_in_default_project"... pass + run "test_monitoring_with_alert_policies_created_in_scoping_project"... pass + run "test_monitoring_with_auth_proxy"... pass + run "test_monitoring_with_auth_proxy_and_vpc_access_connector"... fail +╷ +│ Error: Error waiting to create Subnetwork: Error waiting for Creating +│ Subnetwork: Invalid IPCidrRange: 10.124.0.0/28 conflicts with existing +│ subnetwork 'test-209abb96-vpc-connector' in region 'europe-west2'. +│ +│ +│ with google_compute_subnetwork.vpc_connector[0], +│ on static_egress_ip.tf line 6, in resource "google_compute_subnetwork" +│ "vpc_connector": +│ 6: resource "google_compute_subnetwork" "vpc_connector" { +│ +╵ +```