Site Monitoring for Internal ingress failed to apply in CI pipeline

👓 What did you see?

When merging a change introducing site-monitoring with an internal ingress cloud run instance, the site monitoring failed to apply correctly for that site in the pipeline. It appeared to not find or not have access to the source files it expected. See job output: https://gitlab.developers.cam.ac.uk/uis/devops/iam/activate-account/infrastructure/-/jobs/1945695

And copied out in text:

module.internal_api.module.uptime_monitoring["default"].module.uptime_check_auth_proxy[0].google_storage_bucket_object.function_object: Creating...
╷
│ Error: open /teraform-data/functions//internal-uptime-source.zip: no such file or directory
│ 
│   with module.internal_api.module.uptime_monitoring["default"].module.uptime_check_auth_proxy[0].google_storage_bucket_object.function_object,
│   on /builds/uis/devops/iam/activate-account/infrastructure/terraform_data/modules/internal_api.uptime_monitoring.uptime_check_auth_proxy/main.tf line 67, in resource "google_storage_bucket_object" "function_object":
│   67: resource "google_storage_bucket_object" "function_object" {
│ 
╵
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: command terminated with exit code 1

✅ What did you expect to see?

Terraform apply CI pipeline behaves as expected.

💻 Where does this happen?

See link above, on standard DevOps CI pipeline jobs.

🔬 How do I recreate this?

• Create new infra project using standard DevOps templates, including pipeline to apply and project in GCP to apply to, or use existing test repository already setup (if one exists).
• Add API using gcp-cloud-run module
  • Set enable_monitoring to true
  • Set ingress to INGRESS_TRAFFIC_INTERNAL_ONLY
• Merge changes to main branch, and apply to GCP environment.

📚 Any additional information?

Eventually resolved this by applying directly from my local machine via logan. Job was retried multiple times in pipeline, so not a one-off.