Allow to enable Workload Identity
A new recommended way of accessing secrets from Kubernetes Pods is to use the Kubernetes Secrets Store CSI Driver. This driver allows a Kubernetes workload to access a Google Secret Manager vault (or other enterprise grade secret managers in AWS or Azure).
For Google Secret Manager you need to bind a Kubernetes Service Account (KSA) to a Google Service Account (GSA) using GCP IAM. The Pods running as this KSA can then directly access the secrets stored the Google Secret Manager via the Kubernetes Secrets Store CSI Driver. This improves security as the secrets are only stored Google Secret Manager (superior to Kubernetes Secrets) and can be rotated by just updating those secret entries. Also avoids Terraform knowing about them (only when you have to upload them to Google Secret Manager - more about this on another issue) which then avoids the secret touching the Terraform State.
In order to bind a Kubernetes Service Account (KSA) to a Google Service Account (GSA), we need to enable Workload Identity in our GKE clusters and Pools and thus we need to modify the gke-cluster module to support this.
This can be enabled to existing clusters following these instructions: https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity#enable_on_cluster
This is also the recommended way of executing Cloud SQL Proxy now as well.