Scheduled script run within Google Cloud

This terraform module configures a single-file Python script to be run regularly in Google Cloud using a cron-style scheduling syntax. It supports single-file scripts with requirements specified via a requirements.txt-style list of packages.

The scripts may have secrets passed to them by a Google Secret Manager secret created for this script.

A dedicated service account is created for the script which represents the identity which the script runs as within Google Cloud.

Monitoring is configured with alerts sent if the number of successful invocations of the script within a configurable time period falls below a given threshold.

This module is not suitable for multi-file scripts or scripts which need to be passed parameters which differ from run-to-run.

Versioning

The master branch contains the tip of development. Releases are made automatically using release automation.

Required APIs and services

The following APIs must be enabled to use this module:

• appengine.googleapis.com
• cloudbuild.googleapis.com
• cloudfunctions.googleapis.com
• cloudscheduler.googleapis.com
• secretmanager.googleapis.com

In addition there must be an app engine application configured for your project. This is a requirement to configure Cloud Scheduler jobs. You can ensure an app engine application is configured by means of the google_app_engine_application resource:

resource "google_app_engine_application" "app" {
  location_id = "europe-west2"
}

Monitoring

Cloud Monitoring resources are associated with a Monitoring Workspace which may be hosted in a project which differs from the project containing other resources. The google.monitoring provider is used when creating Monitoring resources and it should be a provider configured with credentials able to create monitoring resources.

Implementation

The script is implemented as a Cloud Function and so it should conform to the Cloud Function Python runtime. The entrypoint to the function can be customised but the default is to use main.

A dedicated Cloud Storage bucket is created to store the source code in. This is created with a random name.

Example

A minimal example of running a script:

module "scheduledscript" {
  source = "..."

  # Name used to form resource names and human readable description.
  name        = "my-script"
  description = "Log a message."

  # Project and region to create resources in.
  project = "..."
  region  = "europe-west2"

  # A directory which can be used to store local files which are not checked in
  # to source control.
  local_files_dir = "/terraform_data/local"

  # The script itself.
  script = <<-EOL
      import logging

      LOG = logging.getLogger()
      logging.basicConfig(level=logging.INFO)

      def main(request):
          LOG.info('I ran!')
          return '{ "status": "ok" }', {'Content-Type': 'application/json'}
  EOL

  # Cron-style schedule for running the job.
  schedule = "*/5 * * * *"

  # Alerting threshold. Specify the minimum number of successful invocations and
  # the period over which this should be measured.
  alert_success_threshold = 5
  alert_success_period    = "3600s"  # == 1hr

  # Email addresses to receive alerts if invocations are failing.
  alert_email_addresses = ["someone@example.com"]

  # Use a separate provider with rights over the Cloud Monitoring workspace for
  # monitoring resource management.
  providers = {
    google.monitoring = google.monitoring
  }
}

See the variables.tf file for all variables.

Additional Python package requirements can be specified in the requirements variable which should be a list of Python packages, one per line, as used by requirements.txt files.

A custom entry point for the script can be configured via the entry_point variable.

Configurations can be passed to the application via the secret_configuration variable. This should be a string which is placed in a Google Secret Manager secret and is made available to the script. The secret is passed as a URL of the form sm://[PROJECT]/[SECRET]#[VERSION] in the SECRET_CONFIGURATION_SOURCE environment variable.

Requirements

Inputs

object({
    retry_count          = optional(number, 1)
    max_retry_duration   = optional(string)
    min_backoff_duration = optional(string)
    max_backoff_duration = optional(string)
    max_doublings        = optional(number)
  })

object({
    account_id   = string
    display_name = optional(string)
    description  = optional(string)
    disabled     = optional(bool)
    project      = optional(string)
    id           = string
    email        = string
    name         = string
    unique_id    = string
    member       = string
  })

Outputs