Scheduled automatic Docker image builds

Description

In order to make update-to-date base images always available, setup automatic builds looseley in-sync with parent base image builds.

Tags should match current formats:

<semantic-version> - mutable
[inherent image sha]

Acceptance criteria

• images are rebuilt from updated base images each week
• semantic release tag is updated to latest base image tag (i.e. a mutable tag)

added issuetypeTask teamIdentity workflowNeeds Refinement labels

added meetingbacklog refinement label

added priority2 Medium label

changed the description

assigned to @st981

changed iteration to IAM Sprint Jul 17, 2024 - Jul 30, 2024

@st981 you and I can refine this together

added workflowSprint Ready label and removed workflowNeeds Refinement label

marking as sprint ready so it's on the board, but still needs refinement

@ek599 and @st981 have informally refined this ticket

removed iteration IAM Sprint Jul 17, 2024 - Jul 30, 2024

changed iteration to IAM Sprint Jul 31, 2024 - Aug 13, 2024

Decided the following requires wider devops discussion and shouldn't block having basic scheduled build (i.e. we're not doing this now):

In an ideal world, the images would be tagged with both the ucam-faas semantic release and an immutable tag that indicated the ucam-faas version and the parent base image version.

<semantic-version> - mutable
<semantic-version>-<base-image-sha> - immutable
[inherent image sha]

Acceptance criteria (not to be addressed in this ticket)

• clear documentation on which tag types are immutable and mutable

changed the description

removed meetingbacklog refinement label

added workflowIn Progress label and removed workflowSprint Ready label

There's one thing that still needs to be clarified:

When doing the weekly job, do we only want to rerelease for the latest tag that we created?

If so, then this means that in the weekly pipeline, we have to determine what the latest tag is and run the job for that. I think there are two options that we can go for:

• For each job that we have specified in our CI/CD pipeline we have to run a before_script that checks what the latest tag is and check the code out. Note that it only needs to do that when running the weekly scheduled pipeline. Possible issues are that this doesn't scale well: if a pipeline in a template gets updated and creates a new job, then we also need to override that in our pipeline.
• We create a new job which is the only job being run during the weekly pipeline that determines what the latest tag is and then this job uses a pipeline trigger token in a curl call to kick off a new pipeline for that specific revision. Possible issue is that the pipeline trigger token is bound to a user.

if a pipeline in a template gets updated and creates a new job, then we also need to override that in our pipeline.

Is this to stop it running more than weekly? I think we can live with that

MVP is just the head tags (so latest + last version) but a stretch goal might be a larger set

Is this to stop it running more than weekly? I think we can live with that

It's to ensure that the job will run on the revision we want to run it on. That's because when setting up a pipeline schedule, you have to select a target branch or tag. It's static, so you can't select "last tag" (which is currently 0.7.1). So the best choice would probably be to select "main". But "main" might contain code that's unreleased and not tagged (as is currently the case for example). So we have to overwrite each job in the pipeline to check out the revision we want (last tag). Forget what I said, I just found out that we can write a global before_script which resolves the biggest problem of the first option.

added workflowSprint Ready label and removed workflowIn Progress label