Explain our approach to secrets management
This issue captures discssions from https://gitlab.developers.cam.ac.uk/uis/devops/digital-admissions/albatross/albatross-deployment/-/issues/18#note_585200 along with private discussions with @rh841 around secrets management.
Although we have some reference documentation on secret-based configuration foir Django libraries we lack a more generic explainer which is language and framework neutral. As noted in uis/devops/tools/logan#28 (comment 586320), the reference documentation itself would benefit from using our gcp-secret-manager
module.
The issue is to create an explainer which covers the following topics:
- What are "secrets" and what is "configuration"
- The different types of secret:
- External (1password)
- Generated (terraform)
- Ways of passing secrets:
- Deployed services
- Local development
- GitLab CI jobs
- Passing external secrets in deployments
- sanctuary exists and is intended to be a poor-man's
data "1password_secret"
resource. - Outline down-sides of using sanctuary to manage secrets mounted directly in applications:
- Unclear when applications will re-read them (launch time versus every use is up to applications)
- If one uses "latest", applications do not re-launch
- If one specifies version, applications re-launch but may break since sanctuary deletes older versions.
- sanctuary exists and is intended to be a poor-man's
- Passing internal secrets in deplotments
- Not the same issues as sanctuary: should always pass with explicit version number
- Having a single canonical place secrets are stored:
- For manually maintained secrets, this is 1password copied into GCP
- For generated secrets, this is GCP
- Prefer generated secrets over copied ones.
- Outline and justify a preferred approach:
- Downsides of environment variables: one secret per environment variable, "leakiness"
- Prefer mounting secrets as files. Allow configuration pointing to them.
- Environment variables are OK if technically required.
- History of our current Django-based approach
- Recommendations for non-Django applications
- Include case studies of Spring Boot, Django and FastAPI-based services.