Make a page which lists the current tech stack used by new apps

Create a page which explicitly lists technologies used by DevOps applications if they're being written tomorrow.

This comment in the Tech Lead private Teams channel is about having diagrams for the boilerplate arch so that we (and others) can reference them in TDA papers to describe where they do / don't deviate from the boilerplate. Part of the 'paved path' approach.

Said comment being:

I'm confident that it doesn't need anything more than boilerplate + SendGrid + MS Entra ID, so the diagram will be in the 2b paper.. This has raised that we don't provide a diagram for the boilerplate for people to reference, so I will raise an issue to create one.

There are a variety of diagrams on https://guidebook.devops.uis.cam.ac.uk/explanations/service-infrastructure/. Could we refine this issue a little?

• What granularity are we wanting? Using C4 terminology, are we talking container, component or system context diagrams?
• What do we include and what do we omit? For example, the diagrams linked above cover the deployment infrastructure and permissions model but don't include components like a Cloud Run service or Cloud DNS record. Should they be on this diagram as well? Or just be referenced in text?

As a strawman, would a diagram with the following suffice? (This list is off the top of my head.)

• Project layout comprising:
  • Meta project
  • Per-environment project
• Cloud Load Balancer, comprising:
  • HTTPS frontend
  • HTTP -> HTTPS redirect frontend
  • TLS certificate
  • TLS policy
  • Serverless NEG backend
  • Dedicated ingress external IP
• Cloud DNS records
• VPC, comprising
  • Region-specific subnetworing
  • (if used) dedicated egress VPC network
  • (if used) dedicated egress external IP
• Cloud Run service, comprising
  • Cloud SQL connector (Google provisioned)
  • (If used) Direct VPC connection
• Cloud SQL instance
• Cloud SQL backup process, comprising
  • Cloud SQL backup bucket
• (if used) Cloud Storage for application
• Cloud Storage used for infrastructure configuration. (a.k.a. the "config bucket")
• Cloud Monitoring comprising:
  • alert policies
  • notification channels
• Cloud IAM policies for all the above

I think that might be a bit much for a single diagram so how about:

• A system context diagram showing the meta and per-environment projects alongside the various X-factories and the config bucket
• A container diagram showing a single environment project with Cloud SQL, load balancer, VPC and Cloud Run service
• A component diagram for a Cloud Run service showing the container, Cloud SQL connection and (if used) direct VPC access and egress IP
• A component diagram for the Cloud SQL instance showing the instance, backup process and backup bucket
• A component diagram for the load balancer showing the ingress external IP, frontends, backend, TLS policy, TLS certificate and DNS records
• A container diagram showing a single environment project and alerting policies

Leaving an IAM diagram to one side for the moment.

@bs461 - is that the sort of thing you'd be looking for?

I was just imagining a diagram with the following:

• Cloud run service, illustrating that containers can scale up and down.
• SQL instance
• Load balancer + DNS
• (User accessing UI, if helpful in illustrating the entry point)

I don't think details of backups, monitoring, or specifics of how the different components are wired together is needed. Just something high-level so that people can see the entry points and major components. Deployment details such as meta project and environments also don't need to be included, I don't think.

Having the rest diagrammed would be great, but in the specific context of supporting TDA papers, I think the level of detail needed is much lower, and the goal is just to allow people with no understanding of GCP to get that there is a thing which does the computing and another thing which does the SQL, and that's basically it. Surrounding words could clarify that the API Gateway and SendGrid could be added if needed, but they aren't in the boilerplate.

OK, so this would be a component diagram in the C4 sense containing the Cloud Run service, SQL, load balancer and DNS records with an external user accessing the service.

Sounds simple enough. Would you like me to Just Do It (TM)?

It'd also be useful to have specific guidance on frontend tech, dependent on the type of application. @rjg21 and @ek599 out of your work on the activate account frontend it'd be useful to get your thoughts here.

We have fairly clear-cut options for backend tech - either Django for anything large or expected to manage many different entities or expose a basic frontend, or FastAPI for anything smaller or which either acts as a proxy API / doesn't directly manage multiple data entities. Although this is just my understanding and needs to be agreed and documented, but it'd be nice to have the same discussion and agreement on the approach for the frontend.

It'd also be useful to have specific guidance on frontend tech, dependent on the type of application.

We have fairly clear-cut options for backend tech - either Django for anything large or expected to manage many different entities or expose a basic frontend, or FastAPI for anything smaller or which either acts as a proxy API / doesn't directly manage multiple data entities. Although this is just my understanding and needs to be agreed and documented, but it'd be nice to have the same discussion and agreement on the approach for the frontend.

This is all very Python ecosystem specific. Do we have a well-formed opinion on Java backends?

Do we have a well-formed opinion on Java backends?

Pinging @snr21 as possibly having opinions in that area.

Do we have a well-formed opinion on Java backends?

AIUI quarkus is the preferred framework for new Java web-applications.

That said, I think we need to consider whether we want to consolidate around Python as our preferred language for backends, especially for greenfield projects. This is the approach we're taking within the Holberton team, we maintain a lot of Java services, but anything new will be developed from either of the Python boilerplates. We gain a lot by trying to shrink the estate of technologies we support and investing in common tooling around our agreed backend and frontend approaches.

That said, I think we need to consider whether we want to consolidate around Python as our preferred language for backends, especially for greenfield projects.

I'm happy for a decision to be made around that elsewhere. While we do have Java-ecosystem products, however, we should remember to be explicit that, e.g., choice of a frontend tech doesn't necessarily mandate a backend one but by all means call out where we have mature solutions for both.

Yes, Quarkus is the preferred back-end tech for Java microservices.

The discussion or decision about Java vs Python belongs elsewhere. Realistically @wgd23 the portfolio you've inherited isn't exactly state-of-the-art and shouldn't be a factor in determining ecosystems for new products.

The discussion or decision about Java vs Python belongs elsewhere.

- I wanted to make sure we don't accidentally pickle assumptions on said in our docs.

assigned to @rjw57

changed due date to October 03, 2024

changed due date to October 09, 2024

added workflowIn Progress label

mentioned in commit 4f71edbf

mentioned in merge request !370 (merged)

@bs461 is https://uis.uniofcam.dev/-/devops/docs/guidebook/-/jobs/1782258/artifacts/public/reference/standard-architecture-diagrams.html the sort of thing you had in mind? It's the result of !370 (merged) which would benefit from someone like you who knows what we need here reviewing. Thanks.

added workflowReview Required label and removed workflowIn Progress label

mentioned in commit a55a72f6

mentioned in commit 2d67ecef

mentioned in issue #278

closed with merge request !370 (merged)

mentioned in commit b07689a5