Synchronise Lookup groups

Along with #7 (closed), we'd like to support full Lookup group integration. Ideally we'd namespace the created Google Groups so we'd have {groupid}@lookup.groups.g.apps.cam.ac.uk as the group name.

added to epic &1 (closed)

changed the description

added 1 deleted label

changed milestone to %Sprint 90

changed milestone to %Sprint 91

changed milestone to %Sprint 92

changed milestone to %Sprint 93

added Development Operations labels

changed time estimate to 4d

added workflowSprint Ready label and removed 1 deleted label

changed milestone to %Sprint 94

changed milestone to %Sprint 95

changed milestone to %Sprint 96

removed milestone

changed milestone to %Sprint 100

assigned to @dar17

Moving to ~"workflow::doing" as @dar17 started to look at this

To expand on the description, as this is a bit short. This is needed based on the request we have got from several users. Groups can be used to share files, folders, drives, docs, sheets, youtube videos, etc in G Suite. Lookup groups are already been used in many applications to share things across many people so it is necessary to have the same groups in G Suite, so that users can share resources with the same people.

This will also generate a single source of truth for groups and ensuring that if you update lookup group membership, this will also get updated in G Suite, avoiding having to update permissions in many places.

added workflowIn Progress label and removed workflowSprint Ready label

changed milestone to %Sprint 101

mentioned in commit 135cc63c

mentioned in commit ad217b6c

mentioned in commit defee11b

I have checked and Google search boxes search on both, the title and the the address of the group.

@dar17 looking at this issue that Rich also opened: #7 (closed) I think it makes sense to use the namespace that rich was proposing, I think it will be tidier if end up having: {instid}@institutions.groups.g.apps.cam.ac.uk and {groupid}@lookup.groups.g.apps.cam.ac.uk.

Do you agree?

So like this: https://admin.google.com/u/1/ac/groups ? Not having {groupName} in the name makes it quite hard to find a group.

Sorry I didn't it explain well. I agree with you that the name of the group should be {groupName}. I think the group (email) address could be the ones proposed by Rich {instid}@institutions.groups.g.apps.cam.ac.uk and {groupid}@lookup.groups.g.apps.cam.ac.uk. Although Rich didn't explain in the issue if he was proposing to add institutions.groups.g.apps.cam.ac.uk and lookup.groups.g.apps.cam.ac.uk as domain aliases or secondary domains, I think he would have proposed them as secondary domains, as we currently have groups.g.apps.cam.ac.uk as a secondary domain in the main G Suite account.

You're now making me wonder if it should've been {id}@{groups,institutions}.lookup.g.apps.cam.ac.uk instead...

I was in two minds about {groupID}@lookup.groups.g.apps.cam.ac.uk vs {groupName}@lookup.groups.g.apps.cam.ac.uk but the former is the right thing to do long-term since lookup groups can have their short name changed. Having the short name appear in the name is probably right too.

(This reflects the current state of !7 (merged) too so I'm happy there.)

You're now making me wonder if it should've been {id}@{groups,institutions}.lookup.g.apps.cam.ac.uk instead...

Yeah, I prefer that because it's more symmetrical. Actually, I would have gone with {id}@{groups,insts}.lookup.g.apps.cam.ac.uk because that matches the naming scheme used in Lookup LDAP.

I wanted to leave room for, e.g., {id}@groups.blue.g.apps.cam.ac.uk. I think I've convinced myself that {groupId}@{groups,insts}.lookup.g.apps.cam.ac.uk is right. I'll create a domain in GSuite. Are you OK to update https://gitlab.developers.cam.ac.uk/uis/devops/gsuite/sync-deploy/-/merge_requests/5 or shall I?

Are you OK to update sync-deploy!5 or shall I?

I'll do it.

I'll create a domain in GSuite.

As I mentioned in a comment last week, I wasn't sure which type of domain we need for this. Is this a secondary domain that we need? How does it get validated?

Secondary. (An alias would only make sense if abc123@cam.ac.uk was also abc123@groups.lookup.cam.ac.uk.) - and G Suite admins have verified cam.ac.uk previously so any subdomain is automatically verified.

Not wanting to re-open the bikeshedding too much but we could actually just have {groupId}@groups.lookup.cam.ac.uk as the email address for a group with {groupName}@groups.lookup.cam.ac.uk as an alias for the group.

What's the length limit on the local part of the email? 64 characters rings a bell, which would be a problem, since groupName has no limit (84 is the current longest).

Answered elsewhere but for people reading the issue in future, we talked about this in #10 (comment 153204) and decided not to use aliases at this time.

mentioned in commit a6bf7dd2

New version uses {groupID}@{groups_domain} as the group email, where {groups_domain} can be configured in the YAML.

mentioned in merge request !7 (merged)

added workflowReview Required label and removed workflowIn Progress label

mentioned in commit 88743813

Moving to ~"workflow::rework" since I've one concrete change on !7 (merged) to suggest re making the group name suffix configurable and a couple of questions about functionality. I tested the tool with a custom synchronisation job and it created/managed groups correctly.

added workflowRework label and removed workflowReview Required label

We had a meeting over Meet to discuss this. Summary is below and I'll update individual comments appropriately:

• No aliases for groups at this time. Rationale:
  • the aliases aren't searchable in the sharing UI and we don't want to offer a lists.cam.ac.uk competitor which is the only place aliases would make a difference.
  • aliases are limited to 100 characters in length while lookup group names can be longer than this.
• At creation time we will set permissions of the group to match the image below. Rationale:
  • we would like latitude to experiment with opening up select groups with more permissions as we experiment with the service and we don't want the sync job clobbering us
  • these settings can only be changed after creation by an admin; there are no owners or managers for the group so no non-admin can't change them
  • there are no owners to contact so the "Contact owner" permission is unused. Set it conservatively to guard against owners appearing in future
  • the "view members" permission mirrors the permissions from Lookup
  • the "view topics" permission allows members to see topics if any were posted. They can't be posted given the default behaviour but that might change
  • the "publish posts" permission is set conservatively to guard against inadvertently making a lists.cam.ac.uk competitor
  • only owners can manage membership. There are no owners at the moment but we may want to add some in future.
  • we do not allow people to request to join the group as membership is managed via lookup
• The email address for groups will be {numeric group id}@groups.lookup.cam.ac.uk. There is no technical difference between groups.lookup.g.apps.cam.ac.uk and groups.lookup.cam.ac.uk and the latter makes more sense given the source of the group.
• The group name will be {group name} from lookup.cam.ac.uk. Rationale: we wanted the name to reflect the "friendly" group name in Lookup and indicate that the group is managed there. We didn't want the name to be email formatted as there was scope for confusion about whether that email formatted identifier would actually get email.
• We'll add MX records to groups.lookup.cam.ac.uk as detailed below. Rationale: currently sharing a Google doc with the group results in the person sharing getting an unfriendly bounce. We believe that we can enable MX records to allow for notications to be sent to the group without enabling general emails to be routed.

Permissions:

MX records:

MX Server address	Priority
ASPMX.L.GOOGLE.COM.	1
ALT1.ASPMX.L.GOOGLE.COM.	5
ALT2.ASPMX.L.GOOGLE.COM.	5
ALT3.ASPMX.L.GOOGLE.COM.	10
ALT4.ASPMX.L.GOOGLE.COM.	10

MX record request being tracked in https://gitlab.developers.cam.ac.uk/uis/devops/gsuite/sync-deploy/-/issues/5.

changed milestone to %Sprint 102

closed via merge request !7 (merged)

mentioned in commit 4917ca18