Google GSuite Synchronisation Tool

This repository contains a custom synchronisation tool for synchronising information from the Lookup service to a Google hosted domain (aka "GSuite").

Configuration is performed via a configuration file. Take a look at the example configuration file for more information.

Usage

The tool can be invoked from the command line:

$ gsuitesync

# To also update the group settings on all groups
$ gsuitesync --group-settings

# To just update users not groups and institutions
$ gsuitesync --just-users

By default this will log what will be done. To actually perform the synchronisation:

$ gsuitesync --write

# And with group settings
$ gsuitesync --group-settings --write

The default socket timeout for batch operations is 300 seconds. To change this value:

# Timeout set to 10 minutes
$ gsuitesync --timeout=600

A folder for caching responses from Google and Lookup APIs can be specified. This is helpful when performing repeated dry-runs while testing code. If the appropriate cache file exists it will be used instead of making the API call. If not, the file is created for later runs.

# Cache results in API
$ gsuitesync --cache-dir ./cache

See the output of gsuitesync --help for more information on valid command-line flags.

Unless overridden on the command line, the tool searches for its configuration file in the following places in the following order:

• A gsuitesync.yaml file in the current directory.
• ~/.gsuitesync/configuration.yaml.
• /etc/gsuitesync/configuration.yaml.

The first located file is used.

Note: The crypt python library behaves differently on MacOS than Linux and fails to create acceptable passwords for new Google accounts. Build the docker image and run gsuitesync via it to overcome this.

Installation

The command-line tool can be installed directly from the git repository:

$ pip3 install git+https://gitlab.developers.cam.ac.uk/uis/gsuite/synctool.git

For developers, the script can be installed from a cloned repo using pip:

$ cd /path/to/this/repo
$ pip3 install -e .

Testing against gdev.apps.cam.ac.uk, (UCam test Google Workspace)

Install as above, and also clone the sync-deploy repo in a sibling folder called sync-deploy.

Download the gsuite-sync-test GCP service account JSON credentials file from 1password and save as credentials.json.

Take the API keys from Directory Sync Tool App in API Gateway and create api_gateway_credentials.json, as described in "Preparing API Gateway credentials" below.

The tool can then be run against the test Google Workspace using the configuration file from the sync-deploy repo. e.g. the following to dry-run sync just users not groups:

$ gsuitesync --configuration ../sync-deploy/test.yaml --just-users

New users

When new users are created they are created with a random password which is immediately thrown away. They are created with a primary email of the form [uid]@[domain] where [uid] is the unique id from lookup (i.e. the CRSid) and [domain] is the name of the Google domain from the configuration.

Required API scopes

This tool requires the following OAuth2 scopes to audition the changes to be made:

• https://www.googleapis.com/auth/admin.directory.user.readonly
• https://www.googleapis.com/auth/admin.directory.group.readonly
• https://www.googleapis.com/auth/admin.directory.group.member.readonly
• https://www.googleapis.com/auth/apps.groups.settings

This tool requires the following OAuth2 scopes to actually perform changes:

• https://www.googleapis.com/auth/admin.directory.user
• https://www.googleapis.com/auth/admin.directory.group
• https://www.googleapis.com/auth/admin.directory.group.member

Additionally, the following scope is required to view and manage licensing:

• https://www.googleapis.com/auth/apps.licensing

See the section on preparing a service account for information on how to grant a service account those scopes on your domain.

Preparing a service account (Admin Roles)

Google have updated the API to allow service accounts direct access to the API without needing domain-wide delegation.

1. Create a service account in the Google Console for this script.
2. Generate and download JSON credentials for the service account.
3. In the GSuite admin panel, go to "Account" > "Admin Roles" and create a new custom role with the following Admin API privileges (not Console privileges):
   • Users - Create, Read, Update and Update Custom Attributes
   • Groups - All privileges
   • Licence Management - All privileges
4. Add the service account to the role using the "Assign service accounts" option when viewing the custom role's admins

Currently, undeletion can only be performed by a superadmin, no Admin Role permission exists for this, so Domain-wide Delegation is required.

Preparing a service account (Domain-wide Delegation)

This tool assumes it will be acting as a service account user. It will use this service account user to then act on behalf of an admin user in GSuite. To prepare such a service account user:

1. Create a service account in the Google Console for this script.
2. Generate and download JSON credentials for the service account.
3. Under "IAM" > "Service Accounts", select the service account, click "Edit", click "Show domain-wide delegation" and "Enable G Suite Domain-wide Delegation". Click "Save" to apply the changes.
4. Hover over the "?" symbol next to the generated client id and click "view client". Copy the Client ID from the popup panel.
5. In the GSuite admin panel, go to "Security Settings" > "API Controls" > "Manage Domain-Wide Delegation", and click "Add new".
6. Paste in the service account Client ID as "Client Name" and add a comma-separated list of scopes. See the section on required API scopes.

The scary-sounding "Enable G Suite Domain-wide Delegation" means that this service account is marked as being willing to "su" to another Google user. By adding the generated Client ID to the GSuite security settings you are, as domain administrator, giving that service account the ability to act as any user in the domain subject to the listed scopes.

Preparing API Gateway credentials

The api_gateway section in the configuration file specifies where the credentials for accessing API Gateway can be found; see the example file. These credentials must be supplied as a file in YAML/JSON format with values for the keys base_url, client_id and client_secret. The key and secret can be found on the Apigee apps page by searching for the "Directory Synchronisation Tool" app (there are staging and production versions). The Lookup API base URL can be found on the Lookup API page on API Gateway.