NB this is so we can put things in LOGFILE at earlier points
than the script does at present

We capture the error code at a few points in the script, and then (intend to)
log something about what went wrong. But if we set -e we immediately bail
before logging anything useful!

Resolves #1

Resolves #3

On a new machine, there's nothing to remove and so rm will return
an error. I'd like our prepare scripts not to return errors
unnecessarily!

changed maintainer
mention name of git repository since it doesn't match package name

This repo used to have a non-standard build procedure. Now it uses
our usual makedeb process so we don't need to add extra documentation
here.

We now e.g. have a cron job on cerebro-backup which calls
these scripts, where /sbin is not on the $PATH.

For cerebro-backup where we do backups by the directory rather than the
whole server. We need to have one machine being backed up
(cerebro-filestore) but a task per directory.

NB arguably we "should" be doing this via debian/rules and
calling dh_fixperms. Doing that is left as an exercise for
whoever volunteers to refactor the way we build all of our
local debs!

The new-backup-rsnapshot script understands a 'postgres' argument, but
this set up a postgres backup in an old style that we no longer use.
This change updates it to do some of the work of setting up a new style
postgres backup and tell the user what else they might need to edit to
make it go; it varies quite a lot depending on server.

Let's not worry if we haven't backed up a machine that has been offline for 3 months (instead of 6 months)

Ignore /usr/share/doc, /usr/share/texlive - takes a long time to enumerate contents which we then exclude anyway due to being deb-provided

Tidy global_ignore - remove some long-since unneeded lines that should have been host-specific in the first place

e.g. /DATA for apps, /home/web for atm webserver. Also /run was in twice...

Our backup logs are generally full of

+ ssh -p 22 -o ConnectTimeout=10 root@openvpn2020.ch.cam.ac.uk '
  umask 077
  FILELIST=`tempfile`
  CONFLIST=`tempfile`
  mkdir -p /var/adm/backup
  # Make logrotate use datestamps
  if ! grep -q dateext /etc/logrotate.conf ; then sed  -i '\''s/^include/dateext\ninclude/'\'' /etc/logrotate.conf ; fi
  # Which packages are installed?
  dpkg --get-selections | awk '\'' { print $1 ; } '\'' >/var/adm/backup/packages
  cat /var/lib/dpkg/info/*.list | while read F ; do [ -f "$F" ] && echo "$F" ; done | sort > $FILELIST
  awk '\''/Description:/ { flag = 0 } ; flag == 1 { print $1 ; }  ; /Conffiles:/ { flag = 1  } ; '\'' </var/lib/dpkg/status |  sort >$CONFLIST
  diff -u $FILELIST $CONFLIST | grep ^-/ | sed s/^-// >/var/adm/backup/package-files
  rm
 '
rm: missing operand

because we're not escaping the args to that final rm command! This means we gradually fill
up /tmp, especially on machines that are infrequently rebooted (e.g. calculon)

From the comment in the script, I think this was an attempt to ignore an
error about setting GIDs. The only place I saw that particular error
occuring in practice was for loopback-mounted isos on OS builders.
We shouldn't be backing those up in the first place! Exit code
23 is "partial transfer due to error" and so can be due to problems
other than failing to set GIDs

Previously just depended on 'postgres' which was OK until we switched to
using the Postgres provided repository for postgres packages, which
makes postgres be postgres 10. Haven't tested any of the infrastructure
for 10 and don't want to start right now.

prepare-olddebian : does not do package file excludes
prepare-database: does not do package file excludes or dump databases

These are for use on machines where we can no longer reinstate them by
quickly and easily installing a new VM from the network and then
layering the backup on top, usually because the OS is obsolete.

The script runs some SQL which ultimately determines the time at which the last
backup with an exit code of zero was. If there has never been a successful
backup, we ended up reporting a clear dot. Whilst this may lead to a few
transitory red dots immediately after adding a new host, I think this is
preferable to not realising for months that a host isn't being backed up!
See ticket 154527