The code used to open a database connection for each thread and leave
them open for as long as the scheduler ran. This worked reasonably well
until we moved to PostgreSQL 13 on Focal, although the scheduler would
fail if the database was restarted because there was no logic to
reconnect after a connection dropped.

On Focal/PG13 the connection for the 'cron' thread steadily consumes
memory until it has exhausted everything in the machine. This appears to
be a Postgres change rather than a Perl DBI change: the problem can be
reproduced by sitting in psql and running 'select * from backup_queue'
repeatedly. Once or twice a minute an instance of this query will cause
the connection to consume another MB of RAM which is not released until
the database connection is closed. The cron thread runs that query every
two seconds. My guess is it's something peculiar about the view that
query selects from - the time interval thing is interesting.
This needs more investigation.

But in the meantime I'd like to have backup servers that don't endlessly
gobble RAM, so this change makes the threads connect to the database
only when they need to, and closes the connection afterwards. This
should also make things work better over database restarts but that's
not been carefully tested.

As of focal a bunch of top-level dirs are symlinks (eg /lib -> /usr/lib) but
the deb packages still deploy files to the symlink rather than the real dir.
Thus if we just take the contents of all the *.list files we end up not
excluding lots of files that are in fact provided by debs

This package depends on a postgres server being available, but annoyingly we
can't use the 'postgresql' metapackage in the dependencies because on Ubuntu
that depends on the specific distro-provided version, which usually isn't the
one we want. So we have to add our supported Postgres versions one by one.

The default prepare script now uses appropriate options if it detects Postgres
12 or higher. The error handling still needs work though.

Update prepare script for newer Postgres

We don't need these backed up, and having them in the list leads to
an error due to not having show_compatibility_56 enabled

Resolves #3

changed maintainer
mention name of git repository since it doesn't match package name

We now e.g. have a cron job on cerebro-backup which calls
these scripts, where /sbin is not on the $PATH.

For cerebro-backup where we do backups by the directory rather than the
whole server. We need to have one machine being backed up
(cerebro-filestore) but a task per directory.

The new-backup-rsnapshot script understands a 'postgres' argument, but
this set up a postgres backup in an old style that we no longer use.
This change updates it to do some of the work of setting up a new style
postgres backup and tell the user what else they might need to edit to
make it go; it varies quite a lot depending on server.

Let's not worry if we haven't backed up a machine that has been offline for 3 months (instead of 6 months)

Our backup logs are generally full of

+ ssh -p 22 -o ConnectTimeout=10 root@openvpn2020.ch.cam.ac.uk '
  umask 077
  FILELIST=`tempfile`
  CONFLIST=`tempfile`
  mkdir -p /var/adm/backup
  # Make logrotate use datestamps
  if ! grep -q dateext /etc/logrotate.conf ; then sed  -i '\''s/^include/dateext\ninclude/'\'' /etc/logrotate.conf ; fi
  # Which packages are installed?
  dpkg --get-selections | awk '\'' { print $1 ; } '\'' >/var/adm/backup/packages
  cat /var/lib/dpkg/info/*.list | while read F ; do [ -f "$F" ] && echo "$F" ; done | sort > $FILELIST
  awk '\''/Description:/ { flag = 0 } ; flag == 1 { print $1 ; }  ; /Conffiles:/ { flag = 1  } ; '\'' </var/lib/dpkg/status |  sort >$CONFLIST
  diff -u $FILELIST $CONFLIST | grep ^-/ | sed s/^-// >/var/adm/backup/package-files
  rm
 '
rm: missing operand

because we're not escaping the args to that final rm command! This means we gradually fill
up /tmp, especially on machines that are infrequently rebooted (e.g. calculon)

Previously just depended on 'postgres' which was OK until we switched to
using the Postgres provided repository for postgres packages, which
makes postgres be postgres 10. Haven't tested any of the infrastructure
for 10 and don't want to start right now.

prepare-olddebian : does not do package file excludes
prepare-database: does not do package file excludes or dump databases

These are for use on machines where we can no longer reinstate them by
quickly and easily installing a new VM from the network and then
layering the backup on top, usually because the OS is obsolete.