Optional config argument to delete mid-process files at the end of a
successful job. There is an associated command-line flag '--clearup'.

The reason for this feature is to provide a way to slim down data
volumes of production runs. A lot of processing files are generated
and production runs may only need to retain configuration, logs and
outputs. Also, the inputs might be archived separately, so they
wouldn't need to be retained in prod runs.

Currently, files are moved to a hard-coded 'recycling bin' directory,
and there is a minimal set of files set for clearup: only the deposition
and environment tar files.

This change requires some standard structure to outputs of
processor_in_job functions, which would be more robust if
a class structure is implemented.

TODO: Implement a test for file clearup.

changing the scp logic and the ssh file checking logic to use a different command if no server name is set

(cherry picked from commit 0f5e125a)

wiring in the newly separated post_processing module into coordinator, rather than the plotting module

1) There is now a python alternative to the R-based wheat-source-
generation, called source_gen. The functionality to call source_gen
has been added (see ProcessorSurveys.pyL939 and L1032-1040), but not
yet switched on while we await testing of
output at the Met Office.

2) ProcessorComponents was huge (2100 lines!) and the structure was
not apparent. Each component's supporting functions are now in a
separate directory. There is also a ProcessorServer.py relating to
file down/up-loads. ProcessorComponents is now essentially a directory
 to import all the functions. This might inspire a more systematic
 (object-oriented?) structure for each component within coordinator
 in the future.