FAQ | This is a LIVE service | Changelog

Skip to content
Snippets Groups Projects
jws52's avatar
J.W. Smith authored
Optional config argument to delete mid-process files at the end of a
successful job. There is an associated command-line flag '--clearup'.

The reason for this feature is to provide a way to slim down data
volumes of production runs. A lot of processing files are generated
and production runs may only need to retain configuration, logs and
outputs. Also, the inputs might be archived separately, so they
wouldn't need to be retained in prod runs.

Currently, files are moved to a hard-coded 'recycling bin' directory,
and there is a minimal set of files set for clearup: only the deposition
and environment tar files.

This change requires some standard structure to outputs of
processor_in_job functions, which would be more robust if
a class structure is implemented.

TODO: Implement a test for file clearup.
bce461a1
History

EWS Coordinator

This project contains all the glue code to run the operational wheat rust Early Warning System.

Main tasks:

  • Cron schedulers to run the EWS every day
  • File transfers between servers (FTP pulls and staging to servers)
  • Job management and launchers - running the EWS components located on our servers

To configure for a new region:

  1. Create a new config file (for example, see config_SouthAsia.json).
  2. Modify crontab to run with the new config files.