- Dec 21, 2023
-
-
L. Bower authored
-
- Dec 12, 2023
-
-
L. Bower authored
splitting out some csv parsing functions
-
- Nov 30, 2023
-
-
L. Bower authored
splitting out some csv parsing functions
-
- Oct 17, 2023
-
-
L. Bower authored
-
- Oct 11, 2023
-
-
L. Bower authored
-
- Jul 26, 2023
-
-
L. Bower authored
changing the optional SPATIAL_POINTS_FILE config parameter to SPATIAL_POINTS_FILE_TEMPLATE and allowing ${} value subsiitution in the paremeter value removing some points from the ethiopia post processing landpoint file, to allow checking that the filtering for post-processing is working
-
- Jul 21, 2023
-
-
L. Bower authored
It is possible to specify which spatial points file to use in the main config json. The default file will be used if not specified. This will allow us to perfrom historic runs with the old grid and specify manually. The country-level points filtering used by the postprocessing steps (beofre plotting) now round down the acuracy when filtering points. This allows the country filtering to work with both the new and the old grid, as the differece is lost in the rounding
-
- Jun 20, 2023
-
-
L. Bower authored
-
- Jun 14, 2023
-
-
Dr T. Mona authored
-
Dr T. Mona authored
-
Dr T. Mona authored
-
Dr T. Mona authored
-
Dr T. Mona authored
-
- Jan 24, 2023
-
-
J.W. Smith authored
Optional config argument to delete mid-process files at the end of a successful job. There is an associated command-line flag '--clearup'. The reason for this feature is to provide a way to slim down data volumes of production runs. A lot of processing files are generated and production runs may only need to retain configuration, logs and outputs. Also, the inputs might be archived separately, so they wouldn't need to be retained in prod runs. Currently, files are moved to a hard-coded 'recycling bin' directory, and there is a minimal set of files set for clearup: only the deposition and environment tar files. This change requires some standard structure to outputs of processor_in_job functions, which would be more robust if a class structure is implemented. TODO: Implement a test for file clearup.
-
- Nov 09, 2022
-
-
L. Bower authored
-
- Oct 20, 2022
-
-
L. Bower authored
-
- Oct 03, 2022
- Sep 14, 2022
- Aug 31, 2022
-
-
J.W. Smith authored
-
J.W. Smith authored
1) There is now a python alternative to the R-based wheat-source- generation, called source_gen. The functionality to call source_gen has been added (see ProcessorSurveys.pyL939 and L1032-1040), but not yet switched on while we await testing of output at the Met Office. 2) ProcessorComponents was huge (2100 lines!) and the structure was not apparent. Each component's supporting functions are now in a separate directory. There is also a ProcessorServer.py relating to file down/up-loads. ProcessorComponents is now essentially a directory to import all the functions. This might inspire a more systematic (object-oriented?) structure for each component within coordinator in the future.
-