This is an initial step to have multiple source file setup.
This step is NOT utilising the grouped files yet.

Survey processor to handle WheatRustToolbox download and formating.

New function to allow conversion of a series according to a provided dictionary. (Needed for WRT formating.)

- Always create ExportCSV folder if it is not exist to allow merging.
- Correcting messages to not have ODK as the default download.
- Add ProcessorSurveysWRT import.

moving log file output to before the assert statement, so we can more easily see which file is missing [if a file is missing]

moving the class setup function to half-fat tests, so the TEST_OUT variable is reset for the half-fat tests but not the full-fat ones when being loaded by unittest.main

changing tests to check for success via the TestResult object and changing the exit code - previously failing tests were not  being picked up by the gitlab runner

reseting the IntegrationTestUtils.TEST_OUT_PATH variable upon loading each test class, was having its value carried across from previous test runs when being run as past of a test suite

making gitignore ignore the contents of the log dir rather than the dir itself. Think it may be why the integration tests are failing

making gitignore ignore the contents of the log dir rather than the dir itself. Think it may be why the integration tests are failing

This reverts commit 8c32ad24

This allows the ODK export functionality to append latest changes
to a single database path, instead of downloading the full database
history which can stretch over many years. This saves a lot of time
for the daily survey job.

The WRSIS survey processor can already limit the download size to
the current season(s) with a start-date parameter.

The Kobotoolbox survey processor doesn't have a solution for
avoiding the daily download of extensive historical data. This needs
investigating further at some point.