FAQ | This is a LIVE service | Changelog

Skip to content
Snippets Groups Projects
Select Git revision
  • main default protected
1 result
Blame Permalink
Code owners
Assign users and groups as approvers for specific file changes. Learn more.
README.md 2.29 KiB

Usage:

Note that this is NOW LIVE (and version controlled) but the structure needs tidying.

The current test script can be called using the wrapper reformatSurveyData.sh

Note: I have set this up to use a python environment specifically set up for the EWS (prevents anyone accidentally breaking the EWS by changing the system python, and we can easily replicate the python install on another machine if necessary). The environment is located in /storage/app/Ethiopia-EWS/EWS-python, launched with the wrapper script /storage/app/Ethiopia-EWS/EWS-python/ews-python3.sh

Indended structure for the additional survey data:

In the future we should store the "extra data" files in the workspace, along with all the project working data. To this end, the testing data now lives in

/storage/app/Ethiopia-EWS/Ethiopia-EWS-Workspace/SURVEYDATA_MANUAL/

with the additional data sent so far by Yoseph in the files 20190902_ETH_YosephA.xlsx and 20190905_ETH_YosephA.xlsx

All additional data from Yoseph or any other non-EIAR server data should be processed into the LIVE_SURVEYDATA_TOADD.csv file

The old file of manual additions is located in the EWS-Coordinator directory, which we should remove

Indended structure for the additional survey data reformatting script:

The reformatter script is responsible for getting the data sent by Yoseph into the correct format, then this will need to be merged into the LIVE_SURVEYDATA_TOADD.csv file.

It looks like Yoseph is currently sending the data with only new findings since he sent the last file, rather than a cumulative file of additional data. This means so we should only need to sanitise each additional data file once, process it and merge with the cumulative LIVE_SURVEYDATA_TOADD.csv file.

In the file 20190902_ETH_YosephA.xlsx I have already made the two changes suggested by Yoseph in the email of 5 Sep 2019, 15:03:

"ID 92107 I would use 1ha. For the growth stage I would estimate the growth stage taking in to consideration the time of survey. for instance for ID 92105 I would use flowering"

Missing functionality

Because Yoseph appears to be sending only new data, the script cannot recognise and remove duplicate entries. This would be useful to add as an extra consistency check on the extra survey data sent to us.