Pre-Populate

Introduction

We need to

Types of Data

• Item Catalogue
• Organisation List
• Assets
• Warehouse Supplies
• Human Resources (Staff & Volunteers)
• Locations (Administrative Boundaries)

Methods

URL Import

Data can be imported using URL calls:

http://127.0.0.1:8000/eden/supply/item/create.s3csv
?filename=/home/web2py/applications/eden/static/formats/s3csv/eric.csv
&transform=/home/web2py/applications/eden/static/formats/s3csv/eric.xsl

However this mehtod isn't easily automated

Also see: UserGuidelinesImporter

Python Script

This offers us better options to automate the import of data.

# Function to handle conflicts
s3mgr.resolve = response.s3.item_deduplicate
resource = s3mgr.define_resource("supply", "item")
resource.import_xml(open("/home/web2py/applications/eden/static/formats/s3csv/eric.csv", "r"),
                    format="csv",
                    stylesheet="/home/web2py/applications/eden/static/formats/s3csv/eric.xls")

Should we have a wrapper function for this which is easily accessed from the CLI?

Location for Pre-populate Data

Proposal

/private/

• transform (ALL xsl files. Any CSV should automatically use the xsl with the same name in the same directory in this folder. If multiple csv files need to use the same xsl transfor give the a "XXX" double underscore suffix, eg. eric__complete.csv, eric__sample.csv can both use supply/eric.xls )
  • module 0
  • ...
  • module n
• prepopulate
  • demo
    • module 0
    • ...
    • module n
  • regression
    • module 0
    • ...
    • module n
  • user (private prepopulate data - not shared in bzr branch)
    • module 0
    • ...
    • module n
  • drrpp (deployment with data which can be shared)
    • module 0
    • ...
    • module n
• ... (other deployments with public data)
  • module 0
  • ...
  • module n

/static/prepopulate /

• templates (csv files with header row only)
  • module 0
  • ...
  • module n

Rational:

• People could create their own "Deployment" pre-populate directories with unique (& contextual) data which could be used to easily create simulations. ( And be a great GCI task )
• We should separate csv from xsl (xsl could very easily confuse semi-technical users)
• CSV templates should be easily downloaded - prepopulate data should not (always) be
• All XLS should be in trunk (even if it is for private data) and have a matching CSV Template
• Then a configuration file can be created where you can just list the CSV file names which you want to be imported when Sahana Eden is initialized. Copies of this files could be saved in the private/prepopulate/<deployment>/ directory then a config setting could select which file to use, to make it easy to set up Sahana Eden to have different types of data in it.