Haiti

Spreadsheet Importer

We have access to a lot of relatively unstructured data in Spreadsheets which we need to import into Sahana. Currently this is done via a laborious Data Entry process.

We really need to be able to do Bulk Imports.

Example Data sources:

• ​http://spreadsheets.google.com/ccc?key=0Aq_3OYelM4ZUdHJyc0lUekZLUmEwa3ZSU2dJS3YyWWc&hl=en_GB (Data Entry tab)
  • The OCHA contacts list is a task we need to do regularly, so need to be able to handle duplicates effectively. We can influence them to standardise their format to make it easier.

Options:

CSV Import

cd /path/to/web2py
python web2py.py -S sahana -M -N
db.import_from_csv_file(open('db.csv','rb'))
db.or_organisation.import_from_csv_file(open('orgs.csv','rb'))
db.commit()
Ctrl+D

• Limited access through Web at: /admin/import_data (& doesn't seem to work at all?)
• Problems with the file format (e.g. MS Excel chews, OpenOffice 3.1 ok)
• Hard work to get reference fields lined up
  • Need to export a dataset first

Google Spreadsheets API

• ​http://code.google.com/apis/spreadsheets/

Excel

• ​http://pypi.python.org/pypi/xlrd xlrd]
  • Example Code - not the cleanest

Other

• ​Resolver One

---

Haiti