Haiti

Spreadsheet Importer

We have access to a lot of relatively unstructured data in Spreadsheets which we need to import into Sahana. Currently this is done via a laborious Data Entry process.

We really need to be able to do Bulk Imports.

Example Data sources:

• ​http://spreadsheets.google.com/ccc?key=0Aq_3OYelM4ZUdHJyc0lUekZLUmEwa3ZSU2dJS3YyWWc&hl=en_GB (Data Entry tab)
  • The OCHA contacts list is a task we need to do regularly, so need to be able to handle duplicates effectively. We can influence them to standardise their format to make it easier.

Options:

• CSV Import

  cd /path/to/web2py
  python web2py.py -S sahana -M -N
  db.import_from_csv_file(open('db.csv','rb'))
  db.or_organisation.import_from_csv_file(open('orgs.csv','rb'))
  db.commit()
  Ctrl+D
  

  • Limited access through Web at: /admin/import_data (& doesn't seem to work at all?)
  • Problems with the file format (e.g. MS Excel chews, OpenOffice 3.1 ok)
  • Hard work to get reference fields lined up
    • Need to export a dataset first
• ​Google Spreadsheets API
• ​xlrd
  • Example Code - not the cleanest
• ​Resolver One

---

Haiti