wiki:UserGuidelines/Importer

Version 49 (modified by Fran Boon, 8 years ago) ( diff )

--

Importer

Spreadsheets

Data is normally imported from spreadsheets.

Currently these need to be converted to CSV format, however there is now support for building .xls[x] templates to wrap around the CSV:

Process:

  • Data needs putting into the correct columns
  • Fields should be wrapped with "" quotes to prevent issues
  • There should be no line-breaks in text fields (see [whttp://eden.sahanafoundation.org/raw-attachment/wiki/UserGuidelines/Importer/cleanCSV.py cleanCSV.py] for help with this)
  • Date formats need to be correct: ISO8601
  • If the file is very large, then it can be better to split into multiple files to avoid memory exhaustion (see splitCSV.py for help with this)

This can be done to PrePopulate an instance.

After 1st run, there are menu links for many resources, but any REST controller can have the 'import' method invoked if there is a suitable .xsl file as static/formats/s3csv/module/resource.xsl:

http://host.domain/module/resource/import

Files & Stylesheets can be loaded from alternate sources, such as:

http://host.domain/eden/hrm/person.s3csv/create?fetchurl=http://my.host/HR.csv&transform=/home/web2py/applications/eden/static/formats/s3csv/hrm_import.xsl

Alternately these can be done using CLI scripts:

OpenStreetMap

XSLT available to import Hospitals & Locations into Sahana Eden

ToDo: Provide a UI to select which levels the admin boundary levels correspond to:

Ushahidi

XSLT available to import Ushahidi reports into Sahana Eden's Incident Reporting system

See Also


UserGuidelines

Attachments (2)

  • cleanCSV.py (1.1 KB ) - added by Fran Boon 8 years ago. Remvoe line-breaks within text fields from a CSV file
  • splitCSV.py (1.2 KB ) - added by Fran Boon 8 years ago. Spli a large CSV file into multiple files to avoid memory exhaustion when importing

Download all attachments as: .zip

Note: See TracWiki for help on using the wiki.