wiki:UserGuidelines/Importer

Importer

Spreadsheets

Data is normally imported from spreadsheets.

XLS/X Templates

CSV Files

Process:

  • File should be encoded in 'UTF-8' (with or without BOM)
  • Data needs putting into the correct columns
  • Fields containing commas should be wrapped with "" quotes
  • There should be no line-breaks in text fields (see cleanCSV.py for help with this)
  • Date formats need to be correct: ISO8601
  • If the file is very large, then it can be better to split into multiple files to avoid memory exhaustion (see splitCSV.py for help with this)

This can be done to PrePopulate an instance.

After 1st run, there are menu links for many resources, but any REST controller can have the 'import' method invoked if there is a suitable .xsl file as static/formats/s3csv/module/resource.xsl:

http://host.domain/module/resource/import

Files & Stylesheets can be loaded from alternate sources, such as:

http://host.domain/eden/hrm/person.s3csv/create?fetchurl=http://my.host/HR.csv&transform=/home/web2py/applications/eden/static/formats/s3csv/hrm_import.xsl

Alternately these can be done using CLI scripts:

OpenStreetMap

XSLT available to import Hospitals & Locations into Sahana Eden

ToDo: Provide a UI to select which levels the admin boundary levels correspond to:

Ushahidi

XSLT available to import Ushahidi reports into Sahana Eden's Incident Reporting system

See Also

Last modified 5 years ago Last modified on 10/28/15 10:41:58

Attachments (2)

  • cleanCSV.py (1.1 KB ) - added by Fran Boon 7 years ago. Remvoe line-breaks within text fields from a CSV file
  • splitCSV.py (1.2 KB ) - added by Fran Boon 7 years ago. Spli a large CSV file into multiple files to avoid memory exhaustion when importing

Download all attachments as: .zip

Note: See TracWiki for help on using the wiki.