| | 10 | == Imported unstructured data via an 'Import Job' == |
| | 11 | |
| | 12 | Import jobs provide a mechanism for unstructured data from a CSV file to be imported into a Sahana model. Import jobs work by allowing you to map each column in the CSV (the first line is assumed to be column headers) to the corresponding model field. |
| | 13 | |
| | 14 | An import job follows 5 steps: |
| | 15 | 1. Create Job |
| | 16 | 1. In the administration section of Sahana choose 'Database > Import' from the menu. |
| | 17 | 1. In the bottom (third) section of the resulting page, select the module and resource that you wish to import data to. |
| | 18 | 1. Provide a description for this import job that identifies the source of data (e.g. OCHA Organisations - 2010-01-25) |
| | 19 | 1. Select the file to upload |
| | 20 | 1. Click the 'Upload' button. |
| | 21 | 1. Map CSV Headers to Resource (Model) Fields |
| | 22 | 1. After the upload completes you will be taken to the import job page, where a list of the headers read from the CSV file is displayed in a table. To the right of each CSV header, is a select box listing the available model fields the data can be imported to. Some basic matching is performed to attempt to guess which CSV columns match which model fields. |
| | 23 | 1. Modify the values of the select boxes until they reflect the correct destination for each CSV column. You can select the special 'None (Ignore)' entry to cause all the data in a particular CSV column to be ignored. |
| | 24 | 1. Click the 'Begin Processing' button. |
| | 25 | 1. Data Processing and Validation |
| | 26 | 1. A background task on the Sahana server will process your uploaded file and validate each line. This step may take 5-10 minutes as the background task only runs at specific intervals. While you are waiting the page will automatically refresh every 60 seconds until the uploaded file has been processed. |
| | 27 | 1. Select lines for import |
| | 28 | 1. Once the file has been processed, you will be presented with a list of lines that were found. For each line a select box is presented allowing you to choose to 'import' or 'ignore' the line. |
| | 29 | 1. Lines which failed validation will automatically be ignored. A list of the invalid fields will be displayed in the 'Errors' column. |
| | 30 | 1. For each valid line, satisfy yourself that the data is good and should be imported. If it is not, change the select box to 'ignore' |
| | 31 | 1. Click the 'Import' button at the bottom of the page. |
| | 32 | 1. Data Import |
| | 33 | 1. A background task on the Sahana server will process the lines selected for import. This step may take 5-10 minutes as the background task only runs at specific intervals. While you are waiting the page will automatically refresh every 60 seconds until the uploaded file has been processed. |
| | 34 | |
| | 35 | If all lines were imported sucessfully, you will be taken to a page stating that the job state is 'finished'. A list of all the lines will be displayed below with their status set to 'imported' or 'ignored' as appropriate. |
| | 36 | |
| | 37 | If some lines failed to import, you will be taken back to Step 4. You will note that most lines are now set to 'imported'. The lines that failed to import will have error text in the 'Errors' column. You can choose to attempt a 2nd import of them, or change their status to 'ignore' before clicking the 'Import' button again to retry. |
