wiki:BluePrint/GIS/GPSSharing

Version 48 (modified by Pat Tressel, 14 years ago) ( diff )

--

Table of Contents

  1. Description
  2. Requirements
    1. Constraints
  3. Use Cases
    1. Scenario 1: GPS coordinates exchange within the office
    2. Scenario 2: Data analysis
    3. Scenario 3: Data exchange between offices
    4. Scenario 4: Printing a Map
    5. Scenario 5: Fleet Management
  4. Implementation
    1. High-Level Architecture
    2. Sahana
      1. Ideas from Depiction
    3. Local OSM Server
    4. Edit Base Map
    5. Sync Base Map
    6. Export to Base Map
    7. Other Options Considered
  5. Suggested GSoC Project

Share GPS Data

This BluePrint is a collection of ideas relating to uploading, aggregating, selecting, and exporting GPS info. It contains parts that could be split off into several parts that could be suitable projects for GSoC. See the end of the page for one such suggested project.

Description

Provide a simple way for NGO field workers to share GPS data amongst themselves, with their Headquarters & with other agencies.

  • Field staff for NGOs (both Humanitarian & Development) visit Programme sites with handheld GPS (e.g. Garmin eTrex).
  • They collect data as they go (Tracks & Waypoints).
  • They would like a way of easily sharing this information between each other:
    • uploading data to a central repository
    • managing this master repository to have a reference dataset: no duplicates, names checked for consistency
      • this management should work offline
    • download the reference dataset to each GPS
  • A secondary benefit will be in terms of wider communications (e.g. reporting to Donors)
  • A potential future enhancement would be to be able to download GPX routes from/to specific locations.

This requirement is defined by ​Oxfam in Haiti, but is applicable to many other organisations & locations as evidenced by this combined Problem Definition for RHoK:

It could also be the public version of the WFP GeoPortal (which will be separate from their Intranet):

CARITAS:
In the case of the current flood disaster in Pakistan, a simple system for the real-time creation and development of tailored maps would have been very helpful in enabling us to add important operational information including places with special emergency needs and project sites where large numbers of people have come together to the map and update this crucial information in real time.

This would be a very valuable tool, especially if the functionality would allow for update of the map by various people over the internet. Thus, our desaster relief staff on the ground in Pakistan could enter up to date project information combined with GPS data into our operative map. The same map could be used by the central project management staff in our Freiburg (Germany) headquarters and updated with additional information and used for various purposes.

The result would be an interactive map of great practical value for the program planning and control of the relief operation.

Requirements

Desirable: It would be ideal if as much of the reference dataset as possible could be incorporated into the Base Map - OpenStreetMap is ideal for this.

Constraints

  • System must be easy to use as staff in emergencies cannot receive extensive training (there is often rapid scale-up of both Internationals with experience from disparate organisations & locals who sometimes have realtively little experience of IT at all)
  • Field sites often have poor access to IT support (it could be a day's drive away or available only via expensive SatPhone)
  • Field sites' Internet access is often expensive (BGAN: $5/Mb) or highly erratic (local providers, power problems)
  • Solution must be accessible to Windows users (although could be delivered using a Virtual Machine)
  • System needs to be easy to backup to a single file (which could be a .zip of a .sql dump)
  • Some data can be sensitive , so the system needs to be secure
    • We already have ACL within Sahana - extending this to map layers is already on the imminent roadmap

Use Cases

Scenario 1: GPS coordinates exchange within the office

Problem:

  • An aid worker returns from the field after having assessed the damage in a particular area/needs of the population and includes the GPS coordinates in her report. Coordinates are not shared.

To work in the field, it is sometimes difficult to get appropriate access to information, e.g. where to find a well with clean drinking water to cater thousands of people, for example, as recently needed in Haiti. A member of Caritas Haiti could now enter data with GPS location on where there are intact wells, which would be saved to the map. The same map could be used later on, e.g. by a group of water experts wanting to examine the quality of the water to easier find the wells - with the help of navigation tools in their cars. They could even upload up to date information on the quality of the water or potential repair needs to the map.

Solution:

  • Aid worker uploads a GPX or spreadsheet file to a webpage, with the opportunity to provide metadata (most critically the category/name. Time & Person are defaulted automatically)
    • There should be an option for uploaded data to be validated by an admin before publishing
  • Users can download data by selecting different layers (e.g. all broken bridges) retrieved in different file formats (e.g. GPX to download back to the GPS or KML to share with users of Google Earth)

Implementation Notes:

  • There are already GPX/KML export stylesheets in Eden & the S3XRC engine allows URL-based filtering to make selections.
    • We could usefully develop a little UI to provide a simple front-end for this (either in Ext to integrate with the current client or else a 'cuddly' one in jQuery like Maps-on-a-Stick)
    • User can create a set of their preferred selections (e.g. 'All Schools in Port-au-Prince & All Hospitals in Haiti')
      • Their default selection could be stored in their Profile (like a Subscription)
  • Provide a UI to edit the uploaded Waypoints before importing into the database (a GPX file often just contains the numeric IDs & the metadata is stored in a notebook with pen/paper, so the import stylesheet is less useful)
    • Ext Store to upload data to (fields the same as gis_waypoint: name, description, category)
    • Ext Table with Editable Rows to edit the data in spreadsheet-style (fast for users & familiar interface)
    • AJAX call to import to the gis_waypoint table via the gis/waypoint controller
  • Uploaded data needing to be admin-validated is already in the Eden roadmap for core ACL functionality, so is probably not usefully developed as part of RHoK
  • CSV/OpenOffice data can already be uploaded to Eden, although, inevitably, the structure needs to be quite specific. There is also a graphical wizard for importing Excel spreadsheets which allows field mapping, etc. However this isn't yet robust...making this tool robust would be useful both for this project & for other Sahana applications.
  • Currently the editing of features using the map is clumsy, we could really use the editing toolbar to be working (needs to be enabled in modules/s3gis.py or use one from ​GeoEditor) to allow Features to be:
    • Added
      • Have a popup to select the type of resource being created.
      • Then open up another popup to add the relevant details? Just add Name initially & add this feature to a worklist to go back & fill out other details?
    • Edited
      • Need to add an 'Edit' button to the existing Popups? Separate toolbar button?
    • Moved
      • Simply do an AJAX call to update the coordinates
    • Deleted
      • Need to add a 'Delete' button to the existing Popups? Separate toolbar button?
      • Simply do an AJAX call to delete the record (& then remove the feature from the current display)
  • Provide UI to access the 'copy' functionality from S3XRC to create hospitals/offices/warehouses/activities from Waypoints & vice versa
  • Complete the per-category Icons for both display & GPX export (currently icons are per-resource, although work has started to rectify this)

Scenario 2: Data analysis

Want to be able to view the different layers together to improve Decision-Making.

The Operation coordinator needs to see location and distribution progress of only Emergency Shelters to see if there are any gaps. Solution: being able to toggle layers on/off independently, e.g. topographic layer-demographics-waypoints etc. and extract and create piecharts from metadata.

Implementation Notes:

  • Contours are a key piece of information here. These can be rendered in local OpenStreetMap tiles
  • We want to be able to display graphs on top of the map
    • Small graphs as Icons
    • Larger graphs within Popups
    • Heat Maps
  • We don't currently have tables for Demographic data
    • Would be useful to add models for these. What's a good source of fields to use?

Scenario 3: Data exchange between offices

Problems:

  • HQ needs to write a donor report & wants to be able to show the donor(s) the locations of the Project Activities
  • Country office is doing security planning & wants to have all security incidents visible on a single map
  • Many different agencies are working in a location & wish to share a common operational picture across the cluster (programme) or all-agencies (security)

Solution:

  • Synchronize the data between Sahana instances in a controlled manner
  • Import & Export data using Open Standards (this is usually compressible text files, which are easy to email)

Implementation Notes:

  • There is already a core synchronization engine in Eden which supports Push (to allow a 'server' behind NAT to share data with a central portal) & 'Pull' (to allow a 'server' on an intermittent connection to retrieve data when it is connected)
  • This engine can also be used to control the Import/Export of data to/from Open Standards using XSLT Stylesheets.
    • Alternatively the data can be imported/exported manually using the same stylesheets
    • Import stylesheets for KML/GeoRSS are yet to be developed: S3XRC/S3XML/Transformation

Scenario 4: Printing a Map

Would like to see A4 District-level maps with names and boundaries of sub-districts (e.g. Tehsils for Pakistan) that we can insert text and tables onto top of.

  • There is a basic print.css stylesheet to allow simple screen prints which could possibly be enhanced.
  • There is basic support for using the GeoExt client libraries to interface with the MapFish PDF print library (e.g. as-embedded into GeoServer), however this could use polishing.
  • Would be great to allow users to have better layout control of where Legends, Scalebars & Compasses are positioned to get them out of the way of the meaningful data
  • BluePrintGISPrinting

Scenario 5: Fleet Management

Link to vehicle-mounted GPS sending data back via GPRS/SMS

  • This could be through integration with ​OpenGTS

Implementation

High-Level Architecture

Propose to:

  • Set up a Virtual Machine containing all necessary software (to make it easy to install on any available PC - this setup may need replicating in several offices, although no need to sync between offices except via main OpenStreetMap database at this time)
  • Local OSM Server
  • Build a GUI within Sahana to allow the uploading/editing of GPX data

Sahana

The current GPX layer interface is probably insufficient.

  • Interim improvement would be to make the Tracks upload visible
    • automatically detect whether it contains WayPoints, Track &/or Routes?

Instead have 2 tables:

Can we develop a Flash(?) interface to actually manage the transfer to/from the unit of the GPX &/or basemap?

  • For local servers (either on user's laptop, or where they visit the shared PC in the small office, can use ​pyUSB)

Extend our GeoExplorer integration as the basis for this (adding other missing MapViewingClient functionality as we go).

Ideas from Depiction

​Depiction has some useful functionality with some nice UI ideas...there are videos which can be downloaded to review.

  • Simple UI
    • add new Features to the map via the mouse inc Single/Multiple options
    • QuickAdd toolbar for common resources
    • Manage Elements tool to locate features within viewport (category, tags or name) & edit their properties (global type or individual)
  • Modelling
    • elevation data within the database
      • can we use the same datamodel as for Mapnik's contours?
      • add an easy downloader like srtm2osm
    • models could come from GRASS or R, maybe via WPS?
      • Plumes: Wind speed, direction, volume
      • Flood
      • Liquid flow
      • Antennas (do we need this within Sahana?)
  • Routing
    • road structure within the database
      • imports from OpenStreetMap
      • imports from Shapefiles
      • ​pgRouting for back-end calculations?
      • build on Measure tool (if no better ones available) for front-end UI?
  • GeoReferencing
  • Shapefiles support
    • integrate OGR2OGR (usable even with basic sqlite versions) - e.g. ​Ogre
    • GeoServer via REST? (-> PostGIS where available)
  • Permatext (useful for printing, especially)
  • OnHover popups have a short Description (not just Name)
  • Photos on Map
    • ensure that Image from Static is rendered even if the template is yet to process (=> image div different to metadata div)
  • Draggable property (default based on resource type?)
  • Interactions
    • Barriers affect Routing
    • Floods knock our buildings
    • Explosions knock out People
    • Zones of Influence
      • Polygon which defaults to octagon with a customisable Radius
        • Can create custom polygons by dragging/adding/removing polygons (UI for this already in OpenLayers)
      • Calculated (e.g. Plumes/security cameras)

Local OSM Server

Edit Base Map

  • Potlatch2
    • Editing local database or only main one?
    • Need better integration into Sahana:
      • passing in the area of the current viewport
      • seeing Sahana overlays inside Potlatch (can export as either GPX or OSM. KML support being added to Potlatch?)
      • can pass in a GPX file!

Sync Base Map

Our local install needs to be kept up to date from the global repo:

If we're making local edits then we need to feed those back to the master repo.

Export to Base Map

Other Options Considered

  • Garmin MapSource
    • can't load more than 1 GPX file at a time
  • Google Earth
  • ​Depiction
    • Data primarily only shareable with other Depiction users which means hard to gain traction even within an organisation let alone for interagency work.
    • Installation requires admin rights on the PC, which many field staff won't easily have access to - again lowering the ability to get wider adoption.
    • Can't create new Depictions or use Base Maps at all without an active Internet connection.
    • Revealers seem very gimmicky: I don't find them useful & they're really hard to use in practise (I just wanted a 'full-screen option').
      • GeoExplorer already has the ability to dynamically edit layer opacity, which is the 1 thing I'd use them for.
    • Licensing costs push this beyond local partners to whom we want to handover the tools when moving from Response -> Recovery -> Mitigation/Preparedness (USD 200/machine + add-ons, such as $50 just for icons that come free with Sahana!)
    • No support for KML (or GeoRSS)
    • No ability to edit the OpenStreetMap base layer which adds so much value to these maps
    • Adding Elevation data is clumsy (fairly easy to make this 'seamless' ;) )
    • Can only cover a limited area (< 1500 sq km which is smaller than, say, Pakistan)

Suggested GSoC Project

One useful subset of GPS Sharing would be to implement a convenient workflow and forms through which users could:

  • Upload files of GPX data to share. (Using a file for input and output avoids the issue of expensive hardware.)
  • Select data they're interested in, based on (for instance):
    • Region (e.g. what city it's in, given a polygon boundary, or other location-based queries).
    • Type (e.g. hospital, vehicle).
    • Time (e.g. when the points were recorded in the field).
    • Maybe select using a map.
  • Display on a map.
  • Allow editing the data.
  • Export to a file.
  • Maybe tie this to the "Save Searches" project so the user could record their queries.

GIS BluePrints

Note: See TracWiki for help on using the wiki.