[[TOC]] = User Guidelines for GIS Data = Assumes installation of the relevant tools: [wiki:InstallationGuidelinesGISData] == Import Data == === UUIDs === We need an agreed set of [http://eden.sahanafoundation.org/wiki/UUID UUID]s for GIS Data so that we can share data more easily across systems. * !OpenStreetMap IDs can change over time if records are deleted/recreated * They can [http://wiki.openstreetmap.org/wiki/Humanitarian_OSM_Tags/Humanitarian_Data_Model#ID hold additional uid/uuid fields] though * Geonames data isn't free enough for OSM: http://wiki.openstreetmap.org/wiki/Geonames * Ushahidi doesn't have a common set of IDs across instances * No space for a UUID either? * Longer-term we need a common central repository of UUIDs that is held for the common good. * Propose that Sahana start this off & then give up ownership/branding later: * Sahana can use UUIDs of format: http://geo.sahanafoundation.org/ * These are associated with an OSM ID & Geonames ID for cross-correlation * IDs from Ushahidi instances can be appended to the comments field in Sahana === Geonames === There is an import_geonames() function in S3GIS which downloads/unzips the country file (a TAB-separated list) from http://download.geonames.org/export/dump/ It should be run for the different level sof hierarchy that you wish to import (generally just the lowest level as Geonames just has Point data, so it's best to use other sources for the Polygons 1st, that way the Geonames importer can locate these Points within the correct Polygons of the hierarchy) NB It takes some time to do this import! Pakistan imports 95000 locations! Python 2.5 doesn't support Zipfile.extract() & Zipfile.read() isn't unicode-safe. Until this is fixed, download the file manually 1st: {{{ cd ~web2py/applications/eden/cache wget http://download.geonames.org/export/dump/PK.zip unzip PK.zip }}} In Web2py CLI: {{{ gis.import_geonames('PK', 'L5') db.commit() }}} Alternate approach: 1. Transform each line in this file into XML by regular expression: {{{ ^(\d*)\t([^\t]*)\t([^\t]*)\t([^\t]*)\t([0-9\.]*)\t([0-9\.]*)\t[^\t]*\t([A-Z]*).* into: $1 $2 $3 $4 $5 $6 $7 }}} This can be done using an RE-capable editor (e.g. Kate), Perl or even Python. Note: Need to replace & with & and to remove any invalid characters 2. Transform into S3XRC-XML using XSLT, stylesheet is available at * [http://pub.nursix.org/eden/geonames/geonames.xsl] === !OpenStreetMap === We have an XSLT [http://eden.sahanafoundation.org/browser/static/xslt/import/osm.xsl stylesheet] to import .osm files, but this needs work to understand the admin hierarchy properly. * http://wiki.openstreetmap.org/wiki/Map_Features#Places * http://wiki.openstreetmap.org/wiki/Placename_hierachies is_in tag Geofabrik have updated extracts dailty for Pakistan: * http://download.geofabrik.de/osm/asia/pakistan.osm.bz2 Otherwise pull a BBOX directly using Osmosis: * http://wiki.openstreetmap.org/wiki/Osmosis === Shapefiles === Inspect the data using [http://qgis.org qGIS]. Use ogr2ogr to convert the data: {{{ ogr2ogr -f CSV CSV TM_WORLD_BORDERS-0.3.shp -lco GEOMETRY=AS_WKT ogr2ogr -f geojson TM_WORLD_BORDERS-0.3.json TM_WORLD_BORDERS-0.3.shp }}} If needing to reproject (e.g. for the [https://www.geoint-online.net/community/haitiearthquake/Geospatial%20Data%20Files/haiti_departments01132010.zip Haiti Departements]): {{{ ogr2ogr -f CSV haiti_departments Haiti_departementes_edited_01132010.shp -s_srs EPSG:32618 -t_srs EPSG:4326 -lco GEOMETRY=AS_WKT }}} NB AS_WKT requires OGR v1.6+ There is a function available in {{{modules/s3gis.py}}} to import from CSV. Example for Pakistan: {{{ db.gis_location.truncate() # L0 import csv csv.field_size_limit(2**20 * 10) # 10 megs db.import_from_csv_file(open('L0.csv','rb')) db.commit() # L1 gis.import_csv('pak_adm1.csv') db.commit() # L2 db(db.gis_location.name=="Baluchistan").update(name="Balochistan") db(db.gis_location.name=="Northern Areas").update(name="Gilgit Baltistan") db(db.gis_location.name=="N.W.F.P.").update(name="Khyber Pakhtunkhwa") db(db.gis_location.name=="F.A.T.A.").update(name="FATA") db(db.gis_location.name=="F.C.T.").update(name="Islamabad") db(db.gis_location.name=="Azad Kashmir").update(name="AJK") gis.import_csv('pak_adm2.csv') db(db.gis_location.name=="Sind").update(name="Sindh") db(db.gis_location.name=="AJK").update(name="Pakistan Administered Kashmir") db(db.gis_location.name=="Islamabad").update(name="Federal Capital Territory") db(db.gis_location.name=="FATA").update(name="Federally Administered Tribal Areas") db.commit() # L2 db(db.gis_location.name=="Islamabad Capital Territory").update(name="Islamabad") db(db.gis_location.name=="Jaccobabad").update(name="Jacobabad") db(db.gis_location.name=="Tando Allahyar").update(name="Tando Allah Yar") db(db.gis_location.name=="Qambar Shahdad kot").update(name="Qambar Shahdadkot") gis.import_csv('pak_adm3.csv') db(db.gis_location.name=="Islamabad").update(name="Islamabad Capital Territory") db(db.gis_location.name=="Tando Allah Yar").update(name="Tando Allahyar") db(db.gis_location.name=="Qambar Shahdadkot").update(name="Qambar Shahdad Kot") db.commit() }}} === WFS === It is possible to use the WFS Plugin to get data into qGIS & thence export into other formats. May need to use a Custom CRS (in Settings menu - remember to Save!) such as: * ESRI's Spherical Mercator ([http://www.cadmaps.com/gisblog/?p=81 different] to 900913) http://spatialreference.org/ref/esri/54004/proj4/ Can then go to the Layer Properties & Specify CRS to this User Defined Coordinate System. Can then Save As and change the CRS to something like the standard WGS84. == Display Data == === !GeoServer === !GeoServer can provide geospatial data in Raster (WMS) or Vector (WFS/KML) formats. Once you have installed ([wiki:InstallationGuidelinesGISDataLinux#GeoServer Linux], [wiki:InstallationGuidelinesGISDataWindows#GeoServer Windows]), then login: * l: admin * p: geoserver Configure: * [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.web.security.user.UserPage Admin Password] * [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.web.admin.ContactPage Contact Details] * [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.wfs.web.WFSAdminPage WFS Details] * [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.wms.web.WMSAdminPage WMS Details] inc the Limited SRS List - probably to just: {{{4326, 900913}}} * Disable the demo [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.web.data.layer.LayerPage Layers] & [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.web.data.layergroup.LayerGroupPage Layer Groups] * Add [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.web.data.workspace.WorkspacePage Workspace] * Add [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.web.data.store.StorePage PostGIS Store] * http://docs.geoserver.org/stable/en/user/gettingstarted/web-admin-quickstart/index.html ==== Import Shapefiles ==== e.g. Country Outlines: * http://thematicmapping.org/downloads/TM_WORLD_BORDERS-0.3.zip These can be loaded direct into !GeoServer, however there will be better performance by importing into PostGIS: [[BR]](can also use pgAdmin III GUI's Shapefile loader on plugins menu) {{{ su postgres shp2pgsql -s 4326 -I TM_WORLD_BORDERS-0.3.shp public.countries | psql -d gis }}} To reproject the data into 900913 for a slight performance advantage: {{{ drop constraint srid; update table set geomcolumn=transform(geomcolumn,900913); }}} ==== Configure !GeoServer ==== * Add [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.web.data.layer.LayerPage Layer] * Add [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.wms.web.data.StylePage Style]: * http://docs.geoserver.org/stable/en/user/styling/sld-introduction.html * Publish * Apply Style ==== Add WMS Layer to Sahana Eden ==== * ''tbc'' === Grid === ''tbc'': [wiki:BluePrintGISGrid] === Topographic Maps === * http://wiki.openstreetmap.org/wiki/Contours * Uses [http://wiki.openstreetmap.org/wiki/SRTM SRTM] ([http://wiki.openstreetmap.org/wiki/ASTER ASTER] is a future possibility) * Alternate approach: [http://wiki.openstreetmap.org/wiki/Srtm2Osm SRTM 2 OSM] === Old Printed Maps === Old Printed Maps can be 'Rectified' to be overlaid on the base maps: * http://warper.geothings.net/ * http://mike.teczno.com/notes/canvas-warp.html * Code: http://github.com/migurski/canvas-warp * http://labs.metacarta.com/rectifier/ == PostgreSQL management == * Connect to Database (or use pgAdmin III GUI's PSQL Console on plugins menu) {{{ su postgres psql }}} or {{{ psql -d my_spatial_db -U username }}} * List Databases {{{ \l }}} * List Tables {{{ \d }}} * List Table structure {{{ \d tablename select field/* from tablename where field = 'value'; }}} === PostGIS functions === * Centroids {{{ SELECT name, iso2, asText(ST_Transform(ST_Centroid(the_geom), 4326)) AS centroid FROM countries; }}} === Data Sources === * VMap0: http://en.wikipedia.org/wiki/Vector_Map * Admin 0 & 1, Hydro features, Raster & Vector: http://www.naturalearthdata.com/ * Admin 1-4: http://geonames.org ==== OGC (WMS/WFS) ==== * http://geonetwork-opensource.org/geonetwork-nodes * http://www.ogc-services.net * !BlueMarble & Mosaic (!LandSat7): http://onearth.jpl.nasa.gov/ * Hazard Layers: http://preview.grid.unep.ch:8080/geoserver/ows?service=WMS&request=GetCapabilities * Hazard Layers: http://www.pdc.org/mde/services.jsp * Massachusetts: http://lyceum.massgis.state.ma.us/wiki/doku.php ---- [wiki:GIS]