User Guidelines for GIS Data

Assumes installation of the relevant tools: InstallationGuidelinesGISData

Import Data

See current urgent task: PakistanUnionCouncils

UUIDs

See: wiki:UUID#Mapping

Geonames

There is an import_geonames() function in S3GIS which downloads/unzips the country file (a TAB-separated list) from ​http://download.geonames.org/export/dump/

It should be run for the different level sof hierarchy that you wish to import (generally just the lowest level as Geonames just has Point data, so it's best to use other sources for the Polygons 1st, that way the Geonames importer can locate these Points within the correct Polygons of the hierarchy)

NB It takes some time to do this import! Pakistan imports 95000 locations!

Python 2.5 doesn't support Zipfile.extract() & Zipfile.read() isn't unicode-safe. Until this is fixed, download the file manually 1st:

cd ~web2py/applications/eden/cache
wget http://download.geonames.org/export/dump/PK.zip
unzip PK.zip

In Web2py CLI:

gis.import_geonames('PK', 'L5')
db.commit()

Alternate approach:

1. Transform each line in this file into XML by regular expression:

   ^(\d*)\t([^\t]*)\t([^\t]*)\t([^\t]*)\t([0-9\.]*)\t([0-9\.]*)\t[^\t]*\t([A-Z]*).*
   
   into:
   
   <location>
          <id>$1</id>
          <name>$2</name>
          <asciiName>$3</asciiName>
          <localNames>$4</localNames>
          <lat>$5</lat>
          <lon>$6</lon>
          <featureClass>$7</featureClass>
   </location>
   
   

This can be done using an RE-capable editor (e.g. Kate), Perl or even Python. Note: Need to replace & with & and to remove any invalid characters

2. Transform into S3XRC-XML using XSLT, stylesheet is available at
   • ​http://pub.nursix.org/eden/geonames/geonames.xsl

OpenStreetMap

We have an XSLT stylesheet to import .osm files, but this needs work to understand the admin hierarchy properly.

• ​http://wiki.openstreetmap.org/wiki/Map_Features#Places
• ​http://wiki.openstreetmap.org/wiki/Placename_hierachies is_in tag

Geofabrik have updated extracts dailty for Pakistan:

• ​http://download.geofabrik.de/osm/asia/pakistan.osm.bz2

Otherwise pull a BBOX directly using Osmosis:

• ​http://wiki.openstreetmap.org/wiki/Osmosis

Ruby script to generate KML of recently-added locations by a group of users:

• ​http://github.com/vertis/osm_export/tree/master

Shapefiles

Inspect the data using ​qGIS.

Use ogr2ogr to convert the data:

ogr2ogr -f CSV CSV TM_WORLD_BORDERS-0.3.shp -lco GEOMETRY=AS_WKT
ogr2ogr -f geojson TM_WORLD_BORDERS-0.3.json TM_WORLD_BORDERS-0.3.shp

If needing to reproject (e.g. for the ​Haiti Departements):

ogr2ogr -f CSV haiti_departments Haiti_departementes_edited_01132010.shp -s_srs EPSG:32618 -t_srs EPSG:4326 -lco GEOMETRY=AS_WKT

NB AS_WKT requires OGR v1.6+

There is a function available in modules/s3gis.py to import from CSV.

Example for Pakistan:

db.gis_location.truncate()
# L0
import csv
csv.field_size_limit(2**20 * 10)  # 10 megs
db.import_from_csv_file(open('L0.csv','rb'))
db.commit()
# L1
gis.import_csv('pak_adm1.csv')
db.commit()
# L2
db(db.gis_location.name=="Baluchistan").update(name="Balochistan")
db(db.gis_location.name=="Northern Areas").update(name="Gilgit Baltistan")
db(db.gis_location.name=="N.W.F.P.").update(name="Khyber Pakhtunkhwa")
db(db.gis_location.name=="F.A.T.A.").update(name="FATA")
db(db.gis_location.name=="F.C.T.").update(name="Islamabad")
db(db.gis_location.name=="Azad Kashmir").update(name="AJK")
gis.import_csv('pak_adm2.csv')
db(db.gis_location.name=="Sind").update(name="Sindh")
db(db.gis_location.name=="AJK").update(name="Pakistan Administered Kashmir")
db(db.gis_location.name=="Islamabad").update(name="Federal Capital Territory")
db(db.gis_location.name=="FATA").update(name="Federally Administered Tribal Areas")
db.commit()
# L2
db(db.gis_location.name=="Islamabad Capital Territory").update(name="Islamabad")
db(db.gis_location.name=="Jaccobabad").update(name="Jacobabad")
db(db.gis_location.name=="Tando Allahyar").update(name="Tando Allah Yar")
db(db.gis_location.name=="Qambar Shahdad kot").update(name="Qambar Shahdadkot")
gis.import_csv('pak_adm3.csv')
db(db.gis_location.name=="Islamabad").update(name="Islamabad Capital Territory")
db(db.gis_location.name=="Tando Allah Yar").update(name="Tando Allahyar")
db(db.gis_location.name=="Qambar Shahdadkot").update(name="Qambar Shahdad Kot")
db(db.gis_location.name=="Leiah").update(name="Layyah")
db(db.gis_location.name=="Leiah Tehsil").update(name="Layyah Tehsil")
db(db.gis_location.name=="Kalur Kot Tehsil").update(name="Kallur Kot Tehsil")
db(db.gis_location.name=="De-excluded Area").update(name="Tribal Area")
db(db.gis_location.name=="De-excluded Area D.g Khan").update(name="Tribal Area")
db.commit()

Yahoo

• ​http://developer.yahoo.com/geo/geoplanet/data/
• Script to process ​http://pastie.org/1139680

WFS

It is possible to use the WFS Plugin to get data into qGIS & thence export into other formats.

May need to use a Custom CRS (in Settings menu - remember to Save!) such as:

• ESRI's Spherical Mercator (​different to 900913) ​http://spatialreference.org/ref/esri/54004/proj4/

Can then go to the Layer Properties & Specify CRS to this User Defined Coordinate System.

Can then Save As and change the CRS to something like the standard WGS84.

Display Data

GeoServer

GeoServer can provide geospatial data in Raster (WMS) or Vector (WFS/KML) formats.

Once you have installed (Linux, Windows), then login:

• l: admin
• p: geoserver

Configure:

• ​Admin Password
• ​Contact Details
• ​WFS Details
• ​WMS Details inc the Limited SRS List - probably to just: 4326, 900913
• Disable the demo ​Layers & ​Layer Groups
• Add ​Workspace
• Add ​PostGIS Store

• ​http://docs.geoserver.org/stable/en/user/gettingstarted/web-admin-quickstart/index.html

Import Shapefiles

e.g. Country Outlines:

• ​http://thematicmapping.org/downloads/TM_WORLD_BORDERS-0.3.zip

These can be loaded direct into GeoServer, however there will be better performance by importing into PostGIS:
(can also use pgAdmin III GUI's Shapefile loader on plugins menu)

su postgres
shp2pgsql -s 4326 -I TM_WORLD_BORDERS-0.3.shp public.countries | psql -d gis

To reproject the data into 900913 for a slight performance advantage:

drop constraint srid;
update table set geomcolumn=transform(geomcolumn,900913);

Configure GeoServer

• Add ​Layer
• Add ​Style:
  • ​http://docs.geoserver.org/stable/en/user/styling/sld-introduction.html
• Publish
  • Apply Style

Add WMS Layer to Sahana Eden

• tbc

Grid

tbc: BluePrintGISGrid

Topographic Maps

• ​http://wiki.openstreetmap.org/wiki/Contours
  • Uses ​SRTM (​ASTER is a future possibility)
• Alternate approach: ​SRTM 2 OSM

Old Printed Maps

Old Printed Maps can be 'Rectified' to be overlaid on the base maps:

• ​http://warper.geothings.net/
• ​http://mike.teczno.com/notes/canvas-warp.html
  • Code: ​http://github.com/migurski/canvas-warp
• ​http://labs.metacarta.com/rectifier/

PostgreSQL management

• Connect to Database (or use pgAdmin III GUI's PSQL Console on plugins menu)

  su postgres
  psql
  

or

psql -d my_spatial_db -U username

• List Databases

  \l
  

• List Tables

  \d
  

• List Table structure

  \d tablename
  select field/* from tablename where field = 'value';
  

PostGIS functions

• Centroids

  SELECT name, iso2, asText(ST_Transform(ST_Centroid(the_geom), 4326)) AS centroid FROM countries;
  

Data Sources

• VMap0: ​http://en.wikipedia.org/wiki/Vector_Map
• Admin 0 & 1, Hydro features, Raster & Vector: ​http://www.naturalearthdata.com/
• Admin 1-4: ​http://geonames.org

OGC (WMS/WFS)

• ​http://geonetwork-opensource.org/geonetwork-nodes
• ​http://www.ogc-services.net
• BlueMarble & Mosaic (!LandSat7): ​http://onearth.jpl.nasa.gov/
• Hazard Layers: ​http://preview.grid.unep.ch:8080/geoserver/ows?service=WMS&request=GetCapabilities
• Hazard Layers: ​http://www.pdc.org/mde/services.jsp
• Massachusetts: ​http://lyceum.massgis.state.ma.us/wiki/doku.php

---

GIS