[[TOC]] = User Guidelines for GIS Data = Assumes installation of the relevant tools: [wiki:InstallationGuidelinesGISData] == Import Data == See current urgent task: PakistanUnionCouncils === UUIDs === See: [wiki:UUID#Mapping] === Geonames === There is an import_geonames() function in S3GIS which downloads/unzips the country file (a TAB-separated list) from http://download.geonames.org/export/dump/ It should be run for the different levels of hierarchy that you wish to import (generally just the lowest level as Geonames just has Point data, so it's best to use other sources for the Polygons 1st, that way the Geonames importer can locate these Points within the correct Polygons of the hierarchy) NB It takes some time to do this import! Pakistan imports 95000 locations! Update: Geonames schema 2.2 supports parentADM(1-4): http://geonames.wordpress.com/2010/09/29/geonames-ontology-2-2/ * will be good for when we only have hierarchy, not polygons * need to check whether much data has this populated though. Python 2.5 doesn't support Zipfile.extract() & Zipfile.read() isn't unicode-safe. Until this is fixed, download the file manually 1st: {{{ cd ~web2py/applications/eden/cache wget http://download.geonames.org/export/dump/PK.zip unzip PK.zip }}} In Web2py CLI: {{{ gis.import_geonames('PK', 'L5') db.commit() }}} Alternate approach: 1. Transform each line in this file into XML by regular expression: {{{ ^(\d*)\t([^\t]*)\t([^\t]*)\t([^\t]*)\t([0-9\.]*)\t([0-9\.]*)\t[^\t]*\t([A-Z]*).* into: $1 $2 $3 $4 $5 $6 $7 }}} This can be done using an RE-capable editor (e.g. Kate), Perl or even Python. Note: Need to replace & with & and to remove any invalid characters 2. Transform into S3XRC-XML using XSLT, stylesheet is available at * [http://pub.nursix.org/eden/geonames/geonames.xsl] === !OpenStreetMap === See below: [wiki:UserGuidelinesGISData#Import] === Shapefiles === Inspect the data using [http://qgis.org qGIS]. Use ogr2ogr to convert the data: {{{ ogr2ogr -f CSV CSV TM_WORLD_BORDERS-0.3.shp -lco GEOMETRY=AS_WKT ogr2ogr -f geojson TM_WORLD_BORDERS-0.3.json TM_WORLD_BORDERS-0.3.shp }}} If needing to reproject (e.g. for the [https://www.geoint-online.net/community/haitiearthquake/Geospatial%20Data%20Files/haiti_departments01132010.zip Haiti Departements]): {{{ ogr2ogr -f CSV haiti_departments Haiti_departementes_edited_01132010.shp -s_srs EPSG:32618 -t_srs EPSG:4326 -lco GEOMETRY=AS_WKT }}} NB AS_WKT requires OGR v1.6+ There is a function available in {{{modules/s3gis.py}}} to import from CSV. Example for Pakistan: {{{ tablename = "gis_location" table = db[tablename] db.executesql("DROP INDEX name__idx on %s;" % tablename) # L0 db(table.name != "Pakistan").delete() db.commit() import csv csv.field_size_limit(2**20 * 10) # 10 megs db.import_from_csv_file(open("L0.csv", "rb")) db.commit() # L1 gis.import_csv("pak_adm1.csv", check_duplicates=False) db.commit() # L2 db(table.name == "Baluchistan").update(name="Balochistan") db(table.name == "Northern Areas").update(name="Gilgit Baltistan") db(table.name == "N.W.F.P.").update(name="Khyber Pakhtunkhwa") db(table.name == "F.A.T.A.").update(name="FATA") db(table.name == "F.C.T.").update(name="Islamabad") db(table.name == "Azad Kashmir").update(name="AJK") gis.import_csv("pak_adm2.csv", check_duplicates=False) db(table.name == "Sind").update(name="Sindh") db(table.name == "AJK").update(name="Pakistan Administered Kashmir") db(table.name == "FATA").update(name="Federally Administered Tribal Areas") db((table.name == "Islamabad") & (table.level == "L1")).update(name="Federal Capital Territory") db.commit() # L3 db(table.name == "Jaccobabad").update(name="Jacobabad") db(table.name == "Tando Allahyar").update(name="Tando Allah Yar") db(table.name == "Qambar Shahdad kot").update(name="Qambar Shahdadkot") gis.import_csv("pak_adm3.csv", check_duplicates=False) db(table.name == "Islamabad").update(name="Islamabad Capital Territory") db(table.name == "Tando Allah Yar").update(name="Tando Allahyar") db(table.name == "Qambar Shahdadkot").update(name="Qambar Shahdad Kot") db(table.name == "Leiah").update(name="Layyah") db(table.name == "Leiah Tehsil").update(name="Layyah Tehsil") db(table.name == "Kalur Kot Tehsil").update(name="Kallur Kot Tehsil") db(table.name == "De-excluded Area").update(name="Tribal Area") db(table.name == "De-excluded Area D.g Khan").update(name="Tribal Area") db.commit() # L4 db(table.name == "Noorpur Tehsil").update(name="Noorpur Thal Tehsil") jhang = db((table.name == "Jhang") & (table.level==L2)).select(table.id, limitby=(0, 1)).first().id table.insert(name="Ahmadpur Sial", parent=jhang, level="L3", url="http://en.wikipedia.org/wiki/Ahmedpur_Sial_Tehsil") gis.import_csv("punjab_l4.csv", check_duplicates=False) db.commit() db(table.name == "Mirwah Taluka").update(name="Thari Mirwah Taluka") db(table.name == "Shah Bunder Taluka").update(name="Shah Bandar Taluka") badin = db((table.name == "Badin") & (table.level==L2)).select(table.id, limitby=(0, 1)).first().id table.insert(name="Talhar", parent=badin, level="L3", url="http://en.wikipedia.org/wiki/Talhar") jamshoro = db((table.name == "Jamshoro") & (table.level==L2)).select(table.id, limitby=(0, 1)).first().id table.insert(name="Manjhand Taluka", parent=jamshoro, level="L3", url="http://en.wikipedia.org/wiki/Jamshoro_District") gis.import_csv("sindh_l4.csv", check_duplicates=False) db.commit() db(table.name == "F.r Kala Dhaka").update(name="F.R. Kala Dhaka") db(table.name == "Martoong Tehsil").update(name="Martung Tehsil") db(table.name == "Takhat Nasrati Tehsil").update(name="Takht-e-Nasrati Tehsil") dikhan = db((table.name == "D. I. Khan") & (table.level == "L2")).select(table.id, limitby=(0, 1)).first().id table.insert(name="Daraban Tehsil", parent=dikhan, level="L3") table.insert(name="Paroa Tehsil", parent=dikhan, level="L3") lowerdir = db((table.name == "Lower Dir") & (table.level == "L2")).select(table.id, limitby=(0, 1)).first().id table.insert(name="Adenzai", parent=lowerdir, level="L3") table.insert(name="Balambat", parent=lowerdir, level="L3") table.insert(name="Khal", parent=lowerdir, level="L3") table.insert(name="Lal Qila", parent=lowerdir, level="L3") table.insert(name="Munda", parent=lowerdir, level="L3") table.insert(name="Samar Bagh", parent=lowerdir, level="L3") table.insert(name="Tazagram", parent=lowerdir, level="L3") table.insert(name="Timargara", parent=lowerdir, level="L3") upperdir = db((table.name == "Upper Dir") & (table.level == "L2")).select(table.id, limitby=(0, 1)).first().id table.insert(name="Barawal Tehsil", parent=upperdir, level="L3") table.insert(name="Chapar Tehsil", parent=upperdir, level="L3") table.insert(name="Dir Tehsil", parent=upperdir, level="L3") table.insert(name="Khal Tehsil", parent=upperdir, level="L3") table.insert(name="Kalkot Tehsil", parent=upperdir, level="L3") table.insert(name="Wari Tehsil", parent=upperdir, level="L3") gis.import_csv("khyber_l4.csv", check_duplicates=False) db.commit() # L5 gis.import_csv("punjab_l5.csv", check_duplicates=False) gis.import_csv("sindh_l5.csv", check_duplicates=False) gis.import_csv("khyber_l5.csv", check_duplicates=False) db.commit() field = "name" db.executesql("CREATE INDEX %s__idx on %s(%s);" % (field, tablename, field)) }}} === Yahoo === * http://developer.yahoo.com/geo/geoplanet/data/ * Script to process http://pastie.org/1139680 === WFS === It is possible to use the WFS Plugin to get data into qGIS & thence export into other formats. May need to use a Custom CRS (in Settings menu - remember to Save!) such as: * ESRI's Spherical Mercator ([http://www.cadmaps.com/gisblog/?p=81 different] to 900913) http://spatialreference.org/ref/esri/54004/proj4/ Can then go to the Layer Properties & Specify CRS to this User Defined Coordinate System. Can then Save As and change the CRS to something like the standard WGS84. == Display Data == === !GeoServer === !GeoServer can provide geospatial data in Raster (WMS) or Vector (WFS/KML) formats. Once you have installed ([wiki:InstallationGuidelinesGISDataLinux#GeoServer Linux], [wiki:InstallationGuidelinesGISDataWindows#GeoServer Windows]), then login: * l: admin * p: geoserver Configure: * [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.web.security.user.UserPage Admin Password] * [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.web.admin.ContactPage Contact Details] * [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.web.admin.GlobalSettingsPage Disable Global Settings] (2.1+ only) * [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.wfs.web.WFSAdminPage WFS Details] * [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.wms.web.WMSAdminPage WMS Details] inc the Limited SRS List - probably to just: {{{4326, 900913}}} * Disable the demo [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.web.data.layer.LayerPage Layers] & [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.web.data.layergroup.LayerGroupPage Layer Groups] * Add [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.web.data.workspace.WorkspacePage Workspace] * Add [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.web.data.store.StorePage PostGIS Store] * http://docs.geoserver.org/stable/en/user/gettingstarted/web-admin-quickstart/index.html ==== Import Shapefiles ==== e.g. Country Outlines: * http://thematicmapping.org/downloads/TM_WORLD_BORDERS-0.3.zip These can be loaded direct into !GeoServer, however there will be better performance by importing into PostGIS: [[BR]](can also use pgAdmin III GUI's Shapefile loader on plugins menu) {{{ su postgres shp2pgsql -s 4326 -I TM_WORLD_BORDERS-0.3.shp public.countries | psql -d gis }}} To reproject the data into 900913 for a slight performance advantage: {{{ drop constraint srid; update table set geomcolumn=transform(geomcolumn,900913); }}} ==== Configure !GeoServer ==== * Add [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.web.data.layer.LayerPage Layer] * Add [http://geo.eden.sahanafoundation.org/geoserver/web/?wicket:bookmarkablePage=:org.geoserver.wms.web.data.StylePage Style]: * http://docs.geoserver.org/stable/en/user/styling/sld-introduction.html * Publish * Apply Style ==== Add WMS Layer to Sahana Eden ==== * ''tbc'' === Grid === ''tbc'': [wiki:BluePrintGISGrid] === Topographic Maps === * http://wiki.openstreetmap.org/wiki/Contours * Uses [http://wiki.openstreetmap.org/wiki/SRTM SRTM] ([http://wiki.openstreetmap.org/wiki/ASTER ASTER] is a future possibility) * Alternate approach: [http://wiki.openstreetmap.org/wiki/Srtm2Osm SRTM 2 OSM] === Old Printed Maps === Old Printed Maps can be 'Rectified' to be overlaid on the base maps: * [wiki:BluePrintGISRectifier] == !OpenStreetMap == === Base Map === We have out-of-the-box the ability to use !OpenStreetMap Tiles as base layer. This can include local OSM sites (OSM Taiwan is included as an example) === Vector Overlays === Can have OSM Vectors displayed over the top of other Base Layers (e.g. Satellite Images) * [wiki:BluePrintGISOSMLayers] === Import === We have an XSLT [http://eden.sahanafoundation.org/browser/static/xslt/import/osm.xsl stylesheet] to import .osm files, but this needs work to understand the admin hierarchy properly. * http://wiki.openstreetmap.org/wiki/Map_Features#Places * http://wiki.openstreetmap.org/wiki/Placename_hierachies is_in tag Geofabrik have updated extracts daily for Pakistan: * http://download.geofabrik.de/osm/asia/pakistan.osm.bz2 Otherwise pull a BBOX directly using Osmosis: * http://wiki.openstreetmap.org/wiki/Osmosis Ruby script to generate KML of recently-added locations by a group of users: * http://github.com/vertis/osm_export/tree/master === Basemap for Garmin GPS === * [wiki:UserGuidelinesGISDataOpenStreetMapGarmin] == PostgreSQL management == * [wiki:InstallationGuidelinesPostgreSQL#Maintenance] === PostGIS functions === * Centroids {{{ SELECT name, iso2, asText(ST_Transform(ST_Centroid(the_geom), 4326)) AS centroid FROM countries; }}} === Data Sources === * VMap0: http://en.wikipedia.org/wiki/Vector_Map * Admin 0 & 1, Hydro features, Raster & Vector: http://www.naturalearthdata.com/ * Admin 1-4: http://geonames.org ==== OGC (WMS/WFS) ==== * http://geonetwork-opensource.org/gallery.html#geonetwork-nodes * http://www.ogc-services.net * !BlueMarble & Mosaic (!LandSat7): http://onearth.jpl.nasa.gov/ * Hazard Layers: http://preview.grid.unep.ch:8080/geoserver/ows?service=WMS&request=GetCapabilities * Hazard Layers: http://www.pdc.org/mde/services.jsp * Massachusetts: http://lyceum.massgis.state.ma.us/wiki/doku.php ---- [wiki:GIS]