= !OpenStreetMap = [[TOC]] '''Wikipedia of Maps''' http://openstreetmap.org == Base Map == We have out-of-the-box the ability to use !OpenStreetMap Tiles as base layer. This can include local OSM sites (OSM Taiwan is included as an example) === Local Tile Server === If you need to run offline then you need to set up your own Tile Server: * http://switch2osm.org/serving-tiles/ * http://switch2osm.org/serving-tiles/manually-building-a-tile-server-12-04/ * http://blog.gravitystorm.co.uk/2010/07/05/map-rendering-on-ec2/ * https://help.openstreetmap.org/questions/11949/server-spec-for-running-osm * http://weait.com/content/build-your-own-openstreetmap-server * http://weait.com/content/make-your-first-map * http://dbsgeo.com/foss4g2010/pdf/osm-rendering-foss4g2010.pdf !OpenStreetMap data can be stored in a PostGIS database & rendered using Mapnik. To support !Key/Value pairs: {{{ apt-get install postgresql-contrib-9.3 su postgres psql -d gis -f /usr/share/postgresql/9.3/contrib/hstore.sql }}} To support Planet diff files: {{{ su postgres psql gis < /usr/share/postgresql/9.3/contrib/_int.sql }}} === Osm2pgsql === * http://wiki.openstreetmap.org/wiki/Osm2pgsql It is recommended to use the latest version from source rather than an out of date package: {{{ apt-get install build-essential libxml2-dev libgeos-dev libpq-dev libbz2-dev proj autoconf subversion libtool svn export http://svn.openstreetmap.org/applications/utils/export/osm2pgsql/ Optional (patch updated from original here): http://www.mail-archive.com/dev@openstreetmap.org/msg12768.html #wget http://eden.sahanafoundation.org/raw-attachment/wiki/InstallationGuidelinesGISDataLinux/osm2pgsql_centroid.patch cd osm2pgsql #patch -p0 < ../osm2pgsql_centroid.patch ./autogen.sh ./configure sed -i 's/-g -O2/-O2 -march=native -fomit-frame-pointer/' Makefile make make install # Fix needed for SVN version 0.70.5 cp default.style /usr/local/share }}} === Mapnik === * http://wiki.openstreetmap.org/wiki/Mapnik * http://mike.teczno.com/notes/mapnik.html (Advanced Tips) 0.7.1 is packaged for Squeeze: {{{ apt-get install python-mapnik unzip }}} Download the !OpenStreetMap extensions for Mapnik: {{{ cd svn export http://svn.openstreetmap.org/applications/rendering/mapnik }}} Download/decompress Coastlines: {{{ cd wget http://tile.openstreetmap.org/world_boundaries-spherical.tgz wget http://tile.openstreetmap.org/processed_p.tar.bz2 wget http://tile.openstreetmap.org/shoreline_300.tar.bz2 wget http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/10m-populated-places.zip wget http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/110m/cultural/110m-admin-0-boundary-lines.zip cd ~/mapnik tar zxvf ~/world_boundaries-spherical.tgz tar jxvf ~/processed_p.tar.bz2 -C world_boundaries tar jxvf ~/shoreline_300.tar.bz2 -C world_boundaries unzip ~/10m-populated-places.zip -d world_boundaries unzip ~/110m-admin-0-boundary-lines.zip -d world_boundaries }}} Disable autovacuum: {{{ vim /etc/postgresql/8.4/main/postgresql.conf #autovacuum = on /etc/init.d/postgresql restart }}} Download/Import area of interest, e.g.: {{{ #wget http://downloads.cloudmade.com/north_america/haiti/haiti.osm.bz2 #su postgres #osm2pgsql -s -d gis haiti.osm.bz2 wget http://labs.geofabrik.de/haiti/2010-11-24-14-43.osm.bz2 su postgres osm2pgsql -s -d gis 2010-11-24-14-43.osm.bz2 }}} Re-enable autovacuum: {{{ vim /etc/postgresql/8.4/main/postgresql.conf autovacuum = on /etc/init.d/postgresql restart }}} Setup XML file: {{{ cd ~/mapnik ./generate_xml.py --accept-none --dbname gis --symbols ./symbols/ --world_boundaries ./world_boundaries/ }}} Render tiles: {{{ cd ~/mapnik vim generate_tiles.py render_tiles(bbox, mapfile, tile_dir, 0, 8, "World") #minZoom = 10 #maxZoom = 16 #bbox = (-2, 50.0,1.0,52.0) #render_tiles(bbox, mapfile, tile_dir, minZoom, maxZoom) # Haiti bbox = (-74.668, 17.884, -71.659, 20.232) render_tiles(bbox, mapfile, tile_dir, 9, 17, "Haiti") # Muenchen # comment all under here mkdir /var/www/tiles chown postgres /var/www/tiles ln -s /var/www/tiles su postgres MAPNIK_MAP_FILE="osm.xml" MAPNIK_TILE_DIR="tiles/" ./generate_tiles.py }}} === Contours === * http://wiki.openstreetmap.org/wiki/Contours#The_PostGIS_approach {{{ apt-get install gdal-bin }}} [http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ SRTM3] data: {{{ mkdir ~/mapnik/srtm cd ~/mapnik/srtm wget http://mapnik-utils.googlecode.com/svn@170/sandbox/testing/hillshading/srtm_generate_hdr.sh chmod +x srtm_generate_hdr.sh cp srtm_generate_hdr.sh /usr/local/bin vim process_srtm3.sh #!/bin/bash PREP_TABLE="1" for X in *.hgt.zip; do yes | srtm_generate_hdr.sh $X rm -f "${X%%.zip}" # Import 10m contours rm -f "${X%%.hgt.zip}.shp" "${X%%.hgt.zip}.shx" "${X%%.hgt.zip}.dbf" gdal_contour -i 10 -snodata 32767 -a height "${X%%.hgt.zip}.tif" "${X%%.hgt.zip}.shp" [ "$PREP_TABLE" ] && shp2pgsql -p -I -g way "${X%%.hgt.zip}" contours | psql -q gis shp2pgsql -a -g way "${X%%.hgt.zip}" contours | psql -q gis rm -f "${X%%.hgt.zip}.shp" "${X%%.hgt.zip}.shx" "${X%%.hgt.zip}.dbf" rm -f "${X%%.hgt.zip}.bil" rm -f "${X%%.hgt.zip}.hdr" rm -f "${X%%.hgt.zip}.prj" rm -f "${X%%.hgt.zip}.tif" unset PREP_TABLE done # Haiti wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N17W072.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N18W072.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N18W073.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N18W074.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N18W075.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N19W072.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N19W073.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N19W074.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N19W075.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N20W073.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N20W074.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N20W075.hgt.zip chown postgres . su postgres sh process_srtm3.sh }}} Since we are non-commercial, we should be able to use the SRTM4.1 data from [http://srtm.csi.cgiar.org CGIAR], however the Shapefile production crashed when it reached 2Gb.[[BR]] Example for Haiti: {{{ cd ~/mapnik/srtm wget http://srtm.csi.cgiar.org/SRT-ZIP/SRTM_V41/SRTM_Data_GeoTiff/srtm_22_09.zip unzip srtm_22_09.zip gdal_contour -i 10 -snodata 32767 -a height "srtm_22_09.tif" "srtm_22_09.shp" su postgres shp2pgsql -p -I -g way "srtm_22_09" contours | psql -q gis shp2pgsql -a -g way "srtm_22_09" contours | psql -q gis }}} Configure Mapnik: {{{ vim ~/mapnik/osm.xml # Under line &layer-shapefiles; &layer-contours; vim ~/mapnik/inc/layer-contours.xml.inc contours10 contours-text10 (select way,height from contours WHERE height::integer % 10 = 0 AND height::integer % 50 != 0 AND height::integer % 100 != 0) as "contours-10" &datasource-settings; contours50 contours-text50 (select way,height from contours WHERE height::integer % 50 = 0 AND height::integer % 100 != 0) as "contours-50" &datasource-settings; contours100 contours-text100 (select way,height from contours WHERE height::integer % 100 = 0) as "contours-100" &datasource-settings; }}} ==== Hillshading ==== * http://wiki.openstreetmap.org/wiki/HikingBikingMaps#Hill_Shading === Serve Tiles === Tiles can simply be served by Apache: {{{ vim /etc/apache2/sites-available/mysite ServerName mysite.org Alias /tiles /var/www/tiles Order Allow,Deny Allow from all apache2ctl restart }}} === WMS === In order to serve the OSM data via WMS then can use mod_mapnik_wms: * http://wiki.openstreetmap.org/wiki/Mod_mapnik_wms {{{ gpg --gen-key apt-get install pbuilder debsigs apache2-prefork-dev libmapnik-dev libgd2-xpm-dev svn export http://svn.openstreetmap.org/applications/utils/mod_mapnik_wms cd mod_mapnik_wms sh autogen.sh vim debian/mapnik_wms.load LoadFile /usr/lib/libmapnik.so.0.7 debuild }}} This data can then be converted by !GeoServer into a KML !SuperOverlay to display in Google Earth: * http://geoserver.org/display/GEOS/GSIP+47+-+WMS+cascading * http://docs.geoserver.org/stable/en/user/googleearth/tutorials/superoverlaysgwc.html === Rails Port === This is needed to allow editing of the data using Potlatch, JOSM, etc * http://wiki.openstreetmap.org/wiki/The_Rails_Port {{{ apt-get install imagemagick libmagick9-dev apt-get install ruby ruby1.8-dev libxml2-dev libxml-ruby1.8 libxml-parser-ruby1.8 rubygems librmagick-ruby gem install -v=2.3.8 rails #gem install libxml-ruby #gem install composite_primary_keys #gem install rmagick gem install timecop gem install pg gem install oauth #svn co http://railsexpress.de/svn/plugins/sql_session_store/trunk sql_session_store apt-get install postgresql-contrib libpq-dev su postgres createuser openstreetmap -s -P createdb -E UTF8 -O openstreetmap openstreetmap createdb -E UTF8 -O openstreetmap osm_test createdb -E UTF8 -O openstreetmap osm psql -d openstreetmap < /usr/share/postgresql/8.4/contrib/btree_gist.sql apt-get install git cd /home git clone git://git.openstreetmap.org/rails.git cd rails cp config/postgres.example.database.yml config/database.yml vim config/database.yml rake gems:install rake db:migrate env RAILS_ENV=production rake db:migrate rake test osmosis --read-xml-0.6 file="planet.osm.bz2" --write-apidb-0.6 populateCurrentTables=yes host="localhost" database="openstreetmap" user="openstreetmap" password="openstreetmap" validateSchemaVersion=no select setval('acls_id_seq', (select max(id) from acls)); select setval('changesets_id_seq', (select max(id) from changesets)); select setval('countries_id_seq', (select max(id) from countries)); select setval('current_nodes_id_seq', (select max(id) from current_nodes)); select setval('current_relations_id_seq', (select max(id) from current_relations)); select setval('current_ways_id_seq', (select max(id) from current_ways)); select setval('diary_comments_id_seq', (select max(id) from diary_comments)); select setval('diary_entries_id_seq', (select max(id) from diary_entries)); select setval('friends_id_seq', (select max(id) from friends)); select setval('gpx_file_tags_id_seq', (select max(id) from gpx_file_tags)); select setval('gpx_files_id_seq', (select max(id) from gpx_files)); select setval('messages_id_seq', (select max(id) from messages)); select setval('sessions_id_seq', (select max(id) from sessions)); select setval('user_tokens_id_seq', (select max(id) from user_tokens)); select setval('users_id_seq', (select max(id) from users)); cd /home/rails ruby script/server cd db/functions make libpgosm.so * Log into PgSQL and execute the CREATE FUNCTION statement from maptile.c's comment: CREATE FUNCTION maptile_for_point(int8, int8, int4) RETURNS int4 AS '/path/to/rails-port/db/functions/libpgosm', 'maptile_for_point' LANGUAGE C STRICT; CREATE FUNCTION tile_for_point(int4, int4) RETURNS int8 AS '/path/to/rails-port/db/functions/libpgosm', 'tile_for_point' LANGUAGE C STRICT; }}} == Vector Overlays == Can have OSM Vectors displayed over the top of other Base Layers (e.g. Satellite Images) * [wiki:BluePrint/GIS/OSMLayers] == Import == We have an XSLT [http://eden.sahanafoundation.org/browser/static/formats/osm/import.xsl stylesheet] to import .osm files e.g for hospitals and clinics: {{{ osmosis --read-xml country.osm --tf accept-nodes amenity=hospital,clinic --tf reject-ways --tf reject-relations --write-xml nodes.osm osmosis --read-xml country.osm --tf reject-relations --tf accept-ways amenity=hospital,clinic --used-node --write-xml ways.osm osmosis --rx nodes.osm --rx ways.osm --merge --wx country_hospitals.osm http://myhost.com/eden/hms/hospital/create.osm?filename=country_hospitals.osm }}} This needs more work to understand the admin hierarchy properly to be able to import Places. * http://wiki.openstreetmap.org/wiki/Map_Features#Places * http://wiki.openstreetmap.org/wiki/Placename_hierachies is_in tag Geofabrik have updated extracts daily for Pakistan: * http://download.geofabrik.de/osm/asia/pakistan.osm.bz2 Otherwise pull a BBOX directly using Osmosis: * http://wiki.openstreetmap.org/wiki/Osmosis Osmosis requires Java. Python options for filtering based on tag, which would be more suitable for integration within Sahana, however we need to add Polygon filtering using Shapely: * http://dev.omniscale.net/imposm.parser/ * [http://pbf.raggedred.net ParsePBF] * http://blog.lifeeth.in/2011/02/extract-pois-from-osm-pbf.html Ruby script to generate KML of recently-added locations by a group of users: * http://github.com/vertis/osm_export/tree/master Polygons: * http://help.openstreetmap.org/questions/8273/how-do-i-extract-the-polygon-of-an-administrative-boundary == Basemap for Garmin GPS == * [wiki:UserGuidelinesGISDataOpenStreetMapGarmin] == Mirror == How to set up a local mirror for Africa with selected Feature types, suitable for use by Import PoI: {{{ apt-get install -y postgresql-8.4-postgis postgresql-contrib mkdir -p /home/osm/planet/replication cd /home/osm wget http://dev.openstreetmap.org/~bretth/osmosis-build/osmosis-latest.tgz tar zxvf osmosis-latest.tgz ln -sf /home/osm/osmosis-0.41 /home/osm/osmosis chown -R postgres /home/osm su postgres psql CREATE USER osm WITH PASSWORD 'planet'; \q createdb -O osm -E UTF8 osm -T template0 createlang plpgsql -d osm psql -d osm -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql psql -d osm -f /usr/share/postgresql/8.4/contrib/postgis-1.5/spatial_ref_sys.sql psql -d osm -f /usr/share/postgresql/8.4/contrib/hstore.sql psql -d osm -f /home/osm/osmosis/script/pgsnapshot_schema_0.6.sql psql \c osm ALTER TABLE geography_columns OWNER TO osm; ALTER TABLE geometry_columns OWNER TO osm; ALTER TABLE nodes OWNER TO osm; ALTER TABLE relation_members OWNER TO osm; ALTER TABLE relations OWNER TO osm; ALTER TABLE schema_info OWNER TO osm; ALTER TABLE spatial_ref_sys OWNER TO osm; ALTER TABLE users OWNER TO osm; ALTER TABLE way_nodes OWNER TO osm; ALTER TABLE ways OWNER TO osm; \q exit cd /home/osm/planet wget http://download.geofabrik.de/openstreetmap/africa.osm.pbf /home/osm/osmosis/bin/osmosis --read-pbf /home/osm/planet/africa.osm.pbf --tf accept-nodes amenity=hospital,place_of_worship,school --tf accept-ways amenity=hospital,place_of_worship,school --write-pbf /home/osm/planet/eurosha.osm.pbf su postgres /home/osm/osmosis/bin/osmosis --read-pbf /home/osm/planet/eurosha.osm.pbf --write-pgsql database=osm user=osm password=planet cd /home/osm/planet/replication /home/osm/osmosis/bin/osmosis --rrii sudo -H -u postgres psql -d osm -c "select max(tstamp) from nodes"; http://toolserver.org/~mazder/replicate-sequences/ Put timestamp into site, select hourly vim state.txt # paste in contents from website vim configuration.txt baseUrl=http://planet.openstreetmap.org/replication/hour /home/osm/osmosis/bin/osmosis --rri workingDirectory=/home/osm/planet/replication/ --simc --read-pbf /home/osm/planet/eurosha.osm.pbf --ac --tf accept-nodes amenity=hospital,place_of_worship,school --tf accept-ways amenity=hospital,place_of_worship,school --tf reject-relations --write-pbf /home/osm/planet/eurosha-new.osm.pbf /home/osm/osmosis/bin/osmosis --read-pbf /home/osm/planet/eurosha-new.osm.pbf --read-pbf /home/osm/planet/eurosha.osm.pbf --dc --write-pgsql-change database=osm user=osm password=planet mv /home/osm/planet/eurosha-new.osm.pbf /home/osm/planet/eurosha.osm.pbf vim /home/osm/replicate.sh #!/bin/sh n=`ps -ef | grep -v grep | grep /home/osm/osmosis/ | wc -l` if [ $n -le 0 ] then if [ -s /home/osm/planet/eurosha-new.osm.pbf ] then mv /home/osm/planet/eurosha-new.osm.pbf /home/osm/planet/eurosha.osm.pbf fi /home/osm/osmosis/bin/osmosis --rri workingDirectory=/home/osm/planet/replication/ --simc --read-pbf /home/osm/planet/eurosha.osm.pbf --ac --tf accept-nodes amenity=hospital,place_of_worship,school --tf accept-ways amenity=hospital,place_of_worship,school --tf reject-relations --write-pbf /home/osm/planet/eurosha-new.osm.pbf /home/osm/osmosis/bin/osmosis --read-pbf /home/osm/planet/eurosha-new.osm.pbf --read-pbf /home/osm/planet/eurosha.osm.pbf --dc --write-pgsql-change database=osm user=osm password=planet mv /home/osm/planet/eurosha-new.osm.pbf /home/osm/planet/eurosha.osm.pbf fi chmod 0755 /home/osm/replicate.sh vim /etc/crontab 1 * * * * root /home/osm/replicate.sh }}} Monitor: {{{ sudo -H -u postgres psql -d osm -c "select max(tstamp) from nodes"; }}} --- [wiki:GIS GIS]