Version 12 (modified by 10 years ago) ( diff ) | ,
---|
OpenStreetMap
Table of Contents
Wikipedia of Maps
Base Map
We have out-of-the-box the ability to use OpenStreetMap Tiles as base layer.
This can include local OSM sites (OSM Taiwan is included as an example)
Local Tile Server
If you need to run offline then you need to set up your own Tile Server:
- http://switch2osm.org/serving-tiles/
- http://switch2osm.org/serving-tiles/manually-building-a-tile-server-12-04/
- http://blog.gravitystorm.co.uk/2010/07/05/map-rendering-on-ec2/
- https://help.openstreetmap.org/questions/11949/server-spec-for-running-osm
- http://weait.com/content/build-your-own-openstreetmap-server
- http://weait.com/content/make-your-first-map
- http://dbsgeo.com/foss4g2010/pdf/osm-rendering-foss4g2010.pdf
OpenStreetMap data can be stored in a PostGIS database & rendered using Mapnik.
To support Key/Value pairs:
apt-get install postgresql-contrib-9.3 su postgres psql -d gis -f /usr/share/postgresql/9.3/contrib/hstore.sql
To support Planet diff files:
su postgres psql gis < /usr/share/postgresql/9.3/contrib/_int.sql
Osm2pgsql
It is recommended to use the latest version from source rather than an out of date package:
apt-get install build-essential libxml2-dev libgeos-dev libpq-dev libbz2-dev proj autoconf subversion libtool git clone https://github.com/openstreetmap/osm2pgsql Optionally investgate updating this outdated patch (updated from original here): http://www.mail-archive.com/dev@openstreetmap.org/msg12768.html #wget http://eden.sahanafoundation.org/raw-attachment/wiki/GIS/InstallationGuidelines/Linux/osm2pgsql_centroid.patch cd osm2pgsql #patch -p0 < ../osm2pgsql_centroid.patch ./autogen.sh ./configure sed -i 's/-g -O2/-O2 -march=native -fomit-frame-pointer/' Makefile make make install # Fix needed for SVN version 0.70.5 cp default.style /usr/local/share
Mapnik
- http://wiki.openstreetmap.org/wiki/Mapnik
- http://mike.teczno.com/notes/mapnik.html (Advanced Tips)
2.0 is packaged for Wheezy, but we want the latest:
apt-get install -y libboost-dev libboost-filesystem-dev libboost-python-dev libboost-regex-dev libboost-system-dev libboost-thread-dev libpq-dev libfreetype6-dev libxml2-dev libjpeg-dev libpng-dev libtiff-dev libproj-dev wget http://mapnik.s3.amazonaws.com/dist/v2.2.0/mapnik-v2.2.0.tar.bz2 tar jxvf mapnik-v2.2.0.tar.bz2 cd mapnik-v2.2.0 ./configure make make test-local make install
Download the OpenStreetMap extensions for Mapnik:
cd svn export http://svn.openstreetmap.org/applications/rendering/mapnik
Download/decompress Coastlines:
cd wget http://tile.openstreetmap.org/world_boundaries-spherical.tgz wget http://tile.openstreetmap.org/processed_p.tar.bz2 wget http://tile.openstreetmap.org/shoreline_300.tar.bz2 wget http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/10m-populated-places.zip wget http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/110m/cultural/110m-admin-0-boundary-lines.zip cd ~/mapnik tar zxvf ~/world_boundaries-spherical.tgz tar jxvf ~/processed_p.tar.bz2 -C world_boundaries tar jxvf ~/shoreline_300.tar.bz2 -C world_boundaries unzip ~/10m-populated-places.zip -d world_boundaries unzip ~/110m-admin-0-boundary-lines.zip -d world_boundaries
Disable autovacuum:
vim /etc/postgresql/8.4/main/postgresql.conf #autovacuum = on /etc/init.d/postgresql restart
Download/Import area of interest, e.g.:
#wget http://downloads.cloudmade.com/north_america/haiti/haiti.osm.bz2 #su postgres #osm2pgsql -s -d gis haiti.osm.bz2 wget http://labs.geofabrik.de/haiti/2010-11-24-14-43.osm.bz2 su postgres osm2pgsql -s -d gis 2010-11-24-14-43.osm.bz2
Re-enable autovacuum:
vim /etc/postgresql/8.4/main/postgresql.conf autovacuum = on /etc/init.d/postgresql restart
Setup XML file:
cd ~/mapnik ./generate_xml.py --accept-none --dbname gis --symbols ./symbols/ --world_boundaries ./world_boundaries/
Render tiles:
cd ~/mapnik vim generate_tiles.py render_tiles(bbox, mapfile, tile_dir, 0, 8, "World") #minZoom = 10 #maxZoom = 16 #bbox = (-2, 50.0,1.0,52.0) #render_tiles(bbox, mapfile, tile_dir, minZoom, maxZoom) # Haiti bbox = (-74.668, 17.884, -71.659, 20.232) render_tiles(bbox, mapfile, tile_dir, 9, 17, "Haiti") # Muenchen # comment all under here mkdir /var/www/tiles chown postgres /var/www/tiles ln -s /var/www/tiles su postgres MAPNIK_MAP_FILE="osm.xml" MAPNIK_TILE_DIR="tiles/" ./generate_tiles.py
Contours
apt-get install gdal-bin
SRTM3 data:
mkdir ~/mapnik/srtm cd ~/mapnik/srtm wget http://mapnik-utils.googlecode.com/svn@170/sandbox/testing/hillshading/srtm_generate_hdr.sh chmod +x srtm_generate_hdr.sh cp srtm_generate_hdr.sh /usr/local/bin vim process_srtm3.sh #!/bin/bash PREP_TABLE="1" for X in *.hgt.zip; do yes | srtm_generate_hdr.sh $X rm -f "${X%%.zip}" # Import 10m contours rm -f "${X%%.hgt.zip}.shp" "${X%%.hgt.zip}.shx" "${X%%.hgt.zip}.dbf" gdal_contour -i 10 -snodata 32767 -a height "${X%%.hgt.zip}.tif" "${X%%.hgt.zip}.shp" [ "$PREP_TABLE" ] && shp2pgsql -p -I -g way "${X%%.hgt.zip}" contours | psql -q gis shp2pgsql -a -g way "${X%%.hgt.zip}" contours | psql -q gis rm -f "${X%%.hgt.zip}.shp" "${X%%.hgt.zip}.shx" "${X%%.hgt.zip}.dbf" rm -f "${X%%.hgt.zip}.bil" rm -f "${X%%.hgt.zip}.hdr" rm -f "${X%%.hgt.zip}.prj" rm -f "${X%%.hgt.zip}.tif" unset PREP_TABLE done # Haiti wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N17W072.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N18W072.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N18W073.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N18W074.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N18W075.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N19W072.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N19W073.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N19W074.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N19W075.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N20W073.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N20W074.hgt.zip wget http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/North_America/N20W075.hgt.zip chown postgres . su postgres sh process_srtm3.sh
Since we are non-commercial, we should be able to use the SRTM4.1 data from CGIAR, however the Shapefile production crashed when it reached 2Gb.
Example for Haiti:
cd ~/mapnik/srtm wget http://srtm.csi.cgiar.org/SRT-ZIP/SRTM_V41/SRTM_Data_GeoTiff/srtm_22_09.zip unzip srtm_22_09.zip gdal_contour -i 10 -snodata 32767 -a height "srtm_22_09.tif" "srtm_22_09.shp" su postgres shp2pgsql -p -I -g way "srtm_22_09" contours | psql -q gis shp2pgsql -a -g way "srtm_22_09" contours | psql -q gis
Configure Mapnik:
vim ~/mapnik/osm.xml # Under line &layer-shapefiles; &layer-contours; vim ~/mapnik/inc/layer-contours.xml.inc <Style name="contours10"> <Rule> &maxscale_zoom14; &minscale_zoom17; <LineSymbolizer> <CssParameter name="stroke">#9cb197</CssParameter> <CssParameter name="stroke-width">0.5</CssParameter> </LineSymbolizer> </Rule> </Style> <Style name="contours50"> <Rule> &maxscale_zoom14; &minscale_zoom17; <LineSymbolizer> <CssParameter name="stroke">#9cb197</CssParameter> <CssParameter name="stroke-width">0.6</CssParameter> </LineSymbolizer> </Rule> <Rule> &maxscale_zoom12; &minscale_zoom13; <LineSymbolizer> <CssParameter name="stroke">#747b90</CssParameter> <CssParameter name="stroke-width">0.6</CssParameter> </LineSymbolizer> </Rule> </Style> <Style name="contours100"> <Rule> &maxscale_zoom14; &minscale_zoom17; <LineSymbolizer> <CssParameter name="stroke">#9cb197</CssParameter> <CssParameter name="stroke-width">0.7</CssParameter> </LineSymbolizer> </Rule> <Rule> &maxscale_zoom12; &minscale_zoom13; <LineSymbolizer> <CssParameter name="stroke">#747b90</CssParameter> <CssParameter name="stroke-width">0.7</CssParameter> </LineSymbolizer> </Rule> <Rule> &maxscale_zoom10; &minscale_zoom11; <LineSymbolizer> <CssParameter name="stroke">#855d62</CssParameter> <CssParameter name="stroke-width">0.7</CssParameter> </LineSymbolizer> </Rule> </Style> <Style name="contours-text50"> <Rule> &maxscale_zoom14; &minscale_zoom17; <TextSymbolizer name="height" face_name="DejaVu Sans Book" size="8" fill="#747b90" halo_radius="1" placement="line" /> </Rule> </Style> <Style name="contours-text100"> <Rule> &maxscale_zoom14; &minscale_zoom17; <TextSymbolizer name="height" face_name="DejaVu Sans Book" size="8" fill="#747b90" halo_radius="1" placement="line" /> </Rule> <Rule> &maxscale_zoom12; &minscale_zoom13; <TextSymbolizer name="height" face_name="DejaVu Sans Book" size="8" fill="#855d62" halo_radius="1" placement="line" /> </Rule> </Style> <Layer name="srtm_10" status="on" srs="+proj=latlong +datum=WGS84"> <StyleName>contours10</StyleName> <StyleName>contours-text10</StyleName> <Datasource> <Parameter name="table">(select way,height from contours WHERE height::integer % 10 = 0 AND height::integer % 50 != 0 AND height::integer % 100 != 0) as "contours-10"</Parameter> &datasource-settings; </Datasource> </Layer> <Layer name="srtm_50" status="on" srs="+proj=latlong +datum=WGS84"> <StyleName>contours50</StyleName> <StyleName>contours-text50</StyleName> <Datasource> <Parameter name="table">(select way,height from contours WHERE height::integer % 50 = 0 AND height::integer % 100 != 0) as "contours-50"</Parameter> &datasource-settings; </Datasource> </Layer> <Layer name="srtm_100" status="on" srs="+proj=latlong +datum=WGS84"> <StyleName>contours100</StyleName> <StyleName>contours-text100</StyleName> <Datasource> <Parameter name="table">(select way,height from contours WHERE height::integer % 100 = 0) as "contours-100"</Parameter> &datasource-settings; </Datasource> </Layer>
Hillshading
Serve Tiles
Tiles can simply be served by Apache:
vim /etc/apache2/sites-available/mysite <VirtualHost *:80> ServerName mysite.org Alias /tiles /var/www/tiles <LocationMatch "/tiles/"> Order Allow,Deny Allow from all </LocationMatch> </VirtualHost> apache2ctl restart
WMS
In order to serve the OSM data via WMS then can use mod_mapnik_wms:
- http://wiki.openstreetmap.org/wiki/Mod_mapnik_wms
gpg --gen-key apt-get install pbuilder debsigs apache2-prefork-dev libmapnik-dev libgd2-xpm-dev svn export http://svn.openstreetmap.org/applications/utils/mod_mapnik_wms cd mod_mapnik_wms sh autogen.sh vim debian/mapnik_wms.load LoadFile /usr/lib/libmapnik.so.0.7 debuild
This data can then be converted by GeoServer into a KML SuperOverlay to display in Google Earth:
- http://geoserver.org/display/GEOS/GSIP+47+-+WMS+cascading
- http://docs.geoserver.org/stable/en/user/googleearth/tutorials/superoverlaysgwc.html
Rails Port
This is needed to allow editing of the data using Potlatch, JOSM, etc
apt-get install imagemagick libmagick9-dev apt-get install ruby ruby1.8-dev libxml2-dev libxml-ruby1.8 libxml-parser-ruby1.8 rubygems librmagick-ruby gem install -v=2.3.8 rails #gem install libxml-ruby #gem install composite_primary_keys #gem install rmagick gem install timecop gem install pg gem install oauth #svn co http://railsexpress.de/svn/plugins/sql_session_store/trunk sql_session_store apt-get install postgresql-contrib libpq-dev su postgres createuser openstreetmap -s -P createdb -E UTF8 -O openstreetmap openstreetmap createdb -E UTF8 -O openstreetmap osm_test createdb -E UTF8 -O openstreetmap osm psql -d openstreetmap < /usr/share/postgresql/8.4/contrib/btree_gist.sql apt-get install git cd /home git clone git://git.openstreetmap.org/rails.git cd rails cp config/postgres.example.database.yml config/database.yml vim config/database.yml rake gems:install rake db:migrate env RAILS_ENV=production rake db:migrate rake test osmosis --read-xml-0.6 file="planet.osm.bz2" --write-apidb-0.6 populateCurrentTables=yes host="localhost" database="openstreetmap" user="openstreetmap" password="openstreetmap" validateSchemaVersion=no select setval('acls_id_seq', (select max(id) from acls)); select setval('changesets_id_seq', (select max(id) from changesets)); select setval('countries_id_seq', (select max(id) from countries)); select setval('current_nodes_id_seq', (select max(id) from current_nodes)); select setval('current_relations_id_seq', (select max(id) from current_relations)); select setval('current_ways_id_seq', (select max(id) from current_ways)); select setval('diary_comments_id_seq', (select max(id) from diary_comments)); select setval('diary_entries_id_seq', (select max(id) from diary_entries)); select setval('friends_id_seq', (select max(id) from friends)); select setval('gpx_file_tags_id_seq', (select max(id) from gpx_file_tags)); select setval('gpx_files_id_seq', (select max(id) from gpx_files)); select setval('messages_id_seq', (select max(id) from messages)); select setval('sessions_id_seq', (select max(id) from sessions)); select setval('user_tokens_id_seq', (select max(id) from user_tokens)); select setval('users_id_seq', (select max(id) from users)); cd /home/rails ruby script/server cd db/functions make libpgosm.so * Log into PgSQL and execute the CREATE FUNCTION statement from maptile.c's comment: CREATE FUNCTION maptile_for_point(int8, int8, int4) RETURNS int4 AS '/path/to/rails-port/db/functions/libpgosm', 'maptile_for_point' LANGUAGE C STRICT; CREATE FUNCTION tile_for_point(int4, int4) RETURNS int8 AS '/path/to/rails-port/db/functions/libpgosm', 'tile_for_point' LANGUAGE C STRICT;
Vector Overlays
Can have OSM Vectors displayed over the top of other Base Layers (e.g. Satellite Images)
Import
We have an XSLT stylesheet to import .osm files
e.g for hospitals and clinics:
osmosis --read-xml country.osm --tf accept-nodes amenity=hospital,clinic --tf reject-ways --tf reject-relations --write-xml nodes.osm osmosis --read-xml country.osm --tf reject-relations --tf accept-ways amenity=hospital,clinic --used-node --write-xml ways.osm osmosis --rx nodes.osm --rx ways.osm --merge --wx country_hospitals.osm http://myhost.com/eden/hms/hospital/create.osm?filename=country_hospitals.osm
This needs more work to understand the admin hierarchy properly to be able to import Places.
- http://wiki.openstreetmap.org/wiki/Map_Features#Places
- http://wiki.openstreetmap.org/wiki/Placename_hierachies is_in tag
Geofabrik have updated extracts daily for Pakistan:
Otherwise pull a BBOX directly using Osmosis:
Osmosis requires Java. Python options for filtering based on tag, which would be more suitable for integration within Sahana, however we need to add Polygon filtering using Shapely:
Ruby script to generate KML of recently-added locations by a group of users:
Polygons:
Basemap for Garmin GPS
Mirror
How to set up a local mirror for a country, optionally with just selected Feature types, suitable for use by Import PoI:
Debian stable apt-get install sudo visudo fran ALL = NOPASSWD: ALL cat << EOF >> "/etc/apt/sources.list.d/pgdg.list" deb http://apt.postgresql.org/pub/repos/apt/ wheezy-pgdg main EOF wget --no-check-certificate https://www.postgresql.org/media/keys/ACCC4CF8.asc apt-key add ACCC4CF8.asc apt-get update apt-get -y install postgresql-9.3 python-psycopg2 postgresql-9.3-postgis ptop postgresql-contrib # Tune PostgreSQL cat << EOF >> "/etc/sysctl.conf" ## Increase Shared Memory available for PostgreSQL # 512Mb kernel.shmmax = 279134208 # 1024Mb (may need more) #kernel.shmmax = 552992768 kernel.shmall = 2097152 EOF sysctl -w kernel.shmmax=279134208 # For 512 MB RAM #sysctl -w kernel.shmmax=552992768 # For 1024 MB RAM sysctl -w kernel.shmall=2097152 sed -i 's|#track_counts = on|track_counts = on|' /etc/postgresql/9.3/main/postgresql.conf sed -i 's|#autovacuum = on|autovacuum = on|' /etc/postgresql/9.3/main/postgresql.conf # 512Mb RAM: sed -i 's|shared_buffers = 28MB|shared_buffers = 56MB|' /etc/postgresql/9.3/main/postgresql.conf sed -i 's|#effective_cache_size = 128MB|effective_cache_size = 256MB|' /etc/postgresql/9.3/main/postgresql.conf sed -i 's|#work_mem = 1MB|work_mem = 2MB|' /etc/postgresql/9.3/main/postgresql.conf # If 1Gb+ RAM, activate post-install via pg1024 script adduser osm mkdir -p /home/osm/planet/replication cd /home/osm wget http://dev.openstreetmap.org/~bretth/osmosis-build/osmosis-latest.tgz mkdir osmosis cd osmosis tar zxvf osmosis-latest.tgz chown -R postgres /home/osm su postgres psql CREATE USER osm WITH PASSWORD 'planet'; \q createdb -O osm -E UTF8 osm -T template0 psql -d osm -f /usr/share/postgresql/9.3/extension/postgis--2.1.4.sql #psql -d osm -f /usr/share/postgresql/9.3/extension/hstore--1.2.sql psql \c osm create extension hstore; \q psql -d osm -f /home/osm/osmosis/script/pgsnapshot_schema_0.6.sql psql \c osm ALTER TABLE geography_columns OWNER TO osm; ALTER TABLE geometry_columns OWNER TO osm; ALTER TABLE nodes OWNER TO osm; ALTER TABLE relation_members OWNER TO osm; ALTER TABLE relations OWNER TO osm; ALTER TABLE schema_info OWNER TO osm; ALTER TABLE spatial_ref_sys OWNER TO osm; ALTER TABLE users OWNER TO osm; ALTER TABLE way_nodes OWNER TO osm; ALTER TABLE ways OWNER TO osm; \q exit cd /home/osm/planet # Either grab snapshot from GeoFabrik or use HOT Export Tool #wget http://download.geofabrik.de/africa/cameroon-latest.osm.pbf -O extract.osm.pbf wget http://export.hotosm.org/download/023803/rawdata.osm.pbf -O extract.osm.pbf apt-get install -y openjdk-7-jre # If wishing to filter the data to just certain types of object #mv extract.osm.pbf full.osm.pbf #/home/osm/osmosis/bin/osmosis --read-pbf /home/osm/planet/full.osm.pbf --tf accept-nodes amenity=hospital,place_of_worship,school --tf accept-ways amenity=hospital,place_of_worship,school --write-pbf /home/osm/planet/extract.osm.pbf su postgres /home/osm/osmosis/bin/osmosis --read-pbf /home/osm/planet/extract.osm.pbf --write-pgsql database=osm user=osm password=planet cd /home/osm/planet/replication /home/osm/osmosis/bin/osmosis --rrii sudo -H -u postgres psql -d osm -c "select max(tstamp) from nodes"; # http://osm.personalwerk.de/replicate-sequences/ # (Code: https://github.com/MaZderMind/replicate-sequences) # Put timestamp into site, select hourly vi state.txt # paste in contents from website vi configuration.txt baseUrl=http://planet.openstreetmap.org/replication/hour # If wishing to filter the data to just certain types of object #/home/osm/osmosis/bin/osmosis --rri workingDirectory=/home/osm/planet/replication/ --simc --read-pbf /home/osm/planet/extract.osm.pbf --ac --tf accept-nodes amenity=hospital,place_of_worship,school --tf accept-ways amenity=hospital,place_of_worship,school --tf reject-relations --write-pbf /home/osm/planet/new.osm.pbf #/home/osm/osmosis/bin/osmosis --read-pbf /home/osm/planet/new.osm.pbf --read-pbf /home/osm/planet/extract.osm.pbf --dc --write-pgsql-change database=osm user=osm password=planet /home/osm/osmosis/bin/osmosis --rri workingDirectory=/home/osm/planet/replication/ --simc --read-pbf /home/osm/planet/extract.osm.pbf --ac --write-pbf /home/osm/planet/new.osm.pbf /home/osm/osmosis/bin/osmosis --read-pbf /home/osm/planet/new.osm.pbf --read-pbf /home/osm/planet/extract.osm.pbf --dc --write-pgsql-change database=osm user=osm password=planet mv /home/osm/planet/new.osm.pbf /home/osm/planet/extract.osm.pbf vi /home/osm/replicate.sh #!/bin/sh n=`ps -ef | grep -v grep | grep /home/osm/osmosis/ | wc -l` if [ $n -le 0 ] then if [ -s /home/osm/planet/new.osm.pbf ] then mv /home/osm/planet/new.osm.pbf /home/osm/planet/extract.osm.pbf fi /home/osm/osmosis/bin/osmosis --rri workingDirectory=/home/osm/planet/replication/ --simc --read-pbf /home/osm/planet/extract.osm.pbf --ac --write-pbf /home/osm/planet/new.osm.pbf /home/osm/osmosis/bin/osmosis --read-pbf /home/osm/planet/new.osm.pbf --read-pbf /home/osm/planet/extract.osm.pbf --dc --write-pgsql-change database=osm user=osm password=planet mv /home/osm/planet/new.osm.pbf /home/osm/planet/extract.osm.pbf fi chmod 0755 /home/osm/replicate.sh vi /etc/crontab 1 * * * * root /home/osm/replicate.sh
Monitor:
sudo -H -u postgres psql -d osm -c "select max(tstamp) from nodes";
Attachments (1)
-
chiwogs.py
(2.3 KB
) - added by 3 years ago.
Script to export Bhutan L3s from OSM
Download all attachments as: .zip