= Installation of GIS Tools on Linux = [[TOC]] Suggest these CLI tools for data manipulation: * GDAL's ogr2ogr, etc * [http://mpa.itc.it/rs/srtm/srtm_generate_hdr.sh Convert SRTM to GeoTIFF] == GDAL == Simple functionality can be got from the Debian package: {{{ apt-get install gdal-bin }}} Advanced functionality requires the latest version (install into /usr/local): {{{ apt-get install -y g++ libproj-dev cd /tmp wget http://download.osgeo.org/gdal/1.11.2/gdal-1.11.2.tar.gz tar zxvf gdal-1.11.2.tar.gz cd gdal-1.11.2 ./configure --with-python CPPFLAGS="-fPIC" make make install cd .. ldconfig }}} Python bindings (e.g. needed for Shapefile Layers and [wiki:UserGuidelinesGISData#GADM GADM importer]) included in above version. e.g. If you are serving GeoTIFFs larger than 2Gb, you should create an [http://docs.geoserver.org/latest/en/user/tutorials/imagepyramid/imagepyramid.html Image Pyramid]: {{{ mkdir pyramid ~/gdal-1.11.2/swig/python/scripts/gdal_retile.py -v -s_srs EPSG:4326 -r bilinear -levels 4 -ps 2048 2048 -co "TILED=YES" -co "COMPRESS=JPEG" -targetDir pyramid mygeotiff.tif }}} == PostGIS == [http://postgis.refractions.net PostGIS] adds Spatial storage to [wiki:InstallationGuidelines/PostgreSQL PostgreSQL] (visit this page to configure PostgreSQL before returning here to continue with the GIS parts) * alternatively these can be installed as part of [http://opengeo.org/community/suite/download/ OpenGeoSuite] NB PostGIS in Debian Lenny or Squeeze is rather old - recommend using the version in Wheezy, ideally the upgraded from external packages (native version shown commented): Wheezy users: {{{ apt-get install -y postgis su postgres createuser -s -P gis createdb -O gis -E UTF8 gis -T template0 #psql -d gis -f /usr/share/postgresql/9.1/contrib/postgis-1.5/postgis.sql #psql -d gis -f /usr/share/postgresql/9.1/contrib/postgis-1.5/spatial_ref_sys.sql psql -d gis -f /usr/share/postgresql/9.3/extension/postgis--2.1.5.sql }}} Squeeze users: {{{ apt-get install -y postgresql-8.4-postgis su postgres createuser -s -P gis createdb -O gis gis createlang plpgsql -d gis psql -d gis -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql psql -d gis -f /usr/share/postgresql/8.4/contrib/postgis-1.5/spatial_ref_sys.sql }}} Ubuntu users (Lucid/PostgreSQL 8.4/Post-GIS 1.4) - the paths are slightly different {{{ psql -d gis -f /usr/share/postgresql/8.4/contrib/postgis.sql psql -d gis -f /usr/share/postgresql/8.4/contrib/spatial_ref_sys.sql }}} Add Spherical Mercator projection (900913. See http://www.cadmaps.com/gisblog/?p=81 for 54004): * Not needed with PostGIS-1.5 :) {{{ psql gis INSERT into spatial_ref_sys (srid, auth_name, auth_srid, srtext, proj4text) values (900913 ,'EPSG',900913,'GEOGCS["WGS 84", DATUM["World Geodetic System 1984", SPHEROID["WGS 84", 6378137.0, 298.257223563,AUTHORITY["EPSG","7030"]], AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich", 0.0, AUTHORITY["EPSG","8901"]], NIT["degree",0.017453292519943295], AXIS["Longitude", EAST], AXIS["Latitude", NORTH],AUTHORITY["EPSG","4326"]], PROJECTION["Mercator_1SP"],PARAMETER["semi_minor", 6378137.0], PARAMETER["latitude_of_origin",0.0], PARAMETER["central_meridian", 0.0], PARAMETER["scale_factor",1.0], PARAMETER["false_easting", 0.0], PARAMETER["false_northing", 0.0],UNIT["m", 1.0], AXIS["x", EAST], AXIS["y", NORTH],AUTHORITY["EPSG","900913"]] |','+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs'); }}} === Using PostGIS in Sahana === * [wiki:InstallationGuidelines/PostgreSQL#AddGeometrycolumntogis_location] == Java == [http://docs.geoserver.org/stable/en/user/production/java.html Update]: As of version 2.0, a Java Runtime Environment (JRE) is sufficient to run !GeoServer. !GeoServer no longer requires a Java Development Kit (JDK). Also note, OpenJDK does not work optimally with !GeoServer. for Debian 7 Wheezy: {{{ # Remove OpenJDK & dependent packages apt-get remove openjdk-6-jre-lib # Install a fake package to tell Debian that a valid Java is installed wget http://babel.ls.fi.upm.es/~vpablos/others/packages/debian/oracle-java-pkg-for-debian_2.0.0+r20140717_all.deb dpkg -i oracle-java-pkg-for-debian_2.0.0+r20140717_all.deb # Download Sun Java via Browser (need to accept T&C) # http://www.oracle.com/technetwork/java/javase/downloads/server-jre7-downloads-1931105.html http://download.oracle.com/otn-pub/java/jdk/7u45-b18/server-jre-7u75-linux-x64.tar.gz mkdir /usr/lib/jvm cd /usr/lib/jvm tar zxvf /tmp/server-jre-7u75-linux-x64.tar.gz ln -s /usr/lib/jvm/jdk1.7.0_75 /usr/lib/jvm/java-sun rm /usr/bin/java ln -s /usr/lib/jvm/java-sun/bin/java /usr/bin/java rm /usr/bin/javac ln -s /usr/lib/jvm/java-sun/bin/javac /usr/bin/javac vi /etc/profile.d/java JAVA_HOME=/usr/lib/jvm/java-sun export JAVA_HOME }}} or, for ubuntu 10.x: {{{ vim /etc/apt/sources.list #uncomment partner repository sudo apt-get update sudo apt-get install sun-java6-jre sudo emacs /etc/environment JAVA_HOME=/usr/lib/jvm/java-6-sun export JAVA_HOME }}} or, for debian 6 Squeeze {{{ vim /etc/apt/sources.list # for sun-java packages in unstable deb http://ftp.debian.org/debian/ unstable non-free deb-src http://ftp.debian.org/debian/ unstable non-free apt-get update apt-get install sun-java6-jdk vim /etc/profile JAVA_HOME=/usr/lib/jvm/java-6-sun export JAVA_HOME }}} The Sun installer will present a license in a curses form. Depending on your terminal emulator, arrow keys may not work for navigation. If not, try !^f and !^b to navigate to the , and enter to select. Then you'll be asked to accept (or not) the license -- again use !^f or !^b to move to and hit enter. or: * [http://java.sun.com/javase/downloads/widget/jdk6.jsp Download the JDK] & install it: {{{ cd /usr/local sh ~/jdk-6u21-linux-i586.bin ln -sf /usr/local/jdk1.6.0_21 /usr/local/java vim /etc/profile JAVA_HOME=/usr/local/java export JAVA_HOME }}} Install the JAI extensions (improves performance for !GeoServer WMS), the downloads available are listed [https://jai.dev.java.net/binary-builds.html here] 64-bit: {{{ cd /usr/lib/jvm/java-sun wget http://download.java.net/media/jai/builds/release/1_1_3/jai-1_1_3-lib-linux-amd64-jdk.bin wget http://download.java.net/media/jai-imageio/builds/release/1.1/jai_imageio-1_1-lib-linux-amd64-jdk.bin sh jai-1_1_3-lib-linux-amd64-jdk.bin # accept license export _POSIX2_VERSION=199209 sh jai_imageio-1_1-lib-linux-amd64-jdk.bin # accept license rm jai-1_1_3-lib-linux-amd64-jdk.bin rm jai_imageio-1_1-lib-linux-amd64-jdk.bin }}} If you installed JRE (sudo sh on Ubuntu) {{{ cd /usr/lib/jvm/java-sun/jre wget http://download.java.net/media/jai/builds/release/1_1_3/jai-1_1_3-lib-linux-amd64-jre.bin sh jai-1_1_3-lib-linux-amd64-jre.bin #accept licence and replace [yes] rm jai-1_1_3-lib-linux-amd64-jre.bin }}} 32-bit: {{{ cd /usr/lib/jvm/java-sun wget http://download.java.net/media/jai/builds/release/1_1_3/jai-1_1_3-lib-linux-i586-jdk.bin wget http://download.java.net/media/jai-imageio/builds/release/1.1/jai_imageio-1_1-lib-linux-i586-jdk.bin sh jai-1_1_3-lib-linux-i586-jdk.bin # accept license export _POSIX2_VERSION=199209 sh jai_imageio-1_1-lib-linux-i586-jdk.bin # accept license rm jai-1_1_3-lib-linux-i586-jdk.bin rm jai_imageio-1_1-lib-linux-i586-jdk.bin }}} == Apache Tomcat == [http://tomcat.apache.org Tomcat] is the recommended way to deploy Java applications: {{{ apt-get install -y tomcat7 libtcnative-1 #echo 'JAVA_HOME="/usr/lib/jvm/java-sun"' >> /etc/environment echo 'JRE_HOME="/usr/lib/jvm/java-sun/jre"' >> /etc/environment # http://docs.geoserver.org/latest/en/user/production/container.html#optimize-your-jvm vim /etc/default/tomcat7 JAVA_HOME=/usr/lib/jvm/java-sun JAVA_OPTS="-Djava.awt.headless=true -Xmx256m -Xms48m -XX:MaxPermSize=128m -XX:SoftRefLRUPolicyMSPerMB=36000 -XX:+UseConcMarkSweepGC" JAVA_OPTS="-Djava.awt.headless=true -Xmx2g -Xms64m -XX:MaxPermSize=128m -XX:SoftRefLRUPolicyMSPerMB=36000 -XX:+UseConcMarkSweepGC" vim /etc/init.d/tomcat7 TOMCAT7_SECURITY=no # Modify logging vim /etc/tomcat7/server.xml vim /etc/logrotate.d/tomcat7 /var/log/tomcat7/catalina.out /var/log/tomcat7/localhost_access_log..txt { copytruncate daily rotate 7 compress missingok create 640 tomcat7 adm } /etc/init.d/tomcat7 restart }}} * Ubuntu 10.x: {{{ sudo apt-get install unzip lynx tomcat6 tomcat6-admin libapache2-mod-jk vim /etc/libapache2-mod-jk/workers.properties workers.java_home=/usr/lib/jvm/java-6-sun /etc/init.d/tomcat6 restart # should see stopping [OK] and starting [OK] # change some permissions cd /var/lib/tomcat6/ sudo chown -R tomcat6:tomcat6 logs work sudo chown tomcat6:tomcat6 /usr/share/tomcat6 # check/change service port - default 8080 /etc/tomcat6/server.xml }}} Enable access to !GeoServer on Port 80 (assuming not using Cherokee): {{{ # if using multiple Tomcats behind a single apache load-balancer apt-get install apache2-mpm-worker a2enmod proxy_ajp a2dissite default vim /etc/tomcat7/server.xml # Uncomment these lines (& add the extra options) # maxThreads here should match the Apache MPM's MaxClients /etc/init.d/tomcat7 restart vim /etc/apache2/ports.conf NameVirtualHost *:80 Listen 80 vim /etc/apache2/sites-available/geo ServerName geo.eden.sahanafoundation.org ServerAdmin webmaster@sahanafoundation.org DocumentRoot /var/www ProxyPreserveHost on # These 2 lines should be disabled in Production ProxyPass /geoserver/ ajp://localhost:8009/geoserver/ ProxyPassReverse /geoserver ajp://localhost:8009/geoserver/ # These lines should be enabled in Production ProxyPass /geoserver/ ajp://localhost:8009/geoserver/gwc ProxyPassReverse /geoserver/gwc ajp://localhost:8009/geoserver/gwc ProxyPass /geoserver/ ajp://localhost:8009/geoserver/wfs ProxyPassReverse /geoserver/wfs ajp://localhost:8009/geoserver/wfs ProxyPass /geoserver/ ajp://localhost:8009/geoserver/wms ProxyPassReverse /geoserver/wms ajp://localhost:8009/geoserver/wms/ ErrorLog /var/log/apache2/geo_error.log LogLevel warn CustomLog /var/log/apache2/geo_access.log combined a2ensite geo /etc/init.d/apache2 restart }}} == WMS/WFS == === [http://geoserver.org GeoServer] === (recommended) {{{ apt-get install unzip wget http://kent.dl.sourceforge.net/project/geoserver/GeoServer/2.7.0/geoserver-2.7.0-war.zip cd /var/lib/tomcat7/webapps unzip ~/geoserver-2.7.0-war.zip /etc/init.d/tomcat7 restart # Allow time to decompress the .war /etc/init.d/tomcat7 stop # Move the data directory somewhere to allow upgrades to happen easily mkdir /var/gis mv /var/lib/tomcat7/webapps/geoserver/data /var/gis/geoserver_data vim /var/lib/tomcat7/webapps/geoserver/WEB-INF/web.xml GEOSERVER_DATA_DIR /var/gis/geoserver_data /etc/init.d/tomcat7 restart vim /etc/iptables.rules -A INPUT -p tcp --dport 8080 -j ACCEPT reboot }}} CSS module: {{{ cd /tmp wget http://kent.dl.sourceforge.net/project/geoserver/GeoServer%20Extensions/2.7.0/geoserver-2.7.0-css-plugin.zip cd /var/lib/tomcat7/webapps/geoserver/WEB-INF/lib/ unzip /tmp/geoserver-2.7.0-css-plugin.zip /etc/init.d/tomcat7 restart }}} Add fonts: {{{ cp *.ttf /usr/lib/jvm/java-6-sun/jre/lib/fonts /etc/init.d/tomcat7 restart }}} GDAL Image Formats (although generally are better off using GDAL to convert to GeoTIFF for serving, if possible): * http://docs.geoserver.org/latest/en/user/services/wcs/basics.html {{{ cd /tmp wget https://java.net/downloads/imageio-ext/Releases/1.0.5/native-libraries/linux64/imageio-ext-1.0.5-linux64-base-lib.tar.gz mkdir -p /usr/local/lib/imageio-ext cd /usr/local/lib/imageio-ext tar zxvf /tmp/imageio-ext-1.0.5-linux64-base-lib.tar.gz vim /etc/default/tomcat7 LD_LIBRARY_PATH=/usr/local/lib/imageio-ext cd /tmp wget http://kent.dl.sourceforge.net/project/geoserver/GeoServer%20Extensions/2.7.0/geoserver-2.7.0-gdal-plugin.zip cd /var/lib/tomcat7/webapps/geoserver/WEB-INF/lib/ unzip /tmp/geoserver-2.7.0-gdal-plugin.zip /etc/init.d/tomcat7 restart }}} Cache of downloadable (if main site has issues): * http://eden.sahanafoundation.org/raw-attachment/wiki/InstallationGuidelinesGISDataLinux/imageio-ext-1.0.5-linux64-base-lib.tar.gz http://geo.host.domain:8080/geoserver Configure: * [wiki:UserGuidelines/GIS/Data#GeoServer] * [http://docs.geoserver.org/stable/en/user/production/index.html Production Performance for GeoServer] * [http://demo.geo-solutions.it/share/foss4g2011/gs_steroids_sgiannec_foss4g2011.pdf Tips to speed-up GeoServer] * [http://geowebcache.org/docs/current/production/index.html Production Performance for GeoWebCache] * [http://blog.opengeo.org/2013/04/18/geoserver-in-a-clustered-configuration-part-1/ Clustering 1] * [http://blog.opengeo.org/2013/04/30/geoserver-in-a-clustered-configuration-part-2/ Clustering 2] Upgrades: {{{ /etc/init.d/tomcat7 stop mv /var/lib/tomcat7/webapps/geoserver ~/geoserver-2.5.1_backup cd /var/lib/tomcat7/webapps unzip ~/geoserver-2.7.0-war.zip /etc/init.d/tomcat7 restart # Allow time to decompress the .war /etc/init.d/tomcat7 stop vim /var/lib/tomcat7/webapps/geoserver/WEB-INF/web.xml GEOSERVER_DATA_DIR /var/gis/geoserver_data /etc/init.d/tomcat7 restart }}} === [http://mapserver.org MapServer] === * [http://www.tinyows.org TinyOWS] can complement this for WFS-T == !OpenStreetMap == * See [wiki:GIS/OpenStreetMap GIS/OpenStreetMap] == Printing == For Printing TMS layers, need to compile the Trunk version of the [http://www.mapfish.org/doc/print/ MapFish Print Module] & then serve the resultant .war with [http://tomcat.apache.org Tomcat]: 1. Ensure that {{{JAVA_HOME}}} points to a JDK. 2. Download & build source: {{{ svn checkout http://www.mapfish.org/svn/mapfish/print/trunk/ cd trunk ./gradlew build }}} 3. Configure Tomcat {{{ vim /etc/tomcat6/policy.d/04webapps.policy permission java.io.FilePermission "/var/lib/tomcat6/webapps/print-servlet-1.2-SNAPSHOT/WEB-INF/classes/logging.properties", "read"; /etc/init.d/tomcat6 restart vim /etc/apache2/sites-available/print ln -s /etc/apache2/sites-available/print /etc/apache2/sites-enabled/print /etc/init.d/apache2 restart }}} 4. Copy the WAR file to your %WEBAPPS% directory: {{{ cp trunk/build/libs/print-servlet-1.2-SNAPSHOT.war /var/lib/tomcat6/webapps }}} 5. (re)Start Tomcat 6. Configure the Map URL for Eden to: * http://host.domain:8080/print-servlet-1.2-SNAPSHOT/pdf/ 7. Configure %WEBAPPS%\print-servlet-1.2-SNAPSHOT\config.yaml: http://www.mapfish.org/doc/print/configuration.html Printing can also be done within !GeoServer: * http://docs.geoserver.org/stable/en/user/community/printing/ The 2.0.x versiondidn't support TMS layers, like !OpenStreetMap. @ToDo: Test this latest version. {{{ wget http://gridlock.opengeo.org/geoserver/2.1.x/community-latest/geoserver-2.1-SNAPSHOT-printing-plugin.zip cd /var/lib/tomcat6/webapps/geoserver/WEB-INF/lib unzip ~/geoserver-2.1-SNAPSHOT-printing-plugin.zip /etc/init.d/tomcat6 restart #wget http://geoserver.org/download/attachments/20938936/printing-data-2.0.x.zip #cd /usr/local/geoserver-2.0.2/data_dir #unzip ~/printing-data-2.0.x.zip #/usr/local/geoserver/bin/shutdown.sh #/usr/local/geoserver/bin/startup.sh & }}} * Configure by editing {{{/var/gis/geoserver_data_dir/printing/config.yaml}}} according to [http://trac.mapfish.org/trac/mapfish/wiki/PrintModuleServer Documentation] === Print to Image === * http://trac.mapfish.org/trac/mapfish/ticket/161 == Proxy / Cache == A Proxy can cache remote servers &/or do Reprojection or Combination of Layers If just wanting to cache the WMS served by !GeoServer, then it's probably best to use the integrated !GeoWebCache: * http://geowebcache.org Alternatives: * http://tilecache.org * http://github.com/migurski/GridTile * http://mapproxy.org * http://mapproxy.org/docs/1.7.0/install.html {{{ adduser --system --disabled-password mapproxy apt-get install -y libgeos-dev libgdal-dev wget http://pypi.python.org/packages/source/M/MapProxy/MapProxy-1.7.1.tar.gz tar zxvf MapProxy-1.7.1.tar.gz cd MapProxy-1.7.1 python setup.py install cd .. cd /home mapproxy-util create -t base-config mapproxy cd mapproxy mapproxy-util create -t wsgi-app -f mapproxy.yaml config.py }}} * Test: http://localhost:8080/demo/ * [http://mapproxy.org/docs/1.7.0/deployment.html#production WSGI configuration] * [http://mapproxy.org/docs/1.7.0/seed.html Cleaning or Seeding]: {{{ mapproxy-seed -s seed.yaml -f mapproxy.yaml }}} NB This can reveal issues such as the max-width limitation on ArcGIS when converting 4326 to 900913: * [http://forums.arcgis.com/threads/19518-Parameter-width-contains-unacceptable-value ArcGIS-side fix] * [http://osdir.com/ml/mapproxy-gis/2011-03/msg00085.html Request a Mercator if you can] (3857, 102100 or 102113) === Apache === {{{ vim /etc/apache2/sites-available/proxy ServerName proxy.sahanafoundation.org ServerAdmin webmaster@sahanafoundation.org WSGIScriptAlias / /home/mapproxy/etc/config.wsgi WSGIDaemonProcess mapproxy user=www-data group=www-data home=/home/mapproxy processes=2 maximum-requests=100 Order deny,allow Allow from all WSGIProcessGroup mapproxy ErrorLog /var/log/apache2/proxy_error.log LogLevel warn CustomLog /var/log/apache2/proxy_access.log combined mkdir -p /home/mapproxy/var chown www-data /home/mapproxy/var a2ensite proxy /etc/init.d/apache2 force-reload }}} === Cherokee/uWSGI === {{{ vim /home/mapproxy/uwsgi.xml /home/mapproxy/ 4 vim /usr/local/etc/cherokee/cherokee.conf vserver!30!rule!300!encoder!deflate = allow vserver!30!rule!300!encoder!gzip = allow vserver!30!rule!300!handler = uwsgi vserver!30!rule!300!handler!balancer = round_robin vserver!30!rule!300!handler!balancer!source!10 = 2 vserver!30!rule!300!handler!check_file = 0 vserver!30!rule!300!handler!error_handler = 1 vserver!30!rule!300!handler!pass_req_headers = 1 vserver!30!rule!300!handler!xsendfile = 0 vserver!30!rule!300!match = directory vserver!30!rule!300!match!directory = /proxy source!2!env_inherited = 1 source!2!group = mapproxy source!2!host = 127.0.0.1:59026 source!2!interpreter = /usr/local/bin/uwsgi -s 127.0.0.1:59026 -x /home/mapproxy/uwsgi.xml source!2!nick = uWSGI 2 source!2!timeout = 300 source!2!type = interpreter source!2!user = mapproxy }}} * http://mapproxy.org/docs/1.4.0/configuration.html * http://lists.osgeo.org/pipermail/mapproxy/2010-December/000301.html {{{ vim /usr/share/proj/epsg # Google Maps Global Mercator <900913> +proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs +over }}} === Configuration for TRMM === TRMM Rainfall Monitoring * http://mapproxy.org/docs/1.4.0/configuration_examples.html {{{ vim /home/mapproxy/etc/mapproxy.yaml layers: - name: TRMM_3B42_V6_DAILY title: Rainfall (1 day) sources: [TRMM_3B42_V6_DAILY_cache, TRMM_3B42_V6_DAILY] - name: TRMM_3B42_V6_10_DAY title: Rainfall (10 days) sources: [TRMM_3B42_V6_10_DAY_cache, TRMM_3B42_V6_10_DAY] caches: TRMM_3B42_V6_DAILY_cache: grids: [GLOBAL_MERCATOR, global_geodetic_sqrt2] sources: [TRMM_3B42_V6_DAILY] TRMM_3B42_V6_10_DAY_cache: grids: [GLOBAL_MERCATOR, global_geodetic_sqrt2] sources: [TRMM_3B42_V6_10_DAY] sources: TRMM_3B42_V6_DAILY: type: wms supported_srs: ['EPSG:4326'] req: url: http://gdata2.sci.gsfc.nasa.gov/daac-bin/wms_trmm? layers: TRMM_3B42_V6_DAILY transparent: true TRMM_3B42_V6_10_DAY: type: wms supported_srs: ['EPSG:4326'] req: url: http://gdata2.sci.gsfc.nasa.gov/daac-bin/wms_trmm? layers: TRMM_3B42_V6_10_DAY transparent: true }}} ---- [wiki:UserGuidelines/GIS/Data]