Changes between Version 192 and Version 193 of UserGuidelines/GIS/Data


Ignore:
Timestamp:
01/22/12 14:34:56 (13 years ago)
Author:
Fran Boon
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • UserGuidelines/GIS/Data

    v192 v193  
    88
    99=== CSV ===
    10 There is a function available in {{{modules/s3/s3gis.py}}} to import from CSV.
     10Locations can be imported using the normal import procedure:
     11 * Prepopulate:
     12  * {{{gis,location,location.csv,location.xsl}}}
     13 * After install:
     14  * http://host/eden/gis/location/import
    1115
    1216The CSV needs to have specific columns:
    13  * WKT column if we have polygon info (or Lat Lon for Points, if not)
    14  * For L1, we need these columns: ADM0_NAME, ADM1_NAME (& WKT)
    15  * For L2, we need these columns: ADM1_NAME, ADM2_NAME (& WKT) [ADM0_NAME can also be used to help separate duplicates]
    16  * For L3, we need these columns: ADM2_NAME, ADM3_NAME (& WKT) [ADM1_NAME can also be used to help separate duplicates]
    17  * CODE column is read, if-present
    18  * POPULATION column is read, if-present
     17 * WKT column if we have polygon info (or Lat and Lon for Points, if not)
     18 * For L1, we need these columns: Country, ADM1_NAME (& WKT)
     19 * For L2, we need these columns: ADM1_NAME, ADM2_NAME (& WKT) [Country can also be used to help separate duplicates]
     20 * For L3, we need these columns: ADM2_NAME, ADM3_NAME (& WKT) [ADM1_NAME and Country can also be used to help separate duplicates]
     21 * For L4, we need these columns: ADM3_NAME, ADM4_NAME (& WKT) [ADM2_NAME, ADM1_NAME and Country can also be used to help separate duplicates]
     22 * For specific lcoations, we need these columns: ADMx_NAME (for appropiate parent level of hierarchy) Name (& Lat/Lon) [ADM2_NAME, ADM1_NAME and Country can also be used to help separate duplicates]
     23 * Code columns are used, if-present
     24  * ADM1_CODE, ADM1_CODE2, ADM2_CODE, ADM2_CODE2, ADM3_CODE, ADM3_CODE2, ADM4_CODE, ADM4_CODE2, Code, Code2
     25 * Population & Elevation columns are read, if-present
    1926 * Ensure that names are consistent between Levels
    2027 * The PROPER() spreadsheet function is useful to get the names in the correct format (then Paste as Text).
    … …  
    2532Basic Hierarchy can often be found from Wikipedia (although currently there's no easy way to download this - a student project to enhance Wikipedia for this would be much appreciated!).
    2633
    27 For the Polygon data, it is normal to get this from Shapefiles (see below).
    28 
    29 Example for Pakistan:
    30 {{{
    31 tablename = "gis_location"
    32 table = db[tablename]
    33 db.executesql("DROP INDEX name__idx on %s;" % tablename)
    34 # L0
    35 import csv
    36 csv.field_size_limit(2**20 * 10)  # 10 megs
    37 db.import_from_csv_file(open("L0.csv", "rb"))
    38 db.commit()
    39 # L1
    40 gis.import_csv("pak_adm1.csv", check_duplicates=False)
    41 db.commit()
    42 # L2
    43 db(table.name == "Baluchistan").update(name="Balochistan")
    44 db(table.name == "Northern Areas").update(name="Gilgit Baltistan")
    45 db(table.name == "N.W.F.P.").update(name="Khyber Pakhtunkhwa")
    46 db(table.name == "F.A.T.A.").update(name="FATA")
    47 db(table.name == "F.C.T.").update(name="Islamabad")
    48 db(table.name == "Azad Kashmir").update(name="AJK")
    49 gis.import_csv("pak_adm2.csv", check_duplicates=False)
    50 db(table.name == "Sind").update(name="Sindh")
    51 db(table.name == "AJK").update(name="Pakistan Administered Kashmir")
    52 db(table.name == "FATA").update(name="Federally Administered Tribal Areas")
    53 db((table.name == "Islamabad") & (table.level == "L1")).update(name="Federal Capital Territory")
    54 db.commit()
    55 # L3
    56 db(table.name == "Jaccobabad").update(name="Jacobabad")
    57 db(table.name == "Tando Allahyar").update(name="Tando Allah Yar")
    58 db(table.name == "Qambar Shahdad kot").update(name="Qambar Shahdadkot")
    59 gis.import_csv("pak_adm3.csv", check_duplicates=False)
    60 db(table.name == "Islamabad").update(name="Islamabad Capital Territory")
    61 db(table.name == "Tando Allah Yar").update(name="Tando Allahyar")
    62 db(table.name == "Qambar Shahdadkot").update(name="Qambar Shahdad Kot")
    63 db(table.name == "Leiah").update(name="Layyah")
    64 db(table.name == "Leiah Tehsil").update(name="Layyah Tehsil")
    65 db(table.name == "Kalur Kot Tehsil").update(name="Kallur Kot Tehsil")
    66 db(table.name == "De-excluded Area").update(name="Tribal Area")
    67 db(table.name == "De-excluded Area D.g Khan").update(name="Tribal Area")
    68 db.commit()
    69 # L4
    70 db(table.name == "Noorpur Tehsil").update(name="Noorpur Thal Tehsil")
    71 jhang = db((table.name == "Jhang") & (table.level==L2)).select(table.id, limitby=(0, 1)).first().id
    72 table.insert(name="Ahmadpur Sial", parent=jhang, level="L3", url="http://en.wikipedia.org/wiki/Ahmedpur_Sial_Tehsil")
    73 gis.import_csv("punjab_l4.csv", check_duplicates=False)
    74 db.commit()
    75 db(table.name == "Mirwah Taluka").update(name="Thari Mirwah Taluka")
    76 db(table.name == "Shah Bunder Taluka").update(name="Shah Bandar Taluka")
    77 badin = db((table.name == "Badin") & (table.level==L2)).select(table.id, limitby=(0, 1)).first().id
    78 table.insert(name="Talhar", parent=badin, level="L3", url="http://en.wikipedia.org/wiki/Talhar")
    79 jamshoro = db((table.name == "Jamshoro") & (table.level==L2)).select(table.id, limitby=(0, 1)).first().id
    80 table.insert(name="Manjhand Taluka", parent=jamshoro, level="L3", url="http://en.wikipedia.org/wiki/Jamshoro_District")
    81 gis.import_csv("sindh_l4.csv", check_duplicates=False)
    82 db.commit()
    83 db(table.name == "F.r Kala Dhaka").update(name="F.R. Kala Dhaka")
    84 db(table.name == "Martoong Tehsil").update(name="Martung Tehsil")
    85 db(table.name == "Takhat Nasrati Tehsil").update(name="Takht-e-Nasrati Tehsil")
    86 dikhan = db((table.name == "D. I. Khan") & (table.level == "L2")).select(table.id, limitby=(0, 1)).first().id
    87 table.insert(name="Daraban Tehsil", parent=dikhan, level="L3")
    88 table.insert(name="Paroa Tehsil", parent=dikhan, level="L3")
    89 lowerdir = db((table.name == "Lower Dir") & (table.level == "L2")).select(table.id, limitby=(0, 1)).first().id
    90 table.insert(name="Adenzai", parent=lowerdir, level="L3")
    91 table.insert(name="Balambat", parent=lowerdir, level="L3")
    92 table.insert(name="Khal", parent=lowerdir, level="L3")
    93 table.insert(name="Lal Qila", parent=lowerdir, level="L3")
    94 table.insert(name="Munda", parent=lowerdir, level="L3")
    95 table.insert(name="Samar Bagh", parent=lowerdir, level="L3")
    96 table.insert(name="Tazagram", parent=lowerdir, level="L3")
    97 table.insert(name="Timargara", parent=lowerdir, level="L3")
    98 upperdir = db((table.name == "Upper Dir") & (table.level == "L2")).select(table.id, limitby=(0, 1)).first().id
    99 table.insert(name="Barawal Tehsil", parent=upperdir, level="L3")
    100 table.insert(name="Chapar Tehsil", parent=upperdir, level="L3")
    101 table.insert(name="Dir Tehsil", parent=upperdir, level="L3")
    102 table.insert(name="Khal Tehsil", parent=upperdir, level="L3")
    103 table.insert(name="Kalkot Tehsil", parent=upperdir, level="L3")
    104 table.insert(name="Wari Tehsil", parent=upperdir, level="L3")
    105 gis.import_csv("khyber_l4.csv", check_duplicates=False)
    106 db.commit()
    107 # L5
    108 gis.import_csv("punjab_l5.csv", check_duplicates=False)
    109 gis.import_csv("sindh_l5.csv", check_duplicates=False)
    110 gis.import_csv("khyber_l5.csv", check_duplicates=False)
    111 db.commit()
    112 field = "name"
    113 db.executesql("CREATE INDEX %s__idx on %s(%s);" % (field, tablename, field))
    114 }}}
    115 
     34For the Polygon data, it is normal to get this from Shapefiles, such as GADM or UN CODS.
    11635=== Shapefiles ===
    11736Inspect the data using [http://qgis.org qGIS].