Pakistan Deployment Cycle

• UAT: ​http://test.eden.sahanafoundation.org
• Live: ​http://pakistan.sahanafoundation.org

Usage Instructions

Login to eden.sahanafoundation.org.

1st time:

cd /home/release
fab test generate_keys
fab test distribute_keys

Subsequently:

cd /home/release
fab test deploy

Wishlist

We want to do daily upgrades of the system from a single script:

1. Upgrade Live with code release from UAT

• Read VERSION from UAT to know which revision to pull to live
  • the 'update' script falls back to 'bzr pull' but supports an 'update XXX' arg to 'bzr pull -r XXX'
• Send a notification to the list (for testers) with a summary of the day's changes: Full list of fixed bugs to be verified, New features to be Tested

2. Upgrade UAT to Trunk code

• Send a notification to the list (for users) with a summary of yesterday's code changes: Focussed on new features & notable bug fixes

3. Refresh the UAT database with data from Live

• Include 'uploads' folder
• Need to ensure that User Accounts in Test are not overwritten
• Need to ensure that Role memberships in Live & Test can be different

Notifications can be built with info from:

• Trac Timeline: http://eden.sahanafoundation.org/timeline
  • LP ChangeLog: ​https://code.launchpad.net/~flavour/sahana-eden/trunk (included in Trac timeline)
  • Investigate a custom Trac script to build the report automatically
    • possible examples which can help with writing this:
      • ​http://trac-hacks.org/wiki/DailyReminderScriptForTracScript
      • ​http://trac-hacks.org/wiki/TracReleasePlugin

• UserGuidelinesUpgrade

Sysadmin ToDo

• Confirm versions of all base software the same on both sides
• Set deployment_settings on UAT to the same as Prod
• Configure Apache Maintenance site

  ln -sf /etc/apache2/sites-available/maintenance ln -sf
  /etc/apache2/sites-enabled/pakistan
  
  This should be enhanced by:
   * keeping access to phpMyAdmin
   * allowing access to site through a browser - but using a different
  name (which we don't publish)
   * improving the text on the maintenance page
  

• Extend scripts:
  • ConfigurationGuidelines#Usefulaliases

    pull
    migrate
    (check for conflicts & copy .THIS over - eiher parse the bzr output or search filesystem - whichever is easier/quicker)
    {CLI web2py load as 'su web2py')
    (Check for migration failures in databases/sql.log)
    migrateoff
    DBsync (tbc)
    

  • Use Fabric:
    • Current status: ​https://gist.github.com/2b6c811432c8c736ad3f
      • Add migration (CLI as web2py)
        • Check sql.log for success before continuing
        • If it fails, it would be lovely to be told which table failed (in sql.log) & also launch a mysql prompt with 'show innodb status;' (parsed?) & potentially even having mysql fix it automatically (all possible for sure, but lower priority than the core)
      • Add rollback() by reading VERSION before bzr pull, so then can bzr revert -r $version
    • Background: ​http://lethain.com/entry/2008/nov/04/deploying-django-with-fabric/

• Rename instance as test.pakistan.sahanafoundation.org
  • keep test.eden.sahanafoundation.org as an alias for now
  • add test.sahanafoundation.org as an alias too

• Create a new dev.pakistan.sahanafoundation.org instance
  • This shouldn't be fully-automated into the upgrades cycle, but does have a script to refresh data from live manually

Other Tasks

Update script for debian packages: SSH into each &

apt-get update; apt-get upgrade

Use Fabric?

Live

• Schedule the Ushahidi imports:
  • ​http://pakistan.sahanafoundation.org/eden/irs/ireport/ushahidi
  • ​http://pakreport.org/ushahidi/api?task=incidents&by=all&resp=xml&limit=1000 (how to avoid this?)
  • Can we pass URL as argument?
• Upgrade Geraldo to 0.4.0 (currently 0.3.9)
• Upgrade ReportLab from the debian-packaged 'python-reportlab 2.1dfsg-2' to current 2.4

• Get MapProxy working (basic install on 'geo' done)

Demo

• Update Demo (whilst keeping the logins there intact - all other data can be dropped)

Trac

• Investigate a fix or alternative to ​http://trac-hacks.org/wiki/MathCaptchaPlugin for allowing Trac users to register bugs anonymously whilst not locking out our testing team.
• Convert from sqlite to PostgreSQL (or MySQL) to improve performance

---

DeveloperGuidelinesDataMigration

Pakistan