Version 6 (modified by Fran Boon, 14 years ago) ( diff )


Data Migration

We want to be able to migrate data from Prod to Dev so that Testers / Documenters have real data to play with.

Also want to be able to migrate from the current sqlite backend to MySQL


Beware DB locks!

  • Don't keep Web2Py shell open longer than necessary
python -S prod -M

Remove reference Tables from CSV file (don't want to import duplicates!)

python -S dev -M

Changing a Column type in Sqlite live

sqlite handles live migrations pretty well, however if there is existing data then a conversion from string to integer can cause problems. For a development system, the easiest solution is simply to remove all databases/* & restart, however this isn't possible for a live system:

vim /home/haiti/prod/models/
    migrate = True

cd /home/haiti/prod
bzr pull

cd /home/haiti/web2py
python -S prod -M

db( > 0).select().export_to_csv_file(open('orgs.csv','wb'))

python -S prod -M


/etc/init.d/apache2 force-reload

vim /home/haiti/prod/models/
    migrate = False

If the system is being heavily used, need to put up a holding page during this time to prevent data entry.

vim /home/haiti/web2py/maintenance.htm
    This site is down for maintenance - back soon!
vim /etc/apache2/sites-enabled/prod
    RewriteRule ^/$ /maintenance.htm [R]

Web Services

Better to avoid locks & do without SSH login

  • S3XRC
  • Currently needs to be done resource by resource (JOINed tables can be done together as Components of main resource)
    • need to do referenced tables before the tables which refer to them
    • migrate the locations first (using the xml interface), you can later migrate the orgs and offices together (through Component JOIN)
  • Ideally we want to develop a way of having all tables loaded together into RAM, refs fixed-up & then imported to db together


Best longer term as we want to be able to have this functionality exposed to UI.

Note: See TracWiki for help on using the wiki.