Version 10 (modified by 15 years ago) ( diff ) | ,
---|
Data Migration
We want to be able to migrate data from Prod to Dev so that Testers / Documenters have real data to play with.
Also want to be able to migrate from the current sqlite backend to MySQL
CSV
Beware DB locks!
- Don't keep Web2Py shell open longer than necessary
python web2py.py -S prod -M db.export_to_csv_file(open('db.csv','wb')) Ctrl+D
Remove reference Tables from CSV file (don't want to import duplicates!)
python web2py.py -S dev -M db.import_from_csv_file(open('db.csv','rb')) db.commit() Ctrl+D
Changing a Column type in Sqlite live
sqlite handles live migrations pretty well, however if there is existing data then a conversion from string to integer can cause problems.
For a development system, the easiest solution is simply to remove all databases/*
& restart, however this isn't possible for a live system:
vim /home/haiti/prod/models/00_db.py migrate = True cd /home/haiti/prod bzr pull cd /home/haiti/web2py python web2py.py -S prod -M db(db.or_organisation.id > 0).select().export_to_csv_file(open('orgs.csv','wb')) db.or_organisation.drop() db.commit() Ctrl+D python web2py.py -S prod -M db.or_organisation.import_from_csv_file(open('orgs.csv','rb')) db.commit() Ctrl+D /etc/init.d/apache2 force-reload vim /home/haiti/prod/models/00_db.py migrate = False
If the system is being heavily used, need to put up a holding page during this time to prevent data entry. For switching on maintenance.
cd /home/haiti ./maintenance.on
For switching off maintenance.
cd /home/haiti ./maintenance.off
Web Services
Better to avoid locks & do without SSH login
- http://localhost:8000/sahana/gis/location/create?format=xml&fetchurl=http://haiti.sahanafoundation.org/prod/gis/location.xml
- http://localhost:8000/sahana/pr/person/create?format=xml&fetchurl=http://haiti.sahanafoundation.org/prod/pr/person.xml
- http://localhost:8000/sahana/or/organisation/create?format=xml&fetchurl=http://haiti.sahanafoundation.org/prod/or/organisation.xml
- http://localhost:8000/sahana/or/office/create?format=xml&fetchurl=http://haiti.sahanafoundation.org/prod/or/office.xml
- http://localhost:8000/sahana/or/contact/create?format=xml&fetchurl=http://haiti.sahanafoundation.org/prod/or/contact.xml
- S3XRC
- Currently needs to be done resource by resource (JOINed tables can be done together as Components of main resource)
- need to do referenced tables before the tables which refer to them
- migrate the locations first (using the xml interface), you can later migrate the orgs and offices together (through Component JOIN)
- Ideally we want to develop a way of having all tables loaded together into RAM, refs fixed-up & then imported to db together
Synchronisation
Best longer term as we want to be able to have this functionality exposed to UI.
- BluePrintSynchronisation - which is to be migrated to standard S3XRC Web Services anyway