Table of Contents
Comparison Script
Before using the comparison script , the user of the script should have 2 eden apps , one the containing the old database schema from which the data is to migrated and one with the new database schema . This can be done by git cloning a new copy of the eden app to which you to migrate .
The apps_db_comparison script compares the db schema of 2 eden apps to tell the differences
Just run the script with 3 necessary arguments in this order
1.WEB2PY_PATH
2.OLD_APP
3.NEW_APP
i.e python applications/eden/static/scripts/Database_migration/apps_db_comparison.py /home/web2py eden_old eden_new
This script also has an test script that makes 2 new web2py with apps to compare
Just run the test script to compare
i.e python applications/eden/tests/dbmigration/TestScript.py
The detected changes are displayed , then using the changes displayed user could figure out the what kind of migration is needed , like if there is a field appeared in the new schema which has the same properties as the one disappeared from the old schema then the migration function for the renaming a field can be called .
Also the change in the properties of the fields are detected , the migration function for those can be called too .
Migration Script
All the functions that does the migration have been added to migration_scripts file . migration_scripts file internally calls the functions of migration_helping_methods file too perform the migration . Thus in order to do the migration one just needs to import migration_scripts and call the method corresponding to the migration he wants . Also the user who wishes to do the migration in the db doesn't have to know the internal functioning of migration_script , thus the functions or the methods declared in migration_helping_methods file doesn't concern him.
Methods in migration_scripts file
migration_renaming_field
Purpose of the migration
Renaming a particular field , while keeping the other properties of the field same. Also if there are some index on that table that should be recreated and other constraints should remain unchanged too.
Method of Calling
import migration_scripts migration_scripts.migration_renaming_field(web2py_path, app, old_table, old_field_name , new_field_name ,attributes_to_copy)
Description of arguments
| Argument | Description |
|---|---|
| web2py_path | The path to the web2py congaing the Eden app (i.e "/home/web2py") |
| app | The name of the eden application of whose database needs to be migrated (i.e "eden") |
| old_table | The name of the table in which the field is renamed |
| old_field_name | The name of the original field before renaming |
| new_field_name | The name of the field after renaming |
| attributes_to_copy | The list of attributes which needs to be copied from the old_field to the new_field (needed only in sqlite) |
migration_renaming_table
Purpose of the migration
Renaming a particular table , Thus if any fields point to that table that should be handled too.
Method of Calling
import migration_scripts migration_scripts.migration_renaming_table(web2py_path, app ,old_table_name,new_table_name)
Description of arguments
| Argument | Description |
|---|---|
| web2py_path | The path to the web2py congaing the Eden app (i.e "/home/web2py") |
| app | The name of the eden application of whose database needs to be migrated (i.e "eden") |
| old_table_name | The name of the original table before renaming |
| new_table_name | The name of the table after renaming |
migrating_to_unique_field
Purpose of the migration
Adding values to a new field and adding values according to the mappings given through the mapping_function
Method of Calling
import migration_scripts import mapping_function migration_scripts.migrating_to_unique_field( web2py_path, app ,field_to_update , changed_table , list_of_tables)
Description of arguments
| Argument | Description |
|---|---|
| web2py_path | The path to the web2py congaing the Eden app (i.e "/home/web2py") |
| app | The name of the eden application of whose database needs to be migrated (i.e "eden") |
| field_to_update | The name of the field to be updated according to the mapping |
| changed_table | The name of the original table in which the new unique field id added |
| list_of_tables | These contains the list of tables which the changed_tables references |
update_field_by_mapping
Purpose of the migration
Update the the values of an existing field according to the mappings given through the mapping_function
Method of Calling
import migration_scripts import mapping_function migration_scripts.update_field_by_mapping( web2py_path, app ,field_to_update , changed_table ,mapping_function, list_of_tables)
Description of arguments
| Argument | Description |
|---|---|
| web2py_path | The path to the web2py congaing the Eden app (i.e "/home/web2py") |
| app | The name of the eden application of whose database needs to be migrated (i.e "eden") |
| field_to_update | The name of the field to be updated according to the mapping |
| changed_table | The name of the original table in which the new unique field id added |
| list_of_tables | These contains the list of tables which the changed_tables references |
list_field_to_reference
Purpose of the migration
Some fields that have types like list:integer or list:reference or list:string. These types encode multiple values into one string field. What's in the field looks something like this: "|5|100|23|" -- it has values separated by vertical bars. This is an official Web2py type, and typically Web2py handles packing and unpacking these fields:
Sometimes people find they need to pull those values out into a separate resource. There are various reasons they might want to do this, e.g. they might find they need more than one item of information, so instead of a single value, which might be easy to pack into a list, they want instead to have a separate table in which they can have multiple columns, and join that to the original table. To start with, they make a new table with a column for the values they'll get from the list field, and maybe some empty columns to be filled in later. That new table has a foreign key reference back to the original table. Then for each value in the list field for each record in the original table, they create one record in the new table that points back to the original record.
Example of the migration
http://pastebin.ubuntu.com/1142568/
Method of Calling
import migration_scripts migration_scripts.list_field_to_reference(web2py_path,app,new_table_name , new_list_field , list_field_name , old_table_id_field , old_table)
Description of arguments
| Argument | Description |
|---|---|
| web2py_path | The path to the web2py congaing the Eden app (i.e "/home/web2py") |
| app | The name of the eden application of whose database needs to be migrated (i.e "eden") |
| new_table_name | The name of the new table to which the list field needs to migrated |
| new_list_field | The name of the field in the new table which will hold the content of the list field |
| list_field_name | The name of the list field in the original table |
| old_table_id_field | The name of the id field in the original table |
| old_table | The name of the original table |
Tests for Migration
The links for all of the migrations
Migration script
https://github.com/anubhav929/Dbmigration-final-/blob/master/tests/dbmigration/test_migrations.py
This script when run shows a menu with all the tests options , choose the migration that you want to test
Links for the scrips
I have made it the database migration in the form of the library ,
- This file contains all the functions for the actual migration
https://github.com/anubhav929/Dbmigration-final-/blob/master/static/scripts/Database_migration/migration_scripts.py
- This file contains all the functions that are used by the actual migration
https://github.com/anubhav929/Dbmigration-final-/blob/master/static/scripts/Database_migration/migration_helping_methods.py
- Test general for all the migration cases , as in made a menu from which the selected one is tested
https://github.com/anubhav929/Dbmigration-final-/blob/master/tests/dbmigration/test_migrations.py
- Database schema comparison script to this library
https://github.com/anubhav929/Dbmigration-final-/blob/master/static/scripts/Database_migration/apps_db_comparison.py
- The tests for the app comparison are added in the tests folder , this folder also contains the new_model and old_model which are used by the test script to build two apps for coparison
Test Folder : https://github.com/anubhav929/Dbmigration-final-/tree/master/tests/dbmigration
Test Scrpit : https://github.com/anubhav929/Dbmigration-final-/blob/master/tests/dbmigration/TestScript.p
- Mapping function file to fill values in a field according to the mappings provided and also the query and fields for the select query to feed the required information to the mapping function https://github.com/anubhav929/Dbmigration-final-/blob/master/static/scripts/Database_migration/mapping_function.py
- Specific Migration script for a particular case in which a string field was changes into a field referencing to a table containing the string field https://github.com/anubhav929/Dbmigration-final-/blob/master/static/scripts/Database_migration/specific_migration.py
