wiki:GSOC2012/DatabaseMigration/Userguildlines

Version 21 (modified by anubhav, 12 years ago) ( diff )

--

Comparison Script

Before using the comparison script , the user of the script should have 2 eden apps , one the containing the old database schema from which the data is to migrated and one with the new database schema . This can be done by git cloning a new copy of the eden app to which you to migrate .

The apps_db_comparison script compares the db schema of 2 eden apps to tell the differences
Just run the script with 3 necessary arguments in this order
1.WEB2PY_PATH
2.OLD_APP
3.NEW_APP
i.e python applications/eden/static/scripts/Database_migration/apps_db_comparison.py /home/web2py eden_old eden_new

This script also has an test script that makes 2 new web2py with apps to compare Just run the test script to compare
i.e python applications/eden/tests/dbmigration/TestScript.py

The detected changes are displayed , then using the changes displayed user could figure out the what kind of migration is needed , like if there is a field appeared in the new schema which has the same properties as the one disappeared from the old schema then the migration function for the renaming a field can be called .

Also the change in the properties of the fields are detected , the migration function for those can be called too .

Migration Script

All the functions that does the migration have been added to migration_scripts file . migration_scripts file internally calls the functions of migration_helping_methods file too perform the migration . Thus in order to do the migration one just needs to import migration_scripts and call the method corresponding to the migration he wants . Also the user who wishes to do the migration in the db doesn't have to know the internal functioning of migration_script , thus the functions or the methods declared in migration_helping_methods file doesn't concern him.

Methods in migration_scripts file

migration_renaming_field

Purpose of the migration

Renaming a particular field , while keeping the other properties of the field same. Also if there are some index on that table that should be recreated and other constraints should remain unchanged too.

Method of Calling

import migration_scripts
migration_scripts.migration_renaming_field(web2py_path, app, old_table, old_field_name , new_field_name ,attributes_to_copy)

Description of arguments

Argument Description
web2py_path The path to the web2py congaing the Eden app (i.e "/home/web2py")
app The name of the eden application of whose database needs to be migrated (i.e "eden")
old_table The name of the table in which the field is renamed
old_field_name The name of the original field before renaming
new_field_name The name of the field after renaming
attributes_to_copy The list of attributes which needs to be copied from the old_field to the new_field (needed only in sqlite)

migration_renaming_table

Purpose of the migration

Renaming a particular table , Thus if any fields point to that table that should be handled too.

Method of Calling

import migration_scripts
migration_scripts.migration_renaming_table(web2py_path, app ,old_table_name,new_table_name)

Description of arguments

Argument Description
web2py_path The path to the web2py congaing the Eden app (i.e "/home/web2py")
app The name of the eden application of whose database needs to be migrated (i.e "eden")
old_table_name The name of the original table before renaming
new_table_name The name of the table after renaming

migrating_to_unique_field

Purpose of the migration

Adding values to a new field and adding values according to the mappings given through the mapping_function

Method of Calling

import migration_scripts
import mapping_function
migration_scripts.migrating_to_unique_field( web2py_path, app ,field_to_update , changed_table , list_of_tables)

Description of arguments

Argument Description
web2py_path The path to the web2py congaing the Eden app (i.e "/home/web2py")
app The name of the eden application of whose database needs to be migrated (i.e "eden")
field_to_update The name of the field to be updated according to the mapping
changed_table The name of the original table in which the new unique field id added
list_of_tables These contains the list of tables which the changed_tables references

update_field_by_mapping

Purpose of the migration

Update the the values of an existing field according to the mappings given through the mapping_function

Method of Calling

import migration_scripts
import mapping_function
migration_scripts.update_field_by_mapping( web2py_path, app ,field_to_update , changed_table ,mapping_function, list_of_tables)

Description of arguments

Argument Description
web2py_path The path to the web2py congaing the Eden app (i.e "/home/web2py")
app The name of the eden application of whose database needs to be migrated (i.e "eden")
field_to_update The name of the field to be updated according to the mapping
changed_table The name of the original table in which the new unique field id added
list_of_tables These contains the list of tables which the changed_tables references

list_field_to_reference

Purpose of the migration

Some fields that have types like list:integer or list:reference or list:string. These types encode multiple values into one string field. What's in the field looks something like this: "|5|100|23|" -- it has values separated by vertical bars. This is an official Web2py type, and typically Web2py handles packing and unpacking these fields:

Sometimes people find they need to pull those values out into a separate resource. There are various reasons they might want to do this, e.g. they might find they need more than one item of information, so instead of a single value, which might be easy to pack into a list, they want instead to have a separate table in which they can have multiple columns, and join that to the original table. To start with, they make a new table with a column for the values they'll get from the list field, and maybe some empty columns to be filled in later. That new table has a foreign key reference back to the original table. Then for each value in the list field for each record in the original table, they create one record in the new table that points back to the original record.

Example of the migration

http://pastebin.ubuntu.com/1142568/

Method of Calling

import migration_scripts
migration_scripts.list_field_to_reference(web2py_path,app,new_table_name , new_list_field , list_field_name , old_table_id_field , old_table)

Description of arguments

Argument Description
web2py_path The path to the web2py congaing the Eden app (i.e "/home/web2py")
app The name of the eden application of whose database needs to be migrated (i.e "eden")
new_table_name The name of the new table to which the list field needs to migrated
new_list_field The name of the field in the new table which will hold the content of the list field
list_field_name The name of the list field in the original table
old_table_id_field The name of the id field in the original table
old_table The name of the original table

Tests for Migration

The links for all of the migrations

Migration script

https://github.com/anubhav929/Dbmigration-final-/blob/master/tests/dbmigration/test_migrations.py

This script when run shows a menu with all the tests options , choose the migration that you want to test

Links for the scrips

I have made it the database migration in the form of the library ,

Note: See TracWiki for help on using the wiki.