= Support for Upgrading Eden Deployments =

== Goal ==

'''Target user''': System Administrator of a production system

'''Goal''': This is one part of making operations a big fan of our software: Make it very easy for them to upgrade. 

== Upgrade Issues ==

The current process is manual: UserGuidelines/Admin/Upgrade

For other software, people are used to automatic upgrades.  Even the (very complex) Windows operating system auto-upgrades.
What can make an Eden upgrade problematic?  There are two types of changes that are visible to the sysadmin or user:

* Structural changes: These are changes that affect data structure and file layout.
  * Database schema changes
  * Directory structure changes
  * Configuration process and parameter changes
* Functional changes: These are changes that would alter the site behavior or layout, in non-trivial ways.
  This would affect users, and might require retraining.

These issues would be present even if the site were running an unmodified version of Eden using a supplied template.
If the site has made modifications to the code, even in a template, and those modifications conflict with changes in
the upgrade, then the two sets of modifications need to be reconciled.  Note configuration changes are expected, and
need to be supported in the normal case above.

Expected issues beyond the above, that can arise if the site has code modifications, are:

* An API change may require the site to alter their code to use the modified API.
* Alteration by the site of code outside of the site's template may lead to merge conflicts.
* If both the site and upgrade have database schema changes, then those changes need to be reconciled and merged.

Of these issues, the most potentially destructive is a database schema change, as without proper preparation,
running the upgraded version of Eden can lead to data loss.  The Web2py migration does not understand changes
such as renaming a column -- that is interpreted as deleting a column and adding a new one.  This can happen
regardless of whether the site has made any changes whatsoever in their code.  Also, requiring the sysadmin to
investigate the upgrade's schema changes, and write scripts to convert their database, is a significant burden.
Addressing database migration is thus likely to be the largest reduction in upgrade effort.

Support for upgrades can proceed in stages:

* Deal with the case in which there are no significant site code changes.
  That is, changes are limited to configuration and relatively cosmetic UI changes.
  * Provide support for database schema changes.
  * Automate configuration changes.
* Assume additional changes are confined to the site's template, but may include use of a now-deprecated
  or altered API.
* Deal with site database schema changes.
* Consider what can be done to ease conflict resolution in UI code.
* Likewise, for conflict resolution in model and controller code, beyond schemas.

Some potential conflicts may be avoided up front by convincing sites not to alter framework code, and to avoid
directly editing models and controllers.  For changes that may be of benefit to other users, encourage sites to
make their changes generic and contribute them back to the main repository.

Here, we consider the first stage, in which the upgrade issues are due to changes in Eden rather than in site
changes.  Since directory layout changes are rare, this can be deferred until such a change is planned, unless
there are sites deferring upgrades due to previous directory reorganizations.

== Configuration Changes ==

The site may have modifications to models/000_config.py, or they may have changes in their
template's config() function in config.py.  These are problematic as they are Python code.
Obtaining the site's settings.* values would require parsing the files in some manner.

An alternative is to replace the current Python code for configuration settings by a
consolidated format such as a JSON or XML structure, or by a standard type of configuration
file used in other Python products.

Options (which may be somewhat dated) are described here:

BluePrint/UpgradeConfig
== Database Schema Changes ==

Since this is typically the most difficult change for a site to cope with, and the most dangerous
(since it risks data loss or corruption), it may be best to tackle this first.

When the database schema changes in a way that requires conversion of the database, then we would
like to provide a script to perform the conversion.  This is not an unusual requirement.  For
instance, in a Ruby on Rails system, migration scripts are mandatory -- the schema is defined as
the result of applying all migration scripts in order.  In an Eden, or more generally, Web2py
system, the schema is defined directly, so there is no onus on the developer to write a migration
script.  Developers typically delete their database and reload sample data from csv files, so do
not have a need to migrate existing databases.  Thus, writing a migration script seems a burden.

The goal is thus to assist the developer in producing migration scripts, which they would then
include along with the changes that necessitate the migration, when they submit those changes to
the main repository.

A site performing an upgrade would then run an overall migration script that would create a copy
of the database, and apply all the individual migration scripts starting from the site's current
Eden revision, to the upgrade revision.  The site would then test using the new database, and if
ok, retire the previous database.

Detailed discussion of database migration (originally for a GSoC project):  BluePrintDatabaseMigration

GSoC 2012 database migration project:  [wiki:Event/2012/GSoC/DatabaseMigration Event/2012/GSoC/DatabaseMigration]

Current state and more discussion:  BluePrint/DatabaseMigration

----
BluePrints