= Automated Tests - Selenium =
[[TOC]]

Selenium provides the ability to test Sahana Eden as users see it - namely through a web browser. This therefore does end-to-end Functional Testing, however it can also be used as Unit Testing

We are building our framework around the new [http://seleniumhq.org/docs/03_webdriver.jsp Selenium WebDriver], despite having some legacy code in the older format:
* http://seleniumhq.org/docs/appendix_migrating_from_rc_to_webdriver.html#why-migrate-to-webdriver

The tests are stored in {{{eden/modules/tests}}}
== Installation of the testing environment in your machine ==
In order to execute the automated Selenium powered test scripts, you must install the Selenium Web Drivers into Python.

*Web2py will need to be installed, and should come with the installation of Sahana Eden.

Download and install the latest Selenium package, which can be found at [http://pypi.python.org/pypi/selenium].
Follow the installation instructions there. Windows users can either install pip and use the pip command shown,
or install 7zip and use it to extract files from the .tar.gz file.

== Running / Executing Automated test scripts: ==

In your {{{models/000_config.py}}}, make these changes:

Uuncomment the lines that disable confirmation popups:
{{{
# Should user be prompted to save before navigating away?
settings.ui.navigate_away_confirm = False
# Should user be prompted to confirm actions?
settings.ui.confirm = False
}}}

Make sure that following settings are enabled in 000_config.py file.
{{{
settings.base.migrate = True
settings.base.debug = True
}}}

Find the settings.base.template line and change it to:
{{{
settings.base.template = "IFRC"
}}}

Then you will need to recreate and populate the database with test data, as follows:

Delete the contents of databases/* , errors/* and sessions/* in your eden directory.

E.g. for Linux, cd to your eden directory and do:
{{{
rm databases/* sessions/* errors/*
}}}

On Windows, use Explorer to delete the contents of the eden\databases, eden\errors, and eden\sessions folders.

To re-populate the database, for either Linux or Windows (using a command prompt window), cd to the web2py directory, and do:
{{{
python web2py.py -S eden -M -R applications/eden/static/scripts/tools/noop.py
}}}

Currently, the tests can run on the following templates -
* IFRC
* default
* !SandyRelief
* DRMP
* CRMT

After the above re-population of the database, change the template to any of the above templates on which you wish to run the Selenium tests.

Start the Web2py server at 127.0.0.1:8000. (Selenium will start a captive browser,
which it will direct to contact the server at 127.0.0.1:8000.)

Run the whole test suite for the current template:
{{{
cd web2py
python web2py.py -S eden -M -R applications/eden/modules/tests/suite.py
}}}

Note : These tests don't run on Firefox 17 and above. So, if you have your default browser set as Firefox 17 or above, you will have to run them on Chrome. To do so, install [http://code.google.com/p/chromedriver/ Chromedriver][[br]]
And add {{{ -A --browser=Chrome }}} to the command.
So, it becomes 
{{{ 
python web2py.py -S eden -M -R applications/eden/modules/tests/suite.py -A --browser=Chrome 
}}}

Run a class and all tests in that class:
{{{
cd web2py
python web2py.py -S eden -M -R applications/eden/modules/tests/suite.py -A -C mytestclass
}}}

Run just one test within a class:
{{{
cd web2py
python web2py.py -S eden -M -R applications/eden/modules/tests/suite.py -A -C mytestclass -M mytestmethod
}}}

=== Command Line Arguments ===

A number of command line arguments have been added and more are being added so to get the latest list of available options use the --help switch, which you can quickly do as follows:
{{{
python web2py.py -S eden -M -R applications/eden/modules/tests/suite.py -A --help
}}}

Important options include:

-C for class to run

-M for test method within a class to run, when you use this option either use the -C option or provide the method in the format class.method

If you have HTMLTestRunner installed then a nicely formatted html report will be generated, should you want to disable this then use the --nohtml option. The HTML report will be written to the path given by the switch --html_path which by default will the web2py folder since that is where the tests scripts are run from. The file name will have a timestamp appended to it, if you want you can have just a date by using the option html_name_date.

The option --suite will run certain predefined tests. At the moment it supports '''smoke''', which runs a test to look for broken urls otherwise it will run all the tests. If a class or method are selected then this option is ignored. For the smoke test to work you need to install two packages, [http://twill.idyll.org/ twill] and [http://pypi.python.org/pypi/mechanize/ mechanize], since twill uses mechanize if you install via a package manager then you probably only need to install twill.

== Useful Information about the test suite ==

=== Tests for each template ===

* The tests to be run on a template are explicitly defined in {{{private/templates/<template_name>/tests.py}}} file.
* The list {{{current.selenium_tests}}} in the above file contains the class names of all the tests which are to be run for the template.
* If the tests are not specified in {{{tests.py}}} file for some template, the tests specified for 'default' template are run. 

=== Functions ===
The test suite provides the following functions - 

==== login ====
* Usage - {{{login(account, nexturl)}}}
* {{{account}}} is the user with whom we wish to login. Typically, "admin".
* {{{nexturl}}} is the url to be visited after logging in the user.

==== create ====

Used to create records in the database.

* Usage - {{{create(table_name, data)}}}
* {{{table_name}}} is the name of the table in which we wish to insert records
* {{{data}}} is a list of tuples, where each tuple represents information about one field in the table.
 * Eg - [ ( column_name1, data1, ... ), ( column_name2, data2, ... ), .. so on ]
* The test suite automatically determines the following field types from the HTML class names in the create form -
 * option(dropdown)
 * autocomplete
 * date
 * datetime
 * normal text input
* Hence, for the above field types, just 2 arguments need to be provided in the tuple - column name and data to be inserted.
 * Eg - {{{ (organisation_id, "International Federation of Red Cross and Red Crescent Societies") }}}
* For other field types listed below, an appropriate third argument needs to be provided.
 * checkbox
 * Embedded form fields - 
  * Eg - "pr_person" embedded in "hrm_human_resource"
 * Specific widgets - 
  * inv_widget
  * supply_widget
  * facility_widget
  * gis_location

* Selection of the option from the dropdown field - 
 * Longest Word Trimmed Search is used - Amongst the options in the dropdown field, the option which has most number of word matches with the input string(in the test data) is selected.
* Note : In case of filling in an organisation name, the data should not contain both name and acronym. 
 * Eg - correct usage : {{{ (organisation_id, "International Federation of Red Cross and Red Crescent Societies") }}}
 * Incorrect usage : {{{ (organisation_id, "International Federation of Red Cross and Red Crescent Societies (IFRC)") }}}

==== Search ====

* Usage - {{{ search(form_type, results_expected, fields, row_count, **kwargs) }}}
* form_type: This can either be search.simple_form or search.advanced_form
* results_expected: Are results expected? : True/False
* fields : a tuple of dictionaries, each dictionary specifying a field which is to be checked/unchecked in the form.
* row_count : Expected row count 
 * {{{ {"tablename":tablename, "key":key, "filters":[(field,value),...]} }}} can be passed to get the resource and eventually the DB row count.

==== report ====

* Usage - {{{ report(fields, report_of, grouped_by, report_fact, *args, **kwargs) }}}
* fields : a tuple of dictionaries, each dictionary specifying a field which is to be checked/unchecked in the form.
* report_of : The field whose report is to be formed. The option in 'Report of'
* grouped_by : The field for which the report is to be grouped by. The option in 'Grouped by'
* report_fact : The option in 'Value'


== Writing / Creating your own test scripts: ==
We aim to make it as easy as possible to write additional tests, which can easily be plugged into the Sahana Eden Testing System Framework.

New tests should be stored in a subfolder per module, adding the foldername to {{{eden/modules/tests/__init__.py}}} & creating an {{{__init__.py}}} in the subfolder.

'''A walk-through example: Creating a Record'''[[BR]]

The canonical example is: {{{eden/modules/tests/staff/create_staff.py}}}

We want to make an automated test script to test the Create Staff module. 
Create Staff falls under Staff feature, therefore we must add this test script module file in the subfolder /staff (eden/modules/tests/staff/)

Steps of automated the test for Create Staff:[[BR]][[BR]]
 
1) Let's call this test file: create_staff.py and save it in the subfolder /staff. [[BR]][[BR]]

2) We must now make sure we add the file name in the __init__.py
Open file __init__.py in the subfolder /staff and add:
{{{
from create_staff import *
}}}
3) create_staff.py
{{{

from tests.web2unittest import SeleniumUnitTest

class CreateStaff(SeleniumUnitTest):
    def test_hrm001_create_staff(self):
        """
            @case: HRM001
            @description: Create a Staff
            
            @TestDoc: https://docs.google.com/spreadsheet/ccc?key=0AmB3hMcgB-3idG1XNGhhRG9QWF81dUlKLXpJaFlCMFE
            @Test Wiki: http://eden.sahanafoundation.org/wiki/DeveloperGuidelines/Testing
        """
        print "\n"

        self.login(account="admin", nexturl="hrm/staff/create")

        self.create("hrm_human_resource", 
                    [( "organisation_id",
                       "International Federation of Red Cross and Red Crescent Societies"),
                     ( "site_id",
                       "AP Zone (Office)"),
                     ( "first_name",
                       "Robert",
                       "pr_person"),
                     ( "middle_name",
                       "James",
                       "pr_person"),
                     ( "last_name",
                       "Lemon",
                       "pr_person"),
                     ( "email",
                       "rjltestdonotusetest99@romanian.com",
                       "pr_person"),
                     ( "job_title_id",
                       "Warehouse Manager"),
                     ]
                     )
}}}

* The create function takes 2 arguments - {{{create(table_name, data)}}}
* {{{table_name}}} - The name of the database table in which we want to insert the record.
* {{{data}}} - The list which contains the record to be inserted.
* Each tuple in the above list contains information about each individual field.
 * Eg - {{{( "organisation_id","International Federation of Red Cross and Red Crescent Societies")}}}. Here, {{{organisation_id}}} is the label of the field in which we want to insert data {{{International Federation of Red Cross and Red Crescent Societies}}}

4) To run this automated test script.
Determine the template(s) on which you wish to run this test on. The tests for each template are defined in {{{private/templates/<template_name>/tests.py}}}.
Open tests.py for these templates and append the class name of the test to the list so it can be executed. (NOTE: We add the class name, not the function name).
{{{
current.selenium_tests.append("CreateStaff")
}}}

5) Run your test script. (Refer above on how to run/execute test scripts from the Test Suite).[[BR]][[BR]][[BR]]

'''Splitting Test Functions:''' [[BR]]
We aim to keep the test script as clean and understandable as possible. Therefore, we must separate the test functions which drives the a particular test script to another file.
As you can see with the Inventory feature tests, there is a file in the INV subdirectory called helper.py. This file contains all functions which helps drive the Inventory module tests such as send_item.py, receive_item.py and so forth.[[BR]][[BR]]

'''Other information on the Test System framework:'''[[BR]]

The key is to make tests which are as least fragile as possible through:
* State (we should be able to run individual tests easilty, which check their current state as-required)
* Deployment_Settings
* Localisation
* Theme

This suggests refactoring tests to centralise common elements into a library to mean fixes should only happen in 1 place.

There are a number of possible selectors to use to find your elements...the 'ID' may be the most stable, so don't be afraid of patching the code to add IDs where you'd like to be able to reach them reliably:
* http://selenium.googlecode.com/svn/trunk/docs/api/py/selenium/selenium.selenium.html

== See Also ==
 * http://code.google.com/p/selenium/wiki/PageObjects - Style suggestion