[[TOC]]
= Testing =
"A bug is a test case you haven't written yet" [[BR]]
"Unit Tests allow merciless [http://diveintopython.org/refactoring/refactoring.html refactoring]"

As it is a disaster management tool, bugs in Sahana that lead to aid/rescue workers in the field receiving incorrect or incomplete information could potentially lead to disaster victims losing their lives.
Therefore, Sahana developers need to place more emphasis on testing than might be required in other types of projects.

This page defines what our current approach versus our [wiki:BluePrintTesting BluePrint for future options]

Test-Driven Development is a programming styles which says that you 1st write your test cases (from the [BluePrints specs]) & then proceed to make them pass.

For manual tests, can pre-populate the database with random data using [http://groups.google.com/group/web2py/browse_thread/thread/4b2e5ed166220ba5# gluon.contrib.populate]

 * [wiki:InstallationGuidelinesDeveloperTesting]
 * TestCases - User Testing - List of things to test

== Functional Tests ==
Building the Right Code

We have integrated [http://seleniumhq.org/projects/remote-control/] into {{{/static/selenium}}} so that Functional Tests can be run easily. For Notes on how to create your own test cases see CreatingTestCasesFirst.

Tests can be developed using [http://seleniumhq.org/projects/ide Selenium IDE], if desired.

See the [http://eden.sahanafoundation.org/attachment/wiki/DeveloperGuidelinesTesting/Regression%20tests.pdf Documentation] for details on how to use this test suite.

Hints:
 * Some wrapper scripts for Win32 attached
 * Improving the stability of Selenium tests: http://googletesting.blogspot.com/2009/06/my-selenium-tests-arent-stable.html
 * Lots of useful tips: http://www.eviltester.com
 * Autocomplete is fiddly to test as need to trigger specific events:
{{{
# Enter the search String
sel.type("gis_location_autocomplete", "SearchString")
# Trigger the event to get the AJAX to send
sel.fire_event("gis_location_autocomplete", "keydown")
# Wait for the popup menu
for i in range(60):
    try:
        if "SearchString" == sel.get_text("css=ul.ui-autocomplete li:first-child a"):
            break
    except:
        pass
    time.sleep(1)
else:
    self.fail("time out")
# Select the Result
sel.fire_event("css=ul.ui-autocomplete li:first-child a", "mouseover")
sel.click("css=ul.ui-autocomplete li:first-child a")
time.sleep(4)
}}}

Systers' approach:
 * http://systers.org/systers-dev/doku.php/automated_functional_testing
 * List of Tests: http://systers.org/systers-dev/doku.php/master_checklist_template
 * GSoC project: http://systers.org/systers-dev/doku.php/svaksha:patches_release_testing_automation

Alternative Options:
 * http://zesty.ca/scrape/
  * [http://pycon.blip.tv/file/3261277 Lightning Talk] (2.30)
== Unit Tests ==
Building the Code Right

[http://seleniumhq.org/projects/remote-control Selenium RC] is great due to ability to handle !JavaScript & also due to having an [http://seleniumhq.org/projects/ide IDE] for generating them (export as Python).[[BR]]
The IDE output needs to be [http://groups.google.com/group/web2py/msg/d8c9fd6008029f6b modified] to work with Web2Py.[[BR]]
NB Custom functions (e.g. for Random) cannot be shared with the Functional Tests (custom=JS) but the rest of the tests can be.[[BR]]
These tests are stored in {{{/tests}}}.[[BR]]
!ToDo:  Port the storeRandom function from JS to Python:
{{{
import random
print "test_%i@xxxxxxxx" % random.randint(1, 10000)
}}}

 * [http://web2py.com/AlterEgo/default/show/260 AltereEgo entry] on testing in Web2Py
  * [http://stackoverflow.com/questions/2762294/unit-testing-in-web2py StackOverflow discussion]
 * [http://web2pyslices.com/main/slices/take_slice/67 Web2Py Slice] on Unit Testing

[http://cherrypy.org CherryPy]'s [http://cherrypy.org/browser/trunk/cherrypy/test/webtest.py WebTest] is good for in-process testing & can be run from a browser-less server(easier for CI-integration).[[BR]]
These tests are stored in {{{/tests/webtest}}}.[[BR]]
NB These are a work-in-progress...need to enable access to Web2Py environment (db, etc) using:
{{{
from gluon.shell import exec_environment
env=exec_environment('the_model_file.py') 
}}}
Or could write as a 'Controller' & call from CLI:
{{{
python web2py.py -S appname -M -R yourscript.py 
}}}

Another similar option could be [http://pylonshq.com/docs/en/0.9.7/thirdparty/webtest/ Pylon's WebTest]
== Doc Tests ==
Agile documentation which can be run using Web2Py's Admin UI.
 * http://www.python.org/doc/2.6/library/doctest.html
 * e.g. http://127.0.0.1:8000/admin/default/test/sahana/default.py
To extend these to web applications, we have a module which uses [http://code.google.com/p/wsgi-intercept wsgi_intercept] & [http://cherrypy.org CherryPy]'s [http://cherrypy.org/browser/trunk/cherrypy/test/webtest.py WebTest]: {{{modules/s3/s3test.py}}} [[BR]]
This can be used from Controllers like:
{{{
def login():
    """ Login
    >>> from applications.sahana.modules.s3.s3test import WSGI_Test
    >>> test=WSGI_Test(db)
    >>> '200 OK' in test.getPage('/sahana/%s/login' % module)
    True
    >>> test.assertHeader("Content-Type", "text/html")
    >>> test.assertInBody('Login')
    """
}}}
This works fine,although if an assert fails then the UI gets stuck :/ [[BR]]
The 'db' access part isn't yet working.

Note that Web2Py uses a big doctest at the end of each file: {{{def test_all()}}}

== Continuous Integration ==
We are starting to use [http://jenkins-ci.org Jenkins] to run the Functional Tests automatically.
 * [SysAdminJenkins Infrasctructure support docs]
 * http://blog.karit.geek.nz/2010/07/selenium-grid-part-1-getting-started.html

Other options: http://stackoverflow.com/questions/225598/pretty-continuous-integration-for-python
 * [http://buildbot.net/trac Buildbot]
 * [http://bitten.edgewall.org Bitten]

----
BluePrintTesting

DeveloperGuidelines