[[TOC]]
= Quality Assurance =

== Testing ==

''"A bug is a test case you haven't written yet"''

''"Unit Tests allow merciless [http://diveintopython.org/refactoring/refactoring.html refactoring]"''

This page defines what our current approach versus our [wiki:BluePrintTesting BluePrint for future options]

Test-Driven Development is a programming style which says that you 1st write your test cases (from the [BluePrint specs]) & then proceed to make them pass.

== Automated Tests ==

=== Unit Tests ===
There are additional unit tests available, which also require running with the IFRC_Train preopulate, 'settings.base.prepopulate = 27':

To run all unit tests:
{{{
python web2py.py -S eden -M -R applications/eden/modules/unit_tests/suite.py
}}}

These unit tests are meant to detect problems early during development, which means you should run them quite often while you're still working on the code (read: before and after every little change you make) rather than expecting them to be run during QA cycles. That again means you would more often need to run tests for particular modules than the whole suite.

For every module in {{{modules/s3}}} and {{{modules/eden}}}, you can find the corresponding unit test module under {{{modules/unit_tests/eden}}} resp. {{{modules/unit_tests/s3}}} (if one exists).

To run tests for particular modules:
{{{
# e.g. for modules in modules/s3
python web2py.py -S eden -M -R applications/eden/modules/unit_tests/s3/s3resource.py
# e.g. for modules in modules/eden
python web2py.py -S eden -M -R applications/eden/modules/unit_tests/eden/pr.py
}}}

It can be a very powerful development strategy - especially for back-end APIs - to first implement unit test cases for the functionality you intend to implement before actually implementing it. Apart from preventing bugs, this helps you to validate your design against requirements, and to keep the implementation simple and focussed. Additionally, the test cases can be a rich source of code samples how to apply your API methods.

=== Smoke Tests ===
{{{
python web2py.py -S eden -M -R applications/eden/modules/tests/suite.py -A --suite smoke --force-debug --link-depth 16 -V 3
}}}

===  Selenium Tests ===
{{{
python web2py.py -S eden -M -R applications/eden/modules/tests/suite.py
}}}
See: wiki:QA/Automated/Selenium for more details 

===  Role Tests ===
This test is used to check the permissions of user roles. Currently is limited to the IFRC (Red Cross) roles for RMS, but could be extended:
{{{
python web2py.py -S eden -M -R applications/eden/modules/tests/suite.py -A --suite roles
}}}

===  Benchmark Tests ===
The Benchmark Tests are a simple way of measuring the performance 
{{{
python web2py.py -S eden -M -R applications/eden/modules/unit_tests/s3/benchmark.py
}}}

===  Load Tests ===
Load Testing will measure the performance of the entire Sahana Eden stack. We recommend using Tsung. See: Testing/Load  for more details 

== Continuous Integration (CI)Server ==
See: SysAdmin/ContinuousIntegration for how we have set up the CI Server and help to find the exact commands to run the tests.

== Test Cases ==
''See:'' TestCases for more details - this page needs updating
*  [https://docs.google.com/spreadsheet/ccc?key=0AmB3hMcgB-3idG1XNGhhRG9QWF81dUlKLXpJaFlCMFE#gid=8 Sahana Eden Test Cases Spreadsheet]
* Test Cases from Historic Deployments
 * TestCasesHaiti
 * TestCasesIndia
 * TestCasesJapan

== See Also ==
Systers' approach:
 * http://systers.org/systers-dev/doku.php/automated_functional_testing
 * List of Tests: http://systers.org/systers-dev/doku.php/master_checklist_template
 * GSoC project: http://systers.org/systers-dev/doku.php/svaksha:patches_release_testing_automation

Alternative Options:
 * http://zesty.ca/scrape/
  * [http://pycon.blip.tv/file/3261277 Lightning Talk] (2.30)
  * [http://pycon.blip.tv/file/3261277 Lightning Talk] (2.30)
  * [http://pycon.blip.tv/file/3261277 Lightning Talk] (2.30)