Changes between Version 2 and Version 3 of DeveloperGuidelines/Testing


Ignore:
Timestamp:
06/29/13 18:19:59 (8 years ago)
Author:
somayjain
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • DeveloperGuidelines/Testing

    v2 v3  
    1 See:
    2 * [wiki:"QA"]
    3 * [wiki:"QA/Automated/Selenium"]
     1= Testing =
     2
     3''"A bug is a test case you haven't written yet"''
     4
     5''"Unit Tests allow merciless [http://diveintopython.org/refactoring/refactoring.html refactoring]"''
     6
     7This page defines what our current approach versus our [wiki:BluePrintTesting BluePrint for future options]
     8
     9Test-Driven Development is a programming style which says that you 1st write your test cases (from the [BluePrint specs]) & then proceed to make them pass.
     10
     11== Automated Tests ==
     12
     13=== Smoke Tests ===
     14Smoke Tests click through on every link within Sahana Eden and can be used to check for errors on pages and broken links. They are a light-weight approach to detecting basic errors, however they do not test form submission or any interaction.
     15To run the Smoke Tests:
     16{{{
     17python web2py.py -S eden -M -R applications/eden/modules/tests/suite.py -A --suite smoke --force-debug --link-depth 16 -V 3
     18}}}
     19''Add details on the dependancies for these''
     20
     21===  Selenium Tests ===
     22Selenium Tests use the [http://seleniumhq.org/docs/03_webdriver.jsp Selenium WebDriver] to simulate user interactions within a browser. They are very thorough as they test interactions in the entire Sahana Eden stack including JS, but can also be fragile (detect false negatives).
     23To run all the Selenium Tests:
     24{{{
     25python web2py.py -S eden -M -R applications/eden/modules/tests/suite.py
     26}}}
     27See: [wiki:QA/Automated/Selenium Automated Tests - Selenium] for more details
     28
     29=== Unit Tests ===
     30Unit Tests can be used to test whether specific "Units" of code are working. They are used extensively to test the Sahana Eden "S3" Framework.
     31Unit Tests require running with the IFRC_Train preopulate, 'settings.base.prepopulate = 27':
     32To run all Unit Tests:
     33{{{
     34python web2py.py -S eden -M -R applications/eden/modules/unit_tests/suite.py
     35}}}
     36
     37These unit tests are meant to detect problems early during development, which means you should run them quite often while you're still working on the code (read: before and after every little change you make) rather than expecting them to be run during QA cycles. That again means you would more often need to run tests for particular modules than the whole suite.
     38
     39For every module in {{{modules/s3}}} and {{{modules/s3db}}}, you can find the corresponding unit test module under {{{modules/unit_tests/s3db}}} resp. {{{modules/unit_tests/s3}}} (if one exists).
     40
     41To run tests for particular modules:
     42{{{
     43# e.g. for modules in modules/s3
     44python web2py.py -S eden -M -R applications/eden/modules/unit_tests/s3/s3resource.py
     45# e.g. for modules in modules/s3db
     46python web2py.py -S eden -M -R applications/eden/modules/unit_tests/s3db/pr.py
     47}}}
     48
     49It can be a very powerful development strategy - especially for back-end APIs - to first implement unit test cases for the functionality you intend to implement before actually implementing it. Apart from preventing bugs, this helps you to validate your design against requirements, and to keep the implementation simple and focussed. Additionally, the test cases can be a rich source of code samples how to apply your API methods.
     50
     51===  Role Tests ===
     52Role tests are used to check the permissions of user roles. Currently is limited to the IFRC (Red Cross) roles for RMS, but could be extended:
     53To run the Role Tests:
     54{{{
     55python web2py.py -S eden -M -R applications/eden/modules/tests/suite.py -A --suite roles
     56}}}
     57
     58===  Benchmark Tests ===
     59The Benchmark Tests are a simple way of measuring the performance of the Sahana Eden "S3" Framework. The result of these tests will give you a measure of the performance relative to the system running them.
     60To run the Benchmark Tests:
     61{{{
     62python web2py.py -S eden -M -R applications/eden/modules/unit_tests/s3/benchmark.py
     63}}}
     64
     65===  Load Tests ===
     66Load Testing will measure the performance of the entire Sahana Eden stack. We recommend using Tsung. See: Testing/Load  for more details
     67
     68== Continuous Integration (CI)Server ==
     69The CI Server will constantly run all Automated Tests on the latest version of Sahana Eden to detect any defects.
     70See: SysAdmin/ContinuousIntegration for how we have set up the CI Server and help to find the exact commands to run the tests.
     71
     72== Test Cases ==
     73''See:'' TestCases for more details - this page needs updating
     74*  [https://docs.google.com/spreadsheet/ccc?key=0AmB3hMcgB-3idG1XNGhhRG9QWF81dUlKLXpJaFlCMFE#gid=8 Sahana Eden Test Cases Spreadsheet]
     75* Test Cases from Historic Deployments
     76 * TestCasesHaiti
     77 * TestCasesIndia
     78 * TestCasesJapan
     79
     80== See Also ==
     81Systers' approach:
     82 * http://systers.org/systers-dev/doku.php/automated_functional_testing
     83 * List of Tests: http://systers.org/systers-dev/doku.php/master_checklist_template
     84 * GSoC project: http://systers.org/systers-dev/doku.php/svaksha:patches_release_testing_automation
     85
     86Alternative Options:
     87 * http://zesty.ca/scrape/
     88  * [http://pycon.blip.tv/file/3261277 Lightning Talk] (2.30)
     89  * [http://pycon.blip.tv/file/3261277 Lightning Talk] (2.30)
     90  * [http://pycon.blip.tv/file/3261277 Lightning Talk] (2.30)