Version 107 (modified by 13 years ago) ( diff ) | ,
---|
Table of Contents
Testing
"A bug is a test case you haven't written yet"
"Unit Tests allow merciless refactoring"
As it is a disaster management tool, bugs in Sahana that lead to aid/rescue workers in the field receiving incorrect or incomplete information could potentially lead to disaster victims losing their lives. Therefore, Sahana developers need to place more emphasis than usual on testing.
This page defines what our current approach versus our BluePrint for future options
Test-Driven Development is a programming styles which says that you 1st write your test cases (from the specs) & then proceed to make them pass.
For manual tests, can pre-populate the database with random data using gluon.contrib.populate
- InstallationGuidelinesDeveloperTesting
- TestCases - User Testing - List of things to test
Functional Tests
Building the Right Code
We have integrated http://seleniumhq.org/projects/remote-control/ into /static/selenium
so that Functional Tests can be run easily. For Notes on how to create your own test cases see CreatingTestCasesFirst.
Tests can be developed using Selenium IDE, if desired.
See the Documentation for details on how to use this test suite.
Hints:
- Some wrapper scripts for Win32 attached
- Improving the stability of Selenium tests: http://googletesting.blogspot.com/2009/06/my-selenium-tests-arent-stable.html
- Lots of useful tips: http://www.eviltester.com
- Autocomplete is fiddly to test as need to trigger specific events:
# Enter the search String sel.type("gis_location_autocomplete", "SearchString") # Trigger the event to get the AJAX to send sel.fire_event("gis_location_autocomplete", "keydown") # Wait for the popup menu for i in range(60): try: if "SearchString" == sel.get_text("css=ul.ui-autocomplete li:first-child a"): break except: pass time.sleep(1) else: self.fail("time out") # Select the Result sel.fire_event("css=ul.ui-autocomplete li:first-child a", "mouseover") sel.click("css=ul.ui-autocomplete li:first-child a") time.sleep(4)
Systers' approach:
- http://systers.org/systers-dev/doku.php/automated_functional_testing
- List of Tests: http://systers.org/systers-dev/doku.php/master_checklist_template
- GSoC project: http://systers.org/systers-dev/doku.php/svaksha:patches_release_testing_automation
Alternative Options:
Unit Tests
Building the Code Right
Selenium RC is great due to ability to handle JavaScript & also due to having an IDE for generating them (export as Python).
The IDE output needs to be modified to work with Web2Py.
NB Custom functions (e.g. for Random) cannot be shared with the Functional Tests (custom=JS) but the rest of the tests can be.
These tests are stored in /tests
.
ToDo: Port the storeRandom function from JS to Python:
import random print "test_%i@xxxxxxxx" % random.randint(1, 10000)
- AltereEgo entry on testing in Web2Py
- Web2Py Slice on Unit Testing
CherryPy's WebTest is good for in-process testing & can be run from a browser-less server(easier for CI-integration).
These tests are stored in /tests/webtest
.
NB These are a work-in-progress...need to enable access to Web2Py environment (db, etc) using:
from gluon.shell import exec_environment env=exec_environment('the_model_file.py')
Or could write as a 'Controller' & call from CLI:
python web2py.py -S appname -M -R yourscript.py
Another similar option could be Pylon's WebTest
Doc Tests
Agile documentation which can be run using Web2Py's Admin UI.
- http://www.python.org/doc/2.6/library/doctest.html
- e.g. http://127.0.0.1:8000/admin/default/test/sahana/default.py
To extend these to web applications, we have a module which uses wsgi_intercept & CherryPy's WebTest: modules/s3/s3test.py
This can be used from Controllers like:
def login(): """ Login >>> from applications.sahana.modules.s3.s3test import WSGI_Test >>> test=WSGI_Test(db) >>> '200 OK' in test.getPage('/sahana/%s/login' % module) True >>> test.assertHeader("Content-Type", "text/html") >>> test.assertInBody('Login') """
This works fine,although if an assert fails then the UI gets stuck :/
The 'db' access part isn't yet working.
Note that Web2Py uses a big doctest at the end of each file: def test_all()
Continuous Integration
We are starting to use Jenkins to run the Functional Tests automatically.
- Infrasctructure support docs
- http://blog.karit.geek.nz/2010/07/selenium-grid-part-1-getting-started.html
Other options: http://stackoverflow.com/questions/225598/pretty-continuous-integration-for-python
Attachments (4)
- Regression tests.pdf (482.2 KB ) - added by 14 years ago.
-
Regression tests.odt
(349.2 KB
) - added by 14 years ago.
Source for the PDF
-
test.cmd
(1.9 KB
) - added by 13 years ago.
Non-Interactive
-
testui.cmd
(1.9 KB
) - added by 13 years ago.
Interactive Wrapper for Selenium
Download all attachments as: .zip