wiki:DeveloperGuidelinesTesting

Version 105 (modified by graeme, 11 years ago) ( diff )

--

Testing

"A bug is a test case you haven't written yet"
"Unit Tests allow merciless refactoring"

This page defines what our current approach versus our BluePrint for future options

Test-Driven Development is a programming styles which says that you 1st write your test cases (from the specs) & then proceed to make them pass.

For manual tests, can pre-populate the database with random data using gluon.contrib.populate

Functional Tests

Building the Right Code

We have integrated http://seleniumhq.org/projects/remote-control/ into /static/selenium so that Functional Tests can be run easily. For Notes on how to create your own test cases see CreatingTestCasesFirst.

Tests can be developed using Selenium IDE, if desired.

See the Documentation for details on how to use this test suite.

Hints:

  • Improving the stability of Selenium tests: http://googletesting.blogspot.com/2009/06/my-selenium-tests-arent-stable.html
  • Lots of useful tips: http://www.eviltester.com
  • Autocomplete is fiddly to test as need to trigger specific events:
    # Enter the search String
    sel.type("gis_location_autocomplete", "SearchString")
    # Trigger the event to get the AJAX to send
    sel.fire_event("gis_location_autocomplete", "keydown")
    # Wait for the popup menu
    for i in range(60):
        try:
            if "SearchString" == sel.get_text("css=ul.ui-autocomplete li:first-child a"):
                break
        except:
            pass
        time.sleep(1)
    else:
        self.fail("time out")
    # Select the Result
    sel.fire_event("css=ul.ui-autocomplete li:first-child a", "mouseover")
    sel.click("css=ul.ui-autocomplete li:first-child a")
    time.sleep(4)
    

Systers' approach:

Alternative Options:

Unit Tests

Building the Code Right

Selenium RC is great due to ability to handle JavaScript & also due to having an IDE for generating them (export as Python).
The IDE output needs to be modified to work with Web2Py.
NB Custom functions (e.g. for Random) cannot be shared with the Functional Tests (custom=JS) but the rest of the tests can be.
These tests are stored in /tests.
ToDo: Port the storeRandom function from JS to Python:

import random
print "test_%i@xxxxxxxx" % random.randint(1, 10000)

CherryPy's WebTest is good for in-process testing & can be run from a browser-less server(easier for CI-integration).
These tests are stored in /tests/webtest.
NB These are a work-in-progress...need to enable access to Web2Py environment (db, etc) using:

from gluon.shell import exec_environment
env=exec_environment('the_model_file.py') 

Or could write as a 'Controller' & call from CLI:

python web2py.py -S appname -M -R yourscript.py 

Another similar option could be Pylon's WebTest

Doc Tests

Agile documentation which can be run using Web2Py's Admin UI.

To extend these to web applications, we have a module which uses wsgi_intercept & CherryPy's WebTest: modules/s3/s3test.py
This can be used from Controllers like:

def login():
    """ Login
    >>> from applications.sahana.modules.s3.s3test import WSGI_Test
    >>> test=WSGI_Test(db)
    >>> '200 OK' in test.getPage('/sahana/%s/login' % module)
    True
    >>> test.assertHeader("Content-Type", "text/html")
    >>> test.assertInBody('Login')
    """

This works fine,although if an assert fails then the UI gets stuck :/
The 'db' access part isn't yet working.

Note that Web2Py uses a big doctest at the end of each file: def test_all()

Continuous Integration

We are starting to use Jenkins to run the Functional Tests automatically.

Other options: http://stackoverflow.com/questions/225598/pretty-continuous-integration-for-python


BluePrintTesting

DeveloperGuidelines

Attachments (4)

Download all attachments as: .zip

Note: See TracWiki for help on using the wiki.