wiki:DeveloperGuidelinesTesting

Version 99 (modified by Fran Boon, 11 years ago) ( diff )

--

Testing

"A bug is a test case you haven't written yet"
"Unit Tests allow merciless refactoring"

This page defines what our current approach versus our BluePrint for future options

Test-Driven Development is a programming styles which says that you 1st write your test cases (from the specs) & then proceed to make them pass.

Can pre-populate the database with random data using gluon.contrib.populate

Functional Tests

Building the Right Code

We have integrated http://seleniumhq.org/projects/remote-control/ into /static/selenium so that Functional Tests can be run easily.

Tests can be developed using Selenium IDE, if desired.

See the Documentation for details on how to use this test suite.

ToDo: create a HandleResults.py for storing results (to make visible to CI):

Hints:

  • Improving the stability of Selenium tests: http://googletesting.blogspot.com/2009/06/my-selenium-tests-arent-stable.html
  • Lots of useful tips: http://www.eviltester.com
  • Autocomplete is fiddly to test as need to trigger specific events:
    # Enter the search String
    sel.type("gis_location_autocomplete", "SearchString")
    # Trigger the event to get the AJAX to send
    sel.fire_event("gis_location_autocomplete", "keydown")
    # Wait for the popup menu
    for i in range(60):
        try:
            if "SearchString" == sel.get_text("css=ul.ui-autocomplete li:first-child a"):
                break
        except:
            pass
        time.sleep(1)
    else:
        self.fail("time out")
    # Select the Result
    sel.fire_event("css=ul.ui-autocomplete li:first-child a", "mouseover")
    sel.click("css=ul.ui-autocomplete li:first-child a")
    time.sleep(4)
    

Systers' approach:

Alternative Options:

Unit Tests

Building the Code Right

Selenium RC is great due to ability to handle JavaScript & also due to having an IDE for generating them (export as Python).
The IDE output needs to be modified to work with Web2Py.
NB Custom functions (e.g. for Random) cannot be shared with the Functional Tests (custom=JS) but the rest of the tests can be.
These tests are stored in /tests.
ToDo: Port the storeRandom function from JS to Python:

import random
print "test_%i@xxxxxxxx" % random.randint(1, 10000)

ToDo: Investigate how we can test multiple browsers.

CherryPy's WebTest is good for in-process testing & can be run from a browser-less server(easier for CI-integration).
These tests are stored in /tests/webtest.
NB These are a work-in-progress...need to enable access to Web2Py environment (db, etc) using:

from gluon.shell import exec_environment
env=exec_environment('the_model_file.py') 

Or could write as a 'Controller' & call from CLI:

python web2py.py -S appname -M -R yourscript.py 

Another similar option could be Pylon's WebTest

Doc Tests

Agile documentation which can be run using Web2Py's Admin UI.

To extend these to web applications, we have a module which uses wsgi_intercept & CherryPy's WebTest: modules/s3/s3test.py
This can be used from Controllers like:

def login():
    """ Login
    >>> from applications.sahana.modules.s3.s3test import WSGI_Test
    >>> test=WSGI_Test(db)
    >>> '200 OK' in test.getPage('/sahana/%s/login' % module)
    True
    >>> test.assertHeader("Content-Type", "text/html")
    >>> test.assertInBody('Login')
    """

This works fine,although if an assert fails then the UI gets stuck :/
The 'db' access part isn't yet working.

Note that Web2Py uses a big doctest at the end of each file: def test_all()

Continuous Integration

We are starting to use the Trac-integrated Bitten to monitor code quality.

ToDo: Write a step to parse/store the results of Selenium's HandleResults.py

<step id="lint" description="Run PyLint">
  <python:exec module="pylint.lint" output="pylint-report.txt" args="/var/www/trac/sahana3/bzr"/>
  <python:pylint file="pylint-report.txt" />
</step>

ToDo: Amend so that it can find the gluon module (or configure to not follow this dependency):
ImportError: Unable to find module for modules/s3/s3validators.py in /tmp/bittenA787wC/build_Trunk_1
ToDo: Fix Windows path (NB plain 'bzr' fails too, even if we're in the folder up from that): F: 1: No module named /var/www/trac/sahana3/bzr We need to write a build/test-results.xml for this one (& it also wants a setup.py?):

<step id="test" description="Run unit tests">
  <python:distutils command="unittest"/>
  <python:unittest file="build/test-results.xml"/>
  <python:trace summary="build/test-coverage.txt" coverdir="build/coverage" include="trac*" exclude="*.tests.*"/>
</step>

This one is pointless for us as we don't build anything using a setup.py:

<step id="build" description="Compile to byte code">
  <python:distutils command="build"/>
</step>

BluePrintTesting

DeveloperGuidelines

Attachments (4)

Download all attachments as: .zip

Note: See TracWiki for help on using the wiki.