Testing

"A bug is a test case you haven't written yet"
"Unit Tests allow merciless ​refactoring"

This page defines what our current approach versus our BluePrint for future options

InstallationGuidelinesDeveloper

Functional Tests

Building the Right Code

We have integrated ​Selenium Core into /static/selenium so that Functional Tests can be run (via 'Test' menu option visible to Admins).

Additional functions (e.g. Random) are added to:

/static/selenium/core/scripts/user-extensions.js

ToDo: create a HandleResults.py for storing results (to make visible to CI):

• e.g. Convert this one from PHP: ​http://wiki.openqa.org/display/SEL/Integrating+Selenium+And+CruiseControl.Net
• e.g. Extract from ​PloneTool's FunctionalTestTool.py

Tests can be developed using ​Selenium IDE, if desired.

If desired they can be maintained in a Python format using ​make_selenium.py. HTML tests run by TestRunner are in /static/selenium/tests. Python format tests are in /static/selenium/src. Convert between these using:

python make_selenium.py src tests
python make_selenium.py -p tests src

Unit Tests

Building the Code Right

​Selenium RC is great due to ability to handle JavaScript & also due to having an ​IDE for generating them (export as Python).
The IDE output needs to be ​modified to work with Web2Py.
NB Custom functions (e.g. for Random) cannot be shared with the Functional Tests (custom=JS) but the rest of the tests can be.
These tests are stored in /tests.
ToDo: Port the storeRandom function from JS to Python:

import random
print "test_%i@xxxxxxxx" % random.randint(1, 10000)

​CherryPy's ​WebTest is good for in-process testing.
These tests are stored in /tests/webtest.
NB These are a work-in-progress...need to enable access to Web2Py environment (db, etc) using:

from gluon.shell import exec_environment
env=exec_environment('the_model_file.py') 

Or could write as a 'Controller' & call from CLI:

python web2py.py -S appname -M -R yourscript.py 

Doc Tests

Agile documentation which can be run using Web2Py's Admin UI.

• e.g. ​http://127.0.0.1:8000/admin/default/test/sahana/default.py

We have a module which uses ​wsgi_intercept & ​CherryPy's ​WebTest: modules/s3_test.py
This can be used from Controllers like:

def login():
    """ Login
    >>> from applications.sahana.modules.s3_test import WSGI_Test
    >>> test=WSGI_Test(db)
    >>> '200 OK' in test.getPage('/sahana/%s/login' % module)
    True
    >>> test.assertHeader("Content-Type", "text/html")
    >>> test.assertInBody('Login')
    """

This works fine,although if an assert fails then the UI gets stuck :/
The 'db' access part isn't yet working.

Note that Web2Py uses a big doctest at the end of each file: def test_all()

Continuous Integration

We are starting to use the Trac-integrated ​Bitten to monitor code quality.

<step id="lint" description="Run PyLint">
  <python:exec module="pylint.lint" output="pylint-report.txt" args="/var/www/trac/sahana3/bzr"/>
  <python:pylint file="pylint-report.txt" />
</step>

ImportError: Unable to find module for /var/www/trac/sahana3/bzr/modules/validators.py in /tmp/bittenA787wC/build_Trunk_1,

<step id="build" description="Compile to byte code">
  <python:distutils command="build"/>
</step>

distutils failed (512)
/usr/bin/python: can't open file '/tmp/bittenk8YjFn/build_Trunk_1/setup.py': [Errno 2] No such file or directory

<step id="test" description="Run unit tests">
  <python:distutils command="unittest"/>
  <python:unittest file="build/test-results.xml"/>
  <python:trace summary="build/test-coverage.txt" coverdir="build/coverage" include="trac*" exclude="*.tests.*"/>
</step>

---

DeveloperGuidelines