wiki:DeveloperGuidelinesTesting

Version 69 (modified by Fran Boon, 13 years ago) ( diff )

--

Testing

"A bug is a test case you haven't written yet"
"Unit Tests allow merciless refactoring"

This page defines what our current approach versus our BluePrint for future options

InstallationGuidelinesDeveloper

Functional Tests

Building the Right Code

We have integrated Selenium Core into /static/selenium so that Functional Tests can be run (via 'Test' menu option visible to Admins).

Additional functions (e.g. Random) are added to:

/static/selenium/core/scripts/user-extensions.js

ToDo: create a HandleResults.py for storing results (to make visible to CI):

Tests can be developed using Selenium IDE, if desired.

If desired they can be maintained in a Python format using make_selenium.py. HTML tests run by TestRunner are in /static/selenium/tests. Python format tests are in /static/selenium/src. Convert between these using:

python make_selenium.py src tests
python make_selenium.py -p tests src

Unit Tests

Building the Code Right

Selenium RC is great due to ability to handle JavaScript & also due to having an IDE for generating them (export as Python).
The IDE output needs to be modified to work with Web2Py.
NB Custom functions (e.g. for Random) cannot be shared with the Functional Tests (custom=JS) but the rest of the tests can be.
These tests are stored in /tests.
ToDo: Port the storeRandom function from JS to Python:

import random
print "test_%i@xxxxxxxx" % random.randint(1, 10000)

CherryPy's WebTest is good for in-process testing.
These tests are stored in /tests/webtest.
NB These are a work-in-progress...need to enable access to Web2Py environment (db, etc) using:

from gluon.shell import exec_environment
env=exec_environment('the_model_file.py') 

Or could write as a 'Controller' & call from CLI:

python web2py.py -S appname -M -R yourscript.py 

Doc Tests

Agile documentation which can be run using Web2Py's Admin UI.

We have a module which uses wsgi_intercept & CherryPy's WebTest: modules/s3_test.py
This can be used from Controllers like:

def login():
    """ Login
    >>> from applications.sahana.modules.s3_test import WSGI_Test
    >>> test=WSGI_Test(db)
    >>> '200 OK' in test.getPage('/sahana/%s/login' % module)
    True
    >>> test.assertHeader("Content-Type", "text/html")
    >>> test.assertInBody('Login')
    """

This works fine,although if an assert fails then the UI gets stuck :/
The 'db' access part isn't yet working.

Note that Web2Py uses a big doctest at the end of each file: def test_all()

Continuous Integration

We are starting to use the Trac-integrated Bitten to monitor code quality.

<step id="lint" description="Run PyLint">
  <python:exec module="pylint.lint" output="pylint-report.txt" args="/var/www/trac/sahana3/bzr"/>
  <python:pylint file="pylint-report.txt" />
</step>

ImportError: Unable to find module for /var/www/trac/sahana3/bzr/modules/validators.py in /tmp/bittenA787wC/build_Trunk_1,

<step id="build" description="Compile to byte code">
  <python:distutils command="build"/>
</step>

distutils failed (512)
/usr/bin/python: can't open file '/tmp/bittenk8YjFn/build_Trunk_1/setup.py': [Errno 2] No such file or directory

<step id="test" description="Run unit tests">
  <python:distutils command="unittest"/>
  <python:unittest file="build/test-results.xml"/>
  <python:trace summary="build/test-coverage.txt" coverdir="build/coverage" include="trac*" exclude="*.tests.*"/>
</step>

ToDo: Configure a Bitten task to run: http://localhost/Selenium/TestRunner.html?auto=true


DeveloperGuidelines

Attachments (4)

Download all attachments as: .zip

Note: See TracWiki for help on using the wiki.