wiki:DeveloperGuidelinesTesting

Version 85 (modified by Fran Boon, 14 years ago) ( diff )

--

Testing

"A bug is a test case you haven't written yet"
"Unit Tests allow merciless refactoring"

This page defines what our current approach versus our BluePrint for future options

Test-Driven Development is a programming styles which says that you 1st write your test cases (from the specs) & then proceed to make them pass.

Can pre-populate the database with random data using gluon.contrib.populate

Functional Tests

Building the Right Code

We have integrated Selenium Core into /static/selenium so that Functional Tests can be run (via 'Test' menu option visible to Admins).

Additional functions (e.g. Random) are added to:

/static/selenium/core/scripts/user-extensions.js

ToDo: create a HandleResults.py for storing results (to make visible to CI):

Tests can be developed using Selenium IDE, if desired.

If desired they can be maintained in a Python format using make_selenium.py. HTML tests run by TestRunner are in /static/selenium/tests. Python format tests are in /static/selenium/src. Convert between these using:

python make_selenium.py src tests
python make_selenium.py -p tests src

Hints on improving the stability of Selenium tests:

Alternative Options:

Unit Tests

Building the Code Right

Selenium RC is great due to ability to handle JavaScript & also due to having an IDE for generating them (export as Python).
The IDE output needs to be modified to work with Web2Py.
NB Custom functions (e.g. for Random) cannot be shared with the Functional Tests (custom=JS) but the rest of the tests can be.
These tests are stored in /tests.
ToDo: Port the storeRandom function from JS to Python:

import random
print "test_%i@xxxxxxxx" % random.randint(1, 10000)

ToDo: Investigate how we can test multiple browsers.

CherryPy's WebTest is good for in-process testing & can be run from a browser-less server(easier for CI-integration).
These tests are stored in /tests/webtest.
NB These are a work-in-progress...need to enable access to Web2Py environment (db, etc) using:

from gluon.shell import exec_environment
env=exec_environment('the_model_file.py') 

Or could write as a 'Controller' & call from CLI:

python web2py.py -S appname -M -R yourscript.py 

Another similar option could be Pylon's WebTest

Doc Tests

Agile documentation which can be run using Web2Py's Admin UI.

To extend these to web applications, we have a module which uses wsgi_intercept & CherryPy's WebTest: modules/s3_test.py
This can be used from Controllers like:

def login():
    """ Login
    >>> from applications.sahana.modules.s3_test import WSGI_Test
    >>> test=WSGI_Test(db)
    >>> '200 OK' in test.getPage('/sahana/%s/login' % module)
    True
    >>> test.assertHeader("Content-Type", "text/html")
    >>> test.assertInBody('Login')
    """

This works fine,although if an assert fails then the UI gets stuck :/
The 'db' access part isn't yet working.

Note that Web2Py uses a big doctest at the end of each file: def test_all()

Continuous Integration

We are starting to use the Trac-integrated Bitten to monitor code quality.

ToDo: Write a step to parse/store the results of Selenium's HandleResults.py

<step id="lint" description="Run PyLint">
  <python:exec module="pylint.lint" output="pylint-report.txt" args="/var/www/trac/sahana3/bzr"/>
  <python:pylint file="pylint-report.txt" />
</step>

ToDo: Amend so that it can find the gluon module (or configure to not follow this dependency):
ImportError: Unable to find module for modules/validators.py in /tmp/bittenA787wC/build_Trunk_1
ToDo: Fix Windows path (NB plain 'bzr' fails too, even if we're in the folder up from that): F: 1: No module named /var/www/trac/sahana3/bzr We need to write a build/test-results.xml for this one (& it also wants a setup.py?):

<step id="test" description="Run unit tests">
  <python:distutils command="unittest"/>
  <python:unittest file="build/test-results.xml"/>
  <python:trace summary="build/test-coverage.txt" coverdir="build/coverage" include="trac*" exclude="*.tests.*"/>
</step>

This one is pointless for us as we don't build anything using a setup.py:

<step id="build" description="Compile to byte code">
  <python:distutils command="build"/>
</step>

DeveloperGuidelines

Attachments (4)

Download all attachments as: .zip

Note: See TracWiki for help on using the wiki.