Version 80 (modified by 15 years ago) ( diff ) | ,
---|
Testing
"A bug is a test case you haven't written yet"
"Unit Tests allow merciless refactoring"
This page defines what our current approach versus our BluePrint for future options
Test-Driven Development is a programming styles which says that you 1st write your test cases (from the specs) & then proceed to make them pass.
Can pre-populate the database with random data using gluon.contrib.populate
InstallationGuidelinesDeveloperTesting
Functional Tests
Building the Right Code
We have integrated Selenium Core into /static/selenium
so that Functional Tests can be run (via 'Test' menu option visible to Admins).
Additional functions (e.g. Random) are added to:
/static/selenium/core/scripts/user-extensions.js
ToDo: create a HandleResults.py
for storing results (to make visible to CI):
- e.g. Convert this one from PHP: http://wiki.openqa.org/display/SEL/Integrating+Selenium+And+CruiseControl.Net
- e.g. Extract from PloneTool's
FunctionalTestTool.py
Tests can be developed using Selenium IDE, if desired.
If desired they can be maintained in a Python format using make_selenium.py. HTML tests run by TestRunner are in /static/selenium/tests
. Python format tests are in /static/selenium/src
. Convert between these using:
python make_selenium.py src tests python make_selenium.py -p tests src
Hints on improving the stability of Selenium tests:
Unit Tests
Building the Code Right
Selenium RC is great due to ability to handle JavaScript & also due to having an IDE for generating them (export as Python).
The IDE output needs to be modified to work with Web2Py.
NB Custom functions (e.g. for Random) cannot be shared with the Functional Tests (custom=JS) but the rest of the tests can be.
These tests are stored in /tests
.
ToDo: Port the storeRandom function from JS to Python:
import random print "test_%i@xxxxxxxx" % random.randint(1, 10000)
ToDo: Investigate how we can test multiple browsers.
CherryPy's WebTest is good for in-process testing & can be run from a browser-less server(easier for CI-integration).
These tests are stored in /tests/webtest
.
NB These are a work-in-progress...need to enable access to Web2Py environment (db, etc) using:
from gluon.shell import exec_environment env=exec_environment('the_model_file.py')
Or could write as a 'Controller' & call from CLI:
python web2py.py -S appname -M -R yourscript.py
Doc Tests
Agile documentation which can be run using Web2Py's Admin UI.
- http://www.python.org/doc/2.6/library/doctest.html
- e.g. http://127.0.0.1:8000/admin/default/test/sahana/default.py
To extend these to web applications, we have a module which uses wsgi_intercept & CherryPy's WebTest: modules/s3_test.py
This can be used from Controllers like:
def login(): """ Login >>> from applications.sahana.modules.s3_test import WSGI_Test >>> test=WSGI_Test(db) >>> '200 OK' in test.getPage('/sahana/%s/login' % module) True >>> test.assertHeader("Content-Type", "text/html") >>> test.assertInBody('Login') """
This works fine,although if an assert fails then the UI gets stuck :/
The 'db' access part isn't yet working.
Note that Web2Py uses a big doctest at the end of each file: def test_all()
Continuous Integration
We are starting to use the Trac-integrated Bitten to monitor code quality.
ToDo: Write a step to parse/store the results of Selenium's HandleResults.py
<step id="lint" description="Run PyLint"> <python:exec module="pylint.lint" output="pylint-report.txt" args="/var/www/trac/sahana3/bzr"/> <python:pylint file="pylint-report.txt" /> </step>
ToDo: Amend so that it can find the gluon
module (or configure to not follow this dependency):
ImportError: Unable to find module for modules/validators.py in /tmp/bittenA787wC/build_Trunk_1
ToDo: Fix Windows path (NB plain 'bzr' fails too, even if we're in the folder up from that):
F: 1: No module named /var/www/trac/sahana3/bzr
We need to write a build/test-results.xml
for this one (& it also wants a setup.py
?):
<step id="test" description="Run unit tests"> <python:distutils command="unittest"/> <python:unittest file="build/test-results.xml"/> <python:trace summary="build/test-coverage.txt" coverdir="build/coverage" include="trac*" exclude="*.tests.*"/> </step>
This one is pointless for us as we don't build anything using a setup.py
:
<step id="build" description="Compile to byte code"> <python:distutils command="build"/> </step>
Attachments (4)
- Regression tests.pdf (482.2 KB ) - added by 14 years ago.
-
Regression tests.odt
(349.2 KB
) - added by 14 years ago.
Source for the PDF
-
test.cmd
(1.9 KB
) - added by 14 years ago.
Non-Interactive
-
testui.cmd
(1.9 KB
) - added by 14 years ago.
Interactive Wrapper for Selenium
Download all attachments as: .zip