Testing

"A bug is a test case you haven't written yet"
"Unit Tests allow merciless ​refactoring"

This page defines what our current approach versus our BluePrint for future options

Test-Driven Development is a programming styles which says that you 1st write your test cases (from the specs) & then proceed to make them pass.

For manual tests, can pre-populate the database with random data using ​gluon.contrib.populate

• InstallationGuidelinesDeveloperTesting
• TestCases - User Testing - List of things to test

Functional Tests

Building the Right Code

We have integrated ​http://seleniumhq.org/projects/remote-control/ into /static/selenium so that Functional Tests can be run easily.

Tests can be developed using ​Selenium IDE, if desired.

See the Documentation for details on how to use this test suite.

ToDo: create a HandleResults.py for storing results (to make visible to CI):

• e.g. Convert this one from PHP: ​http://wiki.openqa.org/display/SEL/Integrating+Selenium+And+CruiseControl.Net
• e.g. Extract from ​PloneTool's FunctionalTestTool.py

Hints:

• Improving the stability of Selenium tests: ​http://googletesting.blogspot.com/2009/06/my-selenium-tests-arent-stable.html
• Lots of useful tips: ​http://www.eviltester.com
• Autocomplete is fiddly to test as need to trigger specific events:

  # Enter the search String
  sel.type("gis_location_autocomplete", "SearchString")
  # Trigger the event to get the AJAX to send
  sel.fire_event("gis_location_autocomplete", "keydown")
  # Wait for the popup menu
  for i in range(60):
      try:
          if "SearchString" == sel.get_text("css=ul.ui-autocomplete li:first-child a"):
              break
      except:
          pass
      time.sleep(1)
  else:
      self.fail("time out")
  # Select the Result
  sel.fire_event("css=ul.ui-autocomplete li:first-child a", "mouseover")
  sel.click("css=ul.ui-autocomplete li:first-child a")
  time.sleep(4)
  

Systers' approach:

• ​http://systers.org/systers-dev/doku.php/automated_functional_testing
• List of Tests: ​http://systers.org/systers-dev/doku.php/master_checklist_template
• GSoC project: ​http://systers.org/systers-dev/doku.php/svaksha:patches_release_testing_automation

Alternative Options:

• ​http://zesty.ca/scrape/
  • ​Lightning Talk (2.30)

Unit Tests

Building the Code Right

​Selenium RC is great due to ability to handle JavaScript & also due to having an ​IDE for generating them (export as Python).
The IDE output needs to be ​modified to work with Web2Py.
NB Custom functions (e.g. for Random) cannot be shared with the Functional Tests (custom=JS) but the rest of the tests can be.
These tests are stored in /tests.
ToDo: Port the storeRandom function from JS to Python:

import random
print "test_%i@xxxxxxxx" % random.randint(1, 10000)

• ​AltereEgo entry on testing in Web2Py
  • ​StackOverflow discussion
• ​Web2Py Slice on Unit Testing

​CherryPy's ​WebTest is good for in-process testing & can be run from a browser-less server(easier for CI-integration).
These tests are stored in /tests/webtest.
NB These are a work-in-progress...need to enable access to Web2Py environment (db, etc) using:

from gluon.shell import exec_environment
env=exec_environment('the_model_file.py') 

Or could write as a 'Controller' & call from CLI:

python web2py.py -S appname -M -R yourscript.py 

Another similar option could be ​Pylon's WebTest

Doc Tests

Agile documentation which can be run using Web2Py's Admin UI.

• ​http://www.python.org/doc/2.6/library/doctest.html
• e.g. ​http://127.0.0.1:8000/admin/default/test/sahana/default.py

To extend these to web applications, we have a module which uses ​wsgi_intercept & ​CherryPy's ​WebTest: modules/s3/s3test.py
This can be used from Controllers like:

def login():
    """ Login
    >>> from applications.sahana.modules.s3.s3test import WSGI_Test
    >>> test=WSGI_Test(db)
    >>> '200 OK' in test.getPage('/sahana/%s/login' % module)
    True
    >>> test.assertHeader("Content-Type", "text/html")
    >>> test.assertInBody('Login')
    """

This works fine,although if an assert fails then the UI gets stuck :/
The 'db' access part isn't yet working.

Note that Web2Py uses a big doctest at the end of each file: def test_all()

Continuous Integration

We are starting to use ​Jenkins to run the Functional Tests automatically.

• Infrasctructure support docs
• ​http://blog.karit.geek.nz/2010/07/selenium-grid-part-1-getting-started.html

Other options: ​http://stackoverflow.com/questions/225598/pretty-continuous-integration-for-python

• ​Buildbot
• ​Bitten

---

BluePrintTesting

DeveloperGuidelines