Smoke Tests

Smoke Tests click through on every link within Sahana Eden and can be used to check for errors on pages and broken links. They are a light-weight approach to detecting basic errors, however they do not test form submission or any interaction. They can be run using ​twill or ​Ghost.py. Ghost is still under development and offers much more features than Twill.

Installation of the testing environment in your machine

You should already have installed Eden and Web2py before starting this.

• Twill
  • Requires ​twill.
  • Requires ​mechanize
• Ghost
  • Requires ​PyQt4 or ​PySide (dependency of Ghost.py)
  • Requires ​Ghost.py

Installation instructions are mentioned on the links

Running / Executing smoke tests:

To run the smoke tests use the following:

python web2py.py -S eden -M -R applications/eden/modules/tests/suite.py -A --suite smoke  --agent t --force-debug --link-depth 7 -V 3

Here,

• link-depth - The recursive depth when looking for links
• force-debug - Run the smoke tests even if debug is set to true. With debug on it can add up to a second per link and given that a full run of the smoke tests will include thousands of links the difference of having this setting on can be measured in hours.
• verbosity, V - The level of verbose reporting
• agent - Agent used to run the smoke tests: g -> Ghost t -> Twill (default)

Issues with Ghost

• Installing PyQt4/PySide might be troublesome.
• Ghost opens a file for every page visited. There is no method to delete the opened files when they are no longer required. If faced with this issue, increase the ulimit of your system as a temporary fix.
• Ghost crashes after a while. The reason is not yet clear. The issue is reported ​here.
• In Mac OS 10.9 [Mavericks], Ghost would throw a warning mentioned below. It is a Qt4 issue.

  Python[6181:1107] CoreText performance note: Client called CTFontCreateWithName() using name "Helvetica Neue" and got font with PostScript name "HelveticaNeue". For best performance, only use PostScript names when calling this API.
  

Future work

• Capitalize the javascript execution capability to improve the smoke tests.
• Add an command line argument for Suite.py which takes the path for a file containing the various optional parameters for Ghost such as setting display on, changing wait timeout etc.