Version 46 (modified by 16 years ago) ( diff ) | ,
---|
Testing
"A bug is a test case you haven't written yet"
"Unit Tests allow merciless refactoring"
Test-Driven Development is a programming styles which says that you 1st write your test cases (from the specs) & then proceed to make them pass.
Behaviour-Driven Development takes this further to focus on the Specification rather than the Verification using something like pyspec or PyFITto provide testable specs:
There are a huge number of Testing Tools available to cover the various parts of the Testing process:
A community available for assistance:
Testing that Developers should be doing:
Unit Tests (must do)
Building the Code Right
- DocTest - inline with code: Agile Documentation
- http://agiletesting.blogspot.com/2005/01/python-unit-testing-part-2-doctest.html
- Web2Py supports running doctests on Controllers from the admin UI, e.g.: http://127.0.0.1:8000/admin/default/test/sahana/default.py
- UnitTest (formerly PyUnit)
- http://agiletesting.blogspot.com/2005/01/python-unit-testing-part-1-unittest.html
- http://diveintopython.org/unit_testing/index.html
- Nose - a discovery-based unittest extension
- Webunit - adds supports for HTTP GET/POST testing to unittest
- http://www.cherrypy.org/browser/trunk/cherrypy/test/webtest.py - CherryPy's extensions
Continuous Integration
Whenever a commit is made it should be checked to see that it doesn't break anything
- Patch Queue Manager - integrates with Bzr
- Bitten - integrates with Trac
Alternate options which could be investigated:
Regression Testing
Fired by dev after certain number of changes or whenever they like.
Documentation
As well as writing DocStrings in all functions, we can generate an overall API using:
If writing a separate manual then we can use:
Testing that Testers should be doing as part of Acceptance:
Boundary Testing (should do)
Checks functionality of modules against specs: Building the Right Code
This sees the application as a black box & so the same tests could be run here against both the Python & PHP versions, for instance.
Sahana is a Web-based application, so testing should be from browser perspective:
Functional tests can be written using:
- Selenium
- Selenium & Twisted: http://agiletesting.blogspot.com/2005/03/web-app-testing-with-python-part-2.html
- discussion on using Selenium with Web2Py: http://groups.google.com/group/web2py/msg/d8c9fd6008029f6b
- make_selenium.py
- Mechanize - library for programming website browsing
- Twill is built on Mechanize
- zope.testbrowser is built on Mechanize (& not Zope-specific
- MaxQ: http://agiletesting.blogspot.com/2005/02/web-app-testing-with-python-part-1.html
- JMeter
- Badboy
- Paste
Integration Testing (good thing)
We depend on various 3rd-party components so we need to ensure that as these components are upgraded this doesn't break any of out functionality:
- Web2Py
- CherryPy
- SimpleJSON
- T2
- OpenLayers
- jQuery
- Ext
Usability Tests
- UI Guidelines - comments on UI issues in Sahana2
Accessibility
- Are we XHTML 1.0 compliant?
- Are we usable without JavaScript?
Performance Tests
Whilst the Web2Py framework is fast, we should check that we're not doing anything stupid to slow it down:
Load Tests
How many simultaneous users can the system support?
Stress Tests
If extreme load is applied to the application, does it recover gracefully?
- Tools above but using more extreme parameters parameters
Security Tests
Whilst the Web2Py framework is secure by design, we should validate this:
Attachments (4)
- Regression tests.pdf (482.2 KB ) - added by 14 years ago.
-
Regression tests.odt
(349.2 KB
) - added by 14 years ago.
Source for the PDF
-
test.cmd
(1.9 KB
) - added by 14 years ago.
Non-Interactive
-
testui.cmd
(1.9 KB
) - added by 14 years ago.
Interactive Wrapper for Selenium
Download all attachments as: .zip