Changes between Version 115 and Version 116 of DeveloperGuidelinesTesting

Timestamp:

08/08/11 21:40:33 (13 years ago)

Author:

Mike A

Comment:

Added more testing advice.

DeveloperGuidelinesTesting

@@ -121 +121 @@
 * Avoid inheritance trees with unittests as the unittest.TestCase isn't really designed for inheritance, more as a convenient place to put the tests.
 * Break test methods up into smaller methods. Non-test code can be extracted out into external procedures/functions which are easier to reuse. 
+* Tests should complete quickly, i.e. much less than a second. Remember they will be run thousands of times. A test that takes a minute to run might therefore cost days of developer time over its lifetime. To do this, encode precise failure cases in unit tests, don't do things like chuck random data at a unit.
+* In fact eliminate all randomness. A failure must be consistently reproducible to be fixable.
 * Read the nose documentation regarding writing tests: [http://readthedocs.org/docs/nose/en/latest/writing_tests.html]
 * Read this too: [http://ivory.idyll.org/articles/nose-intro.html]
+* '''Never use actual output as expected result output in a test.''' This defeats the purpose of the unit tests, as they now test nothing and just become a maintenance headache. Instead, use what you expect.
 
 === Running unit tests ===
-Assume our current directory is web2py/, which contains the applications folder. 
+Assume our current directory is web2py/, which contains the applications folder. We want to test our trunk application.
 
 To run all unit tests:
@@ -151 +154 @@
 }}}
 This is the most minimal version of the command. This is because nose.py is designed to be run from anywhere, i.e. not to be put in the PATH.
+
+'''Advice'''
+* Run specific tests while your are working on something.
+* Always run all tests and make sure they all pass before a merge to a branch like trunk which must be kept stable. 
+  * If they don't pass, fix or leave out the code that is causing failure.
+  * If the tests themselves are broken due to e.g. and API change, update them. Never paste actual output as expected output.
 
 === Javascript unit tests ===