Changes between Version 4 and Version 5 of BluePrint/Testing/TestSuite

Timestamp:

04/19/13 05:44:27 (12 years ago)

Author:

somayjain

Comment:

--

BluePrint/Testing/TestSuite

@@ -44 +44 @@
 *    Adapt tests to meet needs of evolving CI Server(SysAdmin/ContinuousIntegration). The tests should successfully run on the CI Server both locally and remotely. As mentioned before, it should then send the results via email.
 
-*    Load Tests
+*    Load Tests : Will measure performance of Eden given the load under both normal and anticipated peak conditions.
 === Non-functional ===
-http://en.wikipedia.org/wiki/Non-functional_requirements
+
 === Interoperability ===
+
 === Standards ===
+
 === System Constraints ===
 
@@ -56 +58 @@
 
 === Workflows ===
+
+==== General ====
 *    The CI Server runs the tests periodically and sends the results. On a regular basis, the bug marshalls review the test results and see if the reported negatives are false negatives or true negatives. If they are false negatives, they fix the tests. If they are true negatives, they report the bug on the trac or fix the bug themselves.
 
@@ -61 +65 @@
 
 *    The clients run the tests to make sure that the functionality this software is intended to provide is fulfilled.
+
+==== Specefic ====
+
+* Given that Sahana is a web-based tool, Selenium tests should be made robust, easy to use. The main idea is to have a set of functions(create, search, report, edit) which will take the tablename, labels of the field and the record data as input. The Testsuite should automatically check the type of field(option, autocomplete, text, date, datetime, etc) being used in the current template and fill the form accordingly. With this, we can create a code-template which just needs to be copied, pasted and fed in the record when creating a new Selenium test. This will ensure that writing Selenium tests are as easy as giving a test case.
+
+* So, the workflow of the developer will include running the unit tests to see if the new changes break anything and providing a Selenium test for the newly written module, which will be run in Continuous Integration.
+
+* The role tests currently run only on the IFRC template. The test suite should automatically generate data for multiple templates so that the role tests can be run on them as well. 
+
+* To ensure that tests(role, selenium, smoke) work across templates and do not show false negatives when run on a different template, a check has to be made if the module it is to work on is enabled/disabled. If the module is disabled, then the test should automatically deactivate and an appropriate message should be included in the test report.
+
+* The load tests will test the amount of load Eden can take under normal and peak conditions. So, the load tests should be divided into phases, with each phase incrementally increasing the number of users and requests. Features of Tsung can be used to do this effectively.
+
+* The CI Server should run all the test across templates on a daily basis. For each test, it should include the template it is run on, test name and location. In case of failure of the test, it should include the traceback and the record details it was intended to work on also. It should send an aggregate report and a template specefic report for the bug marshalls and developers to investigate.
+
+
 === Technologies ===