Changes between Version 6 and Version 7 of BluePrint/Testing/TestSuite


Ignore:
Timestamp:
04/22/13 14:53:08 (12 years ago)
Author:
somayjain
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • BluePrint/Testing/TestSuite

    v6 v7  
    1818* Bug Marshalls
    1919== User Stories ==
    20 * Developers will run the test suite on making changes to the code to test if it works with the integration of the system. Developers may also see the test results mailed to the list to see the possible bugs introduced into the system.
     20
     21* Developers will run the test suite on making changes to the code to test if their code works with the integration of the system. If their changes do not break the tests, then their code is viable for merging.
     22
     23* Developers may also see the test results mailed to the list to see the possible bugs introduced into the system.
     24
     25*  On a regular basis, the "bug marshals" review the test results sent by the CI Server and see if the reported negatives are false negatives or true negatives. If they are false negatives, they fix the tests or log a ticket on the trac for a bug in the testing code. If they are true negatives, they log a ticket on the trac for a bug in the code on which testing is done or fix the bug themselves.
    2126
    2227* People who want to deploy will run the test suite to check the functionality of the system.
    2328
    24 * The “bug marshalls” will review the test results mailed periodically by the CI Server and fix the bugs or log them.
     29=== Use Case Diagram ===
     30[[Image(Use case testing.png)]]
    2531
    2632== Requirements ==
     
    2834=== Functional ===
    2935
    30 *    Maintain the CI Server to run the tests periodically and send the results.
     36*    Maintain the CI Server to run the tests periodically and send an aggregated report for all the templates via email.
    3137
    32 *    Automatically Create Dummy Data while testing.
     38*    Run Selenium Tests in multiple templates with multiple user accounts.
     39
     40*    Create an abstraction for the Selenium Tests so that they are easier to read, write tests and tests are more robust.
    3341
    3442*    Extend Role Tests to run on different templates(Currently is limited to the IFRC roles for RMS)
    3543
    36 *    Run Selenium and Smoke tests in multiple templates with multiple user accounts. Ideally, these tests can be run against each and every template where the target functionality is available. For templates where the functionality is not available, the test should auto-deactivate.
     44*    Run Smoke tests in multiple templates.
    3745
    38 *    Clearer Error Messages in a way that anyone can reproduce them. The error message will include the traceback and the record information that the test was intended to create/modify/delete so that the user may easily investigate the test.
     46*    Load Tests : Will measure performance of Eden given the load under both normal and anticipated peak conditions.
    3947
    40 *    The CI Server should catch failures in any template. So, it should run tests across templates and include the template name in the aggregated report.
    41 
    42 *    Simplify Selenium Tests and make them easier to read and more robust. This will involve creating an abstraction for Selenium Tests, such that it takes few minutes to write a test for a new module.
     48*    Clearer Error Messages in a way that anyone can reproduce them.
    4349
    4450*    Adapt tests to meet needs of evolving CI Server(SysAdmin/ContinuousIntegration). The tests should successfully run on the CI Server both locally and remotely. As mentioned before, it should then send the results via email.
    4551
    46 *    Load Tests : Will measure performance of Eden given the load under both normal and anticipated peak conditions.
     52
    4753=== Non-functional ===
    4854
    4955=== Interoperability ===
     56
     57The tests should operate between multiple templates.
    5058
    5159=== Standards ===
     
    5361=== System Constraints ===
    5462
    55 == Use-Cases ==
    56 [[Image(Use case testing.png)]]
    5763== Design ==
    5864
    5965=== Workflows ===
    6066
    61 ==== General ====
    62 *    The CI Server runs the tests periodically and sends the results. On a regular basis, the bug marshalls review the test results and see if the reported negatives are false negatives or true negatives. If they are false negatives, they fix the tests. If they are true negatives, they report the bug on the trac or fix the bug themselves.
    63 
    64 *    The developers make changes to the code and run the tests. If their changes do not break the tests, then this code is viable for merging.
    65 
    66 *    The clients run the tests to make sure that the functionality this software is intended to provide is fulfilled.
    67 
    68 ==== Specefic ====
    6967
    7068* Given that Sahana is a web-based tool, Selenium tests should be made robust, easy to use. The main idea is to have a set of functions(create, search, report, edit) which will take the tablename, labels of the field and the record data as input. The Testsuite should automatically check the type of field(option, autocomplete, text, date, datetime, etc) being used in the current template and fill the form accordingly. With this, we can create a code-template which just needs to be copied, pasted and fed in the record when creating a new Selenium test. This will ensure that writing Selenium tests are as easy as giving a test case.