Changes between Version 11 and Version 12 of BluePrint/Testing/TestSuite


Ignore:
Timestamp:
04/26/13 13:56:32 (12 years ago)
Author:
somayjain
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • BluePrint/Testing/TestSuite

    v11 v12  
    1414* People who want to deploy Sahana - They would like to run the tests to ensure that the system is integrated well and ready for deployment.
    1515
    16 * Sahana as a service - Automated testing will provide Quality Assurance to it’s clients.
    17 
    1816* Bug Marshalls
    1917== User Stories ==
    … …  
    2523*  On a regular basis, the "bug marshals" review the test results sent by the CI Server and see if the reported negatives are false negatives or true negatives. If they are false negatives, they fix the tests or log a ticket on the trac for a bug in the testing code. If they are true negatives, they log a ticket on the trac for a bug in the code on which testing is done or fix the bug themselves.
    2624
    27 * People who want to deploy will run the test suite to check the functionality of the system.
     25* Clients who want to deploy will run the test suite to check the functionality of the system.
    2826
    2927==== Use Case Diagram ====
    … …  
    3533
    3634==== CI Server ====
    37 *    Maintain the CI Server to run the tests periodically and send an aggregated report for all the templates via email.
    38 
    39 *    Adapt tests to meet needs of evolving CI Server(SysAdmin/ContinuousIntegration). The tests should successfully run on the CI Server both locally and remotely.
     35*    Maintain the CI Server to-
     36     1. Run the tests daily
     37     2. Generate an aggregated report for all the templates
     38     3. Send the report via email
    4039
    4140==== Selenium Tests ====
    … …  
    6362==== CI Server ====
    6463
    65 * The CI Server should fetch the latest code from [https://github.com/flavour/eden/ Github Repository] and run the tests on the latest code fetched.
     64* The CI Server -
     65     1. Fetches the latest code from [https://github.com/flavour/eden/ Github Repository]
     66     2. Runs the tests daily.
     67     3. Generates the report which includes -
     68        a. Template Name
     69        b. Test Name
     70        c. Location of test script in Eden
     71        d. Traceback, record details in case of failure of test
     72        e. Pass Status in case of passed test
     73     4. Saves the aggregate report of the tests on the server at [http://82.71.213.53/RESULTS/].
     74     5. Mails it to the concerned people.
    6675
    67 * The CI Server should run all the tests across templates on a daily basis both locally and remotely. It should '''save the aggregate report of the tests on the server and as well as mail it to the concerned people.''' People may also subscribe to receive the test results via email. The previous implementation of the CI Server saved the results at [http://82.71.213.53/RESULTS/]. The same location can be used for storing the test results.
    68 
    69 * For each test, the report should include '''the template it is run on, test name and location of the test script in eden'''. In case of failure of the test, '''it should include the traceback and the record details it was intended to work on'''. In case the test passes, it should include the test name and the pass status of the test.
     76* People may also subscribe to receive the test results via email.
    7077
    7178==== Selenium Tests ====
    7279
    73 * These tests should run against each and every template where the target functionality is available. For templates where the functionality is not available, the test should auto-deactivate.
     80* The tests are organized template-wise. There are tests which are intended to work on a specific template and those which are intended to work in general on all templates.
    7481
    75 * Given that Sahana is a web-based tool, Selenium tests should be made robust, easy to use. The main idea is to '''have a set of functions(create, search, edit) which will take the tablename, labels of the field and the record data as input'''.
     82* The general tests run against each and every template where the target functionality is available. For templates where the functionality is not available, the test auto-deactivate.
    7683
    77 * The Test suite should '''automatically check the type of field(option, autocomplete, text, date, datetime, etc) being used''' in the current template and fill the form accordingly. With this, we can create a code-template which just needs to be copied, pasted and fed in the record when creating a new Selenium test. This will ensure that writing Selenium tests are as easy as giving a test case.
     84* There test suite -
     85     1. Provides a set of functions(create, search, edit) which will take the tablename, labels of the field and the record data as input.
     86     2. Automatically checks the type of field(option, autocomplete, text, date, datetime, etc) being used in the current template and fill the form accordingly.
     87     3. Takes the command line arguments to specify which browser the tests should run on(Firefox and Chrome are already implemented)
    7888
    79 * A sample code-template can be of the form -
     89* With this, a code-template is created which just needs to be copied, pasted and fed in the record when creating a new Selenium test. This ensures that writing Selenium tests are as easy as giving a test case. A sample code-template can be of the form -
     90
    8091{{{
    8192# Create Method
    … …  
    105116}}}
    106117
    107 * So, the workflow of the developer will include running the unit tests to see if the new changes break anything and providing a Selenium test for the newly written module, which will be run in Continuous Integration.
    108118
    109 * Command line arguments need to be passed to specify which browser the tests should run (Firefox and Chrome are already implemented)
    110119
    111120==== Role Tests ====
    … …  
    142151
    143152https://github.com/flavour/eden/tree/master/modules/unit_tests
     153
     154== Future Work ==
     155
     156* Adapt tests to meet needs of evolving CI Server(SysAdmin/ContinuousIntegration). The tests should successfully run on the CI Server both locally and remotely.
     157
    144158== References ==
    145159