Changes between Version 32 and Version 33 of BluePrint/Testing/TestSuite


Ignore:
Timestamp:
07/31/13 06:34:30 (8 years ago)
Author:
somayjain
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • BluePrint/Testing/TestSuite

    v32 v33  
    149149 * The content(requests) in these individual sessions can be imported from the helper xml files.
    150150 * See {{{modules/tests/load/search_staff.xml}}} for an example of one such session.
     151* Each use-case of the load test can be written down in separate tests which are to be located in {{{modules/tests/load}}}.
     152 * Eg - Usecase is 20% users creating staff, 40% searching staff, 40% generating reports on staff.
     153 * This usecase is captured by writing a test which has 3 sessions -
     154  * create_staff with probability = 20%
     155  * search_staff with probability = 40%
     156  * report_staff with probability = 40%
     157 * Each of the session is written using helper xml files located in {{{modules/tests/load/helpers}}}
     158 * Eg - Writing {{{&login_page;}}} inside the session will import the login requests.
     159* There needs to be significant amount of data in the database, to incorporate usecases like searching in a database of 10,000 records.
     160* A python script will be used to create such data and add it to the database. This will reside in {{{modules/tests/load/data}}}.
     161 * This will be run prior to the test so that the data is available at the time of testing.
     162* Also, some data is required for the tsung load tests.
     163 * Eg - For load tests which test creation of records, the data needs to be provided in CSV format.
     164 * This will also be done by a similar script as above.
     165 * It writes the data into the CSV file, whereas the script above writes the data into the database.
    151166
    152167===== Running the tests =====
     
    195210* See SysAdmin/ContinuousIntegration#PlansforFuture
    196211
     212Possible options -
     213* Jenkins
     214 * Advantages
     215  * Easy to build Eden
     216  * Easy integration with selenium grid, so that the selenium tests can be run remotely and parallely
     217  * Git plugin -
     218   * Can trigger tests on a commit easily.
     219   * Can automatically publish build status on the git commit.
     220   * Can create tag/push to master automatically on successful tests after the commit.
     221  * Run tests on new pull request (GitHub pull request builder plugin)
     222  * Email notifications about the test results.
     223  * IRC notifications about the test results. (IRC plugin)
     224  * Add more nodes on which jenkins is run, so that the load is distributed
     225 * Setting Up
     226  * Jenkins is easy to install.
     227  * Installation of Selenium Grid needed.
     228  * Setting up some slaves for Selenium grid for cross browser, parallel selenium tests (Can be done on a single machine too)
     229  * Slight modifications in the Selenium test suite to incorporate Selenium Grid.
     230  * Configuring Jenkins to run tests on a git commit, etc, as required.
     231  * Configuration is done by an authenticated web interface.
     232 * Reference - https://wiki.jenkins-ci.org/display/JENKINS/Home
     233* Selenium Grid
     234 * Advantages
     235  * Can run selenium tests across -
     236   * Different browsers
     237   * Different operating systems
     238   * Different machines in Parallel.
     239  * Works on the concept of hub and nodes.
     240  * The tests are run on a single machine - hub.
     241  * Execution will be done on different machines - nodes.
     242  * Will increase the speed of the Selenium tests since they will be distributed across machines.
     243   * Eg - Some nodes will run create, some will run the search tests.
     244  * We can then trigger the tests on a commit/pull request.
     245  * The time taken will roughly decrease depending on the number of nodes.
     246   * Eg - If there are 4 nodes, the tests will run roughly 4 times faster.
     247 * Setting Up
     248  * To install, need to download the Selenium Server jar file - on the hub as well as on all nodes.
     249  * Will have to change some initialization code for the Selenium tests where the Webdriver is initialised.
     250 * Reference - http://code.google.com/p/selenium/wiki/Grid2
     251* Solano
     252 * Comparison with Jenkins -
     253  * Jenkins has a lot more plugins, which are written by a much larger community. So, it becomes more extensible
     254   * Eg - git plugin, IRC plugin, github pull request plugin.
     255  * Jenkins is easier to integrate with Selenium Grid, which enables us to run the tests in parallel.
     256  * Jenkins has a better documentation, easier to use.
     257 * Disadvantages -
     258  * Our tests run using python’s unittest. We will have to change it to run using nosetests or py.test. Also, on changing too, the tests will not be able to run in parallel.
     259  * Reference - http://docs.tddium.com/python/
     260* Sahana Eden to drive the CI
     261
    197262==== Selenium Tests ====
    198263