| | 65 | |
| | 66 | Alternate options which could be investigated: |
| | 67 | * http://buildbot.net/trac |
| | 68 | * http://redsymbol.net/talks/auto-qa-python/ |
| | 69 | * http://confluence.public.thoughtworks.org/display/CC/CI+Feature+Matrix |
| | 70 | * An instance of Eden can also be used which will enable Scheduling, enable subscription of notifications for test results, can also provide formatted results. |
| | 71 | * Jenkins |
| | 72 | * Advantages |
| | 73 | * Easy to build Eden |
| | 74 | * Easy integration with selenium grid, so that the selenium tests can be run remotely and parallely |
| | 75 | * Git plugin - |
| | 76 | * Can trigger tests on a commit easily. |
| | 77 | * Can automatically publish build status on the git commit. |
| | 78 | * Can create tag/push to master automatically on successful tests after the commit. |
| | 79 | * Run tests on new pull request (GitHub pull request builder plugin) |
| | 80 | * Email notifications about the test results. |
| | 81 | * IRC notifications about the test results. (IRC plugin) |
| | 82 | * Add more nodes on which jenkins is run, so that the load is distributed |
| | 83 | * Setting Up |
| | 84 | * Jenkins is easy to install. |
| | 85 | * Installation of Selenium Grid needed. |
| | 86 | * Setting up some slaves for Selenium grid for cross browser, parallel selenium tests (Can be done on a single machine too) |
| | 87 | * Slight modifications in the Selenium test suite to incorporate Selenium Grid. |
| | 88 | * Configuring Jenkins to run tests on a git commit, etc, as required. |
| | 89 | * Configuration is done by an authenticated web interface. |
| | 90 | * Reference - https://wiki.jenkins-ci.org/display/JENKINS/Home |
| | 91 | * Selenium Grid |
| | 92 | * Advantages |
| | 93 | * Can run selenium tests across - |
| | 94 | * Different browsers |
| | 95 | * Different operating systems |
| | 96 | * Different machines in Parallel. |
| | 97 | * Works on the concept of hub and nodes. |
| | 98 | * The tests are run on a single machine - hub. |
| | 99 | * Execution will be done on different machines - nodes. |
| | 100 | * Will increase the speed of the Selenium tests since they will be distributed across machines. |
| | 101 | * Eg - Some nodes will run create, some will run the search tests. |
| | 102 | * We can then trigger the tests on a commit/pull request. |
| | 103 | * The time taken will roughly decrease depending on the number of nodes. |
| | 104 | * Eg - If there are 4 nodes, the tests will run roughly 4 times faster. |
| | 105 | * Setting Up |
| | 106 | * To install, need to download the Selenium Server jar file - on the hub as well as on all nodes. |
| | 107 | * Will have to change some initialization code for the Selenium tests where the Webdriver is initialised. |
| | 108 | * Reference - http://code.google.com/p/selenium/wiki/Grid2 |
| | 109 | * Solano |
| | 110 | * Comparison with Jenkins - |
| | 111 | * Jenkins has a lot more plugins, which are written by a much larger community. So, it becomes more extensible |
| | 112 | * Eg - git plugin, IRC plugin, github pull request plugin. |
| | 113 | * Jenkins is easier to integrate with Selenium Grid, which enables us to run the tests in parallel. |
| | 114 | * Jenkins has a better documentation, easier to use. |
| | 115 | * Disadvantages - |
| | 116 | * Our tests run using python’s unittest. We will have to change it to run using nosetests or py.test. Also, on changing too, the tests will not be able to run in parallel. |
| | 117 | * Reference - http://docs.tddium.com/python/ |
| | 118 | |
| | 155 | |
| | 156 | ==== Selenium Remote Tests ==== |
| | 157 | |
| | 158 | The design for Selenium Remote Tests - |
| | 159 | |
| | 160 | * Prerequisite 1: There are some more instances of Eden with varying configurations (template/python version/database type) running on some servers(preferably in local network with the CI Server, as this will increase the speed of handling requests by the CI Server). |
| | 161 | * Prerequisite 2: There is an 'testing' instance of Eden running on the CI Server which contains the test suite. |
| | 162 | * The 'testing' instance of Eden will shoot the test suite, while changing the base URL in the test suite. The base URL currently is 127.0.0.1. If we change it to the IP address of some of the currently running instances of Eden on other servers, the tests will start running for those servers. |
| | 163 | * The above task can be done as a background process on the CI Server or running the tests using [http://selenium-grid.seleniumhq.org/ SeleniumGrid] to run them in parallel. |
| | 164 | * The selenium tests should work across browsers. Currently, there is support for Firefox Webdriver(upto version 16) and Chrome Webdriver. We need to provide support for Safari Webdriver, Opera Webdriver and Internet Explorer Webdriver. |
| | 165 | |
| | 166 | === Alternatives === |
