| | 11 | == Stakeholders == |
| | 12 | * Developers - With an automatic testing framework setup, it will be relatively easier for the developers to test their changes in the code. |
| | 13 | |
| | 14 | * People who want to deploy Sahana - They would like to run the tests to ensure that the system is integrated well and ready for deployment. |
| | 15 | |
| | 16 | * Sahana as a service - Automated testing will provide Quality Assurance to it’s clients. |
| | 17 | |
| | 18 | * Bug Marshalls |
| | 19 | == User Stories == |
| | 20 | * Developers will run the test suite on making changes to the code to test if it works with the integration of the system. Developers may also see the test results mailed to the list to see the possible bugs introduced into the system. |
| | 21 | |
| | 22 | * People who want to deploy will run the test suite to check the functionality of the system. |
| | 23 | |
| | 24 | * The “bug marshalls” will review the test results mailed periodically by the CI Server and fix the bugs or log them. |
| | 25 | |
| | 26 | == Requirements == |
| | 27 | <Group requirements in subsections, e.g.,, etc.> |
| | 28 | <http://en.wikipedia.org/wiki/Requirements_analysis requirements> |
| | 29 | <Identify different types of requirements:> |
| | 30 | === Functional === |
| | 31 | |
| | 32 | * Maintain the CI Server to run the tests periodically and send the results. |
| | 33 | |
| | 34 | * Automatically Create Dummy Data while testing. |
| | 35 | |
| | 36 | * Extend Role Tests (Currently is limited to the IFRC roles for RMS) |
| | 37 | |
| | 38 | * Run Selenium and Smoke tests in multiple templates with multiple user accounts. Ideally, these tests can be run against each and every template where the target functionality is available. For templates where the functionality is not available, the test should auto-deactivate. |
| | 39 | |
| | 40 | * Clearer Error Messages in a way that anyone can reproduce them. |
| | 41 | |
| | 42 | * The CI Server should catch failures in any template. So, it should run tests across templates and include the template name in the aggregated report. |
| | 43 | |
| | 44 | * Simplify Selenium Tests and make them easier to read and more robust. |
| | 45 | |
| | 46 | * Adapt tests to meet needs of evolving CI Server(SysAdmin/ContinuousIntegration) |
| | 47 | |
| | 48 | * Load Tests |
| | 49 | === Non-functional === |
| | 50 | http://en.wikipedia.org/wiki/Non-functional_requirements |
| | 51 | === Interoperability === |
| | 52 | === Standards === |
| | 53 | === System Constraints === |
| | 54 | |
| | 55 | == Use-Cases == |
| | 56 | [[Image(Use case testing.png)]] |
| | 57 | == Design == |
| | 58 | |
| | 59 | === Workflows === |
| | 60 | * The CI Server runs the tests periodically and sends the results. On a regular basis, the bug marshalls review the test results and see if the reported negatives are false negatives or true negatives. If they are false negatives, they fix the tests. If they are true negatives, they report the bug on the trac or fix the bug themselves. |
| | 61 | |
| | 62 | * The developers make changes to the code and run the tests. If their changes do not break the tests, then this code is viable for merging. |
| | 63 | |
| | 64 | * The clients run the tests to make sure that the functionality this software is intended to provide is fulfilled. |
| | 65 | === Technologies === |
| | 66 | |
| | 67 | * For setting up the CI Server, some of the technologies which can be used are given here - [http://eden.sahanafoundation.org/wiki/BluePrintTesting#ContinuousIntegration ContinuousIntegration] |
| | 68 | |
| | 69 | * For Continuous Integration, an instance of Eden can also be used which will enable Scheduling, enable subscription of notifications for test results, can also provide formatted results. |
| | 70 | |
| | 71 | * For functional tests, currently, Selenium and Smoke Tests are used. The current Selenium tests should be made more robust and they should work across browsers. Currently, there is support for Firefox Webdriver(upto version 16) and Chrome Webdriver. We need to provide support for Safari Webdriver, Opera Webdriver and Internet Explorer Webdriver. |
| | 72 | |
| | 73 | * The Role tests(which currently run only on the IFRC template and uses Selenium Unit Test) are to extend to run on multiple templates. |
| | 74 | == Implementation == |
| | 75 | The seleium tests can be run on mainly IFRC template. WIth some changes, they can be run on the default template as well. However, they don’t work across templates. |
| | 76 | |
| | 77 | The unit tests expect that some particular modules are enabled in the template. If they are not enabled and unit tests are run in that template, then false negatives are reported. |
| | 78 | |
| | 79 | Current implementation of the selenium tests, smoke tests, role tests can be found here - |
| | 80 | |
| | 81 | https://github.com/flavour/eden/tree/master/modules/tests |
| | 82 | |
| | 83 | Unit tests and benchmark tests can be found here - |
| | 84 | |
| | 85 | https://github.com/flavour/eden/tree/master/modules/unit_tests |
| | 86 | == References == |
| | 87 | |
| | 88 | [http://eden.sahanafoundation.org/wiki/BluePrintTesting BluePrintTesting] |
| | 89 | |
| | 90 | [http://eden.sahanafoundation.org/wiki/QA QA] |
| | 91 | |
| | 92 | [http://eden.sahanafoundation.org/wiki/DeveloperGuidelinesTesting DeveloperGuidelinesTesting] |
| | 93 | |
| | 94 | [http://eden.sahanafoundation.org/wiki/SysAdmin/ContinuousIntegration SysAdmin/ContinuousIntegration] |
| | 95 | [[BR]] |
| | 96 | [[BR]] |
| | 97 | BluePrint |
