Contribute: QA

See Testing

Easy

Notes on Bug Reporting Helping find bugs in Sahana Eden is really valuable work as it makes sure that the software works reliably for the people who need it. However sometimes bugs are reported which aren't really bugs or have already been reported. So please make sure (especially if you are reporting bugs as part of the Google Code In (GCI) Program!):

• the bug has not already been reported elsewhere (a bug reported generally for different templates or sites such as Eden or Red Cross are the same - it should not be reported three times). Please search through all tickets to see if the bug has already been reported. If it has, your submission will be marked duplicate and you will not be given credit for reporting it.
• bugs get picked up by the automated testing suite and, because of the quick turnaround, are often fixed without a ticket being raised. So you will also need to check the current ​Sahana Eden CI Server Test Results AND the current version of the code to ensure that the bug is still valid.
• understand that a bug is where the system is not performing as designed. We have received many suggestions for how it should be designed - and these are all valuable - but feature request are not bugs. If the system is performing as designed, but not how you'd like it to work, it is not a bug. A bug will generate a ticket or error message from the system. If you want to suggest an enhancement please see Projects/Design.

A bug is where you get a screen that says something like:

OOPS! SOMETHING WENT WRONG ON OUR SIDE.
Try hitting refresh/reload button or trying the URL from the address bar again.
Please come back after sometime if that doesn't help.
Details - 500, INTERNAL SERVER ERROR
You may want to report a bug about this issue.
View ticket for this problem here - {link}

Report 2 Bugs in Sahana Eden

1. Test your own http://eden.sahanafoundation.org/wiki/InstallationGuidelines local instance or a demo site of Sahana. You can also review the test run on the Continuous Integration Server by either reviewing the ​Test Results or subscribing the the ​Ci Server Mailing List. Make sure that you manually repeat any of the errors in the automated tests and include full instructions for repeating the error in your bug report.
   • if you using your own local instance - consider testing the following Templates:
     • default
     • IFRC (in /models/000_config.py change prepopulate to settings.base.prepopulate = ["IFRC_Train"]
     • DRMP
     • DRPP
     • CRMT
     • Philippines
   • The following demo sites are available
     • ​http://demo.eden.sahanafoundation.org/eden/ (Template = default)
     • ​http://demo.lacrmt.sahanafoundation.org/ (Template = CRMT)
     • ​http://demo.drm.tl/ (Template = DRMP)
     • ​http://demo.drrprojects.net/ (Template = DRRPP)
2. Report them following http://eden.sahanafoundation.org/wiki/BugReportingGuidelines. Add the name of the template you are testing as a keyword

Review Open Tickets in Sahana Eden's Bug Tracker

We want to find out if old bug reports and old feature requests are still relevant.

1. Select *three* tickets from the list of [report/1 Active Tickets] that haven't been reviewed already during GCI 2013, and wasn't reported during GCI 2013. For each ticket:
   1. Review that the the bug that it describes still occurs, or that the feature that it describes has not been implemented and is still relevant.
   2. Update the ticket:
      1. If the description is unclear, add a better description.
      2. If there are several issues in the ticket, say which ones are still broken.
      3. Get screenshots if they're useful.
      4. Add anything else you think would be helpful.
      5. Add "Reviewed for GCI 2013" so people know this one is done.

Write Manual Test Cases for a Sahana Eden Module

Write up the steps to perform the user tasks in the module, record the actions using the Selenium IDE, provide test data where useful. Document the User's workflows as test cases in the ​Sahana Eden Tests spreadsheet (Test Scripts sheet) or in another document. Include any test data you use. You should test as much of the functionality as possible. Add a link to the Test Case in http://eden.sahanafoundation.org/wiki/Contribute/QA#CreateAnAutomatedTestsforaManualTestCase so that this test case can be automated.

Run the Automated Tests

Following this documentation: http://eden.sahanafoundation.org/wiki/QA#AutomatedTests run the automated tests on your own systems:

• Improve the documentation to add in any missing steps to set up the Automated Tests, or make it clearer for other users.
• Register to the Sahana Eden CI Server Test Results which runs these tests daily.
• If the tests fail - manually repeat the test.
  • If the manual test passes, this is a false negative and you should attempt to fix the test
  • If the manual test fails (please be very careful to verify that this is an actual bug), report the bug following the BugReporting Guidelines

Intermediate

Create An Automated Tests for a Test Scripts from the Sahana Eden Tests Spreadsheet

Find 2 test cases in the ​Sahana Eden Tests spreadsheet (On either the Selenium Tests for Templates or Test Scripts sheet) which have not been automated and create automated tests for them. Also check in the eden/modules/tests/<module>/ folder to see if there are any automated tests created. Follow the instructions on DeveloperGuidelines/Testing. Update the spreadsheet with the status, your initials or name and date.

Complete an Automated Tests for a Sahana Eden Module

The following Tests have been started, but have not yet been completed. These have been written, but the syntax used in these tests is not corrected and needs to be fixed. You will need selenium commands to select elements to run the tests instead of the create function in eden/modules/tests/web2unittests.py. Follow the instructions on DeveloperGuidelines/Testing to complete these tests, ensure that they run correctly and add them to the suite.

• Staff:
  • eden/modules/tests/staff/staff_import.py
  • eden/modules/tests/staff/staff_report.py
• Inventory:
  • eden/modules/tests/inv/wh_stock_report.py

Create An Automated Tests for a Manual Test Case

Follow the instructions on DeveloperGuidelines/Testing to complete these tests, ensure that they run correctly and add them to the suite.:

• http://eden.sahanafoundation.org/attachment/wiki/Contribute/QA/Search_doc.docx

Advanced

Create a Test Helper Function for Filter Forms

eg. ​http://demo.drm.tl/eden/default/index/newsfeed See: S3/FilterForms

Create a Test Helper Function for Summary Pages

eg. ​http://demo.lacrmt.sahanafoundation.org/eden/org/facility/summary See:

Create a Generic Test Function for Search Tests

Currently the Sahana test framework has a create function in eden/modules/tests/web2unittests.py which makes it simple to write tests for creating records. A similar function could be written for the Search Function: This would do the following:

• Have parameters for:
  • The search query -- which fields to search on and their values.
    • Look at the create function's data parameter. It has additional information about the form. Do you need anything like that? Could you use the same format for your parameter?
    • Would it make sense to share any code with create?
  • The expected results. It is sufficient to just test that the right number of records are returned and give a list of the first n (determined in test case) values in a specific sorted column.
    • You may want tests that return no results, or one result, or multiple results. What would be a good way to pass in the expected results?
• Fill the search criteria in the advanced search form.
• Execute the search.
• Check that the results are exactly as given.
• If feasible, extend this to also test simple search.
  • Can you use the same data to test the simple search form?
  • Does the caller need to tell you which field the simple search uses?
• Think about what code could be shared between your search function, the create function, and the report function task below.

Test this function in at least one test script.

(Implemented: ​https://github.com/flavour/eden/commit/cec2df2dfeb0ea6779941d9a77fd65c6786e3b96)

Add a helper function to use DAL to replicate the simple search query

Construct a DAL query which will return the records that mirror a simple search. This will need to be a generic solution so that the function needs to work for any given model.

Add a helper function to use DAL to replicate the advance search query

Construct a DAL query which will return the records that mirror an advanced search.

Automatically check that the number of records returned from a search is correct

Use the appropriate helper function, see above, to get the right search query. Use this query to automatically check that the number of records returned are correct.

Automatically check that the records returned from a search are correct

Use the appropriate helper function, see above, to get the right search query. Use this query to check that all the record IDs are correct. NOTE this could be just restricted to the records on the first page if more than one page of records are returned.

Create a Generic Test Function for Report Tests

Currently the Sahana test framework has a create function in eden/modules/tests/web2unittests.py which makes it simple to write tests for creating records. A similar function could be written for the Report Function. Ideally this would re-use features from / share code with the search helper described above -- it would be good to read that description. Because of this dependency, it might be better if this is done after search. Note for GCI: It's ok to claim this task if someone is working on search, and get a start on it. The report helper would do the following:

• Have parameters for:
  • The search query -- which fields to search on and their values. (See the description of parameters for the search helper above.)
  • The report options.
  • The expected results. This doesn't have to be all of the results - just the number of rows and columns (and perhaps optionally the values?) and specified cells in the report table.
• Fill the search criteria in advanced form for the report.
• Fill the report options.
• Generate the report.
• Check that the results are exactly as given.
• If feasible, extend this to also test the simple report.
  • Can you use the same data to test the simple report form?
  • Does the caller need to tell you which field the simple report uses?
• Think about what code could be shared between your report function, the create function, and the search function task above.

Test this function in at least one test script.

(implemented: ​https://github.com/flavour/eden/commit/dc7cc36f488108eb2ce2102b745cf56222e716ca)

---

See:

• Contribute
• QA