Contribute: QA

See Testing

Easy

Notes on Bug Reporting

For the purposes of bug reporting tasks for GCI, please make sure that:

• the bug has not already been reported elsewhere (a bug reported generally for Eden or for Red Cross or for Sandy is the same - it should not be reported three times). Please search through all tickets to see if the bug has already been reported. If it has, your submission will be marked duplicate and you will not be given credit for reporting it.
• understand that a bug is where the system is not performing as designed. We have received many suggestions for how it should be designed - and these are all valuable - but feature request are not bugs. If the system is performing as designed, but not how you'd like it to work, it is not a bug. A bug will generate a ticket or error message from the system. We are creating a new category for usability enhancements (4 will be required) to capture these, because they are valuable, but much easier to find than bugs.

A bug is where you get a screen that says something like:

OOPS! SOMETHING WENT WRONG ON OUR SIDE.
Try hitting refresh/reload button or trying the URL from the address bar again.
Please come back after sometime if that doesn't help.
Details - 500, INTERNAL SERVER ERROR
You may want to report a bug about this issue.
View ticket for this problem here - {link}

Report 2 Bugs in Sahana Eden

Explore either the ​The Sahana Eden Demo Site or a local instance of Sahana Eden as see if you can find any bugs. Report them following http://eden.sahanafoundation.org/wiki/BugReportingGuidelines

Report 2 Bugs in Sahana Eden for Hurricane Sandy Response

Test the ​http://sandyrelieftest.sahanafoundation.org/ (you will need to register and wait for approval). Review Deployments/Sandy/Occupy for a better understanding of what this site is used for. Report them following http://eden.sahanafoundation.org/wiki/BugReportingGuidelines and add the sandy keyword.

Report 2 Bugs in Sahana Eden for Red Cross

Test your local instance of Sahana Eden with the IFRC (International Federation of Red Cross and Red Crescent Societies) deployment setting & . In models/000_config.py set:

settings.base.template = "IFRC"
...
# Over-rides to the Template may be done here
settings.base.prepopulate = ["IFRC_Train"]

Do this before first running Sahana Eden - or delete the contents of your databases/, cache/, and sessions/ folders to force Sahana Eden to prepopulate again. There will be an administrator account generated automatically: ​admin@example.com password: testing. Report them following http://eden.sahanafoundation.org/wiki/BugReportingGuidelines and add the keyword IFRC.

Review Open Tickets in Sahana Eden's Bug Tracker

We want to find out if old bug reports and old feature requests are still relevant.

1. Select a ticket from the list of [report/1 Active Tickets] that hasn't been reviewed already during GCI 2012, and wasn't reported during GCI 2012.
2. Review that the the bug that it describes still occurs, or that the feature that it describes has not been implemented and is still relevant.
3. Update the ticket:
   1. If the description is unclear, add a better description.
   2. If there are several issues in the ticket, say which ones are still broken.
   3. Get screenshots if they're useful.
   4. Add anything else you think would be helpful.
   5. Add "Reviewed for GCI" so people know this one is done.

Write Manual Test Cases for a Sahana Eden Module

Write up the steps to perform the user tasks in the module, record the actions using the Selenium IDE, provide test data where useful. Document the User's workflows as test cases in the ​Sahana Eden Tests spreadsheet (Test Scripts sheet) or in another document. Include any test data you use. You should test as much of the functionality as possible. Add a link to the Test Case in http://eden.sahanafoundation.org/wiki/Contribute/QA#CreateAnAutomatedTestsforaManualTestCase so that this test case can be automated.

Run the Automated Tests

Following this documentation: http://eden.sahanafoundation.org/wiki/QA#AutomatedTests run the automated tests on your own systems:

• Improve the documentation to add in any missing steps to set up the Automated Tests, or make it clearer for other users.
• If the tests fail - manually repeat the test.
  • If the manual test passes, this is a false negative and you should attempt to fix the test
  • If the manual test fails (please be very careful to verify that this is an actual bug), report the bug following the BugReporting Guidelines

Intermediate

Create An Automated Tests for a Test Scripts from the Sahana Eden Tests Spreadsheet

Find 2 test cases in the ​Sahana Eden Tests spreadsheet (Test Scripts sheet) which have not been automated and create automated tests for them. Also check in the eden/modules/tests/<module>/ folder to see if there are any automated tests created. Follow the instructions on DeveloperGuidelines/Testing. Update the spreadsheet with the status, your initials or name and date.

Complete an Automated Tests for a Sahana Eden Module

The following Tests have been started, but have not yet been completed. These have been written, but the syntax used in these tests is not corrected and needs to be fixed. You will need to used selenium commands to select elements to run the tests instead of the create function in eden/modules/tests/web2unittests.py. Follow the instructions on DeveloperGuidelines/Testing to complete these tests, ensure that they run correctly and add them to the suite.

• Asset:
  • eden/modules/tests/asset/asset_report.py
  • eden/modules/tests/asset/asset_search.py (completed by aviraldg)
• Members:
  • eden/modules/tests/member/member_search.py (completed by liezlp)
• Project:
  • eden/modules/tests/project/project_report.py
  • eden/modules/tests/project/project_search.py (in progress - aviraldg)
• Staff:
  • eden/modules/tests/staff/staff_import.py
  • eden/modules/tests/staff/staff_report.py
  • eden/modules/tests/staff/staff_search.py (completed by liezlp)
• Volunteer:
  • eden/modules/tests/volunteer/vol_search.py (completed by aviraldg)
• Inventory:
  • eden/modules/tests/inv/warehouse_search.py (completed by liezlp)
  • eden/modules/tests/inv/wh_stock_report.py

Create An Automated Tests for a Manual Test Case

Follow the instructions on DeveloperGuidelines/Testing to complete these tests, ensure that they run correctly and add them to the suite.:

• http://eden.sahanafoundation.org/attachment/wiki/Contribute/QA/Search_doc.docx

Advanced

Create a Generic Test Function for Search Tests

Currently the Sahana test framework has a create function in eden/modules/tests/web2unittests.py which makes it simple to write tests for creating records. A similar function could be written for the Search Function: This would do the following:

• Have parameters for:
  • The search query -- which fields to search on and their values.
    • Look at the create function's data parameter. It has additional information about the form. Do you need anything like that? Could you use the same format for your parameter?
    • Would it make sense to share any code with create?
  • The expected results. It is sufficient to just test that the right number of records are returned and give a list of the first n (determined in test case) values in a specific sorted column.
    • You may want tests that return no results, or one result, or multiple results. What would be a good way to pass in the expected results?
• Fill the search criteria in the advanced search form.
• Execute the search.
• Check that the results are exactly as given.
• If feasible, extend this to also test simple search.
  • Can you use the same data to test the simple search form?
  • Does the caller need to tell you which field the simple search uses?
• Think about what code could be shared between your search function, the create function, and the report function task below.

Test this function in at least one test script.

(Implemented: ​https://github.com/flavour/eden/commit/cec2df2dfeb0ea6779941d9a77fd65c6786e3b96)

Add a helper function to use DAL to replicate the simple search query

Construct a DAL query which will return the records that mirror a simple search. This will need to be a generic solution so that the function needs to work for any given model.

Add a helper function to use DAL to replicate the advance search query

Construct a DAL query which will return the records that mirror an advanced search.

Automatically check that the number of records returned from a search is correct

Use the appropriate helper function, see above, to get the right search query. Use this query to automatically check that the number of records returned are correct.

Automatically check that the records returned from a search are correct

Use the appropriate helper function, see above, to get the right search query. Use this query to check that all the record IDs are correct. NOTE this could be just restricted to the records on the first page if more than one page of records are returned.

Create a Generic Test Function for Report Tests

Currently the Sahana test framework has a create function in eden/modules/tests/web2unittests.py which makes it simple to write tests for creating records. A similar function could be written for the Report Function. Ideally this would re-use features from / share code with the search helper described above -- it would be good to read that description. Because of this dependency, it might be better if this is done after search. Note for GCI: It's ok to claim this task if someone is working on search, and get a start on it. The report helper would do the following:

• Have parameters for:
  • The search query -- which fields to search on and their values. (See the description of parameters for the search helper above.)
  • The report options.
  • The expected results. This doesn't have to be all of the results - just the number of rows and columns (and perhaps optionally the values?) and specified cells in the report table.
• Fill the search criteria in advanced form for the report.
• Fill the report options.
• Generate the report.
• Check that the results are exactly as given.
• If feasible, extend this to also test the simple report.
  • Can you use the same data to test the simple report form?
  • Does the caller need to tell you which field the simple report uses?
• Think about what code could be shared between your report function, the create function, and the search function task above.

Test this function in at least one test script.

(implemented: ​https://github.com/flavour/eden/commit/dc7cc36f488108eb2ce2102b745cf56222e716ca)

---

See:

• Contribute
• QA