BluePrint: Destructive Testing of Sahana Eden Modules

Introduction

Destructive testing means to test the response of the system to user errors for consistency. When the user performs invalid actions, the system should:

• respond with a proper error message or, where suitable, correct the user's mistake (if the user's intention is clear)
• prevent any unintended data loss or exposing of data for unauthorized access
• give information how the user can do it right (if the user's intention is clear)
• allow the user to resume the normal workflow, or if necessary, to return to the previous stage

The primary goal of the test is to find situations where the user can accidentally loose data or expose them for unauthorized access, or put the system into a state where it is impossible to resume the normal workflow. It is *not* the primary goal of the test to find bugs (=wrong response to right user action) but to confirm a proper error handling (=right response to wrong user action)

The test method is to intentionally perform everything that the user could do wrong, misunderstand or forget when performing a regular user task (e.g. input of invalid values or no values where values are required, clicking on UI elements that are not meant to be clicked on, or unintended navigation actions like clicking the "back" button, bookmarking invalid URLs, and so forth) and document the systems response to the error.

It is important that the test targets "regular user tasks", i.e. the intention of the user must be part of the expected workflow, just actioned the wrong way, and also that the user's mistake has an actual (not potential) consequence on the data integrity or workflow. That means, if the user puts in values in wrong syntax, and the system is able to still save the right values in the record, then that is not an inconsistency but tolerant behavior. If though the user puts in values in wrong syntax and the system saves a wrong value, then that's a bug. And of course, no user action should ever lead to a HTTP 500 "Internal Server Error" (whereas other error messages may be a proper response to the user's mistake).

Ideally, any error messages should:

• name the action that lead to the error
• explain the cause of the error
• explain the consequences of the error for the user action
• explicitly state the impact on data integrity, if any (e.g. "Record has not been deleted")
• where suitable, suggest what the user could do to
  • resolve the cause of the error (e.g. "Please contact the system administrator to ...")
  • mitigate the impact of the error (if necessary)
  • continue with the task the user was going to perform

This BluePrint shall document the procedures for a systematic destructive testing of a particular Eden module, with focus on "systematic", so that the testing covers as many possible user mistakes as possible, and in a way that the test cases can be easily reproduced for any Eden modules.

From this BluePrint, we want to be able to implement a standard testing procedure for Eden releases.

Typical User Mistakes and their Priorities

This section shall document any knowledge about typical user actions and mistakes, and their relative likelihood (=priority).

General Testing Method

Examples of Test Cases

A good example of destructive testing is http://eden.sahanafoundation.org/ticket/1076 . This describes the systems response to an unintended user action (opening a popup link in a tab via right-click instead of left-click to open the popup) - and how it could go wrong (the user cannot resume the normal workflow).

References

---

BluePrints