Changes between Version 1 and Version 2 of Deployments/SAMBRO/Evaluation


Ignore:
Timestamp:
06/14/16 00:09:25 (6 years ago)
Author:
Nuwan Waidyanatha
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Deployments/SAMBRO/Evaluation

    v1 v2  
    1 = SAMBRO Implementation Evaluation Tools =
     1= SAMBRO Evaluation Guidelines =
    22
     3We provide a methodology and a set of evaluation instruments to be used in evaluating the implementation.
     4
     5== Introduction ==
     6Sahana Alerting and Messaging Broker (SAMBRO) is designed foster Cross-Agency Situational-Awareness in support of improving Institutional Responsiveness to All-Hazards: http://eden.sahanafoundation.org/wiki/Deployments/SAMBRO . SAMBRO offers an all-hazard all-media approach whereby consistent, concise, and controlled alerts and warnings are disseminated through multiple channels.
     7
     8To evaluate the performance of SAMBRO and the underlying Common Alerting Protocol (CAP) content standard and procedures, we are introducing an objective and subjective evaluation method. The evaluation is specific to early warning and alerting. Alerting Authorities (Publishers) and warning recipients like the public and first-responders (Subscribers) can interconnect through the SAMBRO for improving their institutional responsiveness.
     9
     10== Methodology ==
     11
     12We have broken the SAMBRO Early Warning System (EMS) dissemination subsystem activities into two components; namely Publishers and Subscribers. Publishers are essentially authorized alerting organizations who would originate or relay a warning/alert message. Both Publishers and Subscribers will undergo a series of tests to determine their aptitude to:
     13publish effective CAP-enabled messages
     14Subscribe to actionable (or inactionable) CAP-enabled alert/warning messages
     15operationalize a cross-agency situational-awareness platform
     16
     17We are applying two Human Computer Interaction (HCI) evaluation methodologies:
     18Technology Acceptance Model (TAM)
     19Goal Oriented HCI
     20Technology Acceptance Model (TAM)
     21TAM provides insights on the ease of use and usability of the technology presented to the users. This applies to both Publishers and Subscribers. This is a subjective analysis of the intervention.
     22Goal Oriented HCI
     23There are two elements to the Goal Oriented HCI:
     24Objectively analyze the functional transition
     25Subjectively analyze the functional transition
     26
     27Screen Capture
     28We will install a screen capture software. The screen capture software will capture the behaviour of the user and the sequence of executing each process. This will provide the evaluators with insights of the ease of use and usability of the system. It may relate to the level or training and expectation of the user’s aptitude.
     29Focus Group Discussion
    330== References ==
    431* [https://www.mindtools.com/pages/article/kirkpatrick.htm Kirk Patrick's Four Level Training Evaluation]