wiki:BluePrintDecisionMakingFunctional

Context Navigation

Version 2 (modified by Connie White, 15 years ago) ( diff )
--

The Delphi Decision Maker - Version 1.0

Functional Specification

Overview

The Delphi Decision Maker module helps groups create a ranked list.

More specifically, it is designed to support the decision making of large groups of Crisis Management Experts. It guides experts to generate, debate and explore alternative solutions producing an real time ranked list of alternative solutions that reflects the group's opinion at any point in time quickly. This system accounts for uncertainty.

Summary

Increasingly, extreme events demand large groups of experts distributed in both location and time to communicate, plan, and coordinate actions. When responding to extreme events, very often there are both alternative actions that might be considered and far more requests for actions than can be executed immediately. The relative desirability of each option for action could be a collaborative expression of a significant number of emergency managers and experts trying to manage the most desirable alternatives at any given time, in real time. This same decision process could be used for a number of tasks but will be designed for distributed dynamic decision making during time critical phases of extreme events. This is because our proposed system is specially designed to save time, remove ambiguities, and decrease uncertainty, major challenges described in the literature on time critical decision making during the volatile time critical phases of emergency.

Scenarios

Scenario 1: Transport Canadian Project (real case)

The problem the group wanted to address was finding a method for: Identifying and Ranking Sustainable Transport Practices in Urban Regions. Making Decisions about Identifying, Adopting, or Implementing Sustainable Transport Practices. Below is the two page description of what the system can do and how the users can use it for their needs.

The Delphi Decision Maker is an online decision support system available to users anywhere there is WEB connectivity. It is designed to support large groups of professionals engaged in urgent, distributed, dynamic decision and option analysis activities.

This process is designed to handle real-world problems, and can be used where distributed subgroups and individuals are determining the options and analyzing them to solve a complex problem or emergency. And as a further important feature of the dynamic Delphi approach, by virtue of being dynamic it provides a real-time mechanism to support continuous planning operations, whereby many individuals add intelligence and new input to the updating of plans, or deal with new products, cost overruns, and other events.

The central idea behind a Delphi process is that the collective opinion of a group of professionals is more accurate and informed than their separate opinions. That is, the group approach produces “collective intelligence”, and is a means for a number of professionals to interact in such a way that: 1) They can offer a feasible and analyzed list of options from which a decision maker can select the mix that satisfies the current problem; and, 2) They can better understand why some options were less satisfactory than those chosen. A dynamic Delphi process can be used to help a group of professionals identify, evaluate and select an optimal-ranked list of options.

This particular method uses voting to identify areas of agreement and disagreement. Exposing areas of disagreement informs the group where they may need to focus their discussion input. On the other hand, letting a group know that they agree on an issue informs participants in a timely manner that consensus has been reached, and directs them to concentrate their effort on the next item on the agenda. While it is always important to respect the participants’ time, it is even more important to do so in this environment because of the urgent aspect that could underlie the problem under consideration.

In a Delphi with heterogeneous professionals, they are asked to vote only on what they feel confident about, or wait until more information on uncertainties is provided by other experts in a field. Participants are informed of how many participants have voted on a given item, as well as the degree to which more votes are expected in the future and how that could affect the results.

This Delphi process is dynamic because of the following reasons:

An expert can participate in any phase of the decision-making process at any time, that is,

1) problem identification, 2) information gathering, 3) solution generation, 4) evaluation.

This feature allows individuals to have discussions in forums where they can present information and debate issues as replies to specific options. Because the dialogue is text-based, others can read and benefit from the content.

An expert can participate online at any time during the day or night, given an Internet connection and web browser.

This feature helps professionals dedicate thoughts and ideas to the discourse as they arise after having time to think about a problem. This means they can choose a time to participate that is convenient for them. However, when some issues call for face-to-face meetings, the Dynamic Delphi approach can be used in preparation for, during, and/or after the group meeting as the meeting agenda or summary instrument.

There is real-time feedback of both the professionals’ individual opinions and of the group’s opinion.

Experts can vote, change their votes or withhold their votes for some reason.

The merits of the situation can change, or new information can sway opinions. Either way, the vote mimics the real-time opinion of the expert and, hence, the group.

Not all members of the group have to interact in order for a decision process to continue.

There may be cases where some of the participants cannot be present, or they may feel they do not have the expertise to engage in a specific option.

Uncertainty as to the status of the current vote (How final are the votes?) is calculated, and produced as feedback to the participants.

The system requests individual comparisons of options for preferences, and converts this rank-order information to an interval scale where distance represents the degree of preference between options.

This method has been modified to handle incomplete data with respect to participation in voting.

As an example for this Transport Canada project, a mission to examine options for implementing sustainable transport practices could entail the need for professionals in 20 to 30 different professional fields, including expertise regarding insurance and liability matters which restrict or prohibit using privatelyowned vehicles for collective uses such as car-pooling, ride-sharing, etc.

When it comes to taking options and discussing them on arelative basis in order to havethe information to choose which options should go forward to implementation, the dynamic Delphi will be the most useful way to involve a large, heterogeneous group necessary to uncover all the potential bottlenecks and concerns that must be addressed and reconciled.

Non Goals

This version 1.0 will not support the following features:

users to create and pose problems.

users outside the voting group to pose solutions-
- i.e. various permissions to handle all the roles of the users.

This version will not be integrated with Sahana, this will be accomplished in version 2.0.

Definitions

Version 1.0: will be used to refer to the first build of the system. This is testing how all of the functionality play together and tests to see if, together this improves group decision making ability in an online asynchronous environment. This is what's due in October.

Subject: will be used synonymously with Users and Experts. This system is created for Experts who are Users of the system. Users will be Subjects in the study that will be conducted on Version 1.0.

List: a list is a problem that is created by someone with this permission. This is not required for Phase I due Oct. Only one discussion will be going on and this is presented to a User group who will all have the same permissions.

Item: an item is a solution offered to a problem (List). Users need to be able to add an Item to the List in Phase I.

User Stories

Pilot Problem on Version 1.0 -- Faculty, staff and students are going to conduct a threat assessment for their campus. These users can interact anytime and from anywhere there is an Internet connection and a browser. The user can:

1.Propose a new option to consider that is added to a List.

2.Comment on any option proposed in a discussion forum - plain text, Linear-Blog, comments should be editable (for 30 minutes after creation or something like that)

3.Vote on a relative comparison of any two options, all options should be available from which to choose on one screen. will provide screenshot idea of interface solution. Once subgroup is selected, then each item needs to be compared with the other items as paired comparisons. (can force Usersto complete all available pairs so that the calculation is easier)

4.View the votes to find both disagreements and agreements using the scale

5.Change their votes. no need to maintain individual old vote. but a 'quality control check' needs to be run to test for any cyclic triad where A is selected over B, B is selected over C, and C is selected over A. error message to User asking them to correct.

6.View a linear interval scale for the options based upon current votes as calculated from the Unit Normal Table.

7.View a modified interval scale based upon the missing votes for any options showing the degree of uncertainty in the relative position of an item in the scale resulting from those that have already voted.One scale showing worst case scenario (lower boundary) and another scale showing best case (upper boundary).

So, three scales are required for Phase I:

1) present calculation based on those who have voted
2) scale of best case scenario (calculation in technical requirements)
3) scale of vote giving worst case scenario of non-voting members

Calculation for vote/revote (their first initial vote is kept, present vote, then a new vote (which is compared to the present vote and updated))

Each item is voted upon with paired comparisons which produces a frequency count which is turned into a percentage number which is matched up with the corresponding number on a Unit Normal Table which finally produces a rank.
A calculation for uncertainty is done: everyone who can vote - those who did

So there are three scales:

one from the calculation from Thurstone (of which code is available for the unit normal conversion table)which takes many individual votes and calculates one group opinion.
one with this, but including uncertainty (those who didn't vote) showing the best case outcome (if all votes go in favor of that item).
and then the third shows the opposing side (uncertainty) where the items will stand on a vote given they receive no more votes.

Delphi Decision Maker Flowchart

Screen by Screen Specification

Design

The following diagram shows how it's important for functionalities to be available with as few 'clicks' as possible. this saves time and gets experts to the information they need as quickly as possible. This shows the 5 primary functionalities required of the system

The Primary Interface Design mimics that of Web2Py - this design can be better seen on the original Research Proposal. link provided under 'background documentation below. Figure 10.6 Screenshot of Integrated Functionality.
-/(+) means that each list below can be expanded or contracted.
This layout corresponds with the design above.

THE DECISION MAKING MODULE USER INTERFACE Primary Page

Item Selection for Users Interface Idea

This next screen shows how paired comparisons can be presented to the user. This can be checkboxes or just something that indicates that one of the pairs has been selected over the other.

The Decision Making System

Scales Visualized for Feedback

This is the way I'd like to see the scales for feedback (3) - This is how Thurstone presented them. However, the lines drawn out to the right which bring out other dimensions is NOT required. This is just to demonstrate that other dimensions can be brought out from the results which could provide further insight. This is for a human to do though.