BluePrint for Human Intelligence Task processing (a.k.a. "Job Jar")

Caution to GSoC students considering this project
This is a difficult project, involving statistics, human performance evaluation, and coming up with a way to allow an administrator to define an arbitrary task. Recommended for advanced students with a background in HCI and data mining / machine learning.

Motivation

During a disaster, people on the scene may report via social media or text messages. Emergency managers might find useful information there, but it's buried in large quantities of incomplete pieces, possibly not in a language known to the EM personnel. Much of this work is not easily automated, e.g. determining whether a message is a request for aid (and specifically what and how much and where). In order to clean up this crowdsourced information, we need...another crowd, but one that's trained, or at least learns as they go, and whose work is cross-checked for accuracy. A similar need holds for data gathering done by volunteers.

These are called "human intelligence tasks" (HITs), and what we want to produce is a system for managing them. Examples of HIT platforms are Amazon's Mechanical Turk and Crowdflower. We want a system tailored to the needs of emergency managers, and to working with crowdsourced data during emergencies.

See also: ​http://www.playsourcing.org/ (only s/Ushahidi/Sahana Eden/ ;-) And in June, see article on Playsourcing in: ​http://www.crcpress.com/product/isbn/9781439853498

Overview

Manage tasks performed on the site, e.g. data entry, cleaning crowdsourced data.

• Assign tasks to users or let users select tasks.
• Track who does what, mark completed.
• Provide administrator UI for defining tasks.
• Cross-check / verify task, or compare result of multiple users on same task.
• Administer tests and training for new workers.

More detailed description coming soon.

---

BluePrints