Changes between Version 20 and Version 21 of BluePrintHumanIntelligenceTasks


Ignore:
Timestamp:
04/06/11 02:09:12 (14 years ago)
Author:
Pat Tressel
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • BluePrintHumanIntelligenceTasks

    v20 v21  
    8080crowdsourcing activities that occurred in actual emergencies.
    8181
     82==== Registration and training of workers ====
     83
     84Basic registration is already available, and there is support for volunteer signup that we
     85can customize. Administrators can list skills they're interested in and volunteers can say
     86what skills they have. We'll need to add whatever fields we want for performance tracking.
     87
     88There are several parts to training:
     89 * Instruction in using the HIT system. This will be "our" part of the training.
     90 * Preliminary instruction in performing the specific task, and online help info for reference
     91   while performing tasks: This will be the responsibility of whoever defines the task.
     92   But we can assist by providing sample training procedures to go with our sample tasks,
     93   a way to direct workers to any external training, and a means of providing help info
     94   via the task form.
     95 * Worker practice on actual tasks: This would be built in to the task queue.
     96
     97(Larger-scale volunteer training is outside the scope of this project.)
     98
    8299==== Managing the task queue ====
    83100
     
    157174==== Evaluating worker performance ====
    158175
    159 ==== Registration and training of workers ====
    160 
    161 Basic registration is already available, and there is support for volunteer signup that we
    162 can customize. Administrators can list skills they're interested in and volunteers can say
    163 what skills they have. We'll need to add whatever fields we want for performance tracking.
    164 
    165 There are several parts to training:
    166  * Instruction in using the HIT system. This will be "our" part of the training.
    167  * Preliminary instruction in performing the specific task, and online help info for reference
    168    while performing tasks: This will be the responsibility of whoever defines the task.
    169    But we can assist by providing sample training procedures to go with our sample tasks,
    170    a way to direct workers to any external training, and a means of providing help info
    171    via the task form.
    172  * Worker practice on actual tasks: This would be built in to the task queue.
    173 
    174 (Larger-scale volunteer training is outside the scope of this project.)
    175 
    176176==== Providing feedback ====
     177
     178Caution: Yer humble author is embarking on a <fanatic expostulation>.
     179
     180One might expect the purpose of providing feedback to be performance improvement,
     181and certainly that's desired. But some common means of supplying feedback are a bad
     182model. A productive worker is not one who's in the dumps from being told their work
     183quality is lousy and they're a bad person for having done it poorly, and they're not
     184as good as that other person over there. Rather, a productive worker is an engaged
     185and enthusiastic worker who feels good about what they're doing.
     186(I'd like to say, a productive worker is a happy and secure worker -- because this
     187is true -- but we're expecting people to be doing this work during disasters.
     188"Happy" is an unlikely state when one has been watching a video feed of the tsunami
     189roaring into Minamisanriku.) Besides just performance improvement, we want to keep
     190our trained workers working, and coming back day after day, and they won't come back
     191as readily if they're made to feel unworthy, and they won't perform as well if
     192they're only there because of grim determination to help.
     193
     194What to do? Well, fortunately, this is a well studied problem, and -- regardless of
     195how well this isn't done in most employment settings -- this is pretty much a solved
     196problem. The context in which it's been studied, and the solutions, come from the
     197game industry.
     198
     199The underlying principle is to deliver rewards -- little jolts of self-satisfaction
     200that let people feel good about themselves and what they're doing. With that, people
     201will absorb the "constructive feedback" as training that lets them get better and so
     202get more rewards. They'll do this even if they know they're being jollied along, and
     203rather enjoy that too. Think of your friends, or your kids (or -- admit it -- yourself)
     204spending far too long playing, oh, say, WoW or Angry Birds or Sudoku or Rabids Go Home
     205or... This isn't a huge industry because people get some material goods or practical
     206benefit out of it. And people are playing avidly without even a Good Cause to motivate
     207them. So, if we take a page from the game industry's playbook on how to get people
     208revved up and wanting to improve, and on top of that have the icing of Doing Good,
     209we should have a hit, not just HITs.
     210
     211Ok, so, what do we actually do here? Since the effectiveness of a game's rewards goes
     212straight to the company's bottom line, they may not give out such details as actual
     213reward pacing and effectiveness of various types of rewards. Yes, yes, there are actual
     214references on this topic. But surely the best methods are proprietary. Fortunately,
     215they're also on display -- extracting them just might take a little research. Now
     216tell me -- where have you heard a better excuse for playing games?
     217
     218</fanatic expostulation>
    177219
    178220=== Some suggested subsets for GSoC ===