wiki:HaitiRMSToDo

Haiti

Haiti RMS (Request Management System)

University of Wisconsin Python club are leading on this: #sahana-py

Keep it real simple to start with - get something usable quickly, which we can then refine later if we gets needs to do so.

Other parallel efforts for which interop would be nice:

Comments:'

  • We might think of how the RMS might be modified to record assessment data and manage requests for doctors, supplies, etc. (HMS is the big live system for us)
  • Closing the loop: RSS feeds regarding status of requests are available at /sahana/rms/sms_complete.rss (Ushahidi feeds) and /sahana/rms/tweet_complete.rss (TtT feeds)

STATUS

Production (100129@8:35UTC)

  • view with sms feeds from Ushahidi
  • view with twitter feeds from TtT
  • SMS alerts on map viewer (need repair)
  • pledge system with pledges as component and requests as resource
  • ServerSidePagination (currently disabled due to filtering problems)
  • Close the loop. Updated status on request (open/fulfilled) is now available as RSS feed.

Development (100129@08:35UTC)

  • added arbitrary key/value table to requests (Omer's code)
  • improved UI (timClick's code)

Experimental (100129@08:35UTC)

  • launchpad branch (~uwthw/sahana/rms)

TODO

Involved:

  • re-enable server side pagination
  • Administer pledges dashboard showing multiple pledge items by pledge organization (currently achieved with search)
  • start moving docs to UserGuidelinesRMS)
  • feed from Tweak the Tweet split feed into 2 tables: #offer & #help
  • report generating functionality assigns id (alpha, beta,...) to pledges every ~6 hours to facilitate tracking by relief organizations
  • allow options for filtering by category/priority/fulfilled/city/type
  • test/repair "search requests"
  • add ability to display "x most recent" requests, particularly useful for map viewer

Quick

  • icon showing status of pledge set with represent in pledge admin (full circle = delivered, 1/2 circle = in transit, open circle = pledged) (copy the priority represent code)
  • change representation of "make pledge" to a button

Strategy

We decided to combine the requests from different feeds into one table via the cron script. This has the benefit of transitioning from the previous prod pledge system without changing the current tables. Also it means that feeds are stored "as is" for posterity. The origins are tracked with source (e.g. sms, tweet) and source id (ushahidi ID and TtTid). The source ids are necessary to close the loop with these groups by sending back info on which requests have been filled. The con is that there is duplication among tables.

Check List of Requirements

  • List copied over from Reqs on 29Jan10

Ability to read a standard GeoRSS tag for aid/pledge requests

  • read/processed queue - track these SMS requests for manual processing and entry
  • click to turn message into a geolocated request (A) or pledge (B)

A) Make Request - we need to capture:

  • request contact details (name, mobile)
  • location - free text* (name of camp, name of org, name of distribution point)
  • priority
  • geolocation (optional taken from GeoRSS)
  • representing what org/group - free text*
  • serving # people (optional)
  • Requested items: list of
  • item (free text*), quantity (numeric), unit
  • see if there are matching pledges (optional) → goto step D)

B) Make Open Pledge:

  • same as above for a pledge (requested/pledged)

C) Fulfillment Request with new pledge

  • pledge contact details (name, mobile)
  • location - free text*
  • representing what org/group - free text*
  • item, quantity (numeric) fullfilled (next to requested items, requested quantity, fixed unit)
  • status - pledged, in transit, delivered

D) Match Open Pledge to Request

  • allocate parts of pledge to request (substitute from pledge and add to request)

E) Report: Unmet requests table, filtered by:

  • requested items
  • date (age)
  • priority
  • smart - priority then oldest first
  • proximity
  • geoRSS feed
  • ones that can be matched by requests highlited

F) Report: New pledge table

  • geoRSS feed
  • filtered by
  • pledged, in transit, delivered

Bugs that need triaging/fixing:

LOGS

100116

[11:29am] nicopresto: would screenshots be helpful?
[11:32am] flavour: screenshots 4 what?
[11:32am] flavour: For the User Manual?
[11:32am] flavour: http://trac.sahanapy.org/wiki/UserGuidelinesRMS
[11:32am] flavour: We have paolo volunteerting to help with user manuals once code has stablised  a bit
[11:33am] nicopresto: screenshots of the rms as we build it .. will check guidelines
[11:33am] flavour: You can ping him directly:
[11:51am] nicopresto: mprutsalis: we briefly outlined were we made it to on the wiki (http://www.sahana.lk/wiki/doku.php/doc:rms:english). We have request and post forms, then wrote a cron job for getting the GeoRSS. We'll update the cron to the new feed link you sent. We're currently checking over the forms to get them ready for dev
[11:52am] nicopresto: the launchpad branch is at (~uwthw/sahana/rms)
[11:54am] mprutsalis: need to keep it simple for haiti-- you've seen the quality of data...
[11:55am] mprutsalis: so we post it all - make it searchable/listable by category....
[11:55am] mprutsalis: if you are on skype - I can add you to the chat room there working on this project
[11:59am] mprutsalis: oh whoa - that haiti.ushahidi.com/feed feed is really a mess 
[12:00pm] nicopresto: yea
[12:04pm] flavour: feed is the main incident feed isn't it
[12:04pm] flavour: Not the 200 RMS line
[12:04pm] nicopresto: ok we'll separate the sms messages from our request table and offer another tab that shows simple/searchable SMS messages. Shouldn't take long and we could get to dev in ~2 hrs.
[12:08pm] nicopresto: ok so  we have 2 feeds http://75.101.195.137/rss.php?key=yqNm7FHSwfdRb8nC2653 (let's call it "4636") and haiti.ushahidi.com/feed (let's call it 'haiti"). We've been using "haiti" and will switch to "4636". We'll display that info in a separate tab. So we'll have 3 tabs at the RMS: "request aid", "pledge aid", and "view SMS requests" ... cool?
[12:09pm] nicopresto: similar to the php side bar (75.101.195.137/add_record.php)
[12:09pm] flavour: sounds good
[12:10pm] flavour: I see I need to work on the Map-based Lat/Lon entry urgently
[12:11pm] flavour: 4636 isn't XML?
[12:11pm] flavour: Just the way it displays in Chrome
[12:11pm] flavour: FF tries to open in Google Reader
[12:12pm] mprutsalis: 4636 replaced 200
[12:13pm] flavour: 200 was suppoosed to be vols accordint o chamindra but the data there seems an MPR
[12:15pm] mprutsalis: hunh... I think 200 is abandoned effort - that was going to be the shortcode in haiti for sms but they couldn't get control of it so migrated to 4636... is new data going into 200?
[12:19pm] flavour: Looks like 3 sample entries to me
[12:31pm] mprutsalis: update from ushahidi... they are making more features to the rss feed "in a bit"
[12:31pm] mprutsalis: [1:29:31 PM] Brian Herbert: but for now, you can do paging
[12:31pm] mprutsalis: [1:29:38 PM] Brian Herbert: just add &limit=0,50
[12:31pm] mprutsalis: [1:29:44 PM] Brian Herbert: that number works just like mysql
[12:31pm] mprutsalis: default is 50 items so you aren't going to surface all reports without using the limit
[12:32pm] mprutsalis: also - Katie Stanton from US State Department - latest project is connecting organizations who can donate things like laptops, generators, phones, etc. with those working on the ground.
[12:32pm] mprutsalis: I have offered up standard RMS functionality - you are working off those specs - for this....
[12:32pm] flavour: Great - we want uers 
[12:32pm] flavour: Users
[12:33pm] mprutsalis: so much going on everywhere... hard to keep up.
[12:34pm] flavour: I think good if nico is the PoC for RMS
[12:35pm] mprutsalis: ok - nico I will introduce you to the 4636 room....
[12:35pm] nicopresto: great thanks
[1:36pm] nicopresto: mprutsalis: typo above (11:51am) I listed a link to an older php rms doc.  Our status is on http://trac.sahanapy.org/wiki/HaitiRMSToDo
[2:04pm] mprutsalis: thanks nicopresto... the older doc is good overview for less technical like katie... that's why I used it.
[2:05pm] nicopresto: yes, that document was also very helpful for us
[2:15pm] }flavour{ joined the chat room.
[2:29pm] }flavour{: <mprutsalis> request management is priority focus...
[2:29pm] }flavour{: <mprutsalis> US State Department wants a registry of pledged in-kind donations from US organizations/companies - that can be matched with needs of organizations on the ground in haiti....
[2:31pm] nicopresto: we've got the georss going ... working on an interface
[2:32pm] nicopresto: we used the built in organization table, correct?
[2:32pm] mprutsalis: sure... if that works.
[2:32pm] mprutsalis: let me know when there is something to look at (in dev)
[2:34pm] nicopresto: ok we'll post our rough draft to branch in a few ... our database
[2:34pm] nicopresto: ... our database structure is flat and not very pretty yet ... working on it
[3:10pm] nicopresto: ok new georss cron just worked ... adding validation for duplicate guid fields then moving over to branch
[3:22pm] scopatz: Ok we are testing our cron job right now....this is the last step (for the moment!)
[3:27pm] nicopresto: starting to upload
[3:27pm] nicopresto: editing wiki
[3:31pm] scopatz: Pushed to lp
[3:32pm] scopatz: ...so you should be able to grab our changes any time.
[3:33pm] scopatz: Note that we added a cron job for rms (rms_entry2record.py) but commented it out
[3:33pm] scopatz: you should uncomment  for it to work
[3:34pm] flavour: ok, I'll get back to you for help if reqd

100116

<chamindra> Basically there will be loads of requests for aid coming from the field 
<chamindra> by sms, through calls and otherwise
<chamindra> and people will have to enter these (millions) of requests quickly into the system for tracking
<chamindra> there is no time for complex workflows and approvals, so the system has to be extremely quick
<chamindra> if we can enter everything for a request in one html form that would be ideal
<chamindra> similarly there will be loads of people offering donations of aid of various things
<chamindra> often they cannot be used immediately 
<chamindra> but they need to be tracked so that when a need arises the donation and the person who offered it can be found
<chamindra> This is the main objective of the request management system, to connect people who need aid and people who are offering it
<chamindra> however to do the matching you need to agree to some measures
<chamindra> I need 100 Kg of Rice

<chamindra> bit the person that pledges can only offer 40Kg
<chamindra> so you match it but note that there is 60Kg remaining to be delivered
<chamindra> but you need to track the remainder that was not fulfilled

<chamindra> so we now have an added part of getting unstructured data from various sources
<chamindra> basicially someone is recording some information about aid, without reviewing the content and send it to us
<chamindra> first step is someone needs to convert that to a request for aid or pledge
<chamindra> so on click of the queue that is coming in (as a GeoRSS feed from Ushidhi right now of their SMS messages) you convert it to the first step of a request or a pledge
<chamindra> once you click that item is no longer in the pending queue (it has been read and processed)
<chamindra> So let's think through what a typical situation would be
<chamindra> someone would make a call with a request or sms a request
<chamindra> fist step would be to enter what is needed
<chamindra> and it will be a list of items
chamindra> sometimes they will just say "food" or "water" without a quantity
<chamindra> but then you need to ask for how many people
<chamindra> which is why that field is there for
<chamindra> units are not compulsory
<scopatz> How specific does, say the food field need to be?  
<scopatz> For instance, is Rice good enough, or do we need to be very specific like Brown Rice
<chamindra> IMO not very
<chamindra> possibly you can have a highlevel drop down category
<nicopresto> AJAX dropdown
<scopatz> Right that is what I was thinking
<chamindra> no ones going to care with the rice is brown or white right now me thinks :-)
<scopatz> because the alternative would be a foods string that then gets parsed
<chamindra> but you are right in some instances
<chamindra> like if it is pork or for religious reasons
<chamindra> so possible as you said one high level category
<scopatz> OK so we'll come up with a representative list for a drop down menu
<chamindra> food, water, aid, medical, etc
<scopatz> Great, that makes life easier for the matching
<chamindra> and a free text box for the sub category
<chamindra> on the free text thing, there reason is that strcuture can create a bottle neck
<chamindra> unless that process can be made easier.. like using an AJAX lookup so that you avoid spelling mistakes
<flavour> We have an AJAX autocomplete widget in SahanaPy
<flavour> I can help with that
<flavour> For use where dropdowns become unwieldy
scopatz> Maybe we'll add that later, I think the goal now is to get the basic RMS up and running
<flavour> scopatz: Yes, that's icing...getting *something* quickly is key

<chamindra> also you will have many types of items per request
<chamindra> one request will consist of multiple items (like a shopping list)
<chamindra> so each one needs type, description, units, quantity
<chamindra> but you can consider the whole request being from one contact, phone, and for a number of people
<chamindra> good to have a backup phone and contact as well, in case the primary disappears
<chamindra> but that is optional

<chamindra> one last bit
<chamindra> and that is the matching part
<chamindra> once you get a request.. you can do a lookup and see if there is a potential match for it in the pledges
<chamindra> and vice versa
<chamindra> so even immidiately as they are on the call, they can respond saying there might be some options
<chamindra> at the end of the request, show a list of potential full and partial matches to the request

<chamindra> remember think simple intuitive interface
<chamindra> no complex workflows and approvals

100130

re: WFP's FoodRequest 

<flavour> I think I should merge RMS back into Trunk - just taking out the Haiti-specific URLs
<flavour> & the frontpage mention of Ushahidi/TtT
<flavour> So it's a manual based system
<nicopresto> that sounds like a great intiative, I'd be happy to help.
<flavour> With some generic feedparsing stuff which can be easily adapted as required
<flavour> Cool :)
<nicopresto> I just looked a the guts of Yatsy
<flavour> We don't have specific requirements yet, so nothing on that side
<nicopresto> could pull a couple of tweaks from there
<flavour> Do you fancy trying the merge back to Trunk of generic stuff?
<flavour> & then focus new deevl there?
<flavour> I guess the 'merge' is pretty easy - just the model, controller, views & cron scripts/crontab entries
<flavour> Then clean a little
<flavour> Really v.strightfwd
<nicopresto> sure do I just propose a merge with trunk once I've made generic?
<nicopresto> should this be a higher priority than polishing the haiti-specific RMS?
<flavour> I would say Yes
<flavour> As WFP is also Haiti
<flavour> & I think the time window when Generic RMS for Haiti was adding value has passed...I may be wrong...you may be more in touch with it than me
<flavour> There has been a merge into HMS of that stuff for us specifically with hospitals
<flavour> WFP want to use for Food requests
<flavour> Those are the active Haiti threads
<flavour> For improved maintainability even in medium term, would be good to merge RMS back into Trunk
<flavour> This removes that anomaly
<flavour> Haiti-specific customisations of Ushahidi/TtT URLs & view index pages remain in Haiti
<flavour> That's how I see it anyway
<flavour> mprutsalis1: Care to comment?
<nicopresto> looks like they'll add specific fields, if we can make that easy for them
<nicopresto> also sounds like we should add back the quantity/unit level detail on requests/pledges that we started designing in the early going of the Haiti work
<flavour> ok, makes sense
<flavour> I like that anyway
<flavour> We know that those fields should be oprtional in some use cases
<flavour> e.g. the SMS/TtT feeds
<flavour> However for actionable requests post Search/Rescue phase then these numbers are key
<flavour> The one thing I think to ponder in WFP, which isn't yet clear, is security
<flavour> My guess is that they may want a view of 'their' requests separately from others
<flavour> THis /may/ need to be able to be excuded from public view
<flavour> It /may/ need to have a different role to enable Edits
<nicopresto> we need groups perhaps
<flavour> Right, We already have security roles
<flavour> We need to be able to implement them easily within RMS
<flavour> So once the core Haiti RMS is available in Trunk, I think that's a really good are to work on - it's similar to the Hospital mini-RMS in terms of being able to have a separate view
<flavour> However I think it's nice to be able to offer an aggegated view too
<flavour> Just some stuff to think about
<nicopresto> it does have the potential to diverge in terms of tasks from RMS
<flavour> You're thinking it's very hard to keep RMS generic?
<flavour> I too worry about this
<flavour> The Ticketing System is really critical for the 1st phase Search & Rescure part
<flavour> HMS have specific needs
<flavour> WFP have specific needs
<flavour> So what can be made core Trunk for RMS to allow easy deployment customisation to meet needsd
<flavour> Ideally being able to support customisation for multiple needs at the same time on the same instance
<flavour> This isn't easy but would be *so* valuable
<nicopresto> I think a core set of code could be generic. Maybe after we build a WFP. We could compare RMS, WFP and HMS and pick best practices for a generic
<nicopresto> might require some attention to generic dbase

Haiti

Last modified 14 years ago Last modified on 06/16/10 20:10:54
Note: See TracWiki for help on using the wiki.