Version 86 (modified by 14 years ago) ( diff ) | ,
---|
Table of Contents
Blueprint for the Messaging Module
The Messaging Module is a modular system for getting Messages in & out of Sahana Eden & routed around it.
Core Architecture
Inbound Messages
- When a message is picked up by the handler, it decides whether it's an XForm or not.
- If it is, then it is routed direct to the relevant resource.
- If not then it is added to the Master Message Log (
msg_log
) - When new messages arrive in the Master Message Log, then the Parser looks for keywords & tries to Route automatically - either to a resource or to a
pe_pentity
(NB 'routing' to a resource is actually just 'tagging').
Routing within Eden
- Users can view the Master Message Log via the 'Ticketing Module' UI & do manual Routing - either to a resource or a
pe_pentity
. - Users can Subscribe to Resources in order to receive copies of all messages which are routed to a resource.
- This is done by pressing the 'Subscribe' button on the page for that Resource.
- Button becomes 'Unsubscribe' if user already subscribed.
- This is done by pressing the 'Subscribe' button on the page for that Resource.
- When a user visits a Resource via it's Web UI, then the RESTController scans the
msg_log
& provides access to associated messages in the views (display/update views have a list of associated messages in a table below the main record. list views have an extra column showing the number of messages tagged to this resource & number unread by this user (unread(read)
) which is hyperlinked to a view which displays these records.
Outbound Messages
- If a pentity has a message sent to them (they subscribed or routed to them automatically or manually) then as well as being tagged for them, a copy is placed in the Outbox with them as recipient. Their list of Contacts is browsed & the appropriate contact method is used, depending on the priority of the message (e..g. a user may want to be sent an SMS for a high priority message, such as a security alert, but just be sent an email if it's just an update to one of their projects). If there are multiple methods available for a priority level then all methods are used at once - each having a separate entry in the outbox.
- XML exports (such as RSS feeds) include tagged messages as linked resources:
- S3ResourceController has a hook for a function that retrieves all message URLs for a resource: s3xrc = s3xrc.S3ResourceController(..., messaging=<function>)
- This function takes the the resource name (=full tablename incl. prefix) and the record ID as arguments, and returns a list of URLs to messages for this resource and user, where each list item is (url, timestamp), with timestamp refering to the message timestamp
- The XML exporter inserts one element <message url="..." unread="True/False"/> per message and resource element as child of the <resource> tag
- This can be used for any feeds that are produced of the XML exporter
- Requires:
make RSS feeds an XML export template
Message Flows
How do we achieve this?
- Table
msg_log
is used as a replacement for the currentmsg_inbox
&ticket_log
.- Fields: sender, subject, body, verified, actionable, assignee (multiple?), actioned, priority
- Q: Do we move some of these 'ticketing' fields to a separate, linked
ticket_msg
table?
- Q: Do we move some of these 'ticketing' fields to a separate, linked
- Fields: sender, subject, body, verified, actionable, assignee (multiple?), actioned, priority
- Table
msg_tag
is used to tag messages for resources.- Fields: message_id, resource, record (uid)
- Q: How to display this nicely?
- A: Give the message a tab with all distribution resources (db.tables). Once a table is selected, a second selector with all UUIDs (represented, of course) in that table appears, so you can choose the target. Use s3xrc.model.configure to add a name_nice for each table.
- Q: How to display this nicely?
- Fields: message_id, resource, record (uid)
- Table
pe_subscription
is used to maintain subscriptions for ape_pentity
.- Fields: pr_pe_id, resource, record (uid)
- Subscriptions can be added/deleted from the linked resource
- Add to Views (layout.html)
- http://myserver.org/eden/msg/subscribe?resource=or_org&uid=XXX
- Controller needs writing
- Add to Views (layout.html)
- Subscriptions can be viewed from the user's subscriptions page
Add to 01_menu.py & pr.py
- Table
msg_outbox
is used to identify messages which need to be routed externally & store delivery statuses.- Fields: message_id, pe_contact_id, status
- Table
msg_read_status
is used to show that a message has been read by a user.- Fields: message_id, auth_user_id ('read' is assumed)
- Q: Do we mark as read if pe_contact_id status suggests person has received message? (Fran thinks not)
- Good to have UI to 'Mark Unread'
- Parser has hooks for modules to plug-into?
- Q: Do we treat a pentity the same way as any other resource?
- Just that when a message is routed to them, it not only tags but also creates an entry in
msg_outbox
, which is scanned by the Cron task for 'pending' messages to be sent out.
- Just that when a message is routed to them, it not only tags but also creates an entry in
shn_rest_controller()
is modified to check msg_log for messages tagged to the resource (list) or record (display/update)- Views modified as above
- 'Ticketing Module' UI
- default view is all un-actioned messages (assigned to this user & unassigned?)
- If a message is routed to a resource/pentity, this results in a 'create' form popup with as many fields auto-populated as possible (subject into 'name' field & body into 'comments' field, if nowhere better)
- Q: If a message is routed to a resource, does this automatically mark ticket as 'actioned'? Provide a popup to ask this?
- Auth restrictions
Parser
Pyparsing is included in the modules folder.
This can decode messages, especially from compressed formats such as Tweets or SMS but also free-from formats such as email.
We could add further text processing such as our implementation of Levenshtein.
Micro-Syntax
- GeoHashes
- Tweets
- Geotagging where hardware/software combination available: place_id from http://twitter.com/{username}/statuses/{tweet_id}.json -> http://api.twitter.com/1/geo/id/{place_id}.json
- otherwise use WW place WW message:
- EPIC's Tweak the Tweet
- SMS codes looked up on a paper wheel: http://ndt.instedd.org/2010/05/it-without-software.html
User Interface
The home page should be a common Inbox view which, by default, shows all new unread Items (the 'Ticketing Module').
Users/Groups should be able to subscribe to incoming messages of a certain type (Project / Location / Organisation) & have these messages forwarded via SMS / Email / Twitter / etc.
Ideally there should be the ability to launch a popup/ticker to notify of new items & a portlet which can be on a user's dashboard when they login.
The 'Send Message/Alert' screen should have a small box to enter the Plain-text message which is sent to all recipients (using all available SMS/Email addresses in the Group specified).
There should be a 2nd box to create a Rich-text message, with the option of adding attachments. This content just gets sent to those members of the group with an Email address specified.
If a user is added to a group we warn about any missing details, but still allow addition (the ref is by pe_pentity.id so addresses can be added/changed later anyway).
When sending we need to provide warning messages about users for whom details weren't available.
Transports
gluon/tools.py
has mail settings:
mail=MailS3() # These settings should be made configurable as part of the Messaging Module mail.settings.server='mail:25' mail.sender='sahana@sahanapy.org'
& a function:
mail.send(to, subject='None', message='None')
Can use this to send emails with dynamic data inserted into templates (like HTML pages):
There is a Cron job to check the contents of the Outbox & send pending messages. On Success it marks the message as 'Sent':
Rich Text
We'd like to be able to send HTML mails for rich-text support (including attachments) This requires the use of smtplib to send MIME-encoded files:
- http://groups.google.com/group/web2py/browse_thread/thread/3d8ed693dd2f29bc
- https://web2py.com/wiki/default/page/98b7448f-059a-47f7-82da-dce4728aa4dd
- Simple app for sending mails from a form (using smtplib): http://web2py.com/appliances/default/show/10
SMS
Support both online gateways (such as Clickatell) & local hardware such as:
- a Bluetooth mobile phone
- MobiGater – Takes 1 sim card. Cost approx EUR 70.
- VoiceBlue Lite – Takes 4 sim cards and can handle 4 concurrent calls. Cost approx EUR 900.
- Multi-Tech MultiModem GPRS MTCBA-G-F4
The SMS Handler Daemon to handle Inbound messages is a separate Python script.
- Routes incoming message according to whether it's using specialist micro-syntax (e.g. submitting an XForm to another controller)
- Non-XForms messages get sent to the Ticketing module's Master Message Log
- Tagged messages also get Routed to the appropriate area (Project / Location / Organisation)
It can distinguish between simple messages (added to the Inbox of the Messaging Module) & those which are encoded in Binary XML (the output of XForms from the J2ME client).
Implementation details are here: http://wiki.sahanafoundation.org/doku.php/dev:pythonsms
We use Cron to check Outbox & signal the daemon to send Pending messages (as for email)
SMS alerts (security alerts more common than natural disasters):
- Being able to trigger an SMS alert broadcast upon reception of an SMS
- Is this just an XForm to a Group? (Can we pre-populate the XForm to do this whenever a certain number is called or just a single word routes here?)
SMS login / data entry / reports
Would be good to add an LCR (Least-Cost Routing engine) for SMS deliveries (so that e.g. local messages routed via modem, but International via Gateway).
Toolkits we can base on:
- SlingshotSMS (PyGSM-based with CherryPy REST interface): http://code.developmentseed.org/slingshotsms/dashboard
- RapidSMS (PyGSM-based): http://rapidsms.org
- PySerial: http://pyserial.wiki.sourceforge.net/pySerial
- Gammu: http://www.gammu.org (Python-gammu: http://www.gammu.org/wiki/index.php?title=Gammu:Python-gammu)
- Gnokii: http://www.gnokii.org (PyGnokii: http://mobilehacking.org/index.php/PyGnokii)
- Kannel: http://kannel.org (PyKannel: http://mobilehacking.org/index.php/PyKannel)
- SMSTools: http://smstools3.kekekasvi.com
Hardware compatibility databases:
APRS
Automatic Packet Reporting System used by Amateur Radio networks:
CAP
Common Alerting Protocol:
- http://xml.coverpages.org/OASIS-CAPv11-200510.pdf
- Use an XSLT?
- Subset of EDXL-DE: http://docs.oasis-open.org/emergency/edxl-de/v1.0/EDXL-DE_Spec_v1.0.pdf
- http://talksahana.com/2009/03/04/firefox-browser-cap-alerting-plugin-sahana-idea-for-gsoc2009/
Adding support for sending out/receiving alerts via Twitter would be nice.
- Simple Web2Py example of posting/reading
- Hashtags can be parsed using pyparsing.
- Twitter API
XForms
We have controllers/xforms.py
to generate XForms automatically out of Sahana models.
These can be used by the J2ME client for data collection.
This can be used to create OCR-able Printed Forms.
XMPP
Instant Messaging & Presence including GoogleTalk:
Alternate implementations
- Agasti design: http://wiki.sahanafoundation.org/doku.php?id=dev:msg_archi
- Agasti User Guide http://wiki.sahanafoundation.org/doku.php?id=doc:message:english
- GeoChat
- Frontline SMS (used by Ushahidi)
Attachments (1)
-
Message Flows.png
(20.8 KB
) - added by 14 years ago.
Message Flows
Download all attachments as: .zip