wiki:BluePrint/Messaging

Version 90 (modified by Fran Boon, 14 years ago) ( diff )

--

Blueprint for the Messaging Module

The Messaging Module is a modular system for getting Messages in & out of Sahana Eden & routed around it.

Core Architecture

Inbound Messages

  • When a message is picked up by the handler, it decides whether it's an XForm or not.
  • If it is, then it is routed direct to the relevant resource.
  • If not then it is added to the Master Message Log (msg_log)
  • When new messages arrive in the Master Message Log, then the Parser looks for keywords & tries to Route automatically - either to a resource or to a pe_pentity (NB 'routing' to a resource is actually just 'tagging').

Routing within Eden

  • Users can view the Master Message Log via the 'Ticketing Module' UI & do manual Routing - either to a resource or a pe_pentity.
  • Users can Subscribe to Resources in order to receive copies of all messages which are routed to a resource.
    • This is done by pressing the 'Subscribe' button on the page for that Resource.
      • Button becomes 'Unsubscribe' if user already subscribed.
  • When a user visits a Resource via it's Web UI, then the RESTController scans the msg_log & provides access to associated messages in the views (display/update views have a list of associated messages in a table below the main record. list views have an extra column showing the number of messages tagged to this resource & number unread by this user (unread(read)) which is hyperlinked to a view which displays these records.

Outbound Messages

  • If a pentity has a message sent to them (they subscribed or routed to them automatically or manually) then as well as being tagged for them, a copy is placed in the Outbox with them as recipient. Their list of Contacts is browsed & the appropriate contact method is used, depending on the priority of the message (e..g. a user may want to be sent an SMS for a high priority message, such as a security alert, but just be sent an email if it's just an update to one of their projects). If there are multiple methods available for a priority level then all methods are used at once - each having a separate entry in the outbox.
  • XML exports (such as RSS feeds) include tagged messages as linked resources:
    • S3ResourceController has a hook for a function that retrieves all message URLs for a resource: s3xrc = s3xrc.S3ResourceController(..., messaging=<function>)
    • This function takes the the resource name (=full tablename incl. prefix) and the record ID as arguments, and returns a list of URLs to messages for this resource and user, where each list item is (url, timestamp), with timestamp refering to the message timestamp
    • The XML exporter inserts one element <message url="..." unread="True/False"/> per message and resource element as child of the <resource> tag
    • This can be used for any feeds that are produced of the XML exporter
    • Requires: make RSS feeds an XML export template

Message Flows

Message Flows

How do we achieve this?

  • Table msg_log is used as a replacement for the current msg_inbox & ticket_log.
    • Fields: sender, subject, body, verified, actionable, assignee (multiple?), actioned, priority
      • Q: Do we move some of these 'ticketing' fields to a separate, linked ticket_msg table?
  • Table msg_tag is used to tag messages for resources.
    • Fields: message_id, resource, record (uid)
      • Q: How to display this nicely?
        • A: Give the message a tab with all distribution resources (db.tables). Once a table is selected, a second selector with all UUIDs (represented, of course) in that table appears, so you can choose the target. Use s3xrc.model.configure to add a name_nice for each table.
  • Table pe_subscription is used to maintain subscriptions for a pe_pentity.
  • Table msg_outbox is used to identify messages which need to be routed externally & store delivery statuses.
    • Fields: message_id, pe_contact_id, status
  • Table msg_read_status is used to show that a message has been read by a user.
    • Fields: message_id, auth_user_id ('read' is assumed)
    • Q: Do we mark as read if pe_contact_id status suggests person has received message? (Fran thinks not)
    • Good to have UI to 'Mark Unread'
  • Parser has hooks for modules to plug-into?
  • Q: Do we treat a pentity the same way as any other resource?
    • Just that when a message is routed to them, it not only tags but also creates an entry in msg_outbox, which is scanned by the Cron task for 'pending' messages to be sent out.
  • shn_rest_controller() is modified to check msg_log for messages tagged to the resource (list) or record (display/update)
    • Views modified as above
  • 'Ticketing Module' UI
    • default view is all un-actioned messages (assigned to this user & unassigned?)
    • If a message is routed to a resource/pentity, this results in a 'create' form popup with as many fields auto-populated as possible (subject into 'name' field & body into 'comments' field, if nowhere better)
      • Q: If a message is routed to a resource, does this automatically mark ticket as 'actioned'? Provide a popup to ask this?
  • Auth restrictions

Parser

Pyparsing is included in the modules folder.

This can decode messages, especially from compressed formats such as Tweets or SMS but also free-from formats such as email.

We could add further text processing such as our implementation of Levenshtein.

Micro-Syntax

User Interface

The home page should be a common Inbox view which, by default, shows all new unread Items (the 'Ticketing Module').

Users/Groups should be able to subscribe to incoming messages of a certain type (Project / Location / Organisation) & have these messages forwarded via SMS / Email / Twitter / etc.

Ideally there should be the ability to launch a popup/ticker to notify of new items & a portlet which can be on a user's dashboard when they login.

The 'Send Message/Alert' screen should have a small box to enter the Plain-text message which is sent to all recipients (using all available SMS/Email addresses in the Group specified).

There should be a 2nd box to create a Rich-text message, with the option of adding attachments. This content just gets sent to those members of the group with an Email address specified.

If a user is added to a group we warn about any missing details, but still allow addition (the ref is by pe_pentity.id so addresses can be added/changed later anyway).

When sending we need to provide warning messages about users for whom details weren't available.

Transports

Email

gluon/tools.py has mail settings:

mail=MailS3()
# These settings should be made configurable as part of the Messaging Module
mail.settings.server='mail:25'
mail.sender='sahana@sahanapy.org'

& a function:

mail.send(to, subject='None', message='None')

Can use this to send emails with dynamic data inserted into templates (like HTML pages):

There is a Cron job to check the contents of the Outbox & send pending messages. On Success it marks the message as 'Sent':

Rich Text

We'd like to be able to send HTML mails for rich-text support (including attachments) This requires the use of smtplib to send MIME-encoded files:

SMS

Support both online gateways (such as Clickatell) & local hardware such as:

The SMS Handler Daemon to handle Inbound messages is a separate Python script.

  • Routes incoming message according to whether it's using specialist micro-syntax (e.g. submitting an XForm to another controller)
  • Non-XForms messages get sent to the Ticketing module's Master Message Log
  • Tagged messages also get Routed to the appropriate area (Project / Location / Organisation)

It can distinguish between simple messages (added to the Inbox of the Messaging Module) & those which are encoded in Binary XML (the output of XForms from the J2ME client).

Implementation details are here: http://wiki.sahanafoundation.org/doku.php/dev:pythonsms

We use Cron to check Outbox & signal the daemon to send Pending messages (as for email)

SMS alerts (security alerts more common than natural disasters):

  • Being able to trigger an SMS alert broadcast upon reception of an SMS
    • Is this just an XForm to a Group? (Can we pre-populate the XForm to do this whenever a certain number is called or just a single word routes here?)

SMS login / data entry / reports

Would be good to add an LCR (Least-Cost Routing engine) for SMS deliveries (so that e.g. local messages routed via modem, but International via Gateway).

Toolkits we can base on:

Hardware compatibility databases:

APRS

Automatic Packet Reporting System used by Amateur Radio networks:

CAP

Common Alerting Protocol:

Twitter

Adding support for sending out/receiving alerts via Twitter would be nice.

XForms

We have controllers/xforms.py to generate XForms automatically out of Sahana models.

These can be used by the J2ME client for data collection.

This can be used to create OCR-able Printed Forms.

XMPP

Instant Messaging & Presence including GoogleTalk:

Alternate implementations


BluePrints

Attachments (1)

Download all attachments as: .zip

Note: See TracWiki for help on using the wiki.