Version 19 (modified by Ashwyn, 9 years ago) ( diff )


Inbound Message Parsing


This is a project proposal for GSoC 2012. We need to parse inbound messages, with an initial focus being SMS.

We can test this with Clickatell or a local phone (the Clickatell Inbound SMS fucntionality needs to be developed, which could be in-scope for this project).

Where should the code live?

Currently message parsing is done in the core code: modules/s3/

We want to be able to make this a deployment-specific set of options.

We are still working on developing our Profile Layer for having deployment-specific files separated from core code, so we can start by just making this a deployment-template like copying the file to modules/s3/ for easy import into S3MSG.

How does the code get run?

The Inbound Message receiving tasks should be run via the Scheduler:

Before exiting those threads, it should trigger a new Schduled task (once, now) to do the parsing.

All Parser Workflows should be defined in

The Configuration links these Tasks to Inbound Sources, along with any other args, in a new msg_workflow table.

  • so this can be linking 2 FKs in the scheduler_task table

Tropo, however, is different, since that receives inbound connections to it's controller. In order to make this consistent a schduled task can be created but set to never run on schedule & have that run from the controller (or, even better, have this run the same code that the schduled task runs but inline...since this will be pretty much a no-op other than triggering the parser, so won't hold the Tropo connection up & avoids having to spawn a new thread).


We want to be able to process OpenGeoSMS

If we need to have complex parsing, then we can make use of pyparsing



We want to be able to direct the message to the appropriate module to handle the data.

This could be done either by launching a real REST request or else simulating one via the API.

resource = s3mgr.define_resource("module", "resourcename")


Data Model

Data Model Blueprint: msg_workflow

Field Name Purpose Datatype/table
source_task_idInbound Email/SMS Source(Another workflow in case of chained workflows).String corresponding to username of the source.
workflow_task_idParsing WorkflowString corresponding to name of the parsing function.
s3.meta_fields() metadata

Changes in msg_log:

Field Name Purpose Datatype/table
is_parsedParsing Status of Inbound MessagesBoolean
source_task_idInbound Email/SMS Source(Another workflow in case of chained workflows).String corresponding to username of the source.

Use Cases

This will be used by at least CERT & Tzu Chi.

They wish to process responses to deployment notifications where the recipients send back at least 'Accepted' / 'Reject'.

Additional information that it would be useful to capture are:

  • Current Location
  • ETA to deployment location
  • Questions/Comments (free text)

GSoC Project

Note: See TracWiki for help on using the wiki.