wiki:Projects/Advanced

Version 78 (modified by Fran Boon, 10 years ago) ( diff )

--

Table of Contents

  1. Integration with Sigmah using HXL
  2. Deployment Tools
  3. Out-Of-The-Box Sahana Deployment
  4. Mobile Interface
  5. Workflow Support
  6. NYC Prepared Site
  7. Image & File Management
  8. Dashboards
  9. Search
  10. Client-side validation / error reporting
  11. Export Graphs
  12. Optimisation
  13. Rostering
  14. Survey: Template GUI
  15. Survey: Reporting
  16. Automatic Test Framework
  17. Social Media
    1. Facebook Integration
  18. GIS/Mapping
    1. Wikipedia Locations
  19. Fix Web2Py Cron on Win32 Service
  20. Framework
  21. Tweet / SMS Parsing
  22. Run Eden on a smart phone
  23. New Use Cases
  24. Flash Drive Version
  25. OSM Importer UI
    1. Current Status
    2. Enhancements Needed
  26. Timeline
  27. Usability
  28. Job Jar / Human Intelligence Task Support
  29. Web Setup
  30. Logistics
  31. Build library(ies) to integrate Emergency Data Exchange Language …
  32. Map elements between EDXL-SITREP and EDXL-RM
  33. See Also

Coding Projects

These are bigger projects that will require more effort and commitment to compel

Some things to keep in mind when working on these projects or planning new modules or major features:

  • Engage with the Sahana Eden Mailing List to share your ideas and get input on the project. (Not everything is documented on the wiki - but if you are committed to working on a project the community can help point you in the right direction!)
  • Please ask on the mailing list whether the project you're interested in has already been done, or is being worked on by others. That is not always recorded here, and some completed projects are not removed right away. That doesn't mean there are not other aspects to the project you could work on, and you might join in with a project in progress. But if you are looking for a complete project, e.g. for a college team to work on, it's better to check.
  • Prepare a detailed Blue Print design to outline your design approach for the project
  • Familiarise yourself with the Sahana Eden Framework and make use of it - no need to reinvent the wheel :)
  • Consider what other FOSS tools exist, that could be installed alongside Eden.
  • Look at tools and libraries already in use in Eden, e.g. jQuery and ExtJS for UI work.

Integration with Sigmah using HXL

See BluePrint/HXL

Deployment Tools

Focal Persons: Fran Boon, ...

Extend the toolkit for deploying and maintaining instances of Sahana Eden, eg;

  • select physical servers &/or Amazon EC2 account (walk-through creating keys/certs, region(s))
  • select server roles distribution
  • select Database (MySQL or PostgreSQL/PostGIS)
  • select Webserver (Apache/mod_WSGI or Cherokee/uWSGI)
  • select Template
  • install all necessary/chosen components on the selected servers
  • add optional co-apps, such as GeoServer &/or OptaPlanner
  • optionally run Load Tests & then cleanup
  • add simple UI (e.g. web2py or Tk app)
  • this could then develop into a WebSetup app

Out-Of-The-Box Sahana Deployment

Focal Persons: Michael Howden, ...

Develop a generic template to allow Sahana Eden to be easily deployed in a range of different contexts.

  • Reviewing functionality across a number deployed templates:
  • Developing a complete BluePrint (requirements) for the template in consultation with the community
  • Modules to Enable / Configure
    • Volunteer Management
    • Staff Management
    • Project Management
    • CMS (User Updates)
    • Request Management
  • Automate Test Scripts
  • UI
    • Re-design Homepage
    • Design Menu Hierarchy to match User Stories
    • Design Module Index Pages / Dashboards
    • Design/implement new theme (Using Bootstrap/JqueryUI?)
  • Support Material
    • User Manual
    • Instruction Videos

Mobile Interface

Focal Persons: Michael Howden, Graeme Foster, Fran Boon

Implement a responsive HTML interface for mobile devices. This will involve:

  • Style standard CRUD + Search + Report for mobile devices
  • Represent Menus in a meaningful way for Mobile Devices
  • Possible options include Bootstrap, jQuery Mobile & Sencha

Workflow Support

Focal Persons: Dominic König

  • UI design to indicate steps in a workflow
  • Functionality to split the completely of large forms up into smaller steps (possibly using JS?)
  • Workflow Engine (probably beyond the scope of GSoC - actually I think this should be the primary scope of the project GF)

NYC Prepared Site

See: Deployments/Sandy/Recovery

Image & File Management

Focal People: Michael Howden, ...

Support better management of Images and Files in Sahana Eden, including:

  • Multiple-File Async Upload with Previews: ​https://github.com/blueimp/jQuery-File-Upload
  • Being able to drag images into rich text edit boxes
    • Mobile Photo/Video Upload: Design a widget (to be included in an Eden form) that can be used to select images or video, with any needed detection of the type of phone.
  • Represent function for full image or thumbnail
  • Lightbox viewer for full sized image
    • e.g. Make use of the one within Vulnerability module
  • Image Library
  • Use Mapping API to display images on Map
  • Show Icons for Files

Dashboards

Focal Persons: Michael Howden, Fran Boon

Focal Persons: Pat Tressel, Dominic König

  • Full Text Search [Completed as a 2013 Google Summer of Code project by Vishrut Mehta.]
    • e.g. using Apache Lucene
    • Search uploaded documents (must follow authorization rules!)
  • Global Search
    • Over multiple resources (must follow authorization rules!)
    • Search box in the top right corner

Client-side validation / error reporting

Focal Persons: Fran Boon, ...

It is nicer for both users & server load if they get told of data entry problems (such as not filling-in mandatory fields) before they submit the form and get the server-side validation results.

We make a little use of jQuery.validate already - currently just for the registration page. It would be good to extend this and provide a nice consistent API to report errors, which should work on both Bootstrap & other Themes. The existing 'required' flag should be picked-up & validated automatically.

NB We already have an API for displaying 'page-level' errors: S3.showAlert() in S3.js. What is missing is a field-level display mechanism.

Started as S3.fieldError()

Export Graphs

Export Graphs as Images.

Focal Person: Dominic König

We currently use the Flot JavaScript library to produce interactive graphs - primarily via the S3Report (/report) method:

We have 2 usecases for exporting graphs:

  1. A user would like to download the report they're seeing on screen as a PDF
  2. A user would like to download the report they're seeing on screen as an XLS
    • to have both the Pivot Table & associated Graph
  3. A user would like to subscribe to a report to receive by email daily/weekly/monthly

So, the task, as I see it, is to take the settings for a Report & produce the PDF &/or XLS version thereof.

We convert our internal S3XML data format to PDF or XLS using codecs:

I would focus on just one of these to keep scope feasible.

  • personally I'd be more interested in XLS as this gives users a more useful tool to explore the data than just a static PDF.

Options:

  • ​Use Flot to Export graphs
  • parallel matplotlib method (extend modules/s3chart.py)
  • server-side webkit browser to take a screenshot (ClimateDataPortal used this solution)

See Also

Optimisation

Focal Persons: Dominic König, Fran Boon

Performance enhancements to handle 10,000's of records:

  • Server-side clustering of map features with different report-style aggregate representations
  • Optimize Report generation (probably beyond GSoC)
    • Data Warehouse? (Flattened tables structure, NoSQL)
    • Stored Procedures?

Concurrency:

See Also:

Rostering

Focal Persons: Dominic König

  • Staff/Volunteer Roster GUI
    • a prototype was done for GSoC 2012 which can be used as a point of reference - what was good about it? What was bad?
  • Development of/Integration with a wCSP engine for automatic rostering

Survey: Template GUI

Focal Persons: Graeme Foster

  • Currently the templates are added via prepopulated data, to create the prepop data it can be done via a script and manual editing
    • Create a frontend that will step the user through the script logic so that new templates can be created
    • Extract this data for use in prepop
  • Allow the import of template details outside of the prepop process. This will us the standard import process but, will add a special interface because there are two files that may be imported the questions and then the layout and the import order is important.
  • Allow the removal of a template (which should be permissible if it has not been used)
  • Support the editing of the wording of the template, this would be simple changes to the phrasing of a section a question or a possible reply. (not structural changes to the questions)

GSoC Students: If you are interested in this project then you will benefit by looking at the ​script. Sample files that can be converted by this script can be requested from me via the mailing list or IRC, and then you should learn how to generate your own file that will be converted by this script. That will then give you the background knowledge required to start on and be successful with this project.

Could ​KoboForm UI be used with our back-end? Or could we just borrow UI ideas?

Survey: Reporting

Focal Persons: Graeme Foster

  • Currently the reports in the survey module use matplotlib, now that Sahana has it's own reporting framework S3Report2 it should be used to deliver this information, so that we have the ability to explore data interactively client-side, users see a consistent interface & developers have less code to maintain.
    • Write test script to for current functionality
    • Re-engineer code to use S3Report2 direct or the underlying S3PivotTable if more control is needed
    • Enhance S3PivotTable/S3Report2 where appropriate

Automatic Test Framework

Focal Persons: Michael Howden, Dominic König See:

Ideas

  • Automatically create Dummy Data
  • Extend Role Tests
  • Run Selenium Tests & Smoke tests in multiple templates with multiple user accounts
  • Clearer Error Messages
  • Simplify Selenium Tests and make them easier to read and more robust.
  • Adapt tests to meet needs of evolving CI Server (SysAdmin/ContinuousIntegration)
  • Load Tests
  • Selenium Test show web2py error messages
  • Write Functional Tests using Selenium
  • Figure out how to do unit tests against a web application.

Social Media

Focal Persons: Fran Boon, Michael Howden, Connie White See:

Facebook Integration

GIS/Mapping

These are a number of GIS/Mapping Tasks to be done:

Wikipedia Locations

Allow Sahana to import Wikipedia locations easily: both Hierarchy & Lat/Lon info

Fix Web2Py Cron on Win32 Service

Web2Py has a nice Cron utility, but this doesn't work currently on Win32 in Service mode: #438

Framework

  • Provide localisation of jquery.ui.datepicker
    • Means that user profile will need extending to support locales
  • Replace jquery.cluetip.js with jquery.ui.tooltip.js (1.9 once released)

Tweet / SMS Parsing

Allow users to provide info via tweets or SMS messages

Run Eden on a smart phone

New Use Cases

Extensions to support:

  • Organizations such as ​Ten Thousand Villages assist craftspeople in disadvantaged countries organize to purchase supplies and market their crafts. What would such a crafts coop need to help run their business? What's missing from Eden to support this?
  • ​Maitri provides support to victims of domestic violence.
    • They have a ​page listing resources but part of the page is organized topically and the rest by location. Consider how to store the resource data to allow displaying it either by topic or by location, and also make it searchable.
    • Look at the ​volunteer page. Compare with the existing (incomplete) volunteer module. What additional features might Maitri need to use the Eden volunteer module?

Flash Drive Version

Extend and configure the flash drive version to support:

  • Different sets of Prepop Data
  • Multiple user access configuration
  • Sync with online instance (auto-set up)

OSM Importer UI

Current Status

Enhancements Needed

5 separable Tasks here:

  1. We need to add the Sahana UUID to these features in the OSM database.
    • This is needed so that we can do deduplication on subsequent imports (the OSM ID is insufficient for this as it can change over time)
    • This means we need to read the OSM credentials (currently a deployment_setting in 000_config.py) & do a REST API call to the main OSM database with the modified feature(s)
  2. Move the OSM credentials from a global 000_config.py deployment_setting to a per-user setting (component resource of db.auth_user)
  3. We would like to be able to control which features are imported from this UI (currently this is a deployment_setting in [000_]config.py - that should become the list of avaialble options rather than the absolute list)
  4. We would like to be able to download features for a Location from main OSM database
    • We want to use the Polygon (falling-back to BBOX (min_lat, max_lon, etc)) to trigger a download of a suitably-filtered .osm file via the ​Overpass API
  5. We would like to be able to download features for a Polygon
    • Provide a map window with a Polygon selector tool which then triggers a download of a suitably-filtered .osm file via the ​Overpass API

Timeline

Usability

Job Jar / Human Intelligence Task Support

Focal People: Pat Tressel,...

Web Setup

Logistics

Build library(ies) to integrate Emergency Data Exchange Language Distribution Element

Proposed by: ​| Nuwan
Specific : ​| EDXL-DE is the final wrapper (envelope) of all ​| EDXL data package. We may be delivering a EDXL Resource Management (RM) information and Situational Reporting (SITREP) information to the managers of several emergency organizations. The DE will contain who, when, and where those RM and SITREP data parcels would be delivered. Every EDXL message (data package) must carry this information. Otherwise, it cannot use available distribution methods and would need to rely on its own protocol; that's not very user friendly.
Measurable: It is purely a coding task that involves playing with XML and developing a class, possibly within or using the ??? 3R ??? framework It is basically a set of procedures for packing and unpacking EDXL-DE wrapped data.
Attainable: run through the steps
Step 1 :: select one of the existing EDXL-based applications in Eden, I recommend the EDXL-RM
Step 2 :: discuss each of the attributes/elements; then determine how EDXL-DE would be added to EDXL-RM as a pop-up GUI.
Step 3 :: create some example XML files to get a feel for the inputs and outputs
Step 4 :: write code to add to take the inputs EDXL-RM and EDXL-DE to package the data for delivery
Step 5 :: test the code, fix bugs, and generalize the functions
Step 6 :: Apply to EDXL-HAVE and EDXL-SITREP to generalize the library
Relevant : Applies to Sahana Interoperability policy. Present developments to investigate are: HAVE, RM, and SITREP ​http://www.oasis-open.org/standards#edxl
Time-bound: If steps 1 - 4 are completed that can be a full accomplishment; additional work is a bonus
Evaluate : Produce XML files with the EDXL-DE element appended to the EDXL-RM, SITREP, or HAVE
Reevaluate: Use an API to add and strip the EDXL-DE to any EDXL data standard. Use the same set of XML files to run through this process

Map elements between EDXL-SITREP and EDXL-RM

Proposed by: ​| Nuwan
Specific : There are several elements within the ​| EDXL-SITREP data standard that are identical to that of ​| EDXL-RM. The objective is to create a descriptive table of those elements that a programmer can use to develop a set of procedures to strip the RM data from SITREP to manage records in the relational database
Measurable : Requires diligently investigating each and every data element then comparing them with the two data standards SITREP and RM. Requires knowledge of data types and XML. If not, this exercise will help the student learn about XML and data standards
Attainable :
Step 1 :: Go through EDXL-RM and EDXL-SITREP documentations
Step 2 :: create example files to get an understanding of the data and structure
Step 3 :: develop the table with RM and SITREP elements with a description
Step 4 :: develop a simple XSL file to strip the RM data from SITREP
Relevant : Applies to Sahana interoperability policy. Given that Eden does support resource and incident management, it is important to derive the response resources and resource requirements from situational reports. This function would help automate some of those data extraction functions. That requires integrating the SITREP and RM components with underlying Eden schema
Time-bound : The exercise of understanding the data standards and mapping the elements should not take more than one week. developing a XSL to test the mapping may take another week, depending on the level of expertise with XML. (NB this is far too large a task for GCI. -- Pat)
Evaluate : The mapping table is the determining output. However, the example RM and SITREP files are also required.
Reevaluate : If the XSL transformation is developed, then the mapping can be tested with the sample RM and SITREP files

See Also

Note: See TracWiki for help on using the wiki.