[[TOC]] = Coding Projects = These are bigger projects that will require more effort and commitment to compel Some things to keep in mind when working on these projects or planning new modules or major features: * Engage with the [MailingList Sahana Eden Mailing List] to share your ideas and get input on the project. (Not everything is documented on the wiki - but if you are committed to working on a project the community can help point you in the right direction!) * Please ask on the mailing list whether the project you're interested in has already been done, or is being worked on by others. That is not always recorded here, and some completed projects are not removed right away. That doesn't mean there are not other aspects to the project you could work on, and you might join in with a project in progress. But if you are looking for a complete project, e.g. for a college team to work on, it's better to check. * Prepare a detailed [BluePrint/Guidelines Blue Print] design to outline your design approach for the project * Familiarise yourself with the [DeveloperGuidelines#Framework Sahana Eden Framework] and make use of it - no need to reinvent the wheel :) * Consider what other FOSS tools exist, that could be installed alongside Eden. * Look at tools and libraries already in use in Eden, e.g. jQuery and ExtJS for UI work. == Integration with Sigmah using HXL == See [wiki:BluePrint/HXL#SigmahtoSahana3W4Wdataexportation] == Out-Of-The-Box Sahana Deployment == Focal Persons: Michael Howden, ... Develop a generic [http://eden.sahanafoundation.org/wiki/DeveloperGuidelines/Templates template] to allow Sahana Eden to be easily deployed in a range of different contexts. * Reviewing functionality across a number deployed templates: * {{{IFRC}}} * {{{DRMP}}} * {{{CRMT}}} * {{{Yolanda}}} - Deployments/Philippines/Haiyan * {{{...}}} * Developing a complete BluePrint (requirements) for the template in consultation with the community * Modules to Enable / Configure * Volunteer Management * Staff Management * Project Management * CMS (User Updates) * Request Management * Automate Test Scripts * UI * Re-design Homepage * Design Menu Hierarchy to match User Stories * Design Module Index Pages / Dashboards * Design/implement new theme (Using Bootstrap/JqueryUI?) * Support Material * User Manual * Instruction Videos == Mobile Interface == Focal Persons: Michael Howden, Graeme Foster, Fran Boon Implement a responsive HTML interface for mobile devices. This will involve: * Style standard CRUD + Search + Report for mobile devices * Represent Menus in a meaningful way for Mobile Devices * Possible options include Bootstrap, jQuery Mobile & Sencha == Workflow Support == Focal Persons: Dominic König * Workflow Engine * UI design to indicate steps in a workflow * Functionality to split the completely of large forms up into smaller steps (possibly using JS?) == NYC Prepared Site == See: Deployments/Sandy/Recovery == Image & File Management == Focal People: Michael Howden, ... Support better management of Images and Files in Sahana Eden, including: * Multiple-File Async Upload with Previews: https://github.com/blueimp/jQuery-File-Upload * Being able to drag images into rich text edit boxes * Mobile Photo/Video Upload: Design a widget (to be included in an Eden form) that can be used to select images or video, with any needed detection of the type of phone. * Represent function for full image or thumbnail * Lightbox viewer for full sized image * e.g. Make use of the one within Vulnerability module * Image Library * Provide a .represent for the view to display image thumbnail with option to zoom large (such as Fancyzoom) * Provide a beautiful image viewer to flip between images * http://blog.lieldulev.com/2010/05/21/parallel-image-preloading-in-js/ * Use Mapping API to display images on Map * Show Icons for Files == Dashboards == Focal Persons: Michael Howden, Fran Boon * Allow pages which can currently be built by a developer using [wiki:S3/S3Profile S3Profile] to be customised by the User * Dynamically load Dashboard components * See: * http://eden.sahanafoundation.org/wiki/BluePrint/UserInterface#ModuleIndexPages * http://eden.sahanafoundation.org/wiki/BluePrint/UserInterface/Dashboard * https://docs.google.com/document/d/1EKt6lSs1d2kuyCfCM8hQZdeL2zfCJ2JKvPU-dm81TUQ/edit?usp=sharing * https://github.com/flavour/eden/blob/master/modules/s3/s3profile.py (Current implementation used in DRMP template) == Global Search == Focal Persons: Pat Tressel, Dominic König * Global Search * Over multiple resources (must follow authorization rules!) * Search box in the top right corner * could use Lucene, which was integrated in a previous GSoC for Full-text searching == Client-side validation / error reporting == Focal Persons: Fran Boon, ... It is nicer for both users & server load if they get told of data entry problems (such as not filling-in mandatory fields) before they submit the form and get the server-side validation results. We make a little use of jQuery.validate already - currently just for the registration page. It would be good to extend this and provide a nice consistent API to report errors, which should work on both Bootstrap & other Themes. The existing 'required' flag should be picked-up & validated automatically. NB We already have an API for displaying 'page-level' errors: S3.showAlert() in S3.js. What is missing is a field-level display mechanism. Started as S3.fieldError() == Export Graphs == Export Graphs as Images. Focal Person: Dominic König We currently use the D3 !JavaScript library to produce interactive graphs - primarily via the S3Report (/report) method: * [wiki:S3/S3Report S3Report] We have 2 usecases for exporting graphs: 1. A user would like to download the report they're seeing on screen as a PDF 2. A user would like to download the report they're seeing on screen as an XLS - to have both the Pivot Table & associated Graph 3. A user would like to subscribe to a report to receive by email daily/weekly/monthly - again this report could be in PDF or XLS - Subscription will be done using this: * https://github.com/flavour/eden/blob/master/modules/s3db/pr.py#L2661 - Notification will be done using this: * https://github.com/flavour/eden/blob/master/modules/s3/s3notify.py So, the task, as I see it, is to take the settings for a Report & produce the PDF &/or XLS version thereof. - ideally this would work for both the interactive & subscription usecases i.e. subscription would store the Report options extending the existing Saved Filter options: * https://github.com/flavour/eden/blob/master/modules/s3db/pr.py#L2619 We convert our internal S3XML data format to PDF or XLS using codecs: * https://github.com/flavour/eden/blob/master/modules/s3/codecs/pdf.py * https://github.com/flavour/eden/blob/master/modules/s3/codecs/xls.py I would focus on just one of these to keep scope feasible. - personally I'd be more interested in XLS as this gives users a more useful tool to explore the data than just a static PDF. Options: * parallel matplotlib method (extend modules/s3chart.py) * server-side webkit browser to take a screenshot (!ClimateDataPortal used this solution) See Also * [wiki:BluePrint/Chart] * [wiki:BluePrint/Reporting] == Optimisation == Focal Persons: Dominic König, Fran Boon Performance enhancements to handle 10,000's of records: * Server-side clustering of map features with different report-style aggregate representations * Optimize Report generation (probably beyond GSoC) * Data Warehouse? (Flattened tables structure, NoSQL) * Stored Procedures? Concurrency: * Non-blocking web server setup: http://uwsgi-docs.readthedocs.org/en/latest/Gevent.html See Also: * [wiki:Projects/Advanced/Optimisation] == Rostering == Focal Persons: Dominic König * Staff/Volunteer Roster GUI * a prototype was done for GSoC 2012 which can be used as a point of reference - what was good about it? What was bad? * Bad was that it didn't integrate sufficiently with the framework * Development of/Integration with a wCSP engine for automatic rostering == Survey: Template GUI == Focal Persons: Graeme Foster * Currently the templates are added via prepopulated data, to create the prepop data it can be done via a script and manual editing * Create a frontend that will step the user through the script logic so that new templates can be created * Extract this data for use in prepop * Allow the import of template details outside of the prepop process. This will us the standard import process but, will add a special interface because there are two files that may be imported the questions and then the layout and the import order is important. * Allow the removal of a template (which should be permissible if it has not been used) * Support the editing of the wording of the template, this would be simple changes to the phrasing of a section a question or a possible reply. (not structural changes to the questions) '''GSoC Students:''' If you are interested in this project then you will benefit by looking at the [https://github.com/flavour/eden/blob/master/static/scripts/tools/ADAT_CSV_generator.py script]. Sample files that can be converted by this script can be requested from me via the mailing list or IRC, and then you should learn how to generate your own file that will be converted by this script. That will then give you the background knowledge required to start on and be successful with this project. Could [http://www.kobotoolbox.org/products/koboform KoboForm] UI be used with our back-end? Or could we just borrow UI ideas? == Survey: Reporting == Focal Persons: Graeme Foster * Currently the reports in the survey module use matplotlib, now that Sahana has it's own reporting framework S3Report2 it should be used to deliver this information, so that we have the ability to explore data interactively client-side, users see a consistent interface & developers have less code to maintain. * Write test script to for current functionality * Re-engineer code to use S3Report2 direct or the underlying S3PivotTable if more control is needed * Enhance S3PivotTable/S3Report2 where appropriate == Automatic Test Framework == Focal Persons: Michael Howden, Dominic König See: * [wiki:QA] * BluePrint/Testing * BluePrint/Testing/Load * BluePrint/Testing/TestSuite Ideas * Automatically create Dummy Data * Extend Role Tests * Run Selenium Tests & Smoke tests in multiple templates with multiple user accounts * Clearer Error Messages * Simplify Selenium Tests and make them easier to read and more robust. * Adapt tests to meet needs of evolving CI Server (SysAdmin/ContinuousIntegration) * Load Tests * Selenium Test show web2py error messages * Write Functional Tests using Selenium * Figure out how to do unit tests against a web application. == Social Media == Focal Persons: Fran Boon, Michael Howden, Connie White See: * BluePrint/SocialMedia#Projects * Projects/Advanced/MediaFeed === Facebook Integration === * Be able to read Facebook Posts from Sahana * use the Facebook Channel in {{{modules/s3db/msg.py}}} * Complete the Posting to Facebook * log messages in msg_facebook * make it easier for people to link a Sahana feed to a Facebook page: Documentation? Improved API? (fn to call onaccept) == GIS/Mapping == These are a number of GIS/Mapping Tasks to be done: * complete [BluePrintMobileClient#Mapping support for Mobile devices] * [wiki:BluePrint/GIS/OSMLayers OpenStreetMap PoI Layers] * Tracking people (John Smith was registered at Camp A, then collected food at Camp B, then volunteers at Site C), or Items (The shipment of Tarpaulins moved from site C to site D, where it was split, and then half of them have been distributed from site E) * Having a playable [wiki:BluePrint/GIS/TimePlay timeline slider] * Photo Bulk Uploader * Add EXIF support to geolocate images * Provide opportunity to add Metadata for all files * Add support for GPX Track timestamp correlation * More: BluePrint/GIS === Wikipedia Locations === Allow Sahana to import Wikipedia locations easily: both Hierarchy & Lat/Lon info * Probably more about work in Wikipedia/Geohack to get these stored in a strctured way & an API to extract the data. * Similar need to: http://wiki.rhok.org/Population_Centers_in_Disaster == Framework == * Replace jquery.cluetip.js with jquery.ui.tooltip.js (1.9 once released) == Tweet / SMS Parsing == Allow users to provide info via tweets or SMS messages * Parse formatted SMS messages or tweets (e.g. key value pairs). * Detect when message interpretation isn't certain and dispatch to a human reader (use existing "task" database, or Job Jar, below!). * Create database records from parsed messages. * BluePrintMessagingModule#Twitter * [wiki:NCSU#Projectsbeingworkedon NCSU Students] working on this * BluePrintMessagingModule#Micro-Syntax == Run Eden on a smart phone == #Mobile * See BluePrintMobileClient. * Alternate Layout for web view: [wiki:BluePrintCSS#MobileView] * [wiki:NCSU#Projectsbeingworkedon NCSU Students] working on this * Android Application: MobileAndroid * [wiki:NCSU#Projectsbeingworkedon NCSU Students] working on this == New Use Cases == Extensions to support: * Organizations such as [http://www.tenthousandvillages.com/ Ten Thousand Villages] assist craftspeople in disadvantaged countries organize to purchase supplies and market their crafts. What would such a crafts coop need to help run their business? What's missing from Eden to support this? * [http://www.maitri.org Maitri] provides support to victims of domestic violence. * They have a [http://www.maitri.org/res_usa.html page listing resources] but part of the page is organized topically and the rest by location. Consider how to store the resource data to allow displaying it either by topic or by location, and also make it searchable. * Look at the [http://www.maitri.org/secure/join.html volunteer page]. Compare with the existing (incomplete) volunteer module. What additional features might Maitri need to use the Eden volunteer module? == Flash Drive Version == Extend and configure the flash drive version to support: * Different sets of Prepop Data * Multiple user access configuration * Sync with online instance (auto-set up) == OSM Importer UI == === Current Status === * There is an import stylesheet for .osm files: static/xslt/import/osm.xsl * This currently has hard-coded mappings between OSM admin levels & Sahana admin levels * http://wiki.openstreetmap.org/wiki/Tag:boundary%3Dadministrative#10_admin_level_values_for_specific_countries * There is an S3ImportPOI method to allow features to be extracted from either an uploaded .osm file or an OSM mirror: * https://github.com/flavour/eden/blob/master/modules/s3/s3gis.py#L7938 * this is accessed from the 'Import from OpenStreetMap menu option in /eden/gis === Enhancements Needed === 5 separable Tasks here: 1. We need to add the Sahana UUID to these features in the OSM database. * This is needed so that we can do deduplication on subsequent imports (the OSM ID is insufficient for this as it can change over time) * This means we need to read the OSM credentials (currently a deployment_setting in 000_config.py) & do a REST API call to the main OSM database with the modified feature(s) 2. Move the OSM credentials from a global 000_config.py deployment_setting to a per-user setting (component resource of db.auth_user) 2. We would like to be able to control which features are imported from this UI (currently this is a deployment_setting in [000_]config.py - that should become the list of avaialble options rather than the absolute list) 3. We would like to be able to download features for a Location from main OSM database * We want to use the Polygon (falling-back to BBOX (min_lat, max_lon, etc)) to trigger a download of a suitably-filtered .osm file via the [http://wiki.openstreetmap.org/wiki/Overpass_API Overpass API] 4. We would like to be able to download features for a Polygon * Provide a map window with a Polygon selector tool which then triggers a download of a suitably-filtered .osm file via the [http://wiki.openstreetmap.org/wiki/Overpass_API Overpass API] == [wiki:Projects/Advanced/Timeline Timeline] == == [wiki:Projects/Advanced/Usability Usability] == == [wiki:Projects/Advanced/JobJar Job Jar / Human Intelligence Task Support] == Focal People: Pat Tressel,... * [wiki:Projects/Advanced/JobJar] * [wiki:BluePrintHumanIntelligenceTasks] == Resource Planning == Interfacing with [http://www.optaplanner.org OptaPlanner]. * Assigning volunteers to shifts (e.g. at facilities - the Mayon usecase). - This would extend the manual Rostering tool) - http://docs.jboss.org/drools/release/6.1.0.Final/optaplanner-docs/html/useCasesAndExamples.html#nurseRostering * Hospital Bed Assignment - http://docs.jboss.org/drools/release/6.1.0.Final/optaplanner-docs/html/useCasesAndExamples.html#pas * Routing trucks to deliver goods to shelters (came up in Japan but is an obvious extension to Req/Inv Send) - http://docs.jboss.org/drools/release/6.1.0.Final/optaplanner-docs/html/useCasesAndExamples.html#vehicleRouting Either [http://docs.jboss.org/drools/release/6.1.0.Final/optaplanner-docs/html/integration.html#integrationWithSoaAndEsb create a REST API for OptaPlanner] using [http://camel.apache.org/optaplanner.html Camel] or use [http://py4j.sourceforge.net Py4J] to [http://docs.jboss.org/drools/release/6.1.0.Final/optaplanner-docs/html/integration.html#integrationOverview call the Java classes] from Python. == [wiki:BluePrintWebSetup Web Setup] == Extend the Deployment co-app ({{{modules/s3db/setup.py}}}) to do on-going Configuration (editing {{{models/000_config.py}}}, handling DB migrations) == [wiki:BluePrint/Logistics Logistics] == == Build library(ies) to integrate Emergency Data Exchange Language Distribution Element == Proposed by: [http://lirneasia.net/profiles/nuwan-waidyanatha | Nuwan] [[BR]] '''Specific : '''[http://docs.oasis-open.org/emergency/edxl-de/v1.0/EDXL-DE_Spec_v1.0.pdf | EDXL-DE] is the final wrapper (envelope) of all [http://en.wikipedia.org/wiki/EDXL | EDXL] data package. We may be delivering a EDXL Resource Management (RM) information and Situational Reporting (SITREP) information to the managers of several emergency organizations. The DE will contain who, when, and where those RM and SITREP data parcels would be delivered. Every EDXL message (data package) must carry this information. Otherwise, it cannot use available distribution methods and would need to rely on its own protocol; that's not very user friendly. [[BR]] '''Measurable: ''' It is purely a coding task that involves playing with XML and developing a class, possibly within or using the '''??? 3R**** ???''' framework It is basically a set of procedures for packing and unpacking EDXL-DE wrapped data. [[BR]] '''Attainable: ''' run through the steps [[BR]] Step 1 :: select one of the existing EDXL-based applications in Eden, I recommend the EDXL-RM [[BR]] Step 2 :: discuss each of the attributes/elements; then determine how EDXL-DE would be added to EDXL-RM as a pop-up GUI. [[BR]] Step 3 :: create some example XML files to get a feel for the inputs and outputs [[BR]] Step 4 :: write code to add to take the inputs EDXL-RM and EDXL-DE to package the data for delivery [[BR]] Step 5 :: test the code, fix bugs, and generalize the functions [[BR]] Step 6 :: Apply to EDXL-HAVE and EDXL-SITREP to generalize the library [[BR]] '''Relevant : ''' Applies to Sahana Interoperability policy. Present developments to investigate are: HAVE, RM, and SITREP http://www.oasis-open.org/standards#edxl [[BR]] '''Time-bound: ''' If steps 1 - 4 are completed that can be a full accomplishment; additional work is a bonus [[BR]] '''Evaluate : ''' Produce XML files with the EDXL-DE element appended to the EDXL-RM, SITREP, or HAVE[[BR]] '''Reevaluate: ''' Use an API to add and strip the EDXL-DE to any EDXL data standard. Use the same set of XML files to run through this process == Map elements between EDXL-SITREP and EDXL-RM == Proposed by: [http://lirneasia.net/profiles/nuwan-waidyanatha | Nuwan][[BR]] '''Specific : ''' There are several elements within the [http://tinyurl.com/bn9nxty | EDXL-SITREP] data standard that are identical to that of [http://docs.oasis-open.org/emergency/edxl-rm/v1.0/EDXL-RM-SPEC-V1.0.pdf | EDXL-RM]. The objective is to create a descriptive table of those elements that a programmer can use to develop a set of procedures to strip the RM data from SITREP to manage records in the relational database [[BR]] '''Measurable : ''' Requires diligently investigating each and every data element then comparing them with the two data standards SITREP and RM. Requires knowledge of data types and XML. If not, this exercise will help the student learn about XML and data standards [[BR]] '''Attainable : ''' [[BR]] Step 1 :: Go through EDXL-RM and EDXL-SITREP documentations [[BR]] Step 2 :: create example files to get an understanding of the data and structure [[BR]] Step 3 :: develop the table with RM and SITREP elements with a description [[BR]] Step 4 :: develop a simple XSL file to strip the RM data from SITREP [[BR]] '''Relevant : ''' Applies to Sahana interoperability policy. Given that Eden does support resource and incident management, it is important to derive the response resources and resource requirements from situational reports. This function would help automate some of those data extraction functions. That requires integrating the SITREP and RM components with underlying Eden schema [[BR]] '''Time-bound : ''' The exercise of understanding the data standards and mapping the elements should not take more than one week. developing a XSL to test the mapping may take another week, depending on the level of expertise with XML. (NB this is far too large a task for GCI. -- Pat)[[BR]] '''Evaluate : ''' The mapping table is the determining output. However, the example RM and SITREP files are also required. [[BR]] '''Reevaluate : ''' If the XSL transformation is developed, then the mapping can be tested with the sample RM and SITREP files [[BR]] == See Also == * [wiki:Contribute/Code] * [wiki:BluePrint BluePrints]