Version 40 (modified by 13 years ago) ( diff ) | ,
---|
Table of Contents
Blueprint - Data Analysis, Visualisation and Reporting
Description
Sahana Eden can be used to collect and manage a large variety of data. For this data to add value it needs to be processed and shared in a way in which it can present the information people need to make decisions and plan activities. To achieve this Sahana Eden should be able to produce reports which analyse and visualize data in a flexible, user friendly and configurable way.
Requirements
Types of Reports
- List
- Pivot Tables
- Bar Charts
- Pie Charts
- Scatter Charts
- Maps
- Timeline
- Motion Chart
What else?
Functionality
- The ability to easily switch between displaying the same data on various types of reports. eg:
- List -> Map
- Pivot Table -> PIe Chart
- Map -> Graph ( for a particular location on the map )
- Drill down within an report to set search criteria, eg:
- Select a single bar in a chart
- Select a point, box, polygon on a map
- Select a wedge in a pie chart
- Select a single bar in a chart
- Export Reports
- Tables -> XLS
- Maps/Graphs -> PNG
- All report types should be fully integrated with the S3Search module, to allow results to be filtered.
- Provide documentation on how to integrate data from Sahana Eden with other reporting & visualisation tools & APIs
Report Gallery
Integration with ADAT
e.g. for BluePrintImpact
These can be implemented as an S3XRC Custom method
Use-Cases
Design
Design Options
Custom S3XRC Method ===
The custom S3XRC would allow reporting for Sahana Eden resources. Vars could be be used to specify how the report is generated, eg:
- What type of report to generate.
- What fields would be represented on what graph axises.
Chart Module
OLD - this section may now be redundant, but my provide some ideas This module could provide the user with the ability to create and embed charts in Sahana Eden. The idea is to provide the user with the ability display any set of data; it need not be tied to a resource or may be a more complex combination of variables than provided in the Sahana Eden framework. The graphs could be in svg format allowing for interactivity. The files themselves could also be stand alone so that they could be easily exported and saved locally.
The basic workflow would be:
- Create a charts with a web-form or file upload using report/scatter/create. There numerous settings that could be altered to create different a looks and feel.
- Or it can be created with a call to the url with the data as vars, such as report/scatter/create?data={"x": [1, 2, 3], "y": [3, 4, 3], "title": "Test Title"}&settings={"width": 300, "height": 200}
- An entry gets created in the database for the graph. It can be retrieved an and inserted into a web-page in an object or embed tag, such as <object data="/eden/report/scatter/get.svg?id=1" />
- These could appear inside popups on the map
- Or, the graphs can be represented without an entry in the database using report/scatter/create?view.svg={"x": [1, 2, 3], "y": [3, 4, 3], "title": "Test Title"}&settings={"width": 300, "height": 200}
Design Notes
- It may be useful to identify a common set of variables which are needed to generate different Types of Reports
Variable | List | Pivot Tables | Bar Charts | Pie Charts | Scatter Charts | Maps | Timeline | Motion Chart |
Data (list) | n | 1 | 2 | 1 | - | n (layers) | 1 (although a record can be represented | 2 (Color & Size) |
X Axis | x | x | x | |||||
Y Axis | x | x | x | |||||
Implementation
Existing Functionality
- S3Chart (no documentation)
- BluePrintSearch
- BluePrintGISLocationsSearch
DeveloperGuidelines/SaveSearchAndSubscription BluePrintGISLocationsSearch S3Cube
- DeveloperGuidelinesSVG
- ReportLab
Charts
For quick visualisations in the browser, we have included jqPlot.
We have included the Healthscapes SaVaGe library to easily output SVG graphs:
- DeveloperGuidelinesSVG
- We should have a recommended tested SVG plugin for Internet Explorer < 9
- http://www.savarese.com/software/svgplugin/ (not 64-bit)
- http://www.adobe.com/svg/viewer/install/ (Not maintained - e.g. no Windows 7 support)
- Background page: http://www.w3.org/Graphics/SVG/IG/wiki/SVG_Plugin_for_IE
T2 includes a basic barchart function:
def barchart(data,width=400,height=15,scale=None, label_width=50,values_width=50): if not scale: scale=max([m for n,c,m in data]) if not scale: return None return TABLE(_class='barchart', *[TR(TD(n,_width=label_width,_style="text-align: right"), TABLE(TR(TD(_width=int(m*width/scale),_height=height, _style='background-color:'+c))),TD(m,_width=values_width), _style="vertical-alignment: middle") for n,c,m in data])
- Matplotlib can generate visualisations
- Cairo can generate nice charts from the back-end
Map
We want to be able to provide reports suitable for displaying inside popups on Map Polygons:
For more formal reports (which can also be retrieved via webservices using the PDF representation of the REST Controller), we currently use ReportLab. This also supports Graphics (via PIL)
Spreadsheets
Excel files are currently written using xlwt.
References
Other Available Tools
Other options:
- http://www.reynoldsftw.com/2009/02/6-jquery-chart-plugins-reviewed/
- Recommends Flot: "From an interaction perspective, Flot by far will get you as close as possible to Flash graphing as you can get with jQuery."
- Pisa uses ReportLab to generate PDF from HTML
- jsPDF uses pure JavaScript to create PDFs
- Geraldo builds on ReportLab, and we used to use this, but no longer
- Use S3XRC & hence XSLT to generate XML-FO which can be converted to PDF
Spreadsheets
- PyExcelerator
- Docs on Python Excel libraries: http://www.python-excel.org/
- Thread recommending xlwt: http://groups.google.com/group/web2py/browse_thread/thread/375307e92df40ca8
- Pod to create Open Document Format
Flash
Could be something to use Flash? (although best-avoided, if possible, to minimise client support)
- PyAMF: http://pyamf.org
- Web2Py integration: http://mdp.cti.depaul.edu/AlterEgo/default/show/22
- BirdEye: http://code.google.com/p/birdeye/
- FusionCharts Free: http://www.fusioncharts.com/Free/
- Open Flash Chart: http://teethgrinder.co.uk/open-flash-chart-2/
- from Python: http://code.google.com/p/open-flash-chart-python/
Sahana Phase 2
http://wiki.sahanafoundation.org/phase2/doku.php/doc:reports:english