[[TOC]] = Blueprint - Data Analysis, Visualisation and Reporting = == Description == == Requirement == We need to be able to generate Pretty & Informative Reports from the database. e.g. for BluePrintImpact These can be implemented as an [wiki:S3XRC_ResourceReport S3XRC Custom method] === Filtering === We will want to be able to customise reports by allowing the user to filter the data (this could include fields in the resource, components and reference resources) An example is: eg http://demo.drrprojects.net/drrp/drrpp/project/search Code: https://code.launchpad.net/~michael-howden/sahana-eden/adpc ==== Location Filtering ==== We need to be able to fitler by a location so that it shows data in all locations which are children of that location (eg villages in a district) == Use-Cases == == Design == === Custom s3xrc method === The custom s3xrc Would allow quick visualisations of resources on the server. There would need to be vars specifying which variables go in which axis in each resource. === Report model/view/controller === This module could provide the user with the ability to create and embed graphs in their instance of sahana. The idea is to provide the user with the ability display any set of data; it need not be tied to a resource or may be a more complex combination of variables than provided in the sahana framework. The graphs could be in svg format allowing for interactivity. The files themselves could also be stand alone so that they could be easily exported and saved locally. The basic workflow would be: * Create a graph with a web-form or file upload using report/scatter/create. There numerous settings that could be altered to create different a looks and feel. * Or it can be created with a call to the url with the data as vars, such as report/scatter/create?data={"x": [1, 2, 3], "y": [3, 4, 3], "title": "Test Title"}&settings={"width": 300, "height": 200} * An entry gets created in the database for the graph. It can be retrieved an and inserted into a web-page in an object or embed tag, such as * These could appear inside popups on the map * Or, the graphs can be represented without an entry in the database using report/scatter/create?view.svg={"x": [1, 2, 3], "y": [3, 4, 3], "title": "Test Title"}&settings={"width": 300, "height": 200} == Implementation == === Misc Existing Functionality === (To be sorted) * DeveloperGuidelinesSVG * ReportLab === Charts === For quick visualisations in the browser, we have included [http://www.jqplot.com jqPlot]. We have included the [http://healthscapes.org Healthscapes] !SaVaGe library to easily output SVG graphs: * [wiki:DeveloperGuidelinesSVG] * We should have a recommended tested SVG plugin for Internet Explorer < 9 * http://www.savarese.com/software/svgplugin/ (not 64-bit) * http://www.adobe.com/svg/viewer/install/ (Not maintained - e.g. no Windows 7 support) * Background page: http://www.w3.org/Graphics/SVG/IG/wiki/SVG_Plugin_for_IE T2 includes a basic barchart function: {{{ def barchart(data,width=400,height=15,scale=None, label_width=50,values_width=50): if not scale: scale=max([m for n,c,m in data]) if not scale: return None return TABLE(_class='barchart', *[TR(TD(n,_width=label_width,_style="text-align: right"), TABLE(TR(TD(_width=int(m*width/scale),_height=height, _style='background-color:'+c))),TD(m,_width=values_width), _style="vertical-alignment: middle") for n,c,m in data]) }}} * [http://matplotlib.sourceforge.net/faq/howto_faq.html#matplotlib-in-a-web-application-server Matplotlib] can generate visualisations * [http://www.web2pyslices.com/main/slices/take_slice/12 Cairo] can generate nice charts from the back-end === Map === We want to be able to provide reports suitable for displaying inside popups on Map Polygons: * [wiki:BluePrintGISFeatureLayers] === PDF === For more formal reports (which can also be retrieved via webservices using the PDF representation of the REST Controller), we currently use [http://reportlab.org ReportLab]. This also supports Graphics (via [http://www.pythonware.com/products/pil/ PIL]) === Spreadsheets === Excel files are currently written using [http://pypi.python.org/pypi/xlwt xlwt]. == References == === Other Available Tools === === PDF === Other options: * http://www.reynoldsftw.com/2009/02/6-jquery-chart-plugins-reviewed/ * Recommends [http://code.google.com/p/flot/ Flot]: "''From an interaction perspective, Flot by far will get you as close as possible to Flash graphing as you can get with jQuery.''" * [http://www.xhtml2pdf.com Pisa] uses [http://reportlab.org ReportLab] to generate PDF from HTML * [http://code.google.com/p/jspdf jsPDF] uses pure !JavaScript to create PDFs * [http://geraldo.sourceforge.net Geraldo] builds on !ReportLab, and we used to use this, but no longer * Use S3XRC & hence XSLT to generate XML-FO which can be converted to PDF * http://xmlgraphics.apache.org/fop/ * http://pypi.python.org/pypi/zopyx.convert/ * [http://osdir.com/ml/python.reportlab.user/2003-10/msg00206.html Recommend ReportLab over FOP] ==== Spreadsheets ==== * [http://sourceforge.net/projects/pyexcelerator PyExcelerator] * Docs on Python Excel libraries: http://www.python-excel.org/ * Thread recommending xlwt: http://groups.google.com/group/web2py/browse_thread/thread/375307e92df40ca8 * [http://appyframework.org/pod.html Pod] to create Open Document Format ==== Flash ==== Could be something to use Flash? (although best-avoided, if possible, to minimise client support) * PyAMF: http://pyamf.org * Web2Py integration: http://mdp.cti.depaul.edu/AlterEgo/default/show/22 * !BirdEye: http://code.google.com/p/birdeye/ * !FusionCharts Free: http://www.fusioncharts.com/Free/ * Open Flash Chart: http://teethgrinder.co.uk/open-flash-chart-2/ * from Python: http://code.google.com/p/open-flash-chart-python/ === Sahana Phase 2 === http://wiki.sahanafoundation.org/phase2/doku.php/doc:reports:english ---- BluePrints