S3XRC Mini-Tutorial

• S3XRC Recipes and Tutorials

A report function for your resource

Use-case

You have a module xxx and within it a resource yyy, which has a number of components. You're providing CRUD functions for the resource and its components using s3_rest_controller().

Now you want to provide a reporting function for this resource, where the user can specify report parameters in a form and then get a summary report auto-generated:

Our example: Your resource contains a "timestmp" field (type datetime) and a field "cost" (type double), and in your reporting function the user shall be able to select a time interval, and get a report of the sum of all "cost"s within the selected time interval auto-generated.

This tutorial shows you how this can be integrated into the REST interface of your resource.

Techniques

Create a custom method handler

Reporting is a method of your resource, the reporting function will therefore be implemented as a custom REST method handler function.

To achieve this, add the reporting function to your model file, and specify it as a method to your resource using s3xrc.model.set_method, like:

def s3_xxx_yyy_report(r, **attr):

    # Report generating code goes here

s3xrc.model.set_method("xxx", "yyy", method="report", action=s3_xxx_yyy_report)

A custom method handler has to take two arguments:

• r is the respective S3Request object, which represents the current request
• attr is the same dict of named arguments that have been passed to shn_rest_controller

Once you have done this, you can invoke your method handler from the URL:

http://localhost:8000/eden/xxx/yyy/report

But it even understands URLs like:

http://localhost:8000/eden/xxx/yyy/1/report
http://localhost:8000/eden/xxx/yyy/report?yyy.id=1,2,3
http://localhost:8000/eden/xxx/yyy/report?yyy.field1__like=abc

meaning: it already implements a RESTful API for your reporting function, e.g. does the parsing/validating of the URL for you, implements the full range of URL queries for your resource and so forth. Nothing you need to care about.

Provide different report formats

In a later improvement, you want perhaps to provide the report in various formats, hence you need to know what format the user has requested. The best way is to check for the format that has been specified in the URL.

To know which format has been specified, simply use r.representation inside the method handler:

def s3_xxx_yyy_report(r, **attr):

    if r.representation == "html":
        # HTML report generating code goes here:
        # "output" gets a dict of variables to be passed to the view

        response.view = "xxx/yyy_report.html" # <== choose the view template

        return output # <== return the output to the view

    elif r.representation == "xls":
        # XLS report generating code goes here
        # "output" gets the XLS contents

        return output # <== return the output to the view

    elif r.representation == "pdf":
        # PDF report generating code goes here
        # "output" takes the PDF contents

        return output # <== return the output to the view

    elif r.representation == "svg":
        # SVG report generating code goes here
        # "output" takes the SVG as a string

        return output # <== return the output to the view

    else:
        # Unsupported format
        raise HTTP(501, body=s3xrc.ERROR.BAD_FORMAT)

s3xrc.model.set_method("xxx", "yyy", method="report", action=s3_xxx_yyy_report)

Now your method handler recognises the requested format as specified in the URL by either a filename extension or the format variable:

http://localhost:8000/eden/xxx/yyy/4/report.xls
http://localhost:8000/eden/xxx/yyy.pdf/6
http://localhost:8000/eden/xxx/yyy/report?format=svg

It is important to raise an HTTP error status in case of an unsupported format rather than providing a nicely formatted error message: while an interactive user who just clicks on links and buttons would rarely request something else than the interactive formats we expose to him, it would rather be a non-interactive client which requests other formats - and that would hardly understand a nicely formatted error message but expect a proper HTTP status in the response header.

Get at the data

How can you find out which data have to be processed by your reporting function, and yet more important: how can you get at them?

r (the S3Request object) contains an interface to your resource as S3Resource object in: r.resource

And this is what you can use to easily access the resource data. Some examples:

def s3_xxx_yyy_report(r, **attr):

    resource = r.resource # <== Get the resource

    if r.representation == "html":

        resource.load()
        rows = resource.records() # <== get all records

    elif r.representation == "xls":

        for record in resource: # <== interate through the records
            ...

    elif r.representation == "pdf":

        # Access components record-wise:
        for record in resource:
            component_set = resource(record, component="component_name")
            for component_record in component_set:
                ...

    elif r.representation == "svg":

        # Modifying the resource query before accessing records
        filter = resource.get_query()
        filter = filter & (db.xxx_yyy.field5 == "value")
        resource.build_query(filter = filter)

        # ...and then iterate through the records:
        for record in resource:
            ...

    else:
        # Unsupported format
        raise HTTP(501, body=s3xrc.ERROR.BAD_FORMAT)

s3xrc.model.set_method("xxx", "yyy", method="report", action=s3_xxx_yyy_report)

Step by step: total costs within a time interval

Your resource contains a "timestmp" field (type datetime) and a field "cost" (type double), and now the user shall select a time interval in aform, and your report function shall provide the sum of all "cost" for those records with a timestmp within the selected time interval.

Let's go:

1. Create a method handler

First of all, we implement our method handler as mentioned before:

def s3_xxx_yyy_report(r, **attr):

    # Report generating code goes here

s3xrc.model.set_method("xxx", "yyy", method="report", action=s3_xxx_yyy_report)

2. Add representations

Then we add the HTML representation:

def s3_xxx_yyy_report(r, **attr):

    if r.representation == "html":
        output = dict()
        return output

s3xrc.model.set_method("xxx", "yyy", method="report", action=s3_xxx_yyy_report)

3. Add the form

The next step would be to provide a form to select the first and the last date of the interval. This would look like that:

    if r.representation == "html":

        # Filter form
        form = FORM(
                 TABLE(
                   TR(
                     T("Date from:"),
                     INPUT(_type="text", _name="date_from", _class="date", _value=datetime.now().date(), requires=IS_DATE()),
                     T("until:"),
                     INPUT(_type="text", _name="date_until", _class="date", _value=datetime.now().date(), requires=IS_DATE())
                   )
                 )
               )

        output = dict(form=form)

        if form.accepts(request.vars, session, keepvalues=True):
            # Processing of the form data goes here

        return output

4. Process the form data

From this, we need to process the form data into two dates. We use datetime for that:

        if form.accepts(request.vars, session, keepvalues=True):

            from datetime import date, datetime, time

            # Date from
            if form.vars.date_from:
                from = datetime.combine(form.vars.date_from, time(0, 0, 0))

            # Date until
            if form.vars.date_until:
                until = datetime.combine(form.vars.date_until + timedelta(days=1), time(0, 0, 0))

5. Extend the resource query

To select the corresponding records, we need to extend our query:

        if form.accepts(request.vars, session, keepvalues=True):

            # Get the initial query from the resource
            # => this implements authorisation as well as URL queries
            resource = r.resource
            table = resource.table
            query = resource.get_query()

            from datetime import date, datetime, time

            # Date from
            if form.vars.date_from:
                from = datetime.combine(form.vars.date_from, time(0, 0, 0))
                date_query = (table.timestmp >= from)

            # Date until
            if form.vars.date_until:
                until = datetime.combine(form.vars.date_until + timedelta(days=1), time(0, 0, 0))
                q = (table.timetmp <= until)
                if date_query:
                    date_query = date_query & q
                else:
                    date_query = q

            # Add our filter
            query = query & date_query

6. Retrieve the sum of costs

From there, it is easy to select the sum of the "cost" field:

        if form.accepts(request.vars, session, keepvalues=True):

            # Get the initial query from the resource
            # => this implements authorisation as well as URL queries
            resource = r.resource
            table = resource.table
            query = resource.get_query()

            from datetime import date, datetime, time

            # Date from
            if form.vars.date_from:
                from = datetime.combine(form.vars.date_from, time(0, 0, 0))
                date_query = (table.timestmp >= from)

            # Date until
            if form.vars.date_until:
                until = datetime.combine(form.vars.date_until + timedelta(days=1), time(0, 0, 0))
                q = (table.timetmp <= until)
                if date_query:
                    date_query = date_query & q
                else:
                    date_query = q

            # Add our filter
            query = query & date_query

            # Get the total costs
            costs = table.costs.sum()
            total = db(query).select(costs)

            # Build the result string
            result = "%s: %s" % (T("The total costs of the selected projects are"), total)

            output.update(result=result)

7. Add a view

Done with the method handler! Here at a glance:

def s3_xxx_yyy_report(r, **attr):

    if r.representation == "html":

        # Filter form
        form = FORM(
                 TABLE(
                   TR(
                     T("Date from:"),
                     INPUT(_type="text", _name="date_from", _class="date", _value=datetime.now().date(), requires=IS_DATE()),
                     T("until:"),
                     INPUT(_type="text", _name="date_until", _class="date", _value=datetime.now().date(), requires=IS_DATE())
                   )
                 )
               )

        output = dict(form=form)

        if form.accepts(request.vars, session, keepvalues=True):

            # Get the initial query from the resource
            # => this implements authorisation as well as URL queries
            resource = r.resource
            table = resource.table
            query = resource.get_query()

            from datetime import date, datetime, time

            # Date from
            if form.vars.date_from:
                from = datetime.combine(form.vars.date_from, time(0, 0, 0))
                date_query = (table.timestmp >= from)

            # Date until
            if form.vars.date_until:
                until = datetime.combine(form.vars.date_until + timedelta(days=1), time(0, 0, 0))
                q = (table.timetmp <= until)
                if date_query:
                    date_query = date_query & q
                else:
                    date_query = q

            # Add our filter
            query = query & date_query

            # Get the total costs
            costs = table.costs.sum()
            total = db(query).select(costs)

            # Build the result string
            result = "%s: %s" % (T("The total costs of the selected projects are"), total)

            output.update(result=result)

            # Select the corresponding view:
            response.view = "xxx/report.html"

    else:
        raise HTTP(501, body=s3xrc.ERROR.BAD_FORMAT)

    return output

s3xrc.model.set_method("xxx", "yyy", method="report", action=s3_xxx_yyy_report)

We have added a line to raise an HTTP error in case an unsupported format gets requested, and added a line to select a proper view template "xxx/report.html", which would look like that:

{{extend "layout.html"}}
<div class='form-container'>
{{try:}}{{=form}}{{except:}}{{pass}}
</div>
<div id='table-container'>
{{try:}}{{=result}}{{except:}}{{pass}}
</div>

...and that's it.

A View of Sahana-Eden

Their are a number of bits and pieces that make up the view and understanding what they are and how to interact with them is essential when developing Sahana-Eden. So taking the screenshot below of a standard screen, listing all of the instances of a particular resource. The various pieces that make up the screen are:

• Main menu
• Sub menu or the menu specific to the model
• Title
• Action Buttons
• Sub title
• dataTable

The menus (the first two bits) are common for all screens with the model specific menu being defined in the models/01_menu.py file. The rest of the view lies within the content div.

The screen title and subtitle

The first element in the content is the Title, this is a simple H2 tag but it has been nicely formatted by the style sheet to stand out. The title can set up with a simple assignment as follows:

output["title"] = "List of Assessment Series" Within the controller logic the output variable is returned by the call to the s3_rest_controller() and it is also available in the post processor function, so this line of code can be included in either the preprocessor or after the call to s3_rest_controller().

The output variable is a simple dictionary and so setting up a key in this dictionary with the value of “title” is sufficient. But this piece of code is not at all friendly for people who work in another language. The string needs to be translatable which is done by passing it to the T() function.

output["title"] = T("List of Assessment Series") The same string may be used in a number of different screens and so to help with consistency the string can be defined in one place and then reused. This is done in the model

{{{s3.crud_strings[tablename] = Storage(

title_list = T("List of Assessment Series"), )}}}

The tablename variable has been set up to the name of the table. Other strings can be added here, so it becomes a convenient holder for all the string to be used by the model. Finally, because the title is such a common requirement in a form if a title_list value is set up for a list view then that string will be automatically used. That means that the output["title"] assignment is not necessary unless you want to use a different string.

That's quite a bit of background to only produce a title, but what you have learnt for the title is the same for the subtitle. The special crud name for a subtitle belonging to a list view is subtitle_list, so all you need to do is define that in the s3.crud_strings variable, and should you want to override the default value then assign the new value to output["subtitle"].

Crud generated buttons

Their are a number of buttons that are automatically added to the screen depending upon the context. For a list the add resource button will be displayed. If you want this button then you don't need to do anything. However, if you wish to suppress the button from being displayed then the following will help:

    s3mgr.configure(tablename,
                    listadd=False,
                   )

This code will go in the controller before the call to s3_rest_controller().

dataTable

This displays all of the resources and adds buttons to open (or edit) and delete the selected resource. Again all of this comes as part of the framework and can whilst the default settings are often sufficient it is possible to adjust the the way the dataTable looks by changing various settings.

Resource Details

Opening a resource on the dataTable will then display details about this resource. From the workflow perspective a resource consists of a number of actions that can be performed on the resource. These actions might, for example display the resource data, generate more data or trigger off more action. The screenshot below of a standard screen, showing the details of a particular resource. The various pieces that make up the screen are remarkably similar to the first screen. The main additions are:

• rHeader
• tabs (within the rHeader)

This particular screen also has some extra details after the subtitle. This was added by creating a new view based on the standard list view but adding an extra slot to hold the help information about the list.

rHeader

The rHeader is an area on the screen to display summary details about the resource. The details of the rHeader will be set up in a function and the function can then be passed as a parameter to s3_rest_controller(). The rHeader callback function receives an S3Request object and should return the HTML which will be displayed on the screen. Typically the HTML returned would be a DIV object, with one HTML element within this being the tabs.

tabs

These are a convenient mechanism for displaying the options (links) available, and where appropriate they can follow the workflow of the resource. The tabs are set up by building a list of tuples, each tuple consists of the label followed by the action. This list is then passed to rheader_tabs(), which renders it into HTML ready to be added to the rHeader DIV.

So the key to the tabs is the list of tuples, an example of one looks like this:

if auth.s3_has_permission("create", "survey_series"):
    tabs = [(T("Series Details"), None),
            (T("New Assessment"), "newAssessment/"),
            (T("Completed Assessments"), "complete"),
            (T("Series Summary"),"summary"),
            (T("Series Graph"),"graph"),
            (T("Series Map"),"map"),
           ]

The labels that appear on the screen are wrapped in a T() so that they can be translated. The action is either a string that relates to the type of action that will be performed, or None which means that the default action will be performed. The actions though are the real power of the tabs. They can be any from:

• A default value
• A component of the resource,
• Another controller or
• A method.

default tab action

The default action will be an edit of the full resource details if the user has sufficient permissions otherwise it will be a read of the resource details.

component tab action

In the example given the database has a table called survey_complete and so complete is a component of the series resource. This will then display, in a list (dataTable), all the complete records that belong to the resource that had been selected. The component records can then be edited or deleted and this is all managed with the main resource UI, providing a consistent interface for the user.

another controller tab action

Not all parts of the workflow are coupled as tightly as a resource component relationship. For such cases it might be easier to switch to a completely different controller. To ensure that the consistency of the interface is retained the same rHeader should be used, but as already explained setting that up is a cinch. In the example above newAssessment is an example of an external controller. Note that in the tab it is followed by a slash.

method tab action

The method will be called by the same controller so it will inherit the pre and post processing. It is typically a function that will work with the same table but needs to perform something beyond the basic CRUD actions. In the example above summary, graph and map are all method actions. Because they are not controller functions they need to be set up a little differently. The method will be set up with the models (not the controllers) and the outline of the method is:

def seriesMap(r, **attr):

    # retain the rheader
    rheader = attr.get("rheader", None)
    if rheader:
        rheader = rheader(r)
        output = dict(title=T("Do Something with the Template"), rheader=rheader)
    else:
        output = dict()
    crud_strings = response.s3.crud_strings["survey_series"]

    # code specific to this method - creates the map

    output["title"] = crud_strings.title_map
    output["subtitle"] = crud_strings.subtitle_map
    output["form"] = form
    output["map"] = map

    response.view = "survey/series_map.html"
    return output

This method now needs to be linked to the controller, again this code should be in the model.

s3mgr.model.set_method("survey", "series", method="map", action=seriesMap)

Because the method gets and then stores the rHeader it is possible for the call to originate from different controllers but the UI will remain consistent.