wiki:S3/S3ResourceFilter

Version 12 (modified by Dominic König, 10 years ago) ( diff )

--

S3ResourceFilter

S3ResourceFilter is a class representing the DAL queries and Python filters to select the records for a resource.

Note: class names may still change as this functionality is still under development.

Overview

S3ResourceFilters are a part of S3Resources:

They contain the queries and filters to extract the data from the database.

At the time an S3Resource gets defined, a S3ResourceFilter is automatically added to it. This default filter (so called "master query") covers:

  • available records (i.e. not deleted)
  • accessible records (i.e. access permitted)
  • selected records by record ID and/or UID
  • joined records (e.g. components)

It is possible to extend this filter during the request:

  • by a URL query
  • by the S3Resource.add_filter() method

S3ResourceFilters can filter for both real fields in the database table as well as virtual (computed) fields.

Design

S3ResourceFilters consist of a hierarchical set of web2py Query and S3ResourceQuery objects, where:

  • Query instances represent the DAL queries ("query")
  • S3ResourceQuery instances represent virtual field filters ("vfltr")

It is possible to define S3ResourceQueries even for real fields. When extracting rows from the database table, those parts of a S3ResourceQuery which (only) concern real fields get automatically converted into Query instances and added to the DAL query. Furthermore, where S3ResourceQueries need joins, those get automatically added to the real query as well.

The process for the extraction of rows from the database is as follows:

This process supports pagination (slicing), although that would necessarily not only retrieve the requested page from the database, but all records which match the DAL query. Therefore, the virtual filters should always only be the "last mile", while the DAL query reduces the number of candidates to the necessary minimum.

Field Selectors

S3ResourceQuery instances are based on S3QueryField field selectors. A field selector is a string with the so-called "list_fields syntax":

alias.{foreign key$}field
  • alias is the alias of the component without prefix, e.g. "contact" for the pr_contact component (note that components may use aliases different from their table name). For the primary table of a resource (master table), "alias" is just the tablename without prefix.
  • foreign key is the name of a foreign key field in the table specified by alias, both together addressing the table referenced by this foreign key. It is possible to chain foreign key selectors to address tables which are more than one reference level away. Note that only real foreign key fields can be used (i.e. neither integer fields nor virtual fields).
  • field is the name of the field or a virtual field in the table addressed by alias (plus foreign key where present).

With this syntax, it is possible to address fields in the primary table, in component tables and in directly referenced tables.

It is not yet possible to address fields in tables linked by link tables - this is though subject to change in the near future.

URL queries

Filters can be specified in URLs simply by adding URL query variables in the form:

<field selector>__<operator>=<value>

(Note: these are two underscores between field selector and operator)

The following operators are supported:

OperatorMeaningComment
ltless than
leless or equal
eqequalcan be omitted (=default)
nenot equal
gegreater or equal
gtgreater than
containscontainmentgiven value belongs to field value (which is a list)
belongsreverse containmentfield value belongs to a list of given values
likestring containmentcase-insensitive

No negate an operator, simply append a ! to the operator like in:

?person.first_name__like!=Dominic

The value can be a single value, a comma-separated list of values.

Date-time-values must be specified as YYYY-MM-DDThh:mm:ss, dates as YYYY-MM-DD and times as hh:mm:ss. Other types use their standard Python notation. NONE specifies a null-value (undefined, Python None). Values in quotes are treated as strings, i.e.

?person.gender=NONE

specifies the value None, whereas

?person.gender="NONE"

specifies the string "NONE".

It is currently not possible to escape quotation marks in order to enclose them in strings - thus strings must not contain quotation marks.

For all operators except __belongs, any comma-separated lists of values are treated as alternatives (OR) unless the field contains a list as well. All operators except __like are case-sensitive.

Extending filters at the back-end

S3QueryField instances can also be generated at the back-end, simply by:

selector = S3QueryField(<field selector>)

Such selectors can then be used to create filter queries, using a syntax very similar to web2py queries:

afilter = S3QueryField("person.first_name") == "Dominic"
bfilter = ~(S3QueryField("person.first_name").like("Dominic"))  # negation by ~ operator

The equivalent query operators are:

Python OperatorURL Operator
<__lt
<=__le
==__eq
!=__ne
>=__ge
>__gt
.like()__like
.contains()__contains
.belongs()__belongs

These filter queries can also be joined together by & and | operators:

cfilter = (S3QueryField("person.first_name") == "Dominic") & (S3QueryField("contact.value") == "dominic@nursix.org")

Note: be diligent with brackets here (& and | operators take precedence).

To add a filter to a resource is as simple as:

myfilter = S3QueryField("person.first_name") == "Dominic"
resource.add_filter(myfilter)

If your filter involves a component table, you may need to add the respective joins. This can be done simply by specifying the component alias in the add_filter call:

myfilter = S3QueryField("contact.value") == "dominic@nursix.org"
resource.add_filter(myfilter, c="contact")  # also adds the join for the "contact" component to the resource query

For URL filters, this happens automatically.

Note that filters as such (i.e. S3ResourceQuery instances) are resource-agnostic, so they can be re-used across resources (provided the same field selector applies in all of them).

Filters for fields which are not defined in a resource will be ignored quietly when retrieving records from the resource.

Attachments (2)

Download all attachments as: .zip

Note: See TracWiki for help on using the wiki.