= Consent Tracking =
[[TOC]]

== Introduction ==

Storing and processing of personally identifiable data (PID) may require explicit consent by the person in question. The Consent Tracking framework provides the means to request and track such consent.
== Data Model ==

=== Types of Data Processing ===

The {{{auth_processing_type}}} table records types of data processing that require consent. Each processing type is identified by a unique '''code''', which can be used to hard-code filters and consent checks throughout the application.

=== Consent Options ===

The {{{auth_consent_option}}} table holds a short title for each processing type (e.g. "Store my personal data") and an explanation what exactly that means. These two are used to request consent from the user, so formulations may be subject to legal requirements and guidelines.

Once a user has consented (or declined to consent) to a consent option, title and explanations can no longer be changed.

If they must be changed, e.g. for legal reasons or because the application has changed with regard to this type of data processing, then a new consent option for this processing type must be created, and the old version be marked as obsolete. Consent recorded for obsolete consent options will no longer be valid, thus, after such a change, the user must be asked for their consent again.
=== Consent Record ===

The {{{auth_consent}}} table records the user response to a consent option (yes or no). Each consent record refers to a specific consent option, hence type of data processing.

A consent record will be created even if the user declines to consent (in this case with the {{{consented}}}-flag set to False). Therefore, the absence of a consent record for a processing type does not mean the user has declined, but rather that they have not been asked.

Consent records can expire after a certain date, and will always expire when a new consent response of the same user for the same processing type is recorded.
== Managing Consent Options ==

Types of Data Processing and Consent Options can be managed by Administrators through the Web GUI. 

To enable the corresponding sub-menu in the Admin section, the template must set:
{{{#!python
settings.auth.consent_tracking = True
}}}

Typically, however, consent options (or at least processing types) would be pre-populated by the template - for the CSV structure see static/formats/s3csv/auth.
== Requesting Consent ==

For the implementation of consent tracking, the framework provides the {{{auth_Consent}}} class which is available through {{{s3db}}}:
{{{#!python
consent = s3db.auth_Consent()
}}}

The constructor can be given a list of processing type codes to limit it to these types of data processing, e.g.:
{{{#!python
consent = s3db.auth_Consent(processing_types=["STOREPID", "SHAREPID"])
}}}
=== Embedding the Widget ===

The auth_Consent instance can generate a form widget with consent options. To embed it into a form, inject an additional string field like this:

{{{#!python
# Instantiate auth_Consent (limit to certain processing types if required)
consent = s3db.auth_Consent()

# Produce a Field instance with the consent widget
consent_field = Field("consent_question",
                      label = T("Consent"),
                      widget = consent.widget,
                      )
formfields.append(consent_field)
}}}

The widget will then store the consent response as JSON string in POST vars ({{{request.post_vars.consent_question}}}, in this case).
=== Mandatory Consent Options ===

Consent options can be marked as mandatory (either in prepop, or in the Admin UI), e.g. when a consent option is required to even process the form that asks for it (e.g. consent to store personal data is required for registration, which can only be asked during the registration).

If the user forgets (or declines) to consent to a mandatory consent option, the form submission will fail and an error will be shown to the user.
=== Post-processing the Widget ===

After successful form submission, the consent response must be post-processed in order to get recorded. For this, a {{{person_id}}} for the person in question is required, so the post-processing can only happen after the person record has been established.

To post-process the response, simply pass it to {{{auth_Consent.track}}} (a class method, instance not required):
{{{#!python
auth_Consent.track(person_id, form.vars.consent_question)
}}}

This will record the new response, and expire previous consent records of that person for the same processing types.
== Checking for Consent ==

=== has_consented ===

To check whether a person has consented to a particular type of data processing, you can use the {{{auth_Consent.has_consented}}} method with the processing type code:

{{{#!python
if s3db.auth_Consent.has_consented(person_id, "PIDSHARE"):
    # perform PIDSHARE processing
    pass
}}}

This is a class method, so instantiation of auth_Consent is not required.

{{{has_consented}}} makes sure that the consent record has not expired, and any consent option the user has consented to is still valid (obsolete=False). It does, however, not verify the integrity of the consent record.
=== consent_query ===

The {{{auth_Consent.consent_query}}} method can be used to filter a DAL table for records belonging to persons that have consented to a particular processing type:

{{{#!python
# Get HR records where the person has consented to the HRDSHARE processing type:
table = s3db.hrm_human_resource
query = s3db.auth_Consent.consent_query(table, "HRDSHARE", field="person_id") & (table.deleted==False)
rows = db(query).select(...)
}}}

If {{{field}}} is not specified, {{{person_id}}} will be assumed (or {{{id}}} when the table is pr_person).

{{{consent_query}}} is a class method, so instantiation of {{{auth_Consent}}} is not required.

The consent sub-query will include an implicit inner join with the {{{auth_consent}}} table (not aliased), which may need to be taken into account when the overall query constructs additional joins.

{{{consent_query}}} takes into account whether the consent record or the consent option have expired, but it will not verify the integrity of any consent records.
=== consent_filter ===

The {{{auth_Consent.consent_filter}}} method can be used to filter S3Resources by consent to a particular processing type:

{{{#!python
resource.add_filter(s3db.auth_Consent.consent_filter("PIDSHARE", "~.person_id"))
}}}

The field selector should identify a reference to pr_person (i.e. person_id).

Only one consent filter can be added to the same S3Resource instance. 

{{{consent_filter}}} is a class method, so instantiation of {{{auth_Consent}}} is not required.

{{{consent_filter}}} will take into account whether consent record or consent option have expired, but it will not verify the integrity of any consent record.

== Consent Expiry ==

It is possible to define a standard expiry period for consent options (both through the Web GUI and prepop). If set, any recorded consent would expire automatically after that period - and subsequent consent checks would result in a "no".

The {{{auth_consent.expires_on}}} field can be used to check for expired consent records, e.g. to notify users about due renewal.

Note that changing the expiry period in the consent option does not change the expiry date of existing consent records.
== Verifying Integrity ==

Every consent record contains an encrypted hash of itself and the consent option it refers to. If the text of the consent option is changed after users have given their consent, or the consent record is manipulated, then this hash becomes invalid.

To verify the hash of a consent record, you can use the {{{auth_Consent.verify}}} method with the consent record ID:
{{{#!python
is_valid = s3db.auth_Consent.verify(consent_record_id)
}}}

This check is not required for regular use of the framework - but in case of a dispute, it can be used to prove the integrity of consent records. Note that this does not disprove willful manipulation of consent records, but it precludes crude/accidental modifications outside of the framework (e.g. DB modifications through the Web GUI).