Consent Tracking

Introduction

Storing and processing of personally identifiable data (PID) may require explicit consent by the person in question. The Consent Tracking framework provides the means to request and track such consent.

Data Model

Types of Data Processing

The auth_processing_type table records types of data processing that require consent. Each processing type is identified by a unique code, which can be used to hard-code filters and consent checks throughout the application.

Consent Options

The auth_consent_option table holds a short title for each processing type (e.g. "Store my personal data") and an explanation what exactly that means. These two are used to request consent from the user, so formulations may be subject to legal requirements and guidelines.

Once a user has consented (or declined to consent) to a consent option, title and explanations can no longer be changed.

If they must be changed, e.g. for legal reasons or because the application has changed with regard to this type of data processing, then a new consent option for this processing type must be created, and the old version be marked as obsolete. Consent recorded for obsolete consent options will no longer be valid, thus, after such a change, the user must be asked for their consent again.

Consent Record

The auth_consent table records the user response to a consent option (yes or no). Each consent record refers to a specific consent option, hence type of data processing.

A consent record will be created even if the user declines to consent (in this case with the consented-flag set to False). Therefore, the absence of a consent record for a processing type does not mean the user has declined, but rather that they have not been asked.

Consent records can expire after a certain date, and will always expire when a new consent response of the same user for the same processing type is recorded.

Managing Consent Options

Types of Data Processing and Consent Options can be managed by Administrators through the Web GUI.

To enable the corresponding sub-menu in the Admin section, the template must set:

settings.auth.consent_tracking = True

Typically, however, consent options (or at least processing types) would be pre-populated by the template - for the CSV structure see static/formats/s3csv/auth.

Requesting Consent

For the implementation of consent tracking, the framework provides the auth_Consent class which is available through s3db:

consent = s3db.auth_Consent()

The constructor can be given a list of processing type codes to limit it to these types of data processing, e.g.:

consent = s3db.auth_Consent(processing_types=["STOREPID", "SHAREPID"])

Embedding the Widget

The auth_Consent instance can generate a form widget with consent options. To embed it into a form, inject an additional string field like this:

# Instantiate auth_Consent (limit to certain processing types if required)
consent = s3db.auth_Consent()

# Produce a Field instance with the consent widget
consent_field = Field("consent_question",
                      label = T("Consent"),
                      widget = consent.widget,
                      )
formfields.append(consent_field)

The widget will then store the consent response as JSON string in POST vars (request.post_vars.consent_question, in this case).

Mandatory Consent Options

Consent options can be marked as mandatory (either in prepop, or in the Admin UI), e.g. when a consent option is required to even process the form that asks for it (e.g. consent to store personal data is required for registration, which can only be asked during the registration).

If the user forgets (or declines) to consent to a mandatory consent option, the form submission will fail and an error will be shown to the user.

Post-processing the Widget

After successful form submission, the consent response must be post-processed in order to get recorded. For this, a person_id for the person in question is required, so the post-processing can only happen after the person record has been established.

To post-process the response, simply pass it to auth_Consent.track (a class method, instance not required):

auth_Consent.track(person_id, form.vars.consent_question)

This will record the new response, and expire previous consent records of that person for the same processing types.

Checking for Consent

has_consented

To check whether a person has consented to a particular type of data processing, you can use the auth_Consent.has_consented method with the processing type code:

if s3db.auth_Consent.has_consented(person_id, "PIDSHARE"):
    # perform PIDSHARE processing
    pass

This is a class method, so instantiation of auth_Consent is not required.

has_consented makes sure that the consent record has not expired, and any consent option the user has consented to is still valid (obsolete=False). It does, however, not verify the integrity of the consent record.

consent_query

The auth_Consent.consent_query method can be used to filter a DAL table for records belonging to persons that have consented to a particular processing type:

# Get HR records where the person has consented to the HRDSHARE processing type:
table = s3db.hrm_human_resource
query = s3db.auth_Consent.consent_query(table, "HRDSHARE", field="person_id") & (table.deleted==False)
rows = db(query).select(...)

If field is not specified, person_id will be assumed (or id when the table is pr_person).

consent_query is a class method, so instantiation of auth_Consent is not required.

The consent sub-query will include an implicit inner join with the auth_consent table (not aliased), which may need to be taken into account when the overall query constructs additional joins.

consent_query takes into account whether the consent record or the consent option have expired, but it will not verify the integrity of any consent records.

consent_filter

The auth_Consent.consent_filter method can be used to filter S3Resources by consent to a particular processing type:

resource.add_filter(s3db.auth_Consent.consent_filter("PIDSHARE", "~.person_id"))

The field selector should identify a reference to pr_person (i.e. person_id).

Only one consent filter can be added to the same S3Resource instance.

consent_filter is a class method, so instantiation of auth_Consent is not required.

consent_filter will take into account whether consent record or consent option have expired, but it will not verify the integrity of any consent record.

Consent Expiry

It is possible to define a standard expiry period for consent options (both through the Web GUI and prepop). If set, any recorded consent would expire automatically after that period - and subsequent consent checks would result in a "no".

The auth_consent.expires_on field can be used to check for expired consent records, e.g. to notify users about due renewal.

Note that changing the expiry period in the consent option does not change the expiry date of existing consent records.

Verifying Integrity

Every consent record contains an encrypted hash of itself and the consent option it refers to. If the text of the consent option is changed after users have given their consent, or the consent record is manipulated, then this hash becomes invalid.

To verify the hash of a consent record, you can use the auth_Consent.verify method with the consent record ID:

is_valid = s3db.auth_Consent.verify(consent_record_id)

This check is not required for regular use of the framework - but in case of a dispute, it can be used to prove the integrity of consent records. Note that this does not disprove willful manipulation of consent records, but it precludes crude/accidental modifications outside of the framework (e.g. DB modifications through the Web GUI).