wiki:BluePrint/Mobile/P2PSync

BluePrint: Mobile Client Peer-to-Peer Synchronization

Table of Contents

  1. Introduction
  2. Stakeholders
  3. User Stories
  4. Requirements
    1. Functional requirements
      1. 1. Establishing peer relationships
        1. 1.1. Peer visibility
        2. 1.2. Peer discovery
        3. 1.3. Peer selection
        4. 1.4. Peer authentication
        5. 1.5. Peer persistence
      2. 2. Destination servers
        1. 2.1. Destination server configuration
      3. 3. Data synchronization
        1. 3.1. Data selection
        2. 3.2. Data deletion
      4. 4. Managing conflicts
        1. 4.1. Conflict detection
        2. 4.2. Conflict resolution
    2. Non-functional requirements
      1. 1. Security
        1. 1.1. Protection of data in transit
        2. 1.2. Protection of data at rest
        3. 1.3. Access control
      2. 2. Performance
        1. 2.1. Power efficiency
        2. 2.2. Communication efficiency
    3. Interoperability
    4. Standards
    5. System Constraints
  5. Design
      1. 1. Synchronize immutable documents
      2. 2. Synchronize changes to database records
      3. Comparison of the options for each use case
    2. Data Model
    3. Workflows
      1. 1. Peering between mobile devices
        1. 1.1. Peering without stored peer identities
        2. 1.2. Peering with a stored peer identity on one device
        3. 1.3. Peering with stored peer identities on both devices
      2. 2. Configuring a destination server
      3. 3. Activating and deactivating P2P sync
    4. Technologies
      1. 1. Synchronize immutable documents
      2. 2. Synchronize database records
  6. Current Implementation
  7. Planned Implementation
  8. Future Extensions
  9. Outstanding Questions
  10. References

Introduction

Peer-to-peer sync will enable EdenMobile users to synchronize data between mobile devices without being connected to an Eden server. This will allow users to operate and collaborate more easily in areas where internet access is unreliable or unavailable.

Peer-to-peer sync capabilities exist in ​Serval Mesh and several messaging apps including ​Briar, ​Murmur and ​Anonymouse.

Stakeholders

EdenMobile users will be the primary stakeholders. The data subjects will also be stakeholders, and appropriate access controls to protect the confidentiality, integrity and authenticity of data must be built in. Trainers who teach EdenMobile users how to use the application will need to be aware of the new functionality. If the data being synchronized is ultimately delivered to a server then the people deploying and operating the server will also be stakeholders.

User Stories

See the ​user stories document.

Requirements

Functional requirements

1. Establishing peer relationships

1.1. Peer visibility

The P2P sync component must be able to make the device it is running on visible to nearby devices in order to establish a peer relationship.

1.2. Peer discovery

The P2P sync component must be able to discover nearby devices that have made themselves visible.

1.3. Peer selection

When peers have been discovered, the P2P sync component must allow the user to choose a device to establish a peer relationship with.

1.4. Peer authentication

When the user has chosen a device to peer with, the P2P sync component must allow the user to authenticate the peer before giving the peer access to any data. The P2P sync component may allow the user to continue without authenticating the peer.

1.5. Peer persistence

When a peer has been authenticated, the P2P sync component may allow the user to store information about the peer, including information that will allow the peer to be authenticated automatically during subsequent connections. If information about a peer has been stored, the P2P sync component must allow the user to delete the information in order to prevent the peer from being authenticated automatically during subsequent connections.

2. Destination servers

2.1. Destination server configuration

The P2P sync component may allow the user or an administrator to enter the details of a destination server for the data being synced. If a destination server has been configured, the P2P sync component may deliver data to the server automatically when internet access is available.

3. Data synchronization

3.1. Data selection

The P2P sync component may allow the user to choose which data to synchronize. When data is received from a peer, the P2P sync component may automatically choose it for synchronization with other peers.

3.2. Data deletion

The P2P sync component must allow the user to delete data from the device, regardless of whether that data has been chosen for synchronization. The P2P sync component may automatically delete data that has been received from peers, for example on receiving confirmation that the data has been delivered to a destination server.

4. Managing conflicts

4.1. Conflict detection

The P2P sync component must assign an identifier to each item of data chosen for synchronization, such that two peer devices performing synchronization can tell whether they are using the same identifier to refer to the same data.

4.2. Conflict resolution

If a conflict is detected during synchronization, the P2P sync component may resolve the conflict automatically, either by merging the conflicting items or by assigning them distinct identifiers. If a conflict cannot be resolved automatically, the P2P sync component must be able to continue synchronizing other items with the peer. The P2P sync component may inform the user of conflicts that cannot be resolved automatically.

Non-functional requirements

1. Security

1.1. Protection of data in transit

The P2P sync component must ensure the confidentiality, integrity and authenticity of data in transit between peers. If a destination server has been configured, the P2P sync component must ensure the confidentiality, integrity and authenticity of data in transit between mobile devices and the destination server.

1.2. Protection of data at rest

The P2P sync component must ensure the confidentiality, integrity and authenticity of data at rest on mobile devices, to at least the same level of protection as other components of the EdenMobile application.

1.3. Access control

The P2P sync component must authenticate peers, or allow the user to authenticate them, before granting them access to data. If a destination server has been configured, the P2P sync component must authenticate the destination server before granting it access to data.

If a delivery confirmation from a destination server is received, either directly from the server itself or from a peer, the P2P sync component must authenticate the delivery confirmation before acting on it or synchronizing it with peers.

To protect the privacy of data subjects, the P2P sync component may encrypt data before synchronizing it with peers, such that it can only be decrypted by the destination server.

2. Performance

2.1. Power efficiency

The P2P sync component must not significantly reduce the battery life of the device when P2P sync is not being used.

2.2. Communication efficiency

The P2P sync component must not significantly increase the mobile data usage of the device, regardless of whether P2P sync is being used.

Interoperability

For a given deployment, the same version of the EdenMobile app will be installed on all mobile devices, which should minimize interoperability issues. The protocols used between peers, and between mobile devices and the destination server, should use version negotiation so that any interoperability issues are detected as early as possible. Interoperability with other applications is not required.

Standards

Communication between mobile devices and the destination server should use HTTPS. If the P2P sync component protects the privacy of data subjects by encrypting data between the originating device and the destination server, this should be done using OpenPGP. If the destination server creates delivery confirmations, these should be signed using OpenPGP.

System Constraints

The P2P sync component must operate on mobile devices with limited battery life, storage, and network bandwidth. It must be usable with minimal training and background knowledge. It must tolerate intermittent and unreliable connections between peers, and between mobile devices and the destination server.

Design

There are two design options:

1. Synchronize immutable documents

In this option, the data to be synced consists of immutable documents, each of which originates on one mobile device and is synced to other devices without being modified. For example, a completed form or a photo can be treated as an immutable document.

Detecting and resolving conflicts is simple in this option: each data item is identified by its cryptographic hash, which ensures that if two items have the same identifier then they also contain the same data.

This option allows documents to be encrypted on the originating device and decrypted on the destination server.

The destination server confirms delivery of a document by creating a delivery confirmation containing the document's identifier, signed with the server's OpenPGP key. The confirmation can be synced between mobile devices, which can delete the confirmed document if they no longer need a local copy.

2. Synchronize changes to database records

In this option, the data to be synced consists of database records that may be modified independently on any device. This provides maximum flexibility for the application, as anything stored in the database can potentially be synced, but it also requires the application to handle conflicts that may arise from concurrent modification of a given record on multiple devices.

Detecting and resolving conflicts in this option follows a similar approach to ​CouchDB. Non-conflicting updates are applied automatically. If a conflict occurs, the P2P sync component stores all conflicting versions of the record and asks the application to resolve the conflict, which may require user intervention, either on the mobile device or on the destination server.

This option does not allow records to be encrypted on the originating device and decrypted on the destination server, as mobile devices need access to the contents of updates. Individual fields can be encrypted, but if a conflict occurs in an encrypted field it can only be resolved on the destination server.

The destination server confirms delivery of a record by creating a delivery confirmation containing the record's identifier and ​revision, signed with the server's OpenPGP key. The confirmation can be synced between mobile devices, which can delete the confirmed record if they are storing the same revision and no longer need a local copy (they must keep the record if they are storing a different revision). Multiple revisions of a given record may be confirmed by the destination server.

Comparison of the options for each use case

The immutable documents option is well suited to the "Situation Reporting and Assessment" and "Self-Organized Canvassing" use cases, which are primarily concerned with collecting data and delivering it to a central server.

The "Aid Distribution" use case involves updating information as well as collecting it: for example, recording the fact that an individual who was previously eligible for aid has now received it. The database records option is well suited to this use case, but the use case can also be handled with immutable documents by using a log structure: for example, a log of documents representing events is attached to a document representing an individual.

The "Logistics Management" use case is primarily concerned with managing mutable information. This can be handled with immutable documents using a log structure, but for efficiency it may also be necessary to maintain a representation of the current state. The complexity of maintaining two representations of the data may make the database records option preferable for this use case.

Data Model

TODO

Workflows

1. Peering between mobile devices

1.1. Peering without stored peer identities
  • Users: U1 and U2, with devices D1 and D2, respectively
  • U1 and U2 start peer discovery
  • D1 discovers D2 and/or D2 discovers D1
  • U1 chooses to peer with D2 and/or U2 chooses to peer with D1
  • D1 connects to D2 and/or D2 connects to D1
  • D1 and D2 exchange authentication information
  • D1 and D2 display each other's authentication information
  • U1 and U2 confirm the authentication information or skip authentication
  • D1 and D2 become peers
  • U1 and/or U2 may choose to store the peer's identity persistently
1.2. Peering with a stored peer identity on one device
  • Users: U1 and U2, with devices D1 and D2, respectively
  • D1 has stored peering information about D2
  • U1 and U2 start peer discovery
  • D1 discovers D2 and/or D2 discovers D1
  • U1 chooses to peer with D2 and/or U2 chooses to peer with D1
  • D1 connects to D2 and/or D2 connects to D1
  • D1 and D2 exchange authentication information
  • D1 automatically authenticates D2
  • D2 displays D1's authentication information
  • U2 confirms the authentication information or skips authentication
  • D1 and D2 become peers
  • U2 may choose to store the peer's identity persistently
1.3. Peering with stored peer identities on both devices
  • Users: U1 and U2, with devices D1 and D2, respectively
  • D1 has stored peering information about D2
  • U1 and U2 start peer discovery
  • D1 discovers D2 and/or D2 discovers D1
  • U1 chooses to peer with D2 and/or U2 chooses to peer with D1
  • D1 connects to D2 and/or D2 connects to D1
  • D1 and D2 exchange authentication information
  • D1 and D2 automatically authenticate each other
  • D1 and D2 become peers

2. Configuring a destination server

  • User or administrator enters the URL and OpenPGP key of a destination server
  • The URL and OpenPGP key are automatically validated
  • User or administrator may choose to test connectivity to the server
  • User or administrator saves the configuration

3. Activating and deactivating P2P sync

  • P2P sync is automatically activated whenever the user starts peer discovery
  • P2P sync is automatically deactivated when peer discovery is not in progress and no peer connections are open
  • User can see whether P2P sync is currently active
  • User can manually deactivate P2P sync, which cancels peer discovery and closes any open peer connections

Technologies

The P2P sync component will use the ​Bramble library, which will be packaged as a Cordova plugin. The component will only be available on Android.

The Bramble protocols ​BTP and ​BSP will be used for syncing data between peers. In Bramble terminology, the P2P sync component will be a BSP client. New modules for discovering, authenticating and storing peers will be developed, or the existing Bramble modules will be generalised.

The structure of the BSP client, and the interface between the P2P sync component and the EdenMobile app, depend on which design option is chosen.

In both options, the P2P sync component exposes the following capabilities to the EdenMobile app:

  • Activate and deactivate P2P sync.
  • Discover and authenticate peers.
  • Optionally store peer identities in the component's database.
  • Optionally configure a destination server and store the configuration in the component's database.
  • Optionally encrypt the component's database with a password-derived key.

The P2P sync component also implements the following capabilities internally:

  • Connect to the destination server when internet access is available.
  • Synchronize data with peers and the destination server.
  • Authenticate delivery confirmations received from peers and the destination server.

The component has the following dependencies:

  • Bramble
  • OpenPGP

1. Synchronize immutable documents

In this option, the P2P sync component exposes the following extra capabilities to the app:

  • Store immutable documents in the component's database.
  • Optionally encrypt documents with the destination server's public key before storing them.
  • Choose documents from the component's database to be synchronized.

The P2P sync component implements the following extra capabilities internally:

  • Delete documents that were received from peers and have been confirmed by the server.

This option has no extra dependencies.

2. Synchronize database records

In this option, the P2P sync component exposes the following extra capabilities to the app:

  • Choose records from the app's database to be synchronized.

This option requires the app's database to be ported from ​Cordova SQLite to ​PouchDB, which ​wraps Cordova SQLite. ​PouchDB Replication Stream extracts changes from the database to be synced via Bramble.

Alternatively, the component can provide a separate PouchDB database for records that need to be synced.

This option has the following extra dependencies:

  • PouchDB
  • PouchDB Replication Stream

Current Implementation

N/A

Planned Implementation

TODO

Future Extensions

TODO

Outstanding Questions

TODO

References

BluePrint

Last modified 8 years ago Last modified on 05/31/17 09:25:11
Note: See TracWiki for help on using the wiki.