BluePrint for supporting multiple regions or incidents in one server

Motivation

Regions

Humanity Road works on events in more than one area, each with different GIS characteristics. For instance, in one area, a complete GIS hierarchy may be available, and we can associate individual locations with the hierarchy location they fall in. In another case, we may not have hierarchy data other than the country, but perhaps we can designate a general area by a bounding polygon, or by radius around a point.

A person using the site would like to quickly select which region they're working on, and have the map centered on that spot with a region-specific set of layers shown, and search results and drop-down selectors filtered for that region or incident. Selecting a region to work on should not interfere with the site's overall GIS configuration, nor the user's own personal configuration.

Incidents

An "incident" is a more fine-grained concept, involving not just location but also time and assigned personnel and resources. Don Cameron points out the importance of being able to specify an incident and identify what's associated with it, in order to implement an ICS (incident command system).

Incidents can overlap in time, region, personnel, and resources, thus the means we adopt of associating resources with incidents should not in general force a resource to be exclusively tied to one incident.

Please note: This is not a general discussion of incidents nor of an incident command system. Rather, it is about how to accommodate multiple incidents active at the same time in one server.

Common features

Regions and incidents touch on each other in several ways:

For both, we will very likely want to support multiple instances in the same server, and same database. How we handle multiple regions may provide some hints on how to handle multiple incidents.

The user will want a way to select which they're working on at the moment.

The definition of a region or incident should be shared among (appropriate) users. An authorized person should be able to configure a region or incident.

An incident will probably need a region as part of its specification.

Region proposal

(Note this part is being implemented as we speak. Its design is not expected to be controversial. It does not break or alter any existing functionality.)

In order to support configuring and selecting regions:

• Identify a region by a gis_config record. Move any region-specific configuration into gis_config, or tie it to gis_config via a foreign key reference. This currently means:
  • A location representing a region (see below).
  • The hierarchy labels, depth, and settings appropriate for that area. (Language translation is not sufficient here. E.g we don't have "provinces" and "districts" in the US but those are perfectly legal US English words. Different areas also have different depths of hierarchy, and different choices about a "strict" hierarchy.) Hierarchy settings are:
    • "Strict" -- specific locations can only be attached to the hierarchy at its maximum level.
    • "Parent required" -- specific locations need a parent location selected from the hierarchy. (This is used to assist forming regions in the absence of boundary data -- see below.)
  • A label to use on a menu of regions. (Presence of this label indicates the config is intended as a region.)
  • A flag for whether this should appear in the Regions menu. (Presence of a label will be required in validation.)
  • Already present in gis_config, and needed for region selection:
    • Map center and zoom (so map can be positioned appropriately for the region).
  • Allow the set of available base layers and the initial selected base layer to differ per region. (Implementation detail: Take enabled flag out of gis_layer_* and instead have a list multiple field in gis_config that enumerates the enabled layers for that config. Add a field for the selected base layer.)
  • Allow the set of available (resource) overlays (a.k.a. "internal" overlays or "internal features") and the initial selected overlays to differ per region. Options for implementation: (Pat: In the following, I believe we were referring to different aspects -- which overlays were selected, i.e. "visible" versus which were made available to select, i.e. "enabled". In any case, both suggested implementations now seem incorrect.)
    • Pat: Inserting a reference to a region in layer_feature *deselects* the layer for that on the map for that region. This is consistent with an empty region set meaning all regions have that layer, and is easier to maintain if new regions are added -- don't need to do anything.)
    • Fran: Inserting a reference to a region in layer_feature *restricts* the layer to just that region since this is the most usual use-case where a layer only applies to a particular region.
    • Pat: (After getting that confusion cleared up...):
      • Take the "enabled" flag out of gis_layer_feature and instead have a list multiple field in gis_config that enumerates the enabled internal overlays for that config.
      • Take the "visible" field out of gis_layer_feature and add another list multiple field in gis_config for which are initially selected.
  • Allow selection of "external" overlays to be specified in gis_config (both "enabled" and "visible" as for internal overlays). (The set of possible external overlays might be hard coded -- changed when support is added for each.) (Note Fran says adding another "folder" in the map viewer is "not so easy".)

• Generate a regions menu from the gis_config table. Include site and personal configs in the menu. Add deployment setting to enable the regions menu and specify a label for the menu head.

• The selected region persists for the session only (or until the user selects another or clears the selection).
  • Selecting a region records the associated gis_config in the session.
  • To find the gis_config currently in effect look for it in these places in this order:
    • In the session.
    • The user's personal gis_config.
    • The generic gis_config.

• Filter entities that have locations by region.
  • Filter drop-down menus and other selectors.
  • Filter search results.
  • Filter lists.
  • Perhaps provide an option to view all entities? (This isn't necessary as one can deselect the region but might be convenient if one wants to find nearby entities.)

The "region location" associates specific locations to the region. There are four main ways a region location can specify this association:

• The region location can be in the hierarchy, and specific locations can have the region location as an ancestor. (E.g. the region might be a city.) Turning on the "parent required" flag means selecting a parent is mandatory, which would be appropriate in this case.
• The region location can contain a boundary polygon. Specific locations with point or polygon data can be tested for overlapping the region polygon.
• The region can specify a point and a radius. Specific locations with point or polygon data can be tested for overlapping this circle.
• The region location can be a "location group" (a list of locations that are each appropriate as region locations). (For instance, the region might consist of several counties in a state.) A specific location is tested against each of the locations in the group. (This would allow recursion, if we permit a location group to be a member of a location group. That's not required, though.)

If the test for whether a location is in a region is expensive (e.g. if it uses polygon overlap), we may want to cache the membership (similar to caching the location's hierarchy path). A location can belong to more than one region, so this would be a list of regions (specified by their region location ids). Would need to record both positive and negative results, i.e. if we test a location against a region and find it's not in that region, we need to record that to avoid redoing the test. The cache would be invalidated if the location's parentage or lon, lat changes. Invalidating region test results if a region's scope changes (e.g. its border is changed, or a group's membership changes, or if the hierarchy is changed) needs careful consideration to avoid expensive database operations. Fortunately, regions are not expected to be changed frequently.

We may want to record the gis_config currently in effect in the session in any case, not just if it was a region gis_config selected from the menu. In addition, we can cache the gis_config contents to avoid some db reads. Since (rumor has it) the session is transmitted to the client on each request (although it appears it is actually saved on the client), and the gis_config is mainly needed on the server, we can just put the id of the selected gis_config in the session, and cache the actual contents in response. So it will still need to be read on each request, but not repeatedly on each request.

Detail on caching region membership

(Fran asked how this would be implemented...)

The proposed implementation of cached region membership is similar to the "materialized path" path currently in locations, in that it encodes the list of regions that a location has been checked against, and the result of each test, in a string. The difference is that the region membership test can return true or false, and we need to record both -- we need to know that the location was tested against a region, not merely which region tests were true, else we'd re-do the test for regions the location is not in. The format of the string is (in pseudo-BNF):

true := "T"
false := "F"
null := <empty field>
result := true|false
region_id := <id in gis_location of a region location>
region_result := region_id","result
region_result_cache := null|region_result["|"region_result]

That is, it looks like:

5,T|3,F

We may want one more item of data to support invalidating results if a region changes, as follows:

It's easy to invalidate the cached region membership if a location is changed (in a way that affects what actual location it points to) -- just clear it.

If the region definition changes, there is the same issue as changing the hierarchy causes for cached paths -- we would like to avoid having to go through the locations cleaning out the result for that region. But since region (or hierarchy) changes are not going to be frequent, this may not be too much overhead.

One way of avoiding it is to add a timestamp to the region cache giving the last time it was updated, or to each test individually. Then if we find a region has changed after the timestamp, we don't believe the cached result. If we have only a timestamp for the cache as a whole, we clear it. If we have individual timestamps, we just replace the result for the re-tested region.

To maintain the region's last-changed timestamp, we need to know when the location it points to is changed in a way that materially affects the geographic area it represents (boundary, lon / lat, address (?), hierarchy parent, group members. That requires at least on database operation. If a location that belongs to a location group is changed, that might require chaining back to find the region. (The region's timestamp would also be changed if its location were replaced by another, but in that case, we already have the region's id.)

For per-region timestamps, change the BNF thusly:

timestamp := <datetime string>
region_result := region_id","result","timestamp

Current intent is to include the timestamp in each result, to avoid the region change issue.

Incident discussion

Open for ideas -- please add suggestions here!

An incident might involve (at least):

• Personnel
  • Individuals
  • Teams
  • Projects / work assignments / work logs
• Material resources
  • Vehicles assigned
  • Inventory allocated
• Associated entities
  • Hospitals
  • Shelters
  • Other orgs involved
  • Assessments
• Information for responders & public
  • Documents
  • Social network ids / feeds

What else could go with an incident?

Re. associated entities: Fran: Are these simply tagged as being part of this Incident & we filter by default to the already-tagged ones, but can also remove this filter if we want to find a resource from the wider pool? Pat: (NB: If we use relationship tables (see below), no tag is applied to the entity itself.) If the user selects an incident, we'd apply a filter where appropriate -- probably this should be the default, i.e. assume the user is interested in entities already associated with the incident. The user would need a way to turn off the filter in order to find and add entities to the incident.

Use an org

Fran considered creating an org for each incident, as it covers the command hierarchy for the staffing, but says that may not be sufficient. Probably need at least a region in addition.

Use relationship tables

Many types of entities are associated with an incident. The "relational" way to tell which entities go with which incidents, without adding fields to entities, is to use relationship tables that associate an incident with each of the types. That is, there could be an "incident assessment" table in which each record has a foreign key reference to an incident and to an assessment, and so-on for the other associated types. These relationship tables are components of incident. Use of relationships allows overlap between incidents -- one entity can be associated with more than one incident. It's simple to associate another entity type with an incident -- just add a relationship table.

Additional data can be included in the relationship tables, such as a start date and end date. When an entity is no longer needed for an incident, its relationship can be deleted or "expired" by setting an end date.

If a "paper trail" is needed, i.e. a record of what was used for each incident, and when, or who worked on it, and when, there are several ways of providing that. One is to add start and end dates to the relationship tables. If someone / something stops being associated with an incident, its end date is set. If that person / resource later comes back to the incident, a new relationship record is created. (Think of keeping a log of hours worked on each of several consulting projects -- a simple way is to record start and end times for each work session.) Since those old records could clutter the table over time, and slow access, one could dump / pickle historical data, or move it to a parallel set of "expired" relationship tables. Keeping it online would be more convenient for preparing reports. Simplest is to leave them in the main incident relationship tables but set their end dates.

---

BluePrints