Offline Field Collection and Sync Reconciliation

A field crew works for three weeks with no connection while the trunk keeps moving, and the whole difficulty of offline collection lives in that gap: the edits are real, the trunk is real, and neither one is wrong. This guide is part of Conflict Resolution & Team Synchronization Workflows.

Prerequisites & Environment Setup

Before deploying an offline collection workflow, confirm each of the following:

• Device-generated identifiers — UUIDv4 or a prefixed ULID minted on the device at capture; never a server sequence, and never a per-device counter without a device prefix
• A recorded checkout point: each offline package stores the trunk revision it was cut from, because that revision is the merge base three weeks later
• An extent assignment per crew, recorded and enforced, so two crews are not editing the same ground while disconnected
• An operation log on the device, not just final state — created, moved, attribute-changed, retired — with the observation timestamp from the device clock and the GNSS fix quality
• Clock discipline: devices sync time from GNSS rather than the operating system, since a device clock that drifts by hours puts observations in the wrong order
• A quarantine area on the server for packages that fail validation, so a bad upload is never partially applied

Core Algorithmic Patterns

1. Identity is minted at capture, not at arrival

The identifier a crew records in a notebook, photographs, and references in a sketch is the identifier the feature must keep. Any scheme that renumbers on arrival breaks every one of those links, and the breakage is discovered weeks later when someone tries to match a photo to a parcel.

Device-minted UUIDs solve the collision problem outright at the cost of being unreadable. In practice a prefixed identifier reads better in the field and remains collision-free:

EAST-7K3M2P9Q4R    crew prefix + ULID suffix

The crew prefix makes provenance obvious at a glance and gives the sync process a fast way to group a package’s creations. The ULID suffix sorts by creation time, which turns out to be useful when replaying an operation log.

2. The checkout point is the merge base

An offline package that records only its edits leaves the server guessing what the crew started from. Recording the trunk revision at checkout turns sync from a two-way overwrite into a three-way merge — exactly the structure that makes automated conflict detection in merge requests tractable.

With the base revision, the server can distinguish the three cases that matter: the crew changed a feature the trunk did not, the trunk changed a feature the crew did not, and both changed it. Only the third needs a decision, and on a well-assigned extent it is rare.

3. Operations replay; snapshots overwrite

A package that says “here is the final state of 412 features” can only be applied by overwriting. A package that says “I created these 9, moved these 3, retired this 1” can be replayed onto a trunk that has moved, and each operation can succeed or conflict independently.

The difference shows up on the features the crew did not touch. A snapshot silently reasserts them, undoing any trunk change in the same extent; an operation log leaves them alone because it never mentions them.

Production Workflow Implementation

Step 1 — Cut the offline package

import json
import subprocess
from datetime import datetime, timezone
from pathlib import Path


def cut_package(layer_path, extent, crew_prefix, out_dir):
    """Produce an offline package: the data, the extent, and the base revision."""
    base_rev = subprocess.check_output(
        ["git", "rev-parse", "HEAD"], text=True
    ).strip()

    out_dir = Path(out_dir)
    out_dir.mkdir(parents=True, exist_ok=True)

    subprocess.run(
        ["ogr2ogr", "-f", "GPKG", str(out_dir / "collect.gpkg"), str(layer_path),
         "-spat", *map(str, extent), "-nlt", "PROMOTE_TO_MULTI"],
        check=True,
    )

    (out_dir / "package.json").write_text(json.dumps({
        "schema": "gdv-offline/1",
        "base_revision": base_rev,
        "crew_prefix": crew_prefix,
        "extent": list(extent),
        "cut_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
    }, indent=2))
    return base_rev

PROMOTE_TO_MULTI avoids a whole class of problem on return: a crew that splits a parcel into two parts produces a multi-part geometry, and a layer declared as single-part rejects it at exactly the wrong moment.

Step 2 — Record operations on the device

The device writes an append-only log alongside the container. Each entry carries what happened, when it was observed, and how good the fix was:

{"op": "create", "id": "EAST-7K3M2P9Q4R", "observed_at": "2026-07-14T09:12:44Z",
 "fix": {"mode": "rtk", "hrms_m": 0.02}, "by": "crew-east-2"}
{"op": "move",   "id": "EAST-4A9B1C2D3E", "observed_at": "2026-07-14T11:41:02Z",
 "fix": {"mode": "dgps", "hrms_m": 0.6},  "by": "crew-east-2"}
{"op": "retire", "id": "EAST-8Z7Y6X5W4V", "observed_at": "2026-07-15T08:03:19Z",
 "reason": "structure demolished", "by": "crew-east-2"}

The fix block earns its place at reconciliation time: when two observations disagree, an RTK fix with 2 cm accuracy is a stronger claim than a handheld fix with sub-metre accuracy, and the rule that says so needs the number.

Step 3 — Reconcile against the current trunk

def reconcile(operations, base_state, trunk_state):
    """Three-way reconciliation of an offline operation log.

    Returns (applicable, conflicts). Nothing is written here — this function
    decides, and a separate step applies.
    """
    applicable, conflicts = [], []

    for op in operations:
        fid = op["id"]
        base = base_state.get(fid)
        trunk = trunk_state.get(fid)

        if op["op"] == "create":
            if fid in trunk_state:
                conflicts.append((op, "identifier already present on trunk"))
            else:
                applicable.append(op)
            continue

        if trunk is None:
            conflicts.append((op, "feature retired on trunk while offline"))
        elif base is not None and trunk["fingerprint"] == base["fingerprint"]:
            applicable.append(op)                       # trunk untouched — safe
        else:
            conflicts.append((op, "changed on both sides"))

    return applicable, conflicts

A create colliding with an existing identifier should be impossible with device-minted identifiers. Checking anyway is cheap, and when it does fire it means the identifier scheme has broken somewhere — which is worth finding out immediately rather than after the data has landed.

Step 4 — Resolve the genuine conflicts

def resolve(field_op, trunk_record):
    """Decide between a field observation and a trunk change.

    Observation authority first, observation time second. Upload time is
    deliberately not consulted anywhere in this function.
    """
    field_acc = field_op["fix"]["hrms_m"]
    trunk_acc = trunk_record.get("hrms_m", float("inf"))

    if field_acc <= trunk_acc / 2:
        return "field", "materially better fix"
    if trunk_acc <= field_acc / 2:
        return "trunk", "trunk observation materially better"

    if field_op["observed_at"] > trunk_record["observed_at"]:
        return "field", "more recent observation"
    if trunk_record["observed_at"] > field_op["observed_at"]:
        return "trunk", "more recent observation"

    return None, "escalate to steward"

The factor-of-two threshold prevents a marginal accuracy difference from deciding anything. Two observations of similar quality fall through to observation time, and a genuine tie escalates — the same structure used by manual review triggers for critical edits.

Step 5 — Apply, validate, and report back to the crew

# Apply the accepted operations onto a branch, never straight onto the trunk
git switch -c sync/crew-east-2026-07-15
python scripts/apply_operations.py --package packages/crew-east/ --accepted accepted.json

# The normal gates apply — an offline package gets no exemption
python scripts/validate_spatial.py data/parcels.gpkg --crs EPSG:3035 --topology

git add -A && git commit -m "Sync crew-east 2026-07-15: 214 applied, 6 escalated"

The report that goes back to the crew matters as much as the merge. A crew that hears “214 of your 220 edits landed, and these six need your notes” will bring the notes; a crew that hears nothing assumes everything landed and stops checking.

Code Reliability Patterns

Never renumber on arrival. If an identifier collision reaches the server, quarantine the package and fix the minting scheme. Renumbering silently converts a detectable defect into a permanent broken link between the data and the field record.

Treat the device clock as evidence, not as truth. Store the device timestamp and the GNSS-derived timestamp separately when both are available. A device whose clock drifted still produced valid observations; you just cannot order them by that clock.

Apply operations onto a branch. An offline sync is a merge request like any other, and it should be reviewable before it becomes trunk. This also gives you somewhere to put the escalated conflicts while the crew is consulted.

Make partial application impossible. Either the accepted operation set applies completely or nothing does. A half-applied package leaves the trunk in a state neither the crew nor the server can describe.

Performance & Scale Considerations

Offline packages are usually small — an extent’s worth of features and a few hundred operations — so the reconciliation itself is cheap. The costs sit elsewhere.

Cutting packages for a large crew deployment is a bulk clip operation, and doing it per crew from a national layer is wasteful. Clip once per region into an intermediate, then cut crew packages from that.

Retaining base revisions is the constraint that actually limits how long a package can stay out. If history is pruned or rewritten while a package is in the field, its merge base disappears and reconciliation degrades to a two-way compare. Keep a retention window that comfortably exceeds the longest planned deployment, and treat any history rewrite as an event that must check for outstanding packages first.

Photographs and attachments dominate upload size, frequently by two orders of magnitude over the vector data. Upload them separately and asynchronously, referenced by identifier, so a slow attachment transfer never blocks the reconciliation of the edits themselves.

Troubleshooting & Failure Modes

FAQ

Related

• Reconciling Offline Edits from Field Collection Apps — the sync step for QField and ArcGIS Field Maps packages specifically
• Designing Conflict-Free Identifiers for Offline Capture — identifier schemes compared, including what breaks each one
• Attribute Reconciliation for Tabular Spatial Data — the cell-level merge that runs on the attributes a package carries
• Manual Review Triggers for Critical Edits — where escalated field conflicts go
• Geometry Overlap Resolution Techniques — resolving the geometric half that identifier schemes cannot

Back to Conflict Resolution & Team Synchronization Workflows