Diffing Attribute-Only Changes Without Geometry Noise

A land-use code changed on 340 parcels and the diff reports 340 geometry modifications alongside it — which makes a five-minute review into an afternoon of proving nothing moved. This page is a focused companion to spatial diff algorithms for polygon data.

Concept & Context

Most edits to a mature spatial layer are attribute edits. Boundaries settle; classifications, ownership references and status codes keep moving. A diff that cannot distinguish the two makes the common case look like the dangerous one, and reviewers who see geometry changes on every pull request stop treating geometry changes as significant.

The noise has a specific source. Saving a feature through an editing client rewrites the whole record, and the geometry makes a round trip through the driver’s floating-point representation. Nothing about the shape changed; the last digits did. Comparing well-known binary directly reports a difference, and the feature is classified as geometrically modified.

Splitting the digest solves it cleanly. Hash the quantised geometry and the canonical attributes separately, and each feature falls into one of four states with a clear meaning: unchanged, attribute-only, geometry-only, or both. The first two dominate real work, and only the last two need a map.

How a save turns an attribute edit into a geometry change A sequence in which an editor changes one attribute, the client rewrites the whole feature on save, the driver round-trips the geometry through floating point, and a byte-comparing diff reports both halves as changed. Editor Client Driver Diff change land_use on one parcel write the whole feature geometry included, unchanged round-trip coordinates through float bytes differ in the last digits geometry AND attributes changed Nothing here is a bug except the last message, and quantising before hashing removes it.

Core Algorithmic Pipeline

  1. Quantise geometry to the layer’s declared precision before hashing it.
  2. Serialise attributes canonically — sorted keys, explicit nulls, fixed numeric formatting.
  3. Digest the two halves independently for every feature on both sides.
  4. Classify by comparing the pair of digests.
  5. Expand attribute changes to column level, so the report names the field that moved.
Two digests give four states, and each needs different review A grid of geometry digest against attribute digest producing four states: unchanged, attribute-only, geometry-only and both — with the review each one calls for. Attributes same Attributes differ Geometry same unchanged — no review attribute-only — a table Geometry differs geometry-only — a map both — a map and a table Most real work lands in the top-right cell, which is the one a single-digest diff cannot distinguish.

Working Implementation

"""Two-part feature diff: geometry and attributes classified independently."""
from __future__ import annotations

import hashlib
import json
from dataclasses import dataclass, field

import geopandas as gpd
from shapely import set_precision
from shapely.wkb import dumps as wkb_dumps

NULL = "\x00NULL\x00"


def geometry_digest(geom, grid: float) -> str:
    if geom is None or geom.is_empty:
        return "empty"
    snapped = set_precision(geom, grid)
    return hashlib.blake2b(
        wkb_dumps(snapped, hex=False, output_dimension=2, byte_order=1,
                  include_srid=False),
        digest_size=12,
    ).hexdigest()


def attribute_map(row, geom_col: str) -> dict[str, str]:
    out = {}
    for key in sorted(k for k in row.index if k != geom_col):
        value = row[key]
        if value is None or (isinstance(value, float) and value != value):
            out[key] = NULL
        elif isinstance(value, float):
            out[key] = f"{value:.10g}"
        else:
            out[key] = str(value)
    return out


def attribute_digest(attrs: dict[str, str]) -> str:
    payload = json.dumps(sorted(attrs.items()), ensure_ascii=False,
                         separators=(",", ":"))
    return hashlib.blake2b(payload.encode("utf-8"), digest_size=12).hexdigest()


@dataclass
class Diff:
    unchanged: list[str] = field(default_factory=list)
    attribute_only: list[dict] = field(default_factory=list)
    geometry_only: list[str] = field(default_factory=list)
    both: list[dict] = field(default_factory=list)
    added: list[str] = field(default_factory=list)
    removed: list[str] = field(default_factory=list)

    def summary(self) -> str:
        return (f"{len(self.unchanged)} unchanged, "
                f"{len(self.attribute_only)} attribute-only, "
                f"{len(self.geometry_only)} geometry-only, "
                f"{len(self.both)} both, "
                f"{len(self.added)} added, {len(self.removed)} removed")


def index_layer(path: str, key: str, grid: float) -> dict[str, dict]:
    gdf = gpd.read_file(path)
    geom_col = gdf.geometry.name
    out = {}
    for _, row in gdf.iterrows():
        attrs = attribute_map(row, geom_col)
        out[str(row[key])] = {
            "geom": geometry_digest(row.geometry, grid),
            "attr": attribute_digest(attrs),
            "values": attrs,
        }
    return out


def diff_layers(base_path: str, head_path: str, key: str, grid: float = 0.001) -> Diff:
    base = index_layer(base_path, key, grid)
    head = index_layer(head_path, key, grid)
    result = Diff()

    result.added = sorted(set(head) - set(base))
    result.removed = sorted(set(base) - set(head))

    for fid in sorted(set(base) & set(head)):
        b, h = base[fid], head[fid]
        geom_changed = b["geom"] != h["geom"]
        attr_changed = b["attr"] != h["attr"]

        if not geom_changed and not attr_changed:
            result.unchanged.append(fid)
        elif attr_changed and not geom_changed:
            result.attribute_only.append(
                {"id": fid, "columns": column_changes(b["values"], h["values"])}
            )
        elif geom_changed and not attr_changed:
            result.geometry_only.append(fid)
        else:
            result.both.append(
                {"id": fid, "columns": column_changes(b["values"], h["values"])}
            )
    return result


def column_changes(before: dict[str, str], after: dict[str, str]) -> list[dict]:
    """Which columns moved, and to what — the part a reviewer actually reads."""
    return [
        {"column": col,
         "from": None if before.get(col) == NULL else before.get(col),
         "to": None if after.get(col) == NULL else after.get(col)}
        for col in sorted(set(before) | set(after))
        if before.get(col) != after.get(col)
    ]

Rendering it for a pull request comment keeps the two classes visually separate, which is the point:

def render(diff: Diff, limit: int = 20) -> str:
    lines = [f"**Spatial diff** — {diff.summary()}", ""]
    if diff.attribute_only:
        lines += ["| feature | column | from | to |", "|---|---|---|---|"]
        for entry in diff.attribute_only[:limit]:
            for change in entry["columns"]:
                lines.append(
                    f"| `{entry['id']}` | `{change['column']}` | "
                    f"{change['from'] or '—'} | {change['to'] or '—'} |"
                )
        if len(diff.attribute_only) > limit:
            lines.append(f"| … | _{len(diff.attribute_only) - limit} more_ | | |")
    if diff.geometry_only or diff.both:
        lines += ["", f"**Geometry changed on {len(diff.geometry_only) + len(diff.both)} "
                      "feature(s)** — map review required."]
    return "\n".join(lines)

Validation & Output Verification

# An attribute-only edit must be classified as attribute-only
import subprocess
from spatial_diff import diff_layers

subprocess.run(["ogr2ogr", "-f", "GPKG", "/tmp/head.gpkg", "data/parcels.gpkg"],
               check=True)
subprocess.run(["ogrinfo", "/tmp/head.gpkg", "-sql",
                "UPDATE parcels SET land_use='industrial' "
                "WHERE parcel_uid='EAST-7K3M2P9Q4R'"], check=True)

d = diff_layers("data/parcels.gpkg", "/tmp/head.gpkg", key="parcel_uid")
assert not d.geometry_only and not d.both, "a driver round trip leaked into geometry"
assert [e["id"] for e in d.attribute_only] == ["EAST-7K3M2P9Q4R"]
assert d.attribute_only[0]["columns"][0]["column"] == "land_use"

# A genuine geometry edit must NOT be classified as attribute-only
subprocess.run(["ogrinfo", "/tmp/head.gpkg", "-sql",
                "UPDATE parcels SET geom = ST_Translate(geom, 0.5, 0) "
                "WHERE parcel_uid='EAST-4A9B1C2D3E'"], check=True)
d2 = diff_layers("data/parcels.gpkg", "/tmp/head.gpkg", key="parcel_uid")
assert "EAST-4A9B1C2D3E" in d2.geometry_only + [e["id"] for e in d2.both]
print("diff separates the two change classes:", d2.summary())

Both assertions matter equally. The first proves the noise is gone; the second proves the quantisation is not so coarse that it hides a real move — the failure mode described in choosing a coordinate quantisation grid.

When the Split Is Not Enough

Four classes cover most of what a reviewer needs, but two situations deserve their own handling. A feature that was split or merged appears as one removal and two additions, which is technically correct and tells the reviewer nothing about the relationship. Carrying a predecessor reference on the new features lets the report say “parcel 4471 split into 4471-A and 4471-B” instead of listing three unrelated changes.

The second is a bulk attribute update applied by a script, where several thousand features change one column to the same value. Rendering that per feature buries the one row that also changed something else. Group the attribute-only class by column and value transition, show the count, and list only the features whose change does not fit the dominant pattern — those are the ones worth a reviewer’s attention, and they are invisible in a flat listing.

Two cases the four states describe badly Two panels. A split or merge appears as one removal and two additions unless a predecessor reference is carried. A scripted bulk update appears as thousands of identical rows unless the report groups by column and value transition. SPLIT OR MERGE Appears as one removal and two additions Technically correct, and tells the reviewer nothing Carry a predecessor reference on the new features The report can then say 4471 became 4471-A and 4471-B SCRIPTED BULK UPDATE Thousands of features, one column, one value Per-feature rendering buries the exceptions Group by column and value transition, show the count List only the rows that break the dominant pattern In both cases the fix is in the report, not in the classifier. A correct diff nobody can read gets approved on trust, which is the same as not running it.

Failure Modes

  • Every attribute edit reports a geometry change — symptom: two change classes always move together. Root cause: geometry hashed at full precision. Fix: quantise before hashing.

  • A real geometry move reported as attribute-only — symptom: a shifted boundary escapes map review. Root cause: the quantisation grid is coarser than the move. Fix: set the grid from survey accuracy, and keep the second assertion above in CI.

  • Attribute diff reports every column — symptom: unchanged columns listed as changed. Root cause: inconsistent numeric or null formatting between the two reads. Fix: canonicalise attributes as above, with one null marker and one float format.

  • The report is unreadable at scale — symptom: a diff of 4,000 attribute changes as one wall of rows. Root cause: per-feature rendering with no aggregation. Fix: group by column and value transition, then list a sample; reviewers care that 340 parcels moved to industrial, not about each one.

Back to Spatial Diff Algorithms for Polygon Data