Security Boundaries in Spatial Repositories

Enforcing security boundaries in a geospatial repository means controlling access across four simultaneous surfaces — storage, repository metadata, binary pointer objects, and the geometry payload itself — a problem that does not arise in ordinary source-code version control. Part of Geospatial Data Versioning Fundamentals & Architecture.

Prerequisites & Environment Setup

Before implementing the workflow below, verify your environment meets these baseline requirements:

• Git 2.30+ with GPG or SSH signed commits enabled
• DVC 3.x or Git-LFS 3.x installed and configured against a remote object store
• Python 3.10+ with geopandas >= 0.14, shapely >= 2.0, pyproj >= 3.6, and numpy in an isolated virtual environment
• GDAL/OGR 3.6+ accessible on PATH (gdal-config --version to confirm)
• Open Policy Agent (OPA) 0.60+ or an equivalent rule-based access control engine
• Cloud object storage (AWS S3, GCS, or Azure Blob) with IAM/RBAC integration, or an on-premises NAS with equivalent role bindings
• ISO 19115 / OGC-compliant metadata schemas in place for dataset classification
• Precision tolerance standard documented: 1e-8 degrees for EPSG:4326 data, or project-specific equivalent

The four-layer boundary model that this page implements is visualised below. Understanding how the layers nest is critical: the large file handling in DVC for GIS guide explains why large raster datasets bypass standard Git diff engines and why their pointer files introduce a separate access surface.

Core Algorithmic Patterns

Pattern 1 — Tier-Keyed Policy Routing

Tier-keyed policy routing assigns each dataset a classification string (public, internal, restricted, ephemeral) at commit time and routes every subsequent operation — pointer resolution, remote fetch, merge queue decision — through an OPA policy that reads that tag. The spatial complexity is O(1) per access decision because the policy engine evaluates a hash lookup, not a geometry intersection. The critical invariant is that untagged datasets are denied by default, not permitted.

Pattern 2 — Topology-Aware Pre-Commit Gate

Standard byte-level diff tools cannot determine whether a geometry change is a benign coordinate correction or a covert attribute extraction. A topology-aware pre-commit gate runs shapely.is_valid() and CRS reconciliation on every changed vector file before the commit is accepted. This gate also surfaces the interaction between the delta tracking algorithms for vector data and attribute redaction: delta engines reconstruct geometry from incremental patches, so an unredacted attribute on any patch in the chain can survive into the final state. The gate must validate the reconstructed geometry, not only the patch.

Pattern 3 — Pointer Manifest Drift Detection

Pointer synchronization for raster datasets introduces a consistency requirement: the checksum recorded in the pointer file must match the object actually stored in the remote cache. Drift — where the pointer references a deleted, modified, or re-uploaded object — can be exploited to substitute an attacker-controlled payload. A nightly drift-detection job compares all pointer manifests against storage-backend ETags or MD5 checksums and raises an alert on any mismatch.

Production Workflow Implementation

Step 1 — Classify and Tag Every Spatial Asset

Map every dataset to its security tier before the first commit. Enforce classification via a repository-level CODEOWNERS file and a pre-commit hook that rejects untagged spatial assets. Write the tier as a custom metadata field in the dataset’s ISO 19115 record and, for DVC-tracked objects, embed it in the DVC .dvc file as a custom field.

# scripts/classify_spatial_asset.py
import json
import subprocess
import sys
from pathlib import Path

VALID_TIERS = {"public", "internal", "restricted", "ephemeral"}

def classify_asset(dvc_file: Path, tier: str) -> None:
    if tier not in VALID_TIERS:
        raise ValueError(f"Unknown tier '{tier}'. Must be one of {VALID_TIERS}.")
    meta = json.loads(dvc_file.read_text())
    meta.setdefault("meta", {})["security_tier"] = tier
    dvc_file.write_text(json.dumps(meta, indent=2))
    subprocess.run(["git", "add", str(dvc_file)], check=True)
    print(f"[classify] {dvc_file.name} → tier={tier}")

if __name__ == "__main__":
    classify_asset(Path(sys.argv[1]), sys.argv[2])

Step 2 — Gate Pointer Resolution with OPA

Version control systems track pointers, not raw binaries. Securing these pointers prevents unauthorized fetches and pointer drift. Deploy the following OPA Rego policy to gate DVC or Git-LFS pointer resolution based on user identity and dataset classification:

package spatial_repo.access

default allow = false

# Authenticated users may always read public datasets
allow {
    input.user.authenticated == true
    input.dataset.security_tier == "public"
}

# Internal datasets require authenticated org membership
allow {
    input.user.authenticated == true
    input.dataset.security_tier == "internal"
    input.user.org == "my-org"
}

# Restricted datasets require explicit team allowlist
allow {
    input.user.authenticated == true
    input.dataset.security_tier == "restricted"
    input.user.teams[_] == "gis-security-team"
}

# CI service accounts may fetch ephemeral layers during their own job
allow {
    input.user.service_account == true
    input.dataset.security_tier == "ephemeral"
    input.user.job_id == input.dataset.issuing_job_id
}

# Deny anything untagged — no implicit permit
deny[msg] {
    not input.dataset.security_tier
    msg := "Dataset lacks security_tier tag. Pointer resolution denied."
}

Integrate this policy into your CI/CD pipeline and storage gateway. For the format-specific vulnerabilities that arise when teams use shapefiles — including .prj injection and .dbf field truncation — review setting up secure access controls for versioned shapefiles.

Step 3 — Enforce Geometry and Attribute Policies

Repository-level controls are insufficient when geometries themselves leak sensitive information. The following Python implementation validates topology, normalises CRS to EPSG:4326, and applies field-level redaction to restricted layers:

import geopandas as gpd
import pyproj
import hashlib
import hmac
import os
import numpy as np
from shapely.validation import make_valid

CANONICAL_CRS = pyproj.CRS("EPSG:4326")
HMAC_SECRET = os.environ["SPATIAL_HMAC_SECRET"].encode()

# Columns requiring redaction in restricted layers
SENSITIVE_COLS = ["owner_name", "parcel_id", "exact_address", "phone", "email"]

def _hmac_hash(value: str) -> str:
    """Deterministic HMAC-SHA256 preserves referential integrity across joins."""
    return hmac.new(HMAC_SECRET, value.encode(), hashlib.sha256).hexdigest()[:16]

def enforce_spatial_boundary(
    gdf_path: str,
    tier: str,
    reject_unknown_epsg: bool = True,
) -> gpd.GeoDataFrame:
    gdf = gpd.read_file(gdf_path)

    # CRS normalisation — reject commits with non-allowlisted EPSG codes
    if gdf.crs is None:
        raise ValueError(f"File {gdf_path} has no CRS. Assign one before committing.")
    if reject_unknown_epsg and gdf.crs.to_epsg() is None:
        raise ValueError(f"Non-EPSG CRS in {gdf_path} rejected by policy.")
    if gdf.crs != CANONICAL_CRS:
        gdf = gdf.to_crs(CANONICAL_CRS)

    # Topology validation — repair invalid geometries, log repaired count
    invalid_mask = ~gdf.geometry.is_valid
    repaired = int(invalid_mask.sum())
    if repaired:
        gdf.loc[invalid_mask, "geometry"] = (
            gdf.loc[invalid_mask, "geometry"].apply(make_valid)
        )
        print(f"[boundary] Repaired {repaired} invalid geometries in {gdf_path}")

    # Attribute policy — HMAC hash instead of null to preserve join keys
    if tier == "restricted":
        for col in SENSITIVE_COLS:
            if col in gdf.columns:
                gdf[col] = gdf[col].astype(str).apply(_hmac_hash)

    return gdf

Step 4 — Wire the Gate into the Pre-Commit Lifecycle

Embed boundary checks into your Git lifecycle using pre-commit hooks and post-merge webhooks.

.pre-commit-config.yaml:

repos:
  - repo: local
    hooks:
      - id: spatial-boundary-check
        name: Validate Spatial Security Boundaries
        entry: python scripts/validate_spatial_boundary.py
        language: system
        types: [file]
        files: \.(gpkg|shp|geojson|parquet|tif|tiff)$
        pass_filenames: true
      - id: dvc-tier-check
        name: Require security_tier tag on DVC files
        entry: python scripts/classify_spatial_asset.py --check-only
        language: system
        files: \.dvc$
        pass_filenames: true

Step 5 — Automate Audit Trails and Drift Detection

Manual reviews fail at scale. Three mechanisms are required:

1. Immutable access logs — Write all OPA access decisions, pointer fetches, and merge-queue results to an append-only log (AWS CloudTrail, Datadog, or an ELK stack). Log the user identity, tier, dataset path, and decision outcome.
2. Webhook signature verification — Sign all repository webhooks with HMAC-SHA256. Reject unsigned or timestamp-expired (> 300 s old) payloads to prevent replay attacks. Rotate secrets every 90 days.
3. Nightly pointer manifest drift detection — Compare all .dvc pointer checksums against the remote storage ETags. Alert immediately on any mismatch.

# scripts/drift_detect.py
import boto3
import json
from pathlib import Path

def detect_pointer_drift(dvc_root: str, s3_bucket: str) -> list[dict]:
    s3 = boto3.client("s3")
    mismatches = []
    for dvc_file in Path(dvc_root).rglob("*.dvc"):
        meta = json.loads(dvc_file.read_text())
        for entry in meta.get("outs", []):
            md5 = entry.get("md5", "")
            key = f"files/md5/{md5[:2]}/{md5[2:]}"
            try:
                head = s3.head_object(Bucket=s3_bucket, Key=key)
                remote_etag = head["ETag"].strip('"')
                if remote_etag != md5:
                    mismatches.append({"file": str(dvc_file), "expected": md5, "found": remote_etag})
            except s3.exceptions.NoSuchKey:
                mismatches.append({"file": str(dvc_file), "expected": md5, "found": "MISSING"})
    return mismatches

Code Reliability Patterns

CRS tolerance snapping — Always snap coordinate precision to the canonical tolerance (1e-8 degrees for EPSG:4326) before topology validation. Floating-point drift from reprojection can produce near-duplicate vertices that pass is_valid() but generate spurious self-intersections under repeated round-trips.

Rollback on partial failure — Wrap multi-step boundary enforcement in a transaction pattern. If CRS normalisation succeeds but attribute redaction raises a KeyError, roll back the CRS change and re-raise so the commit is rejected cleanly rather than leaving a partially processed file.

Poison-pill sentinel — Prepend a deliberate canary record with a known HMAC value to every restricted export. If the canary appears in a downstream dataset without the expected hash, it signals that redaction was bypassed.

Empty-geometry guard — make_valid() can return GEOMETRYCOLLECTION EMPTY for degenerate inputs. After repair, filter out empty geometries and log them; empty features that slip through can corrupt spatial indexes on merge.

Performance and Scale Considerations

Boundary enforcement adds latency to every commit. Keep the pre-commit gate under 5 seconds for typical dataset sizes by following these guidelines:

• Spatial index acceleration — Build an STRtree (shapely.STRtree) over the incoming geometries before running validity checks. For large polygon layers (> 500 k features), process in 50 k-row chunks using geopandas chunksize parameter.
• Memory-mapped reads — Open raster payloads with rasterio using mode="r" (memory-mapped); avoid loading the full array into RAM when only the metadata header needs inspection for CRS and tier tag validation.
• Parallel CRS reprojection — Use pyproj.Transformer.from_crs(..., always_xy=True) with numpy vectorised coordinate arrays rather than row-wise reprojection; this is typically 10–30× faster on large point datasets.
• Benchmark reference — On a 2023 M2 MacBook Pro, the full gate (CRS normalisation + topology validation + HMAC redaction) processes a 1 M-row polygon GeoPackage in approximately 18 seconds. For repositories where this exceeds the acceptable hook latency, run the full gate only in CI and use a lightweight metadata-only check in the local hook.

Troubleshooting and Failure Modes

FAQ

Related

• Setting Up Secure Access Controls for Versioned Shapefiles — format-specific vulnerability mitigation for .prj, .dbf, and .shx components
• Delta Tracking Algorithms for Vector Data — how incremental patch chains interact with attribute redaction requirements
• Pointer Synchronization for Raster Datasets — pointer manifest consistency and drift detection for binary payloads
• Large File Handling in DVC for GIS — storage architecture decisions that determine where security boundaries can be enforced

Back to Geospatial Data Versioning Fundamentals & Architecture