Automating Basemap Release Tags with CI Pipelines

Cutting a basemap release by hand invites drift — a forgotten dvc push, a tag that predates the data, notes written from memory — so the whole cut belongs in a pipeline, one concrete workflow within release tagging strategies for spatial basemaps.

Concept & Context

A basemap release is a promise: the tag v2.4.0+20260711 should always check out the exact bytes it shipped, complete with the notes and manifest that describe them. Keeping that promise manually is fragile. Someone tags the commit but forgets to push the large artifacts to remote storage, so a colleague checks out the tag and gets dangling pointers. Someone writes the version by hand and picks a number out of sequence. Someone drafts release notes from memory and omits a layer. Each slip erodes trust in the tag as a stable reference.

Automation removes the discretion. When a release is triggered, the pipeline computes a deterministic data fingerprint — a hash over the tracked basemap artifacts in canonical order — so it can tell whether the data actually changed and record exactly what shipped. It derives a combined semantic plus calendar tag, so the version is both ordered and dated. It runs dvc push before tagging, guaranteeing the data exists in remote storage before the tag advertises it. Then it emits a manifest and generates notes from the commit range.

This builds directly on the discipline in feature branching for GIS development teams: feature work is isolated, validated, and merged to a release branch, and only then does the release pipeline fire. Because the basemap artifacts are content-addressed, the fingerprint and the DVC store reinforce each other — a topic explored further in large file handling in DVC for GIS.

Core Release Pipeline

  1. Trigger the release. Fire on a merge to the release branch or a manual dispatch that carries the release type (major, minor, patch) as an input.
  2. Compute the data fingerprint. Hash every tracked artifact in a canonical, sorted order to produce a fingerprint that is independent of file modification times and rebuild noise.
  3. Derive the version tag. Read the latest tag, apply the requested semantic bump, and append the build date to form a combined tag such as v2.4.0+20260711.
  4. Push DVC artifacts. Run dvc push so the large basemap files land in remote storage, addressed by content hash, before the tag is written.
  5. Generate the manifest and notes. Emit a manifest capturing the fingerprint, tag, layer inventory, and DVC pointers, and build human-readable notes from the commit range since the previous tag.
  6. Write and push the annotated tag. Commit the manifest, create an annotated Git tag, and push both so the release is fully reproducible.

Working Implementation

The pipeline pairs a GitHub Actions job with a Python script. The script fingerprints the DVC-tracked artifacts, derives the next combined tag, pushes the data, writes a manifest, and tags the commit — ordered so the tag is only written once the artifacts are safely in remote storage.

# .github/workflows/basemap-release.yml
name: Basemap Release
on:
  workflow_dispatch:
    inputs:
      bump:
        description: "Semantic bump: major | minor | patch"
        default: "patch"
jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }        # full history for tag + note generation
      - run: pip install dvc[s3]
      - name: Cut the release
        env:
          GIT_AUTHOR_NAME: release-bot
          GIT_AUTHOR_EMAIL: release-bot@ci
        run: python tools/cut_release.py --bump "${{ github.event.inputs.bump }}"
#!/usr/bin/env python3
"""
Cut a versioned basemap release: fingerprint -> tag -> dvc push -> manifest -> tag.

Usage:
    python cut_release.py --bump patch
"""

import argparse
import hashlib
import json
import re
import subprocess
from datetime import datetime, timezone
from pathlib import Path

ARTIFACT_GLOB = "basemap/**/*.dvc"     # DVC pointer files under version control


def run(cmd: list[str]) -> str:
    return subprocess.run(cmd, capture_output=True, text=True, check=True).stdout.strip()


def data_fingerprint() -> str:
    """
    Deterministic fingerprint over DVC content hashes in canonical (sorted) order.
    Each .dvc pointer records the md5 of its tracked artifact, so hashing the
    pointers captures the actual data bytes without downloading gigabytes.
    """
    h = hashlib.sha256()
    for ptr in sorted(Path(".").glob(ARTIFACT_GLOB)):
        h.update(ptr.read_bytes())
    return h.hexdigest()[:16]


def latest_semver() -> tuple[int, int, int]:
    """Highest vMAJOR.MINOR.PATCH tag in the repo, or 0.0.0 if none exist."""
    tags = run(["git", "tag", "--list", "v*"]).splitlines()
    best = (0, 0, 0)
    for t in tags:
        m = re.match(r"v(\d+)\.(\d+)\.(\d+)", t)
        if m:
            best = max(best, tuple(int(g) for g in m.groups()))
    return best


def next_tag(bump: str) -> str:
    """Bump the semantic version and append a calendar stamp: vX.Y.Z+YYYYMMDD."""
    major, minor, patch = latest_semver()
    if bump == "major":
        major, minor, patch = major + 1, 0, 0
    elif bump == "minor":
        minor, patch = minor + 1, 0
    else:
        patch += 1
    stamp = datetime.now(timezone.utc).strftime("%Y%m%d")
    return f"v{major}.{minor}.{patch}+{stamp}"


def release_notes() -> str:
    """One-line-per-commit notes since the previous tag."""
    prev = run(["git", "describe", "--tags", "--abbrev=0"]) if run(
        ["git", "tag"]) else ""
    rng = f"{prev}..HEAD" if prev else "HEAD"
    return run(["git", "log", "--no-merges", "--pretty=- %s", rng])


def main() -> None:
    ap = argparse.ArgumentParser()
    ap.add_argument("--bump", default="patch", choices=["major", "minor", "patch"])
    args = ap.parse_args()

    fingerprint = data_fingerprint()
    tag = next_tag(args.bump)

    # Push data FIRST — the tag must never advertise artifacts that aren't stored.
    run(["dvc", "push"])

    manifest = {
        "tag": tag,
        "fingerprint": fingerprint,
        "released_at": datetime.now(timezone.utc).isoformat(),
        "artifacts": sorted(str(p) for p in Path(".").glob(ARTIFACT_GLOB)),
        "notes": release_notes(),
    }
    manifest_path = Path("releases") / f"{tag}.json"
    manifest_path.parent.mkdir(exist_ok=True)
    manifest_path.write_text(json.dumps(manifest, indent=2) + "\n")

    # Commit the manifest, then annotate and push the tag (data already stored).
    run(["git", "add", str(manifest_path)])
    run(["git", "commit", "-m", f"release: basemap {tag} (fp {fingerprint})"])
    run(["git", "tag", "-a", tag, "-m", f"Basemap release {tag}\nfingerprint {fingerprint}"])
    run(["git", "push", "origin", "HEAD", "--follow-tags"])
    print(f"Released {tag} — fingerprint {fingerprint}")


if __name__ == "__main__":
    main()

Validation & Output Verification

After the pipeline runs, confirm the tag, the manifest, and the stored data all agree. First check the annotated tag exists on the remote and its message records the fingerprint:

git ls-remote --tags origin | grep '+20260711'
git show v2.4.0+20260711 --stat | head -5

Prove the release is actually reproducible by checking out the tag in a clean clone and pulling its data from remote storage — the dvc pull must succeed with no missing artifacts:

git checkout v2.4.0+20260711
dvc pull
dvc status   # expected: "Data and pipelines are up to date."

Confirm the fingerprint in the manifest matches a freshly recomputed one, which catches a manifest that drifted from the data it claims to describe:

python - <<'PY'
import hashlib, json, pathlib
m = json.loads(pathlib.Path("releases/v2.4.0+20260711.json").read_text())
h = hashlib.sha256()
for p in sorted(pathlib.Path(".").glob("basemap/**/*.dvc")):
    h.update(p.read_bytes())
assert h.hexdigest()[:16] == m["fingerprint"], "fingerprint mismatch"
print("fingerprint OK:", m["fingerprint"])
PY

Treat the manifest as the release’s contract, the same way how to tag and release versioned OpenStreetMap extracts treats its VERSION.json.

Failure Modes

  • Tag written but dvc pull fails for consumers — the tag was created before dvc push completed, so the remote lacks the artifacts. Fix: keep push strictly before tagging, as the script does, and treat a failed push as a hard stop that aborts the release.
  • Fingerprint changes on every run with no data change — the hash is being computed over the artifacts’ modification times or absolute paths rather than their content hashes. Fix: hash the DVC pointer files (which carry the content md5) in sorted order, so identical data yields an identical fingerprint.
  • Version tag lands out of sequence — the bump logic read a lightweight or mis-formatted tag and reset the counter. Fix: parse only well-formed vX.Y.Z tags with a strict regex, as latest_semver does, and ignore anything that does not match.
  • Release notes are empty — git describe found no previous tag in a shallow clone, so the commit range collapsed. Fix: check out with full history (fetch-depth: 0) so tag discovery and the commit range both resolve.

Back to Release Tagging Strategies for Spatial Basemaps