Versioning PMTiles Archives for Map Releases

A tile archive is one file, which makes it wonderfully simple to publish and completely opaque to diff — so the versioning question becomes what to store, and how to tell one build from another. This page is a focused companion to cloud-native spatial formats for versioned pipelines.

Concept & Context

PMTiles packs an entire tile pyramid into a single file with an internal directory, so a client can fetch any tile with a couple of ranged requests. That design makes publication trivially atomic: one object either exists or does not, and there is no half-published tree of millions of tiles.

For versioning it creates two questions. The first is what to store. An archive is derived data — it can be rebuilt from the vector sources — so tracking every nightly build fills an object store with gigabytes nobody will ever fetch. The second is how to compare two archives, since a byte diff of two builds reports that everything changed even when one road moved.

Both have the same answer: version the recipe continuously and the releases permanently, and compare at tile granularity rather than byte granularity. The recipe is small, reviewable and exactly reproducible; the releases are what consumers hold and must remain retrievable; and a tile-level diff turns a binary blob into a map of what changed.

Version the recipe continuously, the releases permanently Two panels. The recipe — tiler version, options and input hashes — is small, reviewable and belongs in every commit. The archive is large derived data, worth storing only for releases, which consumers hold and bug reports refer to. THE RECIPE — ALWAYS Tiler version, every option, input hashes Kilobytes, and readable in a diff Reproduces the archive exactly Changes only when something real changes THE ARCHIVE — RELEASES ONLY Gigabytes of derived data per build Rebuildable from the recipe at any time Stored permanently once released Retained past its support window for bug reports Track every build and the store fills with archives nobody will fetch. The exception is a release: that is the artifact somebody cached, and it has to remain retrievable.

Core Algorithmic Pipeline

  1. Pin the build: tiler version, every option, and the content hashes of the inputs.
  2. Track the recipe in the repository, and track only released archives as artifacts.
  3. Verify reproducibility by rebuilding and comparing the archive digest.
  4. Diff two releases by tile, reporting counts by zoom and a geographic summary.
  5. Retain released archives beyond their support window, since bug reports outlive releases.

Working Implementation

"""Build, version and compare PMTiles archives."""
from __future__ import annotations

import hashlib
import json
import subprocess
from dataclasses import dataclass, asdict
from pathlib import Path


@dataclass
class Recipe:
    """Everything that determines the archive's bytes."""
    tippecanoe_version: str
    options: list[str]
    inputs: dict[str, str]            # path -> content hash
    name: str

    def digest(self) -> str:
        payload = json.dumps(asdict(self), sort_keys=True, separators=(",", ":"))
        return hashlib.sha256(payload.encode("utf-8")).hexdigest()[:16]


def tiler_version() -> str:
    out = subprocess.check_output(["tippecanoe", "--version"], text=True,
                                  stderr=subprocess.STDOUT)
    return out.strip().split()[-1]


def file_hash(path: str) -> str:
    h = hashlib.sha256()
    with open(path, "rb") as fh:
        while block := fh.read(1 << 20):
            h.update(block)
    return h.hexdigest()


def build(recipe: Recipe, out_path: str) -> dict:
    subprocess.run(
        ["tippecanoe", "-o", out_path, "--force", *recipe.options,
         "--name", recipe.name, *recipe.inputs.keys()],
        check=True,
    )
    archive_sha = file_hash(out_path)
    record = {
        "recipe": asdict(recipe),
        "recipe_digest": recipe.digest(),
        "archive_sha256": archive_sha,
        "bytes": Path(out_path).stat().st_size,
    }
    Path(out_path + ".build.json").write_text(json.dumps(record, indent=2))
    return record


def tile_diff(a_path: str, b_path: str, max_zoom: int = 14) -> dict:
    """Which tiles differ between two archives, summarised by zoom.

    Comparing tile payload hashes rather than archive bytes is what turns an
    opaque diff into something reviewable — and it is immune to the archive
    layout changing when the tiler is upgraded.
    """
    from pmtiles.reader import Reader, MmapSource

    a = Reader(MmapSource(open(a_path, "rb")))
    b = Reader(MmapSource(open(b_path, "rb")))

    changed_by_zoom: dict[int, int] = {}
    added = removed = 0
    samples: list[str] = []

    for z in range(0, max_zoom + 1):
        span = 2 ** z
        for x in range(span):
            for y in range(span):
                ta, tb = a.get(z, x, y), b.get(z, x, y)
                if ta == tb:
                    continue
                if ta is None:
                    added += 1
                elif tb is None:
                    removed += 1
                else:
                    changed_by_zoom[z] = changed_by_zoom.get(z, 0) + 1
                if len(samples) < 12:
                    samples.append(f"{z}/{x}/{y}")

    return {
        "changed_by_zoom": dict(sorted(changed_by_zoom.items())),
        "added": added,
        "removed": removed,
        "sample_tiles": samples,
    }

The recipe lives in the repository; the archive is tracked only when it is released:

# dvc.yaml
stages:
  tiles:
    cmd: python scripts/build_tiles.py --out build/basemap.pmtiles
    deps:
      - data/national/parcels.geojson
      - data/national/roads.geojson
      - scripts/build_tiles.py
    params:
      - tiles.max_zoom
      - tiles.min_zoom
    outs:
      - build/basemap.pmtiles:
          cache: false        # rebuildable; only releases are stored
      - build/basemap.pmtiles.build.json   # the record, which IS versioned

At release time the archive is promoted into tracked storage, which is the point at which permanence starts to matter:

dvc add --to-remote releases/${TAG}/basemap.pmtiles
git add releases/${TAG}/basemap.pmtiles.dvc build/basemap.pmtiles.build.json
git commit -m "Release ${TAG}: basemap tiles"
Reading a tile-level diff between two releases Horizontal bars of tiles changed per zoom level between two releases. A localised source edit changes many tiles at deep zooms and few at coarse ones; a flat profile across all zooms means the whole build shifted rather than the data. TILES CHANGED, BY ZOOM z6 2 z8 9 z10 41 z12 168 z14 604 a real, localised source edit A flat profile — as many z6 tiles as z14 — is a tiler change wearing a data change's clothes.

Validation & Output Verification

# Reproducibility: the same recipe must produce the same archive
python scripts/build_tiles.py --out /tmp/first.pmtiles
python scripts/build_tiles.py --out /tmp/second.pmtiles
diff <(jq -r .archive_sha256 /tmp/first.pmtiles.build.json) \
     <(jq -r .archive_sha256 /tmp/second.pmtiles.build.json) \
  && echo "build is reproducible" \
  || echo "FAIL: identical inputs produced different archives"

# The archive must be a valid PMTiles file that serves tiles
python -m pmtiles.info /tmp/first.pmtiles | head -12
# A source edit must change tiles only where the edit was
from tiles import tile_diff
d = tile_diff("releases/v2026.07/basemap.pmtiles", "build/basemap.pmtiles")
print(json.dumps(d, indent=2))
assert d["changed_by_zoom"], "a known source edit changed no tiles"
assert max(d["changed_by_zoom"]) >= 12, "the edit did not reach the deepest zooms"
# Coarse zooms should change far less than fine ones — an edit that changes as
# many z6 tiles as z14 tiles usually means the whole build shifted.
assert d["changed_by_zoom"].get(6, 0) < d["changed_by_zoom"].get(14, 1)

That last assertion is a cheap detector for the class of failure where a tiler upgrade or an option change alters every tile. The diff will be enormous, the map will look the same, and without the check nobody notices until the release is published and every client re-downloads everything.

Proving a release can be rebuilt Four steps: pin the tiler and record its version, sort the inputs so ordering is fixed, rebuild from the recipe alone, and compare the archive digest against the one recorded at release. 1 Pin and record the tiler a version bump rewrites every tile 2 Sort the inputs feature order reaches the archive bytes 3 Rebuild from the recipe alone on a clean machine 4 Compare the digest against the release record Run it on the current release on a schedule: it fails the day a dependency moves, not the day a user reports it.

Failure Modes

  • Every rebuild produces a different archive — symptom: the reproducibility check fails on unchanged inputs. Root cause: unsorted input features, an embedded build timestamp, or an unpinned tiler. Fix: sort inputs, pin the tiler, and record its version in the build record.

  • The object store fills with tile archives — symptom: storage growth dominated by derived data. Root cause: tracking every build rather than every release. Fix: cache: false on the build output; promote to tracked storage only at release.

  • A tiler upgrade rewrites every tile — symptom: an enormous diff with no visible map change. Root cause: a change in geometry simplification or encoding defaults. Fix: treat a tiler upgrade as a deliberate release with a note, and check the zoom distribution before publishing.

  • A released archive cannot be retrieved — symptom: a bug report about a release nobody can reproduce. Root cause: the release archive was pruned with the intermediates. Fix: retain released artifacts beyond their support window, and check them on a schedule as release tagging strategies recommends.

Back to Cloud-Native Spatial Formats for Versioned Pipelines