Plate № — · observability · diagnostics
A pattern from the gf.cx specimen book
Decode the jump
"I need to code in a JSON page that helps decode large jumps in data — and exposes the types of files, number of files, average file size — as a page. The delta between before/after is the mystery."
The problem
An aggregate metric moves. Overnight the dan-immich R2 bucket ticks up +1.1 GB; the storage sparkline flags the step and captions it with a heuristic guess — "objects replaced with larger." But the metric only ever knew two scalars: total bytes and total object count. It cannot say which files changed. The delta between yesterday's size and today's is a mystery, and the metric alone can never explain itself — a sum has forgotten its terms by the time you read it.
The fix is to keep the terms. Periodically capture a content fingerprint — a manifest that breaks the aggregate down by a meaningful dimension (here, file extension) with a count, total bytes, and average size per group — and keep a short history of them. When the aggregate jumps, diff the two most recent manifests: which groups grew, which shrank, what's new, what's gone. The before/after diff is the decoded answer to the mystery.
| Piece | What it does |
|---|---|
| the content fingerprint (a by-group manifest) | Break the aggregate down by a meaningful dimension — here, file extension. Per group: file count, total bytes, average size. Stored as a short JSON history (the last 12 manifests per bucket). The manifest is exactly what the metric forgot: the terms behind the sum. |
| move-triggered capture | Re-fingerprint only when the aggregate actually moved, using the same notability thresholds the sparkline already computes — ±50 MB absolute, ±20% of prior bytes, or an object count outside a small flat band. The expensive listing rides the cheap metric; a 493,358-object bucket is walked only on the runs where it jumped, never idly. |
| the before/after diff | With two manifests, diff them into a signed Δ per group (Δ files, Δ bytes), tagged new or gone for types that appeared or vanished, sorted by absolute byte impact so the dominant driver becomes the headline. This is the answer the metric couldn't give. |
| an honest empty state | With only one manifest there is nothing to diff yet, and the page says so rather than faking a delta. A brand-new bucket's page still exists — so the link from the bucket page is never broken — and states plainly that the first listing hasn't landed. |
The load-bearing inversion is that the metric you're trying to explain is also the cheapest possible trigger for the explanation. You never schedule the expensive fingerprint; you let the movement you already detect pull it. And the "before" you'll need has to exist before you know you need it — so capture-on-movement doubles as baseline-building: the first jump writes the baseline as a side effect, and the second jump is the first one you can fully decode.
The element, in-page
The first instance is the dan-immich R2 bucket — 493,358 objects across 18 file types. Its manifest page opens on the current breakdown, "Current breakdown · by file type": .jpg, RAW .cr2/.dng, .avif, video .3gp/.avi, each with its file count, total size, average size, and share of bytes. Above it — once a second manifest exists — sits "What changed · the mystery, decoded," the signed before/after diff.
Travel into the live page below. The minimal instance is immich-db: 11 .gz files (gzipped Postgres dumps), one type, 5.5 GB — the case where the breakdown is a single row and a jump can only mean "the dumps got bigger" or "there are more of them." Each /r2/<bucket>/ page carries a "decode file types ↗" link into its manifest page.
status.gf.cx/r2/dan-immich/manifest/ — the first instance: the 493,358-object photo bucket, its 18 file types broken out with count, total size, and average size, and the before/after diff that resolves each jump. status.gf.cx/r2/immich-db/manifest/ is the minimal case — 11 .gz dumps, one row. Both are generated by gfcx_r2_manifest.py.
Where it applies
- Object-storage buckets whose byte count moves without a paper trail — the first instance. A
+1.1 GBstep resolves into which extensions grew and by how many files. - Database table growth — a table's row count or on-disk size jumps; fingerprint by table (or by a category column) to see which one, and whether it's more rows or fatter rows.
- Index or cache size — a search index or CDN cache balloons; break it down by document type or key prefix to find the driver.
- Log volume — a log store's daily bytes spike; group by source or severity to surface the noisy emitter behind the spike.
- Any monitored aggregate whose jumps are opaque — the pattern fits wherever a single scalar has forgotten the terms that sum to it.
- Not needed when the metric already carries its own breakdown — if the dashboard already reports bytes-by-type natively, there is nothing to decode.
Reusable elements
The named, copyable pieces — lift any one without the others:
- the by-group content manifest — a periodic capture that breaks an aggregate down by a meaningful dimension, storing
count+ totalbytes+ average size per group as a short, bounded history (the last N per subject). The terms behind the sum, kept. - the move-triggered capture — use the aggregate metric as its own cheap trigger: re-fingerprint only when the number moved past the same thresholds your alerting already flags (an absolute floor or a percentage or a count band). The expensive listing never runs on an idle subject.
- the two-manifest before/after diff — diff the two most recent captures into a signed Δ per group (Δ count, Δ bytes), tagged
new/gonefor groups that appeared or vanished, sorted by absolute impact so the dominant mover is the headline. - the honest single-manifest empty state — with only one capture, say "nothing to diff yet" instead of rendering a fake zero-delta; the page still exists so the entry link is never broken.
Reference
- Source
- Dan, 2026-08-19 — "I need to code in a JSON page that helps decode large jumps in data — and exposes the types of files, number of files, average file size — as a page. The delta between before/after is the mystery."
- In use / example
- status.gf.cx/r2/dan-immich/manifest/ — the first instance: 493,358 objects across 18 file types (photos, RAW
.cr2/.dng,.avif, video.3gp/.avi). status.gf.cx/r2/immich-db/manifest/ — the minimal case: 11.gzPostgres dumps, one type, 5.5 GB. Both generated bygfcx_r2_manifest.py. - Companion patterns
- Show your work — a sibling "make an opaque number legible" move. Where that gives a derived figure its own method page, this gives an aggregate metric its own decode page.
- Reusable elements
- the by-group content manifest · the move-triggered capture · the two-manifest before/after diff · the honest single-manifest empty state (listed above).
- Origin
- The R2 storage sparkline captioned a jump with a heuristic guess — "objects replaced with larger." This pattern upgrades that guess into a sourced, itemized answer: the actual file types, counts, and average sizes that moved, diffed before against after.