# On-screen deck for report videos A persistent bottom panel ("the deck") for a report cut, rendered as ONE HyperFrames composition over the whole concat: a pip timeline (one subtle, unlabelled pip per clip), a big authored title per clip, an automatic source-and-date subtitle, and the clip's QR. It replaces the old chrome for a cut that opts in — the citation header, the corner QR and the ffmpeg section footer — and the footage is scaled to ~82 % above it. At every clip change the pip travels and the title, subtitle and QR hand over to the next clip's. Opt-in per manifest (`render.chrome`). Every manifest without it builds byte-for-byte as before. Generic, not tied to any one report. Two ways to edit: umtool's On-screen section, or an agent editing the manifest. First application: the ferret-rescue report (`~/reports/ferret-rescue`). ## Slices ``` S0 ─┬─> S1 ─┬─> S3 ─┬─> S6 ─> S7 │ └─> S4 ─┼─> S5 ─┘ └─> S2 ─────────┘ S3 ─> S8 ``` | Slice | What | Owns (files) | |---|---|---| | S0 | This contract; `deck.mjs` + `deck.test.mjs`; `attributionParts` / `deckSubtitle` / `formatDeckDate`; `segmentOffsets` on `scheduleFrom` | `report-to-video/deck*.mjs`, `attribution.mjs` | | S1 | Pipeline: deck framing for clip/image/card segments, chrome skipped when `deckOn`, schedule writer, chapters, `assertChrome` at build start, `reservedFooterHeight`/`chromeRegions` deck branches | `build-video.mjs` (segment builders, chapters, helpers), `render-cards.mjs` | | S2 | Composition: `chrome-deck.mjs` + the compose-chrome port (region dispatch, `--still`, `--from`, workers/quality/format, vendored GSAP, QR, version pin, render cache) | `compose-chrome.mjs`, `chrome-deck.mjs`, `report-to-video/assets/` | | S3 | Build integration: one command composes + renders + overlays; transition-0 `applyChrome`; `--chrome-only` / `--no-chrome` / `--chrome-preview`; driver `chromeOnly` | `build-video.mjs` (`buildVideo`, concat), `umtool/lib/report/driver.mjs` | | S4 | umtool writers and routes | `umtool/lib/report/{manifest,serve}.mjs`, `umtool/app/api/report/{window,chrome,still,video}` | | S5 | umtool UI | `umtool/components/projects/OnscreenSection.tsx`, `ReportProject.tsx`, `ClipBench.tsx` | | S6 | e2e | `umtool/e2e/onscreen.spec.ts`, fixture stubs | | S7 | Docs | `umtool/report-to-video/README.md`, `umtool/docs/report-video.md`, quirks | | S8 | First application | the ferret manifest (outside the repo) | ## S0 contract Everything below is binding on the slices. A slice that needs to change it says so in its report; it does not quietly diverge. ### Manifest: `render.chrome` ```jsonc "render": { "chrome": { "engine": "hyperframes", "layout": "deck", "deck": { // every key optional; defaults shown "height": 190, // px, integer 120–400 "footageScale": 0.82, // 0.5–1, and the footage must fit above the deck "background": "panel", // "panel" (lifted) | "flush" "pip": { "spacing": "even", "size": 6, "activeSize": 12 }, // spacing "even" | "time" "title": { "size": 54, "maxChars": 48 }, // maxChars is the editor's counter, not a refusal "subtitle": { "parts": "auto", "dateFormat": "long" }, // parts "auto" | ["channel","title","date","clock"]; "long" = "Aug 14, 2026" | "iso" "qr": { "show": true, "size": 150 }, // 80–380 and ≤ height − 20 "overCards": "hide", // "hide" | "show" "motion": { "out": 0.3, "in": 0.45, "pip": 0.7 } // seconds } } } ``` - `deckOn(render)` is the ONE switch. `validateChrome(chrome, render)` returns sentences; `assertChrome` throws them. Unknown keys are refused. A deck beside `render.rail` or the legacy `render.chromeEngine` is refused. - `resolveDeck(render)` fills defaults (one level deep). Nothing else hardcodes a default. ### Manifest: per-entry `onscreen` ```jsonc { "type": "clip", "id": "c04", …, "onscreen": { "title": "County approves pre-application", "subtitle": "…optional override…" } } ``` - On any entry (clip, image, card). Nested because `entry.title` already means the stream title in headers and chapters. - `normalizeOnscreen(v)` trims, drops empty strings, and returns `null` for nothing left. A writer stores `null` by DELETING the key. One line each, ≤ 200 chars. - Deck title: `onscreen.title`, else (cards only) `heading`, else empty. - Deck subtitle: `onscreen.subtitle`, else auto — `deckSubtitle(attributionParts(…))` for a clip, `title · date` for an image, `sub` for a card. The channel is in the auto subtitle only when `isMultiChannel` (more than one clip channel slug). - QR: `deckQrUrl(entry, provenance)`. A clip's is the corner QR's rule unchanged (`citeUrl`, else the site link at `floor(start)`); an image only with `citeUrl`; a card never. `qr.show: false` makes every one null. ### `deck.mjs` (pure; no fs, no ffmpeg, no network) | Export | Signature → result | |---|---| | `deckOn` | `(render) → boolean` | | `resolveDeck` | `(render) → deck settings, defaults filled` | | `validateChrome` / `assertChrome` | `(chrome, render) → string[]` / throws | | `normalizeOnscreen` | `(v) → {title?, subtitle?} \| null`, throws on bad shape | | `deckGeometry` | `(render) → {W, H, footage:{x,y,width,height}, deck:{x,y,width,height}}` — 1920×1080 defaults: footage 1574×886 at (173,2), deck 1920×190 at (0,890) | | `deckLayout` | `(render) → {width, height, pipTrack:{x0,x1,y}, text:{x,width,titleSize,subtitleSize}, qr:{x,y,size}\|null}` region-local; S2 may tune the numbers HERE | | `pipXs` | `(n, x0, x1, spacing="even", schedule?) → number[]` | | `scheduleFrom` | `(durs, D) → {starts, total}` — THE arithmetic; `segmentOffsets` calls it | | `estimatedDuration` | `(entry, render) → seconds` | | `transitionOf` | `(render, {noXfade}) → D` | | `isMultiChannel`, `hidesDeck`, `deckText`, `deckQrUrl` | as above | | `deckSchedule` | `({entries, durs, D, render, provenance, metas, estimated}) → schedule doc` | | `estimateSchedule` | `(variantManifest, {metas, noXfade}) → schedule doc, estimated:true` | | `deckChoreography` | `(schedule, render) → {handovers, visibility}` (absolute seconds) | | `pipSegments` | `(schedule) → segments shown with the deck` | | `chromeCacheKey` | `({html, assets:[[name, sha256]], fps, frames, version}) → hex` | | `frameCount` | `(total, fps) → Math.round(total*fps)` | | `hyperframesCommand` | `(env) → {cmd, args, version}`; `HYPERFRAMES_BIN` wins, else `npx --yes ${HYPERFRAMES_PKG ?? "hyperframes@0.8.24"}` | ### `out//schedule.json` when the deck is on Written by the build (S1's `writeChromeSchedule`) from PROBED segment durations and real source metadata; never hand-written. Same shape from `estimateSchedule` with `estimated: true`. ```jsonc { "version": 1, "kind": "deck", "estimated": false, "fps": 30, "transition": 0.5, "total": 351.233, "multiChannel": false, "segments": [ { "id": "c01", "type": "clip", "start": 0, "duration": 19.1, "end": 19.1, "title": "…", "subtitle": "… · Sep 3, 2024", "qrUrl": "https://…", "hideDeck": false } ] } ``` ### Choreography (from `deckChoreography`) With m = `segments[i].start + D/2` (= the cut for D = 0): old title wipes out over [m − out, m]; new title expands over [m, m + in] (power3.out); subtitle trails by 0.08 s; an accent underline grows with the title; the QR flips through [m − 0.125, m + 0.125]; the pip marker eases from i−1 to i over [m − pip/2, m + pip/2]; past pips are filled; a progress fill runs with the clock. Into a hidden segment the deck slides down over [start, start + max(D, in)] (hard cut: finishing at the cut), back up the same way after it. No text handover across a hidden segment. Every segment owns its own DOM nodes — nothing is swapped at runtime. ### The composition (S2) - `composeChrome({ manifestPath, outDir, variant, region: "deck", schedule?, preview?, doRender, fps, workers, quality, format, still, png, from, duration }) → { projDir, frames, still, cached, key, frameCount }`. `schedule` (an object) overrides reading `out//schedule.json` — umtool's preview passes an estimate or a draft. - Render project: `out//chrome/deck/` (`index.html`, `assets/`, `hyperframes.json`). Frames: `out//chrome/deck-frames/frame_%06d.png` plus `deck-frames/.key` holding `chromeCacheKey`. A render is SKIPPED when the key matches and the frame count is `frameCount(total, fps)`. - Preview project (no render, umtool): `out//chrome/deck-preview/`, so a preview never disturbs a build's project or cache. - Region: 1920 × `deck.height` (region-local), transparent outside the panel; overlaid at (0, H − height). Root `data-composition-id="deck"`, `data-width`, `data-height`, `data-duration = total`. One paused `window.__timelines.deck`. - Fonts copied into `assets/` as private families `DeckSans` / `DeckSansBold` (from `render.fontRegular` / `render.fontBold`), fallback `sans-serif` only. GSAP vendored at `report-to-video/assets/gsap.min.js` — no CDN, in this region or the chart band. - `?still=`: seek to t and hold. `?preview=1`: listen for `postMessage` `{type:"deck:seek", t}` and `{type:"deck:text", id, title?, subtitle?}` (patch that segment's nodes, refit), and post `{type:"deck:ready", total, ids}` to the parent. DOM per segment: `[data-seg=""]` holding `.deck-title`, `.deck-sub`, `.deck-qr img`. - Text fit: at load, shrink a title's font until it fits the text column (floor 60 % of `title.size`), then ellipsize. - Stills: system chromium (`CHROME` env, default `/usr/bin/chromium`), headless screenshot of `index.html?still=t`. CLI: `compose-chrome.mjs --region deck [--still --png ] [--render] [--workers 4] [--quality high] [--format png-sequence] [--from ] [--duration ] [--variant v]`. ### The build (S1 + S3) When `deckOn(render)`: 1. `assertChrome` before any fetch. 2. Segments framed into `deckGeometry().footage`: `scale…,pad` into the footage box, then `pad` to W×H in `palette.bg`. No header, no corner QR, no ffmpeg footer assets, no section animation. Cards: full frame when `overCards: "hide"`, else framed like footage. `buildImageSegment` uses the same box. 3. `writeChromeSchedule()` → `out//schedule.json` (above), from `segmentOffsets` durations and `videoMeta` metadata. 4. `composeChrome({ region: "deck", doRender: true })` by dynamic import (cached). 5. Concat: transition > 0 → `concatWithXfade` with `chromeOverlayChain`; transition 0 → hard-cut concat to the `prerail-hardcut` file, then ONE overlay re-encode (`applyChrome`, the diet fork's). This replaces the refusal. 6. Chapters: `entry.chapter`, else `onscreen.title`, else today's fallback. 7. `chromeRegions(render, outDir)` deck branch → `[{ name: "deck", frames: chrome/deck-frames, x: 0, y: H − height, width: W, height }]`; `reservedFooterHeight` deck branch → `overCards === "show" ? height : 0`. Flags: `--chrome-only` (segments must exist: re-probe, re-write the schedule, recompose, re-render if the key changed, re-concat with the overlay, re-mux chapters — no segment is rebuilt), `--no-chrome` (deck framing, no overlay), `--chrome-preview ` (a short window to `.preview.mp4`). NDJSON: `chrome` events with `phase: "schedule" | "compose" | "render" | "cached" | "overlay"`. Driver: `buildSteps(…, { options: { chromeOnly: true } })` → `--chrome-only`, labelled "re-render on-screen". ### umtool (S4 + S5) Writers (all through `withManifestLock` / `backupOnce` / `writeManifestAtomic` and the stale-token guard): - `updateClip` accepts `onscreen` (normalised; `null`/empty deletes the key). - `updateOnscreen(dir, { [entryId]: {title?, subtitle?} | null }, { token })` — any entry type; unknown id refuses the whole batch. - `updateChrome(dir, chrome | null, { token })` — `validateChrome`; `null` removes `render.chrome`. Routes (`ctx.params` is a Promise in this Next): - `PUT /api/report/window` — whitelist gains `onscreen`. - `PUT /api/report/chrome` — `{ project, chrome, token }`. - `PUT /api/report/onscreen` — `{ project, onscreen: {id: {...}}, token }`. - `POST /api/report/chrome/preview` — `{ project, variant?, draft? }` → composes `deck-preview` with the real schedule if one exists (draft `onscreen` overlaid), else an estimate; NO render. Returns `{ src, geometry, layout, schedule }`. - `GET /api/report/chrome/files/[...path]` — serves the deck-preview composition dir, traversal-guarded. - `GET /api/report/still?project&variant&(clip|at)` — a true still PNG. - `GET /api/report/video?project&variant&kind=final|preview` — range-served mp4; the range code becomes `rangeResponse()` in `lib/report/serve.mjs`, shared with the segment route. UI: "On-screen" section on the report project page (between the build chain and Deliver; NOT "deck", which is the song kind's name) — enable toggle + settings, the title/subtitle table (auto subtitle as placeholder, `maxChars` counter, one save), a 16:9 live preview (the composition iframe at the deck rect over a still) with a scrubber, True still, Re-render on-screen, and the final video. The clip bench gets On-screen title/subtitle fields beside ATTRIB with a live deck preview fed by `postMessage`. ## Verification gates - `pnpm test:scripts` (deck, attribution and brand tests unchanged and green). - Byte-identical: a manifest without `render.chrome` builds `--only --skip-fetch` to the same segment md5 before and after. - `pnpm --filter umtool typecheck` and a capped `pnpm --filter umtool build` with the corpus linked. - umtool e2e through its own filter, detached: `SONG_DIR=~/reports/quartering-uh-song/data pnpm --filter umtool run e2e onscreen.spec.ts clip-bench.spec.ts build.spec.ts projects.spec.ts`. - First application: build + `verify-build.mjs` exit 0; stills at every handover m ± 0.2 s and mid-clip; `zbarimg` on each mid-clip QR decodes that clip's URL; ffprobe chapters equal the on-screen titles; an edit-one-title `--chrome-only` round trip re-renders the deck only. ## As built Points where a slice's actual shape differs from, or adds detail this contract left open to, what is written above. None of these broke a verification gate; they are the things a reader of this file alone would not know. - **`deckLayout` fields were left open here on purpose ("S2 may tune the numbers HERE"), and S2 did.** The shipped numbers: 48 px side padding, a 16 px pip track with 14 px of clearance below it for the active pip's halo, the title and subtitle boxes sized off `title.size` (title box `1.2×`, subtitle `48%` of it clamped to 18–34 px, the gap between them `30%`), and the QR (when shown) right-aligned with the text column ending 48 px short of it. None of this is a second contract — `deckLayout` is still the one place that draws it — but a reader of this file alone would not know the actual pixels without reading `deck.mjs`. - **The per-clip progress fill is not a pure export.** The contract's `deckChoreography` returns `{handovers, visibility}` and says in prose that "a progress fill runs with the clock"; what ships is a `fuse` element in `chrome-deck.mjs` whose `scaleX` is computed directly from the pip x positions `deckLayout`/`pipXs` already handed it (burning from pip k to pip k+1 while clip k plays), not a third field `deckChoreography` returns. The timing is still handover-derived and still has no second implementation — it is simply derived in the composition, where it is drawn, rather than handed to it as data. - **`-reinit_filter 0` / `format=rgba` on the deck's overlay input isn't in this contract at all.** S3 found it: HyperFrames' RGB/RGBA-mixed sequence restarts ffmpeg's filtergraph mid-concat and truncates the output (a 3.5 s xfade+overlay came out 2.0 s). It's now load-bearing in `chromeOverlayChain`, on the deck's input only — see `umtool/docs/quirks.md` and the pipeline README's deck section. - **`assertChrome` runs on any `render.chrome`, not only when `deckOn`.** The contract's build section says "When `deckOn(render)`: 1. `assertChrome` before any fetch" — read narrowly, a malformed block with a bad `engine` or `layout` (so `deckOn` is false) would reach the fetch stage unchecked. `buildVideo()` instead calls `assertChrome(render.chrome, render)` whenever `render.chrome` is present at all, `deckOn` or not, which is the stricter and correct reading. - **Two GET routes exist that the contract only specified as PUT.** `GET /api/report/chrome` (the settings form's starting state: the saved block, `deckOn`, the resolved deck, `DECK_DEFAULTS`, and `validateChrome`'s errors against a block somebody hand-edited into an invalid shape) and `GET /api/report/onscreen` (the On-screen table's rows: each entry's saved `onscreen` beside the AUTO title/subtitle it falls back to, `maxChars`, and the token) were needed for the UI to have anything to render and shipped alongside the PUTs the contract named. - **A windowed compose gets its own name, not just its own flag.** `compose-chrome.mjs --from`/`--duration` (used by `--chrome-preview`) writes a project and frame sequence named `deck-from[-frames]` rather than the full cut's `deck`/`deck-frames`, so a preview window never overwrites or races the whole-cut sequence's cache. The contract names `--from`/`--duration` as composeChrome arguments but not this naming rule. ## As shipped Branch `deck/integration`, every slice merged `--no-ff` after review; `main` 3edeb4d0 merged in at 0d1e7d69 before the final gates. | Commit | What | |---|---| | 2cd13c9a, d1e8439e | S0 — this contract, `deck.mjs` + tests, `attributionParts`/`deckSubtitle`/`formatDeckDate`, `segmentOffsets` on `scheduleFrom`, the `./deck` export | | e95f208e | S4 — umtool writers (`updateOnscreen`, `updateChrome`, `updateClip` `onscreen`), the chrome/onscreen/preview/files/still/video routes, `rangeResponse`, `buildSteps` `chromeOnly` | | 0be86732 | S2 — `chrome-deck.mjs`, the compose-chrome port (dispatch, `--still`, windows, workers, vendored GSAP, pinned renderer, render cache, `qrPng`) | | d3bb2804 | S1 — deck framing for clip/image/shown-card segments, chrome skipped under the deck, `writeChromeSchedule`, chapters prefer `onscreen.title` | | f936f799 | S5 — the On-screen section and the clip bench's on-screen fields | | c653e119 | S3 — one command composes, renders and overlays; `applyChrome` for transition 0; `--chrome-only` / `--no-chrome` / `--chrome-preview`; `-reinit_filter 0` + `format=rgba`; verify-build deck checks; absolute concat list | | 89fd4cfa | S7 — README, report-video/build/clip-bench docs, quirks, "As built" | | 66e60c65 | S6 — `onscreen.spec.ts`, deck fixtures and a HyperFrames stub; the 409 reload keeps only real edits; timeline-row locators scoped past the table's `data-entry` | | 2d746cbc | Review fixes — the deck reuses a cached hard-cut concat only for the same segments in the same order (`.segments`, `sameConcatList`); chapters take `onscreen.title` only under the deck | ### Gates On 66e60c65 (everything), then on 2d746cbc after the review fixes (the gates the fixes touch): - `pnpm -r --no-bail --workspace-concurrency=1 exec tsc --noEmit` clean (81 s). - common 2,404/2,404; editor unit 101/101; `test:scripts` 267 pass, 2 skipped (the `queue-lock.test.mjs` timing cases fail intermittently under machine load and pass alone — a known load flake, not touched here); mcp 271/271. - `pnpm --filter editor exec next build` exit 0 (50 s); `pnpm --filter export exec next build` exit 0 (28 s). - umtool `next build` with the corpus linked, under the 5 GB cap: exit 0, 22 s; link removed. - umtool e2e `onscreen clip-bench build projects report-fetch-via-editor`: 87 passed (3.6 min); on S6's branch 87/87 twice (7.8 and 3.3 min). - On 2d746cbc: tsc clean (65 s); `test:scripts` 268 pass, 2 skipped; capped umtool build exit 0 (19 s), link removed; umtool e2e 87 passed (3.3 min). - Review (read-only, Opus): SHIP AFTER FIXES — F1 (a reordered timeline reused the old hard-cut concat under `--chrome-only`) and F2 (chapters took `onscreen.title` without the deck), both fixed in 2d746cbc. - Byte-identical without `render.chrome`: `--only c07` and `--only t00` segment md5s equal to the base's (S1), and `c07` again after S3. - First application, built from this branch with `--skip-fetch`: 17 clips, 350.2 s, deck 10,506/10,506 frames (rendered in 211 s), `verify-build` ok, 17/17 mid-clip QRs decode to the clip's `citeUrl`, chapters equal the on-screen titles in order. An edit-one-title `--chrome-only` round trip (S3, scratch) re-rendered the deck only; an unchanged re-run was `cached` in 0.6 s. ### Found and left - `verify-build` cannot tell whether the overlay was laid: a `--no-chrome` final passes while frames from an earlier render are on disk. - `--chrome-only` writes over the final while it encodes, as `--rail-only` does. - The umtool preview shows saved settings; unsaved title/subtitle drafts preview live, unsaved settings do not (save recomposes). - e2e runs the renderer through a stub; the real renderer is proven by the gates above, not by e2e. - The legacy chart band still refuses `transition: 0`; `applyChrome` could serve it, untested. - The chart band's renderer is now the pinned `hyperframes@0.8.24` (it was `@latest`), through the same `hyperframesCommand`; GSAP is vendored rather than loaded from a CDN. - `render.palette` colours reach the preview composition's CSS unescaped; only a manifest's author can set them. - The rail's `--preview` and `--chrome-preview` both write `.preview.mp4`; a deck refuses a rail, so one manifest cannot produce both. - `chrome/deck-from[-frames]` preview windows are never pruned. ### Rollout Umtool-only: a umtool rebuild (`pnpm --filter umtool exec next build`, capped, corpus linked) and a umtool restart. Nothing under `export/`, `homepage/`, `editor/` (beyond the changelog) or `common/` changed, so no site, hub or homepage rebuild is owed for it.