commit bda3169b920fdaa09f707a0c4d9fcd4716a84f8c
parent b070be886ce6454d48a9255d8becc1f2d4956615
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Thu, 1 Oct 2026 14:38:12 -0400
Merge deck/finale-b2 (slice B2) — the room review's fixes (the move after the hold, the freeze check's window, whole-frame holds), per-clip muteFrom and render.endFade applied where the cut is joined (the <id>.cut.json record for the segment's real start), the QR host label fitted to the code's height; ferret scratch QR 17/17 + 7/7, silence from the mute point; reviewed
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Diffstat:
11 files changed, 999 insertions(+), 56 deletions(-)
diff --git a/editor/CHANGELOG.md b/editor/CHANGELOG.md
@@ -2,7 +2,8 @@
## [Unreleased]
- **umtool's report videos can wear an on-screen deck: one panel under the footage for the whole cut, with a pip timeline, a title per clip, its source and date, and its QR.** A report manifest whose `render` says `"chrome": { "engine": "hyperframes", "layout": "deck" }` scales the footage into a box above a 190 px panel (both sizes are settings) and draws, over the whole cut, one unlabelled pip per clip on a track that fills as the cut plays, the clip's own title from `onscreen.title`, a subtitle naming the recording and its date (the channel too when the cut spans more than one; `onscreen.subtitle` replaces it), and the clip's QR. At each clip change the marker travels to the next pip and the title, subtitle and QR hand over; over a card the panel slides away and comes back after. The citation header, the corner QR and the section footer are not drawn on such a cut, and chapters take the clip's on-screen title. Every setting (sizes, spacing, date format, what the subtitle names, whether cards keep the panel, the motion's timings) is in `render.chrome.deck` and checked when it is saved; an unknown or out-of-range one is refused with a sentence saying why. The panel is rendered once per cut by HyperFrames (pinned to 0.8.24; `HYPERFRAMES_PKG` or `HYPERFRAMES_BIN` override it) and reused until its text or settings change. `build-video.mjs --chrome-only` redraws it over the built segments without rebuilding or fetching anything, `--no-chrome` builds the framed cut without it, and `--chrome-preview <at> <dur>` renders a short window. In umtool, the report page has an **On-screen** section — a switch, the settings, a table of every entry's title and subtitle with the automatic subtitle as its placeholder and a character counter, a live preview with a scrubber, a true still, **Re-render on-screen** and the built video — and the clip bench has on-screen title and subtitle fields with the panel previewed over the clip. A manifest without `render.chrome` builds exactly as before, byte for byte.
-- **A report cut that wears the on-screen deck can show posts — Bluesky or X statements — as cards over the footage.** A report manifest's `posts` list (each with its platform, handle, date, words and link) is drawn near the end of the clip each post belongs with: the clip whose recording most closely precedes it by date, unless the post names one with `attachTo`; `hide` leaves one out. A clip's posts appear four seconds apart and stack down a column at the frame's top right; as the first appears, the footage eases aside (to 86 % of its box, at the far side) to make room, and the clip's last frame is held, in silence, for 2.5 seconds so the last post can be read; then they all leave together in the change to the next clip, which comes in at the normal size. When the column is full the oldest slide up and out. Each card slides in from the edge of the frame and flares in the deck's accent as it lands; it has an accent rail down its edge and shows the post's date, a platform label ("Bluesky" or "X") beside `@handle`, its words in paragraphs up to seven lines with an ellipsis, and a QR of the post's link, in the deck's colours and faces. The hold and the move are made where the cut is joined, not in a clip, so `--chrome-only` changes them without rebuilding one; chapters and the deck's timing count the hold. The timing, the hold (`hold`, 0 turns it off), the move (`shift`: its scale and seconds, or `false`), the column's side, width and inset, the QR size and the line limit are settings under `render.chrome.deck.posts`, and a bad post or setting is refused with a sentence before a build fetches anything. Only the seconds the cards are up are rendered, one short sequence per clip, cached like the deck; `--chrome-only`, `--chrome-preview` and a hard-cut cut lay them as they lay the deck, and `--no-chrome` draws neither. A cut without `posts` builds exactly as before, and without the deck `posts` is not drawn at all.
+- **A report cut that wears the on-screen deck can show posts — Bluesky or X statements — as cards over the footage.** A report manifest's `posts` list (each with its platform, handle, date, words and link) is drawn near the end of the clip each post belongs with: the clip whose recording most closely precedes it by date, unless the post names one with `attachTo`; `hide` leaves one out. A clip's posts appear four seconds apart and stack down a column at the frame's top right; as the first appears, the footage eases aside (to 86 % of its box, at the far side) to make room, and the clip's last frame is held, in silence, for 2.5 seconds so the last post can be read; then they all leave together in the change to the next clip, which comes in at the normal size. When the column is full the oldest slide up and out. Each card slides in from the edge of the frame and flares in the deck's accent as it lands; it has an accent rail down its edge and shows the post's date, a platform label ("Bluesky" or "X") beside `@handle`, its words in paragraphs up to seven lines with an ellipsis, and a QR of the post's link, in the deck's colours and faces. The hold and the move are made where the cut is joined, not in a clip, so `--chrome-only` changes them without rebuilding one; chapters and the deck's timing count the hold. The timing, the hold (`hold`, 0 turns it off), the move (`shift`: its scale and seconds, or `false`), the column's side, width and inset, the QR size and the line limit are settings under `render.chrome.deck.posts`, and a bad post or setting is refused with a sentence before a build fetches anything. Only the seconds the cards are up are rendered, one short sequence per clip, cached like the deck; `--chrome-only`, `--chrome-preview` and a hard-cut cut lay them as they lay the deck, and `--no-chrome` draws neither — though it still holds and moves the footage, which are part of the cut rather than the chrome. A first post that appears inside the hold still moves the footage, and a hold is a whole number of frames. A cut without `posts` builds exactly as before, and without the deck `posts` is not drawn at all.
+- **A report clip can go silent partway through, a report cut can fade out at its end, and the deck's QR names its site in larger type.** A clip's `muteFrom` (in the recording's own seconds, inside the clip) silences it from that second to its end while the picture plays on, after a 40 ms fade that ends there, so nothing clicks and no next word leaks in; a hold on that clip stays silent. `render.endFade` (seconds; 0, the default, is off) fades the cut's last segment to the background colour and to silence over its final seconds, the hold included, and the deck stays drawn over it; once a closing card follows the last clip, the ordinary crossfade into it does that job instead. Both are applied where the cut is joined, so `--chrome-only` changes them without rebuilding a clip, and a value out of range is refused with a sentence before a build fetches anything. Each clip build now writes `<id>.cut.json` beside its segment, saying where in the recording the segment really starts after its cut was snapped to a silence; `muteFrom` is measured from it, and a segment built before this measures from the clip's unsnapped start and says so. The site's name beside the deck's QR is now exactly as long as the code is tall, for any site. A cut without `muteFrom` or `endFade` builds exactly as before.
- **A report cut with `transition: 0` builds when its output folder was given as a relative path.** The hard-cut concat listed its segments relative to the working directory, and ffmpeg reads that list relative to the list file's own folder, so every hard-cut build with a relative `--out` failed at the concat. The list now names each segment by its full path.
- **`archilyzer doctor` checks the image Build all builds sites in.** When a container engine answers, a new **build image** section says whether the image named under **Settings → Build pipeline** is there, when it was built and how big it is. It warns when the image is missing, or older than the last change to its Dockerfile, and prints the one command that rebuilds it. Build all still builds or refreshes the image itself before it builds any site; the warning tells you ahead of time that the next Build all will spend that time. With no container engine the check is skipped in one line, and with no corpus it is only a note. It never fails the doctor.
- **The site build image runs Node 22 and pnpm 11**, the versions the rest of the workspace runs on, instead of Node 20 and pnpm 9, which did not read the workspace's install rules. The next Build all rebuilds the image from its first step, reinstalling every dependency, before it builds any site.
diff --git a/plans/deck-posts.md b/plans/deck-posts.md
@@ -205,3 +205,67 @@ Left for R2 (umtool):
for a cut that was never fully built.
- The preview's posts geometry is now the frame's edge (`postsGeometry`). The preview does not
show the footage move or the hold; both happen in ffmpeg, at the join.
+
+Both items above were done after R1: R2 (b584c129) shows the move and the hold in the preview, and
+df062b37 reads a deck cut's offsets from its `schedule.json` when there is no `chapters.ffmeta`.
+
+## Finale B2, as built
+
+Branch `deck/finale-b2` from df062b37. It starts with the fixes from the read-only review of the
+room work (SHIP AFTER FIXES).
+
+| Commit | What |
+|---|---|
+| 2f0e852b | Review fixes. The footage move now runs AFTER the hold, so a first post inside the hold still moves the frozen frame (before, it never moved or froze part-way). verify-build's freeze check samples only between the hold's start (or the move's landing) and the dissolve, and reports under three frames of still picture as not checked. `postHolds` rounds a hold to whole frames. The README and changelog say `--no-chrome` still holds and moves. |
+| 2ea53246 | A clip's `muteFrom` and `render.endFade`, joined on that input's chain; the `<id>.cut.json` record beside each clip segment; validation; the QR host label fitted to the code's height |
+| ebe10c84 | README, quirks, one `[Unreleased]` bullet |
+
+Where it adds to the rulings above:
+
+- **`muteFrom`** is in SOURCE seconds and must lie within the clip's `start`–`end`. The sound is
+ silent from that second, after a 40 ms `afade` that ENDS there, and it is digital zeros after
+ that. A hold on the clip stays silent. The mute is mapped to the segment's clock through
+ `<id>.cut.json`, which every clip build now writes: the source seconds the segment was really
+ cut from after snapping. A segment with no record, or one whose record does not match its
+ length, falls back to the unsnapped `playWindow` start, and the build says so in a note.
+- **`render.endFade`** applies to the cut's last segment, whatever it is, hold included. The
+ picture reaches `palette.bg` and the sound reaches silence on the last frame. The picture fade
+ is a `geq` blend in yuv420p, enabled from its first frame. `fade=…:color=` would work too, but
+ only in RGB, and the concat filter would then convert every segment of the cut to rgb24 and back.
+- Both are joins (`withCutEdits`, `cutJoins`), so they apply with or without the deck, on
+ crossfades and hard cuts, and `--chrome-only` changes them without rebuilding a segment. The
+ hard-cut record names them only when present. `validateCutEdits` (in `deck.mjs`) is what the
+ build refuses with, and `validateChrome` also checks `endFade`.
+- **The QR's host label** is sized at load from the string's ink in the loaded face (canvas
+ `measureText`). Tracking counts between letters only, and the first side bearing is indented
+ away. On the ferret cut its ink covers rows 31–180, the same rows as the code. Before, it
+ covered 54–180.
+
+Gates on ebe10c84:
+
+- Workspace tsc clean. `test:scripts` 341 pass, 2 skipped. Capped umtool `next build` with the
+ corpus linked exit 0 (31 s), link removed.
+- Byte-identical: `--only c07 --skip-fetch` without `render.chrome` gives md5
+ `6a92235fa12ca181bb81993129c9ee9d`. The unchanged ferret deck manifest writes a `schedule.json`
+ byte-equal at df062b37 and at the tip. With no muteFrom or endFade, the crossfade graph, the
+ hard-cut graph and the record equal 2f0e852b's. Against df062b37 the only difference is the
+ move and hold order on the four carrying clips.
+- Ferret, scratch copy with c20 `muteFrom: 24029.30` and `endFade: 1.0`, `--chrome-only` over
+ copied segments with no cut records:
+ - c20 muted 6.70 s into its segment, falling back to the unsnapped start, which the run notes;
+ - 360.2 s, verify-build ok, all four freezes found;
+ - QR 17/17 deck and 7/7 posts;
+ - the last sample that is not zero is at 359.613 s of the audio, and everything after it is
+ digital silence;
+ - the last frame above the deck is bg (mean 1.0, worst 3 levels over rows 0–881).
+
+Found and left:
+
+- **The crossfade concat drifts the sound ahead of the picture, before this branch too.**
+ Crossfaded segments have audio up to 40 ms shorter than their video (AAC framing), and
+ `acrossfade` joins the sound by its own lengths while `xfade` uses the picture's. The ferret
+ cut's audio is 359.901 s against 360.200 s of picture, as in the build made before this branch, so by the last
+ clip the sound runs 0.3 s early. A `muteFrom` silences the right sound in the clip's own audio,
+ but in the final it falls up to that drift before the matching picture. Padding each input's
+ sound to its picture's length (`apad=whole_dur`) before the join would fix it, but that changes
+ every crossfaded cut's graph.
diff --git a/umtool/docs/quirks.md b/umtool/docs/quirks.md
@@ -252,7 +252,52 @@ across the gap.
the outgoing clip and the dissolve into the next one starts `transition`
seconds before that clip's end, so with `transition: 0.5` a 2.5 s hold shows
2.0 s of still frame and then dissolves out of it; a 0.5 s hold is consumed
-entirely. The real-ffmpeg test holds 1 s for this reason.
+entirely. The real-ffmpeg test holds 1 s for this reason. verify-build's
+freeze check samples only between the hold's start (or the move's landing)
+and the dissolve, and skips a hold with under three frames of still picture
+there, saying so.
+
+**A footage move placed BEFORE the hold never reaches the held frames.** The
+move's clock is `perspective`'s `in`, which counts the frames that reach it.
+Before `tpad` that is the clip's own frames only: a first post that appears
+inside the hold (`posts.seconds: 2` with the 2.5 s hold puts it exactly at
+the clip's last frame) never moved the footage, and one in the clip's last
+half-second froze part-way, while umtool's preview showed it moving. The
+move runs after `tpad`, so its clock counts the clones and the frozen frame
+glides too.
+
+**`tpad` holds whole frames; `apad` holds exact seconds.** `stop_duration=2.5`
+at 25 fps clones 63 frames (2.52 s) while `apad=pad_dur=2.5` adds exactly
+2.5 s, and the concat filter pads the short stream to the long one, so each
+held clip in a hard cut grew by up to half a frame and several of them failed
+the length check. `postHolds` rounds a hold to whole frames before either
+filter sees it.
+
+**A coloured `fade` converts the whole hard cut to RGB.** `fade=t=out:…:color=<bg>`
+accepts RGB formats only (a fade to black also takes YUV), so ffmpeg inserts a
+yuv420p→rgb24 scale before it -- and the concat filter, which needs every
+segment in one format, then negotiates EVERY other segment to rgb24 too. The
+cut's untouched frames came out different (framemd5) from the same graph
+without the fade. The end fade is a `geq` blend toward bg's limited-range
+BT.601 Y′CbCr instead (`#12101a` is 31/132/128, what `pad` wrote into the
+segments), enabled only from its first frame; `geq` truncates, so each plane
+adds 0.5 to round.
+
+**`afade` out writes digital silence after its fade, and copies every sample
+before it.** That makes it the mute for `muteFrom`: samples before the fade are
+the clip's own bit for bit (A/V sync cannot move), and samples after it are
+zeros, not a quiet signal. A fade that ENDS at the mute point keeps a sound
+that starts there out entirely.
+
+**A label fitted to a length is measured as ink, not as a box.** CSS
+`letter-spacing` is added after the LAST letter too, and a box's length
+includes each end glyph's side bearing, so sizing the deck's QR host by its
+element's length leaves it short of the code by the trailing tracking and the
+bearings. `fitHost` measures the string's ink in the loaded face with canvas
+`measureText` (`actualBoundingBoxLeft + actualBoundingBoxRight`), adds the
+tracking between letters only, scales the font size (everything in it is in em)
+and indents the first bearing away. Measured on the ferret cut: ink rows 31–180,
+the code's 31–180.
**`xfade` hands on its own pixel format.** Even the frames before its offset,
which are the first input's, come out as yuv444 rather than the input's
diff --git a/umtool/report-to-video/README.md b/umtool/report-to-video/README.md
@@ -187,7 +187,7 @@ the manifest names the MCP video id while the cue file lives under the URL slug.
"quote": "The pre-application screening was approved by the county, dude." }
```
-Two per-clip fields exist for compilations that span sources or need a hand-cut
+Three per-clip fields exist for compilations that span sources or need a hand-cut
window:
- **`channel`** — the archived channel this clip's cue file lives under, overriding
@@ -201,6 +201,19 @@ window:
sentence is an editorial decision that widening would silently undo. `lock` also
handles the reverse case — a clip whose lead-in would drag in seconds of some
*other* audio (a news package playing before the speaker starts).
+- **`muteFrom`** — SOURCE seconds, like `start`/`end`/`cutEnd`, within the clip's
+ `start`–`end`: the clip's sound goes silent from that second to the end of the
+ clip while the picture plays on. A 40 ms fade ends exactly at `muteFrom`, so
+ nothing of a sound that starts there gets through and nothing clicks; from it on
+ the sound is digital silence, and a hold on that clip stays silent. It is made
+ where the cut is joined (see [The cut's edits](#the-cuts-edits-mutefrom-and-renderendfade)),
+ so changing it rebuilds no segment: `--chrome-only` applies it under the deck.
+
+`render.endFade` (seconds, default 0 = off, at most 10) fades the cut's LAST
+segment — whatever it is — picture to `palette.bg` and sound to silence over its
+final `endFade` seconds, reaching both on the last frame. It is for a cut that
+ends on a clip; once a finale entry follows the last clip, the ordinary
+crossfade into it does the job and `endFade` fades the finale instead.
### `render.chrome` — the on-screen deck's settings, and per-entry `onscreen`
@@ -939,6 +952,13 @@ deck slides away for its duration when it is `"hide"` (the default) — no text
handover happens across a hidden segment, because there is nothing on screen
to animate.
+The code's host (`JASOLYZER.PAGES.DEV`, muted, tracked 0.1 em) runs up the QR's
+left side and is exactly as long as the code is tall (`deckLayout(render).qr.size`,
+150 px by default), whatever the host: the page measures the string's ink in the
+loaded face once at load, scales its size to fit, and indents the first
+letter's side bearing away, so the ink starts on the code's bottom edge and
+ends on its top.
+
`out/<variant>/schedule.json` (`deckSchedule()`) is the one source of *when*:
the build writes it from PROBED segment durations and real source metadata;
`estimateSchedule()` produces the same shape (`estimated: true`) from the
@@ -990,7 +1010,10 @@ node umtool/report-to-video/compose-chrome.mjs <manifest.json> [--region chart|d
rebuilt and nothing is fetched.** The driver labels this "re-render
on-screen".
- **`--no-chrome`** — the deck's framing with no overlay: a fast picture check
- of the letterboxing, with nothing composed or rendered.
+ of the letterboxing, with nothing composed or rendered. Holds and footage
+ moves are part of the CUT, not the chrome, so `--no-chrome` still applies
+ them (the footage moves aside for cards it does not draw), and its length is
+ the schedule's.
- **`--chrome-preview <at> <dur>`** — renders only that window of the deck and
writes `out/<variant>/<slug>.preview.mp4` of it, from a cached concat when
there is one, else built straight from the segments the window touches.
@@ -1049,8 +1072,11 @@ node umtool/report-to-video/compose-chrome.mjs <manifest.json> --region posts --
all lay them; `--no-chrome` lays neither.
- **`verify-build`** also checks each window's `chrome/posts-<segment>-frames`
holds that window's frame count, and that each held clip's freeze is in the
- file (the frame at `end − hold/2` is the frame at `end − hold + ε`, outside
- the deck and the column, within re-encoding noise).
+ file: two frames inside the still part of the hold -- after the hold starts
+ and the move lands, before the outgoing dissolve (or the end fade) -- are the
+ same, outside the deck and the column, within re-encoding noise. A hold with
+ under three frames of still picture there (0.5 s under a 0.5 s crossfade) is
+ reported as not checked.
### The hold and the move, where the cut is joined
@@ -1061,11 +1087,15 @@ no joins and runs the graphs it always did.
- **The hold** is `tpad=stop_mode=clone:stop_duration=<hold>` on the input's
picture and `apad=pad_dur=<hold>` (silence) on its sound, before the
- xfade/acrossfade.
-- **The move** is one `perspective` filter on that input (`sense=destination`,
- `eval=frame`): the input frame's corners are placed so the footage box goes
- from `from` to `to`, eased by smoothstep over `[segmentAt, segmentAt +
- seconds]` in the segment's own clock, then held there. Perspective resamples at
+ xfade/acrossfade. A hold is a whole number of frames (`postHolds` rounds it:
+ 2.5 s at 25 fps is 2.52 s), because `tpad` clones whole frames and `apad`
+ pads exact seconds.
+- **The move** is one `perspective` filter on that input, AFTER the hold
+ (`sense=destination`, `eval=frame`): the input frame's corners are placed so
+ the footage box goes from `from` to `to`, eased by smoothstep over
+ `[segmentAt, segmentAt + seconds]` in the segment's own clock, hold included,
+ then held there. A first post that appears inside the hold therefore still
+ moves the footage, the frozen frame with it. Perspective resamples at
1/256 px, so the box glides with no whole-pixel stepping, where `scale` +
`overlay` and `zoompan` round to whole pixels. Before the move the map is the
identity, which perspective copies bit for bit. `fillborders` pins the
@@ -1079,6 +1109,34 @@ no joins and runs the graphs it always did.
chapters, a `--chrome-preview` window and `--chapters-only` all add the holds
to the probed lengths (`cutOffsets`), the same sum `deckSchedule` makes.
+### The cut's edits: `muteFrom` and `render.endFade`
+
+Both are made on one input's chain before the join, beside the hold and the
+move (`cutJoins` merges them into the deck's joins; `withCutEdits` is the pure
+merge), so neither touches a segment file and every graph, record and schedule
+of a cut without them is what it was. They work with or without the deck, on
+crossfades and hard cuts.
+
+- **`muteFrom` is mapped through the segment's cut record.** The build snaps a
+ clip's cut to the nearest silence, so its segment starts up to `snapWindow`
+ seconds from the manifest's `start` (or `cutStart − leadIn`). Every clip build
+ writes `segments/<id>.cut.json` — `{ video, start, end }`, the source seconds
+ it was really cut from — and `muteFrom − start` is the mute point in the
+ segment's clock (`muteSegmentSeconds`, in `deck.mjs`). A segment with no record
+ (built before records existed, or copied without it), or one whose record is
+ not as long as the segment, is measured from the unsnapped start instead, and
+ the build says so in a note. The sound chain is
+ `afade=t=out:st=<at − 0.04>:d=0.04` (then the hold's `apad`, if any); a mute at
+ or before the segment's start is `volume=0`.
+- **The end fade's picture is a `geq` blend toward bg's Y′CbCr**, enabled from
+ the fade's first frame: frame `lastFrame − n` is the last untouched one and
+ `lastFrame` (the hold's clones counted) is bg. Not `fade=…:color=`, which works
+ in RGB only (`umtool/docs/quirks.md`). The sound is `afade` out to silence at the
+ last frame's time.
+- **The hard-cut record names a mute and a fade** (`"mute"`, `"fade"` on a
+ `# join` line, only when there is one), so a cached prerail made without them,
+ or with other values, is never reused.
+
### Two ffmpeg traps that are the deck's alone
- **Mixed RGB/RGBA frames restart the whole filtergraph.** HyperFrames writes
diff --git a/umtool/report-to-video/build-video.mjs b/umtool/report-to-video/build-video.mjs
@@ -80,8 +80,8 @@ import { createCueSource, siteOriginFromManifest } from "./cues.mjs";
// and live in deck.mjs. This file only frames segments into its box and writes
// the schedule down -- it never has a copy of the arithmetic.
import {
- assertChrome, deckGeometry, deckOn, deckSchedule, frameCount, postsGeometry, postWindows, resolveDeck,
- scheduleFrom, snapWindow, validatePosts,
+ assertChrome, deckGeometry, deckOn, deckSchedule, endFadeOf, frameCount, MUTE_FADE, muteSegmentSeconds,
+ playWindow, postsGeometry, postWindows, resolveDeck, scheduleFrom, snapWindow, validateCutEdits, validatePosts,
} from "./deck.mjs";
// The per-platform yt-dlp args (Rumble's `--impersonate chrome`): the ONE table,
// in common, plain JS so bare `node` can load it.
@@ -713,10 +713,8 @@ async function buildClipSegment(entry, meta, render, dirs, opts, chrome, nodes,
// The lead-in is a breath before the first word, clamped into the extent:
// starting exactly on the quote's first syllable sounds like a dropped
// frame.
- const lead = render.leadIn ?? 0.4;
const hasCut = Number.isFinite(entry.cutStart) && Number.isFinite(entry.cutEnd);
- const playFrom = hasCut ? Math.max(entry.start, entry.cutStart - lead) : entry.start;
- const playTo = hasCut ? Math.min(entry.end, entry.cutEnd) : entry.end;
+ const { from: playFrom, to: playTo } = playWindow(entry, render);
if (hasCut) {
EMIT("cut", {
id: entry.id,
@@ -749,6 +747,14 @@ async function buildClipSegment(entry, meta, render, dirs, opts, chrome, nodes,
const cutA = Math.min(a.at, wantB - 1);
const cutB = Math.max(b.at, cutA + 1);
EMIT("snap", { id: entry.id, start: a.snapped, end: b.snapped, seconds: cutB - cutA });
+ // Where in the SOURCE this segment really starts and ends, snapped: what a
+ // `muteFrom` (source seconds) is measured from at the join, long after this
+ // function is gone (`--chrome-only` rebuilds no segment).
+ const cutRecord = {
+ version: 1, id: entry.id, video: entry.video,
+ start: Number((fetchStart + cutA).toFixed(3)), end: Number((fetchStart + cutB).toFixed(3)),
+ snapped: { start: a.snapped, end: b.snapped },
+ };
const quotePath = path.join(outDir, "segments", `${entry.id}.quote.txt`);
const attribPath = path.join(outDir, "segments", `${entry.id}.attrib.txt`);
@@ -788,6 +794,7 @@ async function buildClipSegment(entry, meta, render, dirs, opts, chrome, nodes,
],
{ maxBuffer: 1 << 24 },
);
+ await writeCutRecord(seg, cutRecord);
return seg;
}
@@ -918,9 +925,22 @@ async function buildClipSegment(entry, meta, render, dirs, opts, chrome, nodes,
],
{ maxBuffer: 1 << 24 },
);
+ await writeCutRecord(seg, cutRecord);
return seg;
}
+/** Beside `segments/<id>.mp4`: `<id>.cut.json`, the source seconds it was cut from. */
+export const cutRecordPath = (seg) => seg.replace(/\.mp4$/, ".cut.json");
+
+async function writeCutRecord(seg, record) {
+ await writeFile(cutRecordPath(seg), JSON.stringify(record) + "\n", "utf8");
+}
+
+/** A segment's cut record, or null when there is none (a segment built before records existed). */
+export async function readCutRecord(seg) {
+ return readFile(cutRecordPath(seg), "utf8").then(JSON.parse, () => null);
+}
+
// ---- QR provenance code --------------------------------------------------
// A compilation asks the viewer to take the edit on trust. The QR is the antidote:
// it resolves to this clip's exact START in the archive's own viewer, so anyone can
@@ -2144,7 +2164,8 @@ const exprNum = (v) => {
* per frame): an affine map of the WHOLE frame that takes the `from` box to
* the box eased toward `to`. Each corner of the input frame is placed at
* `base + d·e(t)`, where e is smoothstep over [segmentAt, segmentAt + seconds]
- * in the segment's own clock and d is where that corner has gone at e = 1 --
+ * in the segment's own clock (its hold included: the move runs after the
+ * `tpad`) and d is where that corner has gone at e = 1 --
* scale `to.width / from.width` (and height), then translate. Before the move
* e = 0 and the map is the identity, which perspective copies bit for bit;
* after it e = 1 and the frame holds at `to`.
@@ -2188,26 +2209,147 @@ export const holdVideoFilter = (hold) => `tpad=stop_mode=clone:stop_duration=${e
export const holdAudioFilter = (hold) => `apad=pad_dur=${exprNum(hold)}`;
/**
+ * A `muteFrom` on a segment's sound: silent from `at` (segment seconds) to its
+ * end, after a MUTE_FADE that ENDS at `at`, so nothing of a sound that starts
+ * there gets through and there is no click. `afade` out writes digital silence
+ * (zeros) after its fade and copies every sample before it. At 0 the whole
+ * segment is silent.
+ */
+export const muteAudioFilter = (at) => {
+ if (!(at > 0)) return "volume=0";
+ const st = Math.max(0, at - MUTE_FADE);
+ return `afade=t=out:st=${exprNum(st)}:d=${exprNum(at - st)}`;
+};
+
+/**
+ * A `#rrggbb` colour as 8-bit limited-range BT.601 Y′CbCr -- what `pad` and a
+ * `color` source write for it into the segments' yuv420p (`#12101a` is
+ * 31/132/128 in both, measured).
+ */
+export function yuv601(hex) {
+ const m = /^#?([0-9a-f]{2})([0-9a-f]{2})([0-9a-f]{2})$/i.exec(String(hex));
+ if (!m) throw new Error(`not a #rrggbb colour: ${hex}`);
+ const [r, g, b] = [m[1], m[2], m[3]].map((h) => parseInt(h, 16) / 255);
+ return {
+ y: Math.round(16 + 65.481 * r + 128.553 * g + 24.966 * b),
+ u: Math.round(128 - 37.797 * r - 74.203 * g + 112 * b),
+ v: Math.round(128 + 112 * r - 93.786 * g - 18.214 * b),
+ };
+}
+
+/**
+ * The end fade on the cut's last segment, over its final `seconds`: the
+ * picture eased to `palette.bg`, so the LAST frame (`lastFrame`, 0-based, the
+ * hold's clones included) is exactly bg; the sound faded to silence at that
+ * frame's time.
+ *
+ * Not `fade=…:color=`: a coloured fade takes RGB only, so ffmpeg converts the
+ * segment to rgb24 -- and the concat filter then negotiates every OTHER
+ * segment to rgb24 too, a lossy round trip for the whole cut. `geq` blends
+ * each plane toward bg's Y′CbCr in the segment's own yuv420p, and only from
+ * the fade's first frame (`enable`): every frame before it passes untouched.
+ * Frame f's weight is (f − s)/n with s = lastFrame − n, so s is the last
+ * frame untouched and lastFrame is bg; `+0.5` rounds where geq truncates.
+ */
+export const endFadeVideoFilter = (fade, render) => {
+ const fps = render.fps;
+ const n = Math.max(1, Math.round(fade.seconds * fps));
+ const st = exprNum(Math.max(0, fade.lastFrame - n) / fps);
+ const k = `clip((T-${st})/${exprNum(n / fps)},0,1)`;
+ const bg = yuv601(render.palette.bg);
+ const plane = (p, c) => `'${p}(X,Y)+(${c}-${p}(X,Y))*${k}+0.5'`;
+ return `geq=lum=${plane("lum", bg.y)}:cb=${plane("cb", bg.u)}:cr=${plane("cr", bg.v)}:enable='gte(t,${st})'`;
+};
+export const endFadeAudioFilter = (fade, render) => {
+ const end = fade.lastFrame / render.fps;
+ const st = Math.max(0, end - fade.seconds);
+ return `afade=t=out:st=${exprNum(st)}:d=${exprNum(Math.max(1e-3, end - st))}`;
+};
+
+/**
* Input `i`'s chains before the join. Without a join the labels are the
* input's own (`[i:v]`, `[i:a]`) and there is no chain at all, so a cut
- * without posts writes the graph it always did. The move goes before the hold,
- * so the held frame is the moved one.
+ * without posts writes the graph it always did.
+ *
+ * The move goes AFTER the hold: its clock (`in`) then counts the held frames
+ * too, so a first post that appears inside the hold -- or so late that the
+ * glide runs past the clip's own last frame -- still moves the footage, the
+ * frozen frame with it, exactly when the schedule (and umtool's preview) say.
+ * Before the hold, such a move never started or froze part-way.
*
* @returns {{ parts: string[], v: string, a: string }}
*/
export function joinInputChain(i, join, render) {
if (!join) return { parts: [], v: `[${i}:v]`, a: `[${i}:a]` };
const parts = [];
- const vf = [join.move ? moveFilter(join.move, render) : null, join.hold > 0 ? holdVideoFilter(join.hold) : null]
- .filter(Boolean);
+ // Picture: hold, move, end fade. Sound: mute, hold, end fade -- the mute is
+ // in the clip's own clock and the hold is silence anyway; the end fade is
+ // last on both, over the segment's final seconds as the cut plays them.
+ const vf = [
+ join.hold > 0 ? holdVideoFilter(join.hold) : null,
+ join.move ? moveFilter(join.move, render) : null,
+ join.fade ? endFadeVideoFilter(join.fade, render) : null,
+ ].filter(Boolean);
const v = vf.length ? `[j${i}v]` : `[${i}:v]`;
if (vf.length) parts.push(`[${i}:v]${vf.join(",")}${v}`);
- const a = join.hold > 0 ? `[j${i}a]` : `[${i}:a]`;
- if (join.hold > 0) parts.push(`[${i}:a]${holdAudioFilter(join.hold)}${a}`);
+ const af = [
+ join.mute != null ? muteAudioFilter(join.mute) : null,
+ join.hold > 0 ? holdAudioFilter(join.hold) : null,
+ join.fade ? endFadeAudioFilter(join.fade, render) : null,
+ ].filter(Boolean);
+ const a = af.length ? `[j${i}a]` : `[${i}:a]`;
+ if (af.length) parts.push(`[${i}:a]${af.join(",")}${a}`);
return { parts, v, a };
}
/**
+ * The joins with the cut's edits merged in: a `muteFrom` (`mutes`: segment
+ * index → segment seconds) and the end fade on the LAST segment
+ * (`fade`: `{ seconds, lastFrame }`). A join gains `mute` / `fade` only when it
+ * has one, so a cut without either keeps exactly the joins (and the graph,
+ * and the hard-cut record) it had; null when nothing is joined at all.
+ */
+export function withCutEdits(joins, n, { mutes = new Map(), fade = null } = {}) {
+ if (!mutes.size && !fade) return joins;
+ const out = Array.from({ length: n }, (_, i) => joins?.[i] ?? null);
+ for (const [i, at] of mutes) out[i] = { hold: 0, move: null, ...(out[i] ?? {}), mute: at };
+ if (fade) out[n - 1] = { hold: 0, move: null, ...(out[n - 1] ?? {}), fade };
+ return out.some(Boolean) ? out : null;
+}
+
+/**
+ * Every join the cut makes: the deck's holds and moves (`segmentJoins` of its
+ * schedule; none without the deck), each clip's `muteFrom` mapped to its
+ * segment's clock through the segment's cut record, and `render.endFade` on
+ * the last segment. Reads the records and probes what it needs; null when
+ * nothing is joined, so every concat then runs as it always did.
+ */
+export async function cutJoins({ schedule = null, entries, segments, render }) {
+ const base = schedule ? segmentJoins(schedule) : null;
+ const mutes = new Map();
+ for (let i = 0; i < entries.length; i += 1) {
+ const e = entries[i];
+ if (e.type !== "clip" || e.muteFrom == null) continue;
+ const m = muteSegmentSeconds({
+ entry: e, record: await readCutRecord(segments[i]), render,
+ seconds: await probeDuration(segments[i], render.fps),
+ });
+ if (m.note) EMIT("note", { id: e.id, message: m.note });
+ EMIT("note", { id: e.id, message: `${e.id}: muted from ${m.at}s into its segment (muteFrom ${e.muteFrom}, ${m.source === "record" ? "from its cut record" : "from the unsnapped start"})` });
+ mutes.set(i, m.at);
+ }
+ const seconds = endFadeOf(render);
+ let fade = null;
+ if (seconds > 0 && segments.length) {
+ const last = segments.length - 1;
+ const frames = Math.round((await probeDuration(segments[last], render.fps)) * render.fps) +
+ Math.round((base?.[last]?.hold ?? 0) * render.fps);
+ fade = { seconds, lastFrame: frames - 1 };
+ }
+ return withCutEdits(base, segments.length, { mutes, fade });
+}
+
+/**
* The segments' lengths IN THE CUT: probed, plus each one's hold -- the sum
* `deckSchedule` makes (it is handed the same probed lengths and adds the same
* holds), so the xfade offsets, the chapters and a preview's window agree with
@@ -2710,8 +2852,14 @@ export const concatListText = (segments) =>
export function concatRecordText(segments, joins = null) {
const list = concatListText(segments);
if (!joins) return list;
+ // `mute` and `fade` only when a join has them: a record made before they
+ // existed, of a cut without them, still matches.
const lines = segments.flatMap((s, i) => (joins[i]
- ? [`# join ${i} ${JSON.stringify({ hold: joins[i].hold, move: joins[i].move })}`]
+ ? [`# join ${i} ${JSON.stringify({
+ hold: joins[i].hold, move: joins[i].move,
+ ...(joins[i].mute != null ? { mute: joins[i].mute } : {}),
+ ...(joins[i].fade ? { fade: joins[i].fade } : {}),
+ })}`]
: []));
return list + lines.join("\n") + "\n";
}
@@ -2780,6 +2928,12 @@ export async function buildVideo({ manifestPath, opts = {}, out, only, fetchOnly
// A `render.chrome` that cannot be built is refused here, before a single
// fetch is spent. Absent, validateChrome has nothing to say.
if (render.chrome !== undefined && render.chrome !== null) assertChrome(render.chrome, render);
+ // A clip's `muteFrom` and `render.endFade`, checked against the WHOLE
+ // manifest, deck or not: both are made where the cut is joined.
+ {
+ const errors = validateCutEdits(whole);
+ if (errors.length) throw new Error(`manifest: ${errors.join("; ")}`);
+ }
const deck = deckOn(render);
// Posts are drawn only under the deck, so only the deck refuses bad ones --
// against the WHOLE timeline, where an `attachTo` has to name a clip.
@@ -2923,8 +3077,9 @@ export async function buildVideo({ manifestPath, opts = {}, out, only, fetchOnly
}
if (!(await exists(prerail))) {
EMIT("concat", { mode: D === 0 ? "hardcut" : "xfade", n: segs.length });
- if (D === 0) await concatHardCut(segs, outDir, prerail);
- else await concatWithXfade(segs, render, prerail, null);
+ const joins = await cutJoins({ entries, segments: segs, render });
+ if (D === 0) await concatHardCut(segs, outDir, prerail, { joins, render });
+ else await concatWithXfade(segs, render, prerail, null, null, joins);
}
const railPlan = await buildRailPlan(manifest, render, entries, segs, D, outDir);
await assertConcatLength(prerail, railPlan.total, render.fps,
@@ -2961,8 +3116,9 @@ export async function buildVideo({ manifestPath, opts = {}, out, only, fetchOnly
}
const schedule = await writeChromeSchedule({ manifest, entries, segments: segs, D, outDir });
EMIT("chrome", { phase: "schedule", total: schedule.total, segments: schedule.segments.length });
- // The holds and the moves, joined on their inputs (null without posts).
- const joins = segmentJoins(schedule);
+ // The holds, the moves, the mutes and the end fade, joined on their
+ // inputs (null without any).
+ const joins = await cutJoins({ schedule, entries, segments: segs, render });
const prerail = prerailPath(outDir, manifest.slug, D);
if (opts.chromePreview) {
@@ -3086,9 +3242,9 @@ export async function buildVideo({ manifestPath, opts = {}, out, only, fetchOnly
schedule = await writeChromeSchedule({ manifest, entries, segments, D, outDir });
EMIT("chrome", { phase: "schedule", total: schedule.total, segments: schedule.segments.length });
}
- // The deck's holds and moves, joined on their inputs; null without posts
- // (and always without the deck), which leaves every concat as it was.
- const joins = deck ? segmentJoins(schedule) : null;
+ // The deck's holds and moves, each clip's muteFrom and the end fade, joined
+ // on their inputs; null without any, which leaves every concat as it was.
+ const joins = await cutJoins({ schedule: deck ? schedule : null, entries, segments, render });
const railPlan = opts.noRail ? null : await buildRailPlan(manifest, render, entries, segments, D, outDir);
@@ -3134,7 +3290,7 @@ export async function buildVideo({ manifestPath, opts = {}, out, only, fetchOnly
// has to be a second pass here whether we like it or not.
const prerail = prerailPath(outDir, manifest.slug, D);
if (railPlan) {
- await concatHardCut(segments, outDir, prerail);
+ await concatHardCut(segments, outDir, prerail, { joins, render });
await assertConcatLength(prerail, railPlan.total, render.fps, "hard-cut concat");
await applyRail(prerail, final, render, railPlan, null);
} else if (chromePlan) {
diff --git a/umtool/report-to-video/chrome-deck.mjs b/umtool/report-to-video/chrome-deck.mjs
@@ -66,6 +66,9 @@ const esc = (s) =>
const r4 = (v) => Math.round(v * 10000) / 10000;
+/** The tracking of the QR's host label, in em: part of the length `fitHost` fits to the code. */
+export const HOST_TRACKING = 0.1;
+
/** The host a QR resolves to -- the one thing on the tile a viewer cannot read off the code. */
export function hostOf(url) {
try {
@@ -371,6 +374,9 @@ export function deckHtml(schedule, render, opts = {}) {
window: windowed ? { from, dur } : null,
titleSize: t.titleSize,
floor: Math.ceil(t.titleSize * 0.6),
+ // The QR's host label is fitted to run the code's full height.
+ hostLength: qr?.size ?? 0,
+ hostTracking: HOST_TRACKING,
ids: schedule.segments.map((s) => s.id),
init,
cues: cues.map(({ why, ...c }) => c),
@@ -455,10 +461,13 @@ export function deckHtml(schedule, render, opts = {}) {
height: 4px; border-radius: 2px; background: ${pal.accent}; }
.deck-qr { position: absolute; left: ${qr?.x ?? 0}px; top: ${qr?.y ?? 0}px;
width: ${qr?.size ?? 0}px; height: ${qr?.size ?? 0}px; transform-style: preserve-3d; }
+ /* The QR's host, reading up its left side. fitHost sizes it at load so
+ its ink runs the code's full height, bottom edge to top edge; 11px is
+ only what shows before the face is in. */
.deck-qr-host { position: absolute; left: ${(qr?.x ?? 0) - 30}px; top: ${qr?.y ?? 0}px; width: 18px;
height: ${qr?.size ?? 0}px; writing-mode: vertical-rl; transform: rotate(180deg);
- text-align: left; white-space: nowrap; font-size: 11px; line-height: 18px;
- letter-spacing: 0.1em; text-transform: uppercase; color: ${rgba(pal.muted, 0.85)}; }
+ text-align: start; white-space: nowrap; font-size: 11px; line-height: 18px;
+ letter-spacing: ${HOST_TRACKING}em; text-transform: uppercase; color: ${rgba(pal.muted, 0.85)}; }
.deck-qr img { display: block; width: ${qr?.size ?? 0}px; height: ${qr?.size ?? 0}px; border-radius: 6px;
image-rendering: pixelated; backface-visibility: hidden;
box-shadow: 0 0 0 1px ${rgba(pal.fg, 0.25)}, 0 6px 18px rgba(0, 0, 0, 0.35); }
@@ -526,10 +535,36 @@ export function deckHtml(schedule, render, opts = {}) {
}
}
const fitAll = () => document.querySelectorAll(".deck-title").forEach(fitTitle);
+
+ // The QR's host runs up beside the code, and its INK is exactly as long
+ // as the code is tall, whatever the host. Every length in it scales with
+ // the font size (the tracking is in em), so one measurement in the
+ // loaded face solves it: the string's ink at a reference size, plus the
+ // tracking between its letters (not after the last), scaled to the
+ // code's height. The first letter's side bearing is indented away, so
+ // the ink starts on the code's bottom edge and ends on its top.
+ const hostCanvas = document.createElement("canvas").getContext("2d");
+ function fitHost(node) {
+ // Measured as drawn: the CSS uppercases it.
+ const text = node.textContent.toUpperCase();
+ if (!text || !(D.hostLength > 0)) return;
+ const ref = 100;
+ hostCanvas.font = ref + "px DeckSans";
+ const m = hostCanvas.measureText(text);
+ const ink = m.actualBoundingBoxLeft + m.actualBoundingBoxRight + D.hostTracking * ref * (text.length - 1);
+ if (!(ink > 0)) return;
+ const k = D.hostLength / ink;
+ node.style.fontSize = ref * k + "px";
+ node.style.textIndent = m.actualBoundingBoxLeft * k + "px";
+ }
const ready = Promise.all([
document.fonts.load(D.titleSize + "px DeckSansBold"),
document.fonts.load("26px DeckSans"),
- ]).catch(() => {}).then(() => { fitAll(); document.documentElement.dataset.fit = "1"; });
+ ]).catch(() => {}).then(() => {
+ fitAll();
+ document.querySelectorAll(".deck-qr-host").forEach(fitHost);
+ document.documentElement.dataset.fit = "1";
+ });
const params = new URLSearchParams(location.search);
// The review still: a seek and nothing else -- the same seek the renderer
diff --git a/umtool/report-to-video/chrome-deck.test.mjs b/umtool/report-to-video/chrome-deck.test.mjs
@@ -307,3 +307,40 @@ appendFileSync(${JSON.stringify(path.join(dir, "runs.log"))}, a.join(" ") + "\\n
rmSync(dir, { recursive: true, force: true });
}
});
+
+const haveChromium = haveTools && spawnSync(process.env.CHROME ?? "/usr/bin/chromium", ["--version"], { stdio: "ignore" }).status === 0;
+
+test("the QR's host label: its ink runs the code's full height, top edge to bottom edge, for a long host and a short one",
+ { skip: !haveChromium }, async () => {
+ const { composeChrome } = await import("./compose-chrome.mjs");
+ const dir = mkdtempSync(path.join(tmpdir(), "deck-host-"));
+ try {
+ const manifest = {
+ slug: "t", title: "t", provenance: {},
+ render: { ...RENDER, fontRegular: path.join(HERE, "fonts", "IBMPlexMono-Regular.ttf"), fontBold: path.join(HERE, "fonts", "IBMPlexMono-Bold.ttf") },
+ timeline: [],
+ };
+ const manifestPath = path.join(dir, "video.manifest.json");
+ writeFileSync(manifestPath, JSON.stringify(manifest));
+ const qr = deckLayout(RENDER).qr;
+ // c01's code goes to jasolyzer.pages.dev, c03's to youtube.com.
+ for (const [t, host] of [[9, "jasolyzer.pages.dev"], [28, "youtube.com"]]) {
+ const png = path.join(dir, `still-${t}.png`);
+ await composeChrome({ manifestPath, region: "deck", schedule: schedule(), still: t, png });
+ // The column beside the code, as 8-bit grey rows, against the plate's own shade a little left of it.
+ const cols = { x: qr.x - 40, w: 36 };
+ const raw = spawnSync("magick", [png, "-crop", `${cols.w}x190+${cols.x}+0`, "+repage", "-colorspace", "gray", "-depth", "8", "gray:-"], { maxBuffer: 1 << 24 }).stdout;
+ const rows = [];
+ for (let y = 0; y < 190; y += 1) {
+ const ref = raw[y * cols.w];
+ for (let x = 4; x < cols.w; x += 1) if (raw[y * cols.w + x] - ref > 25) { rows.push(y); break; }
+ }
+ assert.ok(rows.length, `${host}: no label found`);
+ const top = Math.min(...rows), bottom = Math.max(...rows);
+ assert.ok(Math.abs(top - qr.y) <= 1, `${host}: ink starts at ${top}, the code at ${qr.y}`);
+ assert.ok(Math.abs(bottom - (qr.y + qr.size - 1)) <= 1, `${host}: ink ends at ${bottom}, the code at ${qr.y + qr.size - 1}`);
+ }
+ } finally {
+ rmSync(dir, { recursive: true, force: true });
+ }
+ });
diff --git a/umtool/report-to-video/cut-edits.test.mjs b/umtool/report-to-video/cut-edits.test.mjs
@@ -0,0 +1,326 @@
+// Tests for the cut's edits made where it is joined (slice B2): a clip's
+// `muteFrom` (source seconds → the segment's clock, through the cut record
+// the build writes beside each segment) and `render.endFade` on the cut's
+// last segment. The chains as strings, unchanged without them; the mapping;
+// validation; and real ffmpeg runs showing the sound after `muteFrom` is
+// digital silence, the picture is untouched, the end fade reaches bg and
+// silence on the last frame, and the length and A/V sync do not move.
+//
+// Run with: pnpm test:scripts
+import assert from "node:assert/strict";
+import { spawnSync } from "node:child_process";
+import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import path from "node:path";
+import test from "node:test";
+
+import {
+ concatListText, concatRecordText, cutJoins, cutRecordPath, endFadeAudioFilter, endFadeVideoFilter,
+ hardCutFilterArgs, joinInputChain, muteAudioFilter, sameConcatList, withCutEdits, xfadeGraph, yuv601,
+} from "./build-video.mjs";
+import {
+ endFadeOf, MUTE_FADE, muteSegmentSeconds, playWindow, validateChrome, validateCutEdits, validateEndFade,
+ validateMuteFrom,
+} from "./deck.mjs";
+
+const PALETTE = { bg: "#12101a", fg: "#f4f1ea", muted: "#9a93ad", accent: "#a97bff", amber: "#ffc860" };
+const RENDER = {
+ width: 1920, height: 1080, fps: 30, transition: 0.5, palette: PALETTE,
+ audioRate: 48000, audioChannels: 2, chrome: { engine: "hyperframes", layout: "deck", deck: {} },
+};
+const CLIP = { id: "c20", type: "clip", video: "B36", start: 24022.6, end: 24029.6 };
+
+// ---- validation -------------------------------------------------------------
+
+test("validateMuteFrom: a number of source seconds within the clip's extent; only on a clip", () => {
+ assert.deepEqual(validateMuteFrom(CLIP), []);
+ assert.deepEqual(validateMuteFrom({ ...CLIP, muteFrom: null }), []);
+ assert.deepEqual(validateMuteFrom({ ...CLIP, muteFrom: 24029.3 }), []);
+ assert.deepEqual(validateMuteFrom({ ...CLIP, muteFrom: 24022.6 }), [], "the start is inside");
+ assert.deepEqual(validateMuteFrom({ ...CLIP, muteFrom: 24029.6 }), [], "the end is inside");
+ assert.match(validateMuteFrom({ ...CLIP, muteFrom: 24029.7 })[0], /muteFrom 24029\.7 is outside the clip's 24022\.6–24029\.6/);
+ assert.match(validateMuteFrom({ ...CLIP, muteFrom: 3 })[0], /outside/);
+ assert.match(validateMuteFrom({ ...CLIP, muteFrom: "24029" })[0], /must be a number of source seconds/);
+ assert.match(validateMuteFrom({ ...CLIP, muteFrom: NaN })[0], /must be a number/);
+ assert.match(validateMuteFrom({ id: "t1", type: "card", muteFrom: 2 })[0], /only a clip has sound to mute/);
+});
+
+test("validateEndFade and validateCutEdits: seconds from 0 to 10; every entry named by its place", () => {
+ assert.deepEqual(validateEndFade({}), []);
+ assert.deepEqual(validateEndFade({ endFade: 0 }), []);
+ assert.deepEqual(validateEndFade({ endFade: 1.5 }), []);
+ for (const bad of [-1, 11, "1", NaN]) assert.match(validateEndFade({ endFade: bad })[0], /render\.endFade must be from 0 to 10 seconds/);
+ assert.equal(endFadeOf({}), 0);
+ assert.equal(endFadeOf({ endFade: 1 }), 1);
+ assert.equal(endFadeOf({ endFade: -1 }), 0);
+ const errs = validateCutEdits({
+ render: { endFade: 20 },
+ timeline: [CLIP, { ...CLIP, id: "c21", muteFrom: 1 }],
+ });
+ assert.equal(errs.length, 2);
+ assert.match(errs[0], /^timeline\[1\] \(c21\)\.muteFrom 1 is outside/);
+ assert.match(errs[1], /render\.endFade/);
+ assert.deepEqual(validateCutEdits({ render: {}, timeline: [CLIP] }), []);
+ // The deck's validator refuses a bad end fade too; a good one changes nothing.
+ assert.ok(validateChrome(RENDER.chrome, { ...RENDER, endFade: 99 }).some((e) => /render\.endFade/.test(e)));
+ assert.deepEqual(validateChrome(RENDER.chrome, { ...RENDER, endFade: 1 }), []);
+});
+
+// ---- the mapping, source → segment -------------------------------------------
+
+test("muteSegmentSeconds: from the cut record's snapped start when it matches the segment", () => {
+ const record = { version: 1, id: "c20", video: "B36", start: 24022.5, end: 24029.5 };
+ assert.deepEqual(
+ muteSegmentSeconds({ entry: { ...CLIP, muteFrom: 24029.3 }, record, render: RENDER, seconds: 7 }),
+ { at: 6.8, source: "record" },
+ );
+ // A muteFrom before the segment's real start mutes it from its first sample.
+ assert.equal(muteSegmentSeconds({ entry: { ...CLIP, muteFrom: 24022.6 }, record: { ...record, start: 24022.7, end: 24029.7 }, render: RENDER, seconds: 7 }).at, 0);
+});
+
+test("muteSegmentSeconds: no record, a stale one or another video's -- the unsnapped start, and a note that says so", () => {
+ const entry = { ...CLIP, muteFrom: 24029.3 };
+ const none = muteSegmentSeconds({ entry, record: null, render: RENDER, seconds: 7 });
+ assert.equal(none.at, 6.7);
+ assert.equal(none.source, "window");
+ assert.match(none.note, /^c20: no cut record beside the segment — muteFrom measured from the unsnapped start 24022\.6; the real start may differ by up to 1\.6s/);
+ const stale = muteSegmentSeconds({ entry, record: { video: "B36", start: 24020, end: 24030 }, render: RENDER, seconds: 7 });
+ assert.equal(stale.source, "window");
+ assert.match(stale.note, /the cut record does not match the segment/);
+ assert.equal(muteSegmentSeconds({ entry, record: { video: "other", start: 24022.5, end: 24029.5 }, render: RENDER, seconds: 7 }).source, "window");
+ // Within two frames of the segment's length is a match.
+ assert.equal(muteSegmentSeconds({ entry, record: { video: "B36", start: 24022.6, end: 24029.65 }, render: RENDER, seconds: 7 }).source, "record");
+ // A clip with a tight cut plays from cutStart less the lead-in.
+ const cut = { ...CLIP, cutStart: 24025, cutEnd: 24029, muteFrom: 24028 };
+ assert.deepEqual(playWindow(cut, RENDER), { from: 24024.6, to: 24029 });
+ assert.equal(muteSegmentSeconds({ entry: cut, render: RENDER }).at, 3.4);
+});
+
+// ---- the chains, as strings -------------------------------------------------
+
+test("muteAudioFilter: afade out ENDING at the mute point, silent after; volume=0 from the first sample", () => {
+ assert.equal(MUTE_FADE, 0.04);
+ assert.equal(muteAudioFilter(6.7), "afade=t=out:st=6.66:d=0.04");
+ assert.equal(muteAudioFilter(0.02), "afade=t=out:st=0:d=0.02");
+ assert.equal(muteAudioFilter(0), "volume=0");
+});
+
+test("the end fade: a yuv blend toward bg from frame s = last − n, so the LAST frame is bg; silence at that frame's time", () => {
+ assert.deepEqual(yuv601("#12101a"), { y: 31, u: 132, v: 128 }, "what pad wrote into the ferret segments");
+ assert.deepEqual(yuv601("#000000"), { y: 16, u: 128, v: 128 });
+ assert.deepEqual(yuv601("#ffffff"), { y: 235, u: 128, v: 128 });
+ const fade = { seconds: 1, lastFrame: 284 }; // 7 s + 2.5 s hold at 30 fps = 285 frames
+ assert.equal(endFadeVideoFilter(fade, RENDER), "geq=lum='lum(X,Y)+(31-lum(X,Y))*clip((T-8.4667)/1,0,1)+0.5':cb='cb(X,Y)+(132-cb(X,Y))*clip((T-8.4667)/1,0,1)+0.5':cr='cr(X,Y)+(128-cr(X,Y))*clip((T-8.4667)/1,0,1)+0.5':enable='gte(t,8.4667)'");
+ assert.equal(endFadeAudioFilter(fade, RENDER), "afade=t=out:st=8.4667:d=1");
+});
+
+test("joinInputChain: a mute alone is a chain on the sound only; the picture is the input's own", () => {
+ assert.deepEqual(joinInputChain(2, { hold: 0, move: null, mute: 6.7 }, RENDER), {
+ parts: ["[2:a]afade=t=out:st=6.66:d=0.04[j2a]"], v: "[2:v]", a: "[j2a]",
+ });
+ // Mute, then the hold's silence, then the end fade, in that order; the
+ // picture holds, moves (none here) and fades.
+ const fade = { seconds: 1, lastFrame: 284 };
+ assert.deepEqual(joinInputChain(0, { hold: 2.5, move: null, mute: 6.7, fade }, RENDER), {
+ parts: [
+ "[0:v]tpad=stop_mode=clone:stop_duration=2.5,geq=lum='lum(X,Y)+(31-lum(X,Y))*clip((T-8.4667)/1,0,1)+0.5':cb='cb(X,Y)+(132-cb(X,Y))*clip((T-8.4667)/1,0,1)+0.5':cr='cr(X,Y)+(128-cr(X,Y))*clip((T-8.4667)/1,0,1)+0.5':enable='gte(t,8.4667)'[j0v]",
+ "[0:a]afade=t=out:st=6.66:d=0.04,apad=pad_dur=2.5,afade=t=out:st=8.4667:d=1[j0a]",
+ ],
+ v: "[j0v]",
+ a: "[j0a]",
+ });
+});
+
+test("withCutEdits: nothing to add leaves the joins as they were (null stays null); edits land on their segments", () => {
+ assert.equal(withCutEdits(null, 3), null);
+ const joins = [null, { hold: 2.5, move: null }, null];
+ assert.equal(withCutEdits(joins, 3, {}), joins, "the same joins, not a copy");
+ const fade = { seconds: 1, lastFrame: 209 };
+ const out = withCutEdits(joins, 3, { mutes: new Map([[1, 4], [2, 6.7]]), fade });
+ assert.deepEqual(out, [
+ null,
+ { hold: 2.5, move: null, mute: 4 },
+ { hold: 0, move: null, mute: 6.7, fade },
+ ]);
+ assert.deepEqual(joins[1], { hold: 2.5, move: null }, "the schedule's joins are not written to");
+ assert.deepEqual(withCutEdits(null, 2, { fade }), [null, { hold: 0, move: null, fade }]);
+});
+
+test("the graphs: unchanged without edits; the hard cut's record names a mute and a fade, so a changed one is never reused", () => {
+ // No edits: the crossfade graph the build always wrote.
+ assert.deepEqual(xfadeGraph([10, 12], 0.5, withCutEdits(null, 2), RENDER).parts, [
+ "[0:v][1:v]xfade=transition=fade:duration=0.5:offset=9.500[v1]",
+ "[0:a][1:a]acrossfade=d=0.5:c1=tri:c2=tri[a1]",
+ ]);
+ const segs = ["/s/a.mp4", "/s/b.mp4"];
+ const holdOnly = [null, { hold: 2.5, move: null }];
+ assert.equal(concatRecordText(segs, withCutEdits(null, 2)), concatListText(segs));
+ assert.equal(
+ concatRecordText(segs, holdOnly),
+ concatListText(segs) + '# join 1 {"hold":2.5,"move":null}\n',
+ "a hold's record line is the one it always was",
+ );
+ const fade = { seconds: 1, lastFrame: 359 };
+ const edited = withCutEdits(holdOnly, 2, { mutes: new Map([[0, 3]]), fade });
+ const rec = concatRecordText(segs, edited);
+ assert.match(rec, /^# join 0 \{"hold":0,"move":null,"mute":3\}$/m);
+ assert.match(rec, /^# join 1 \{"hold":2\.5,"move":null,"fade":\{"seconds":1,"lastFrame":359\}\}$/m);
+ assert.equal(sameConcatList(rec, segs, edited), true);
+ assert.equal(sameConcatList(rec, segs, holdOnly), false);
+ assert.equal(sameConcatList(rec, segs, withCutEdits(holdOnly, 2, { mutes: new Map([[0, 3.5]]), fade })), false);
+ assert.equal(sameConcatList(rec, segs, withCutEdits(holdOnly, 2, { mutes: new Map([[0, 3]]) })), false);
+ const fc = hardCutFilterArgs(segs, edited, RENDER, "/o.mp4");
+ assert.equal(fc[fc.indexOf("-filter_complex") + 1],
+ "[0:a]afade=t=out:st=2.96:d=0.04[j0a];" +
+ "[1:v]tpad=stop_mode=clone:stop_duration=2.5,geq=lum='lum(X,Y)+(31-lum(X,Y))*clip((T-10.9667)/1,0,1)+0.5':cb='cb(X,Y)+(132-cb(X,Y))*clip((T-10.9667)/1,0,1)+0.5':cr='cr(X,Y)+(128-cr(X,Y))*clip((T-10.9667)/1,0,1)+0.5':enable='gte(t,10.9667)'[j1v];" +
+ "[1:a]apad=pad_dur=2.5,afade=t=out:st=10.9667:d=1[j1a];" +
+ "[0:v][j0a][j1v][j1a]concat=n=2:v=1:a=1[vc][ac]");
+});
+
+// ---- ffmpeg, for real -------------------------------------------------------
+
+const haveFfmpeg = spawnSync("ffmpeg", ["-version"], { stdio: "ignore" }).status === 0;
+const R = { width: 320, height: 180, fps: 30, palette: PALETTE, crf: 21, preset: "veryfast", audioRate: 48000, audioChannels: 2 };
+const ff = (args, opts = {}) => {
+ const r = spawnSync("ffmpeg", ["-nostdin", "-v", "error", "-y", ...args], { maxBuffer: 1 << 28, ...opts });
+ assert.equal(r.status, 0, String(r.stderr));
+ return r.stdout;
+};
+const md5s = (out, stream = 0) => String(out).split("\n").filter((l) => l && !l.startsWith("#"))
+ .map((l) => l.split(",")).filter((f) => Number(f[0]) === stream).map((f) => f.at(-1).trim());
+
+/** Three 2 s segments, framed as the deck frames them, each with a tone. */
+function segments(dir, ext = "mov") {
+ const make = (name, src, hz) => {
+ const f = path.join(dir, `${name}.${ext}`);
+ const codec = ext === "mov" ? ["-c:v", "ffv1", "-c:a", "pcm_s16le"] : ["-c:v", "libx264", "-preset", "ultrafast", "-pix_fmt", "yuv420p", "-c:a", "aac"];
+ ff([
+ "-f", "lavfi", "-i", `${src}=s=280x150:r=30:d=2`,
+ "-f", "lavfi", "-i", `sine=frequency=${hz}:sample_rate=48000:duration=2`,
+ "-filter_complex", `[0:v]pad=320:180:20:10:color=${PALETTE.bg},format=yuv420p[v];[1:a]aformat=channel_layouts=stereo[a]`,
+ "-map", "[v]", "-map", "[a]", ...codec, f,
+ ]);
+ return f;
+ };
+ return [make("a", "testsrc2", 440), make("b", "smptebars", 550), make("c", "rgbtestsrc", 660)];
+}
+
+/** Run a graph to mono s16 PCM (the picture sunk), and to frame hashes. */
+const pcmOf = (inputs, parts, v, a) =>
+ ff([...inputs, "-filter_complex", `${parts.join(";")};${v}nullsink`, "-map", a, "-f", "s16le", "-ac", "1", "-ar", "48000", "-"], { encoding: "buffer" });
+const framesOf = (inputs, parts, v, a) =>
+ md5s(ff([...inputs, "-filter_complex", `${parts.join(";")};${a}anullsink`, "-map", v, "-f", "framemd5", "-"]));
+const sample = (pcm, i) => pcm.readInt16LE(i * 2);
+const peak = (pcm, a, b) => {
+ let m = 0;
+ for (let i = Math.round(a * 48000); i < Math.min(pcm.length / 2, Math.round(b * 48000)); i += 1) m = Math.max(m, Math.abs(sample(pcm, i)));
+ return m;
+};
+
+test("ffmpeg: after muteFrom the sound is digital silence; before its fade it is the clip's own; the picture is untouched",
+ { skip: !haveFfmpeg }, () => {
+ const dir = mkdtempSync(path.join(tmpdir(), "cut-mute-"));
+ try {
+ const [a, b, c] = segments(dir);
+ const inputs = [a, b, c].flatMap((s) => ["-i", s]);
+ const D = 0.5;
+ const plain = xfadeGraph([2, 2, 2], D, null, R);
+ const joins = withCutEdits(null, 3, { mutes: new Map([[1, 1.2]]) });
+ const muted = xfadeGraph([2, 2, 2], D, joins, R);
+ // The picture: frame for frame the graph without the mute.
+ assert.deepEqual(framesOf(inputs, muted.parts, muted.vlab, muted.alab), framesOf(inputs, plain.parts, plain.vlab, plain.alab));
+ const p0 = pcmOf(inputs, plain.parts, plain.vlab, plain.alab);
+ const p1 = pcmOf(inputs, muted.parts, muted.vlab, muted.alab);
+ assert.equal(p1.length, p0.length, "the sound is as long as it was");
+ // b plays from 1.5 s in the cut, so its mute point is 2.7 s; the dissolve
+ // into c starts at 3.0 s. Before the fade (2.66 s), every sample is the
+ // unmuted cut's -- nothing moved, so A/V sync is what it was.
+ const fadeAt = Math.round((1.5 + 1.2 - MUTE_FADE) * 48000);
+ assert.ok(p1.subarray(0, fadeAt * 2).equals(p0.subarray(0, fadeAt * 2)), "untouched before the fade");
+ assert.ok(peak(p0, 2.7, 3.0) > 1000, "b sounds there without the mute");
+ assert.equal(peak(p1, 2.7, 3.0), 0, "digital silence from the mute point to the dissolve");
+ assert.ok(peak(p1, 2.66, 2.7) > 0 && peak(p1, 2.66, 2.7) < peak(p0, 2.66, 2.7), "a fade, not a click");
+ // From the dissolve on, only c's sound: the cut after it is c's own.
+ const cOnly = pcmOf(["-i", c], ["[0:a]anull[a]"], "[0:v]", "[a]");
+ const tail = p1.subarray(Math.round(3.5 * 48000) * 2, Math.round(5.5 * 48000) * 2);
+ const own = cOnly.subarray(Math.round(0.5 * 48000) * 2, Math.round(2.5 * 48000) * 2);
+ assert.ok(peak(p1, 3.6, 5.4) > 1000);
+ assert.equal(tail.length, own.length);
+ } finally {
+ rmSync(dir, { recursive: true, force: true });
+ }
+ });
+
+test("ffmpeg: the end fade -- the last frame is bg and the sound silent there; with a hold and a hard cut, the length is unchanged",
+ { skip: !haveFfmpeg }, () => {
+ const dir = mkdtempSync(path.join(tmpdir(), "cut-fade-"));
+ try {
+ const segs = segments(dir);
+ const inputs = segs.flatMap((s) => ["-i", s]);
+ // c is 60 frames, held 0.5 s (15 frames): its last frame is 74.
+ const base = [null, null, { hold: 0.5, move: null }];
+ const fade = { seconds: 1, lastFrame: 74 };
+ const joins = withCutEdits(base, 3, { fade });
+ const plainArgs = hardCutFilterArgs(segs, base, R, "-");
+ const fadeArgs = hardCutFilterArgs(segs, joins, R, "-");
+ const fc = (args) => [args[args.indexOf("-filter_complex") + 1]];
+ const f0 = framesOf(inputs, fc(plainArgs), "[vc]", "[ac]");
+ const f1 = framesOf(inputs, fc(fadeArgs), "[vc]", "[ac]");
+ assert.equal(f1.length, f0.length, "as many frames as without the fade");
+ assert.equal(f1.length, 60 + 60 + 75);
+ assert.deepEqual(f1.slice(0, 120 + 44), f0.slice(0, 120 + 44), "every frame before the fade is the same");
+ assert.notDeepEqual(f1[120 + 50], f0[120 + 50], "fading");
+ // The last frame, decoded: bg everywhere.
+ const last = ff([...inputs, "-filter_complex", `${fc(fadeArgs).join(";")};[ac]anullsink;[vc]select=eq(n\\,194)[o]`,
+ "-map", "[o]", "-frames:v", "1", "-f", "rawvideo", "-pix_fmt", "rgb24", "-"], { encoding: "buffer" });
+ const bg = [0x12, 0x10, 0x1a];
+ let worst = 0;
+ for (let i = 0; i < last.length; i += 1) worst = Math.max(worst, Math.abs(last[i] - bg[i % 3]));
+ assert.ok(worst <= 2, `the last frame is bg (worst channel off by ${worst})`);
+ // The sound: as long as without the fade, the same up to it, silent from the last frame's time.
+ const p0 = pcmOf(inputs, fc(plainArgs), "[vc]", "[ac]");
+ const p1 = pcmOf(inputs, fc(fadeArgs), "[vc]", "[ac]");
+ assert.equal(p1.length, p0.length);
+ // The hold is silent already; give c's own tone the fade instead.
+ const toneJoins = withCutEdits([null, null, null], 3, { fade: { seconds: 1, lastFrame: 59 } });
+ const t0 = pcmOf(inputs, fc(hardCutFilterArgs(segs, [null, null, null], R, "-")), "[vc]", "[ac]");
+ const t1 = pcmOf(inputs, fc(hardCutFilterArgs(segs, toneJoins, R, "-")), "[vc]", "[ac]");
+ assert.equal(t1.length, t0.length);
+ const lastAt = 4 + 59 / 30; // c starts at 4 s in the hard cut
+ const fadeStart = Math.round((lastAt - 1) * 48000);
+ assert.ok(t1.subarray(0, fadeStart * 2).equals(t0.subarray(0, fadeStart * 2)), "the same sound up to the fade");
+ assert.ok(peak(t1, lastAt - 0.9, lastAt - 0.8) < peak(t0, lastAt - 0.9, lastAt - 0.8), "fading");
+ assert.equal(peak(t1, lastAt, 6), 0, "silent from the last frame on");
+ assert.ok(peak(t0, lastAt, 6) > 1000);
+ } finally {
+ rmSync(dir, { recursive: true, force: true });
+ }
+ });
+
+test("cutJoins: the record beside the segment places muteFrom; the end fade counts the last segment's frames and hold",
+ { skip: !haveFfmpeg }, async () => {
+ const dir = mkdtempSync(path.join(tmpdir(), "cut-joins-"));
+ try {
+ const segs = segments(dir, "mp4");
+ const entries = [
+ { id: "a", type: "clip", video: "va", start: 100, end: 102 },
+ { id: "b", type: "clip", video: "vb", start: 200, end: 203, muteFrom: 201.5 },
+ { id: "c", type: "clip", video: "vc", start: 300, end: 302 },
+ ];
+ // No record for b: the unsnapped start (200), 1.5 s in.
+ assert.equal(await cutJoins({ entries: entries.slice(0, 1), segments: segs.slice(0, 1), render: R }), null, "nothing to join");
+ let j = await cutJoins({ entries, segments: segs, render: R });
+ assert.deepEqual(j, [null, { hold: 0, move: null, mute: 1.5 }, null]);
+ // b's record says it was cut from 200.3: 1.2 s in.
+ writeFileSync(cutRecordPath(segs[1]), JSON.stringify({ version: 1, id: "b", video: "vb", start: 200.3, end: 202.3 }));
+ j = await cutJoins({ entries, segments: segs, render: R });
+ assert.equal(j[1].mute, 1.2);
+ // The end fade on c, with the deck's hold on it: 60 + 15 frames.
+ const schedule = { segments: [{ id: "a" }, { id: "b" }, { id: "c", hold: 0.5 }] };
+ j = await cutJoins({ schedule, entries, segments: segs, render: { ...R, endFade: 1 } });
+ assert.deepEqual(j[2], { hold: 0.5, move: null, fade: { seconds: 1, lastFrame: 74 } });
+ assert.equal(j[0], null);
+ } finally {
+ rmSync(dir, { recursive: true, force: true });
+ }
+ });
diff --git a/umtool/report-to-video/deck-room.test.mjs b/umtool/report-to-video/deck-room.test.mjs
@@ -76,6 +76,53 @@ test("the cut's lengths: the schedule's starts are the probed lengths plus the h
assert.equal(s.total, 10 + 12 + 4 - 1 + 2.5);
});
+test("a first post inside the hold: the move's segmentAt is past the clip's own last frame, in the held clock", () => {
+ // c2 is 12 s on disk; one post at 2 s with the 2.5 s hold appears 12.0 s
+ // into the segment -- inside the hold. The move runs after the hold, so
+ // that is a moment its clock reaches.
+ const two = { ...RENDER, chrome: { ...RENDER.chrome, deck: { posts: { seconds: 2 } } } };
+ const s = deckSchedule({ entries: CLIPS, durs: [10, 12, 4], D: 0.5, render: two, provenance: PROV, posts: [POST("a", "2024-10-19")] });
+ const [m] = s.moves;
+ assert.equal(m.segment, "c2");
+ assert.equal(m.segmentAt, 12);
+ assert.equal(s.segments[1].duration, 14.5);
+ assert.ok(m.segmentAt + m.seconds <= s.segments[1].duration - 0.5, "the glide lands before the dissolve out");
+ const c = joinInputChain(1, segmentJoins(s)[1], two);
+ assert.ok(c.parts[0].indexOf("tpad=") < c.parts[0].indexOf("perspective="), "hold, then move");
+});
+
+test("holds are whole frames: 2.5 s at 25 fps is 63 frames (2.52 s), and the cut's length counts that", () => {
+ const r25 = { ...RENDER, fps: 25 };
+ const s = deckSchedule({
+ entries: CLIPS, durs: [10, 12, 4], D: 0, render: r25, provenance: PROV,
+ posts: [POST("a", "2024-10-19"), POST("b", "2025-07-01")],
+ });
+ assert.equal(s.segments[1].hold, 2.52);
+ assert.equal(s.segments[2].hold, 2.52);
+ assert.equal(s.total, 10 + 12 + 4 + 2 * 2.52);
+ assert.equal(holdVideoFilter(segmentJoins(s)[1].hold), "tpad=stop_mode=clone:stop_duration=2.52");
+ // At 30 fps 2.5 s is already 75 frames: unchanged.
+ const s30 = deckSchedule({ entries: CLIPS, durs: [10, 12, 4], D: 0, render: RENDER, provenance: PROV, posts: [POST("a", "2024-10-19")] });
+ assert.equal(s30.segments[1].hold, 2.5);
+});
+
+test("freezeSamples: between the hold's start (or the move's end) and the dissolve; skipped when under three frames", async () => {
+ const { freezeSamples } = await import("./verify-build.mjs");
+ const seg = { id: "c2", start: 9.5, end: 24, hold: 2.5 };
+ const close = (a, b) => assert.ok(Math.abs(a - b) < 1e-9, `${a} != ${b}`);
+ const a = freezeSamples(seg, { fps: 30, D: 0.5 });
+ close(a.at[0], 21.5 + 0.05);
+ close(a.at[1], 23.5 - 0.05);
+ // A move that ends inside the hold: the still span starts where it lands.
+ close(freezeSamples(seg, { fps: 30, D: 0.5, moveEnd: 22.1 }).at[0], 22.1 + 0.05);
+ // Hold 0.5 under a 0.5 s crossfade: all of it is the dissolve.
+ assert.match(freezeSamples({ ...seg, hold: 0.5 }, { fps: 30, D: 0.5 }).skip, /^0\.000s of still picture/);
+ // Hard cut: up to the end. The last segment: up to its end fade.
+ close(freezeSamples({ ...seg, hold: 0.5 }, { fps: 30, D: 0 }).at[1], 24 - 0.05);
+ close(freezeSamples(seg, { fps: 30, D: 0.5, last: true, endFade: 1 }).at[1], 23 - 0.05);
+ assert.ok(freezeSamples({ ...seg, hold: 1 }, { fps: 30, D: 0.5, last: true, endFade: 1 }).skip);
+});
+
// ---- the chains, as strings -------------------------------------------------
test("joinInputChain: no join, no chain -- the input's own labels", () => {
@@ -92,10 +139,10 @@ test("joinInputChain: a hold is tpad clone on the picture and apad silence on th
});
});
-test("joinInputChain: the move goes before the hold, so the held frame is the moved one; a move alone leaves the sound alone", () => {
+test("joinInputChain: the move goes after the hold, so its clock counts the held frames; a move alone leaves the sound alone", () => {
const both = joinInputChain(1, { hold: 2.5, move: MOVE(6) }, RENDER);
assert.equal(both.parts.length, 2);
- assert.equal(both.parts[0], `[1:v]${moveFilter(MOVE(6), RENDER)},tpad=stop_mode=clone:stop_duration=2.5[j1v]`);
+ assert.equal(both.parts[0], `[1:v]tpad=stop_mode=clone:stop_duration=2.5,${moveFilter(MOVE(6), RENDER)}[j1v]`);
assert.equal(both.parts[1], "[1:a]apad=pad_dur=2.5[j1a]");
const move = joinInputChain(1, { hold: 0, move: MOVE(6) }, RENDER);
assert.deepEqual(move, { parts: [`[1:v]${moveFilter(MOVE(6), RENDER)}[j1v]`], v: "[j1v]", a: "[1:a]" });
@@ -354,6 +401,36 @@ test("ffmpeg: the move is the identity before it starts, lands on the target box
}
});
+test("ffmpeg: a move that starts inside the hold glides the frozen frame to the target box",
+ { skip: !haveFfmpeg }, () => {
+ const dir = mkdtempSync(path.join(tmpdir(), "deck-room-mh-"));
+ try {
+ const [, b] = segments(dir);
+ // b is 2 s (60 frames), held 1 s; the move starts at 2.1 s -- inside the hold.
+ const move = { segment: "b", at: 0, segmentAt: 2.1, seconds: 0.6,
+ from: { x: 20, y: 10, width: 280, height: 150 }, to: { x: 8, y: 20, width: 240, height: 128 } };
+ const c = joinInputChain(0, { hold: 1, move }, R);
+ c.parts.push(`${c.a}anullsink`);
+ const all = md5s(ff(["-i", b, "-filter_complex", c.parts.join(";"), "-map", c.v, "-f", "framemd5", "-"]));
+ assert.equal(all.length, 90, "the hold's 30 frames are all there");
+ const rgb = (n) => ff(["-i", b, "-filter_complex", `${c.parts.join(";")};${c.v}select=eq(n\\,${n})[o]`, "-map", "[o]",
+ "-frames:v", "1", "-f", "rawvideo", "-pix_fmt", "rgb24", "-"], { encoding: "buffer" });
+ const px = (buf, x, y) => [...buf.subarray((y * 320 + x) * 3, (y * 320 + x) * 3 + 3)];
+ const bg = [0x12, 0x10, 0x1a];
+ const close = (p, q, tol = 3) => p.every((v, i) => Math.abs(v - q[i]) <= tol);
+ // Frozen and not yet moved at 2.0 s; moving from 2.1 s; landed by 2.7 s.
+ assert.equal(all[60], all[62], "held, before the move");
+ assert.ok(!close(px(rgb(62), 290, 150), bg), "footage still at the from box");
+ assert.notEqual(all[66], all[62], "the held frame moves");
+ const post = rgb(85);
+ for (const [x, y] of [[300, 10], [310, 170], [260, 100]]) assert.ok(close(px(post, x, y), bg), `ground at ${x},${y}`);
+ assert.ok(!close(px(post, 245, 145), bg), "footage to the target box's far corner");
+ assert.deepEqual(all.slice(82), Array(8).fill(all[82]), "landed, then still");
+ } finally {
+ rmSync(dir, { recursive: true, force: true });
+ }
+ });
+
test("verify-build: a held clip's freeze is found in the file, and a cut that dropped it is refused", { skip: !haveFfmpeg }, async () => {
const { verifyHolds } = await import("./verify-build.mjs");
const dir = mkdtempSync(path.join(tmpdir(), "deck-room-vb-"));
diff --git a/umtool/report-to-video/deck.mjs b/umtool/report-to-video/deck.mjs
@@ -114,6 +114,9 @@ export function validateChrome(chrome, render = {}) {
if (render.chromeEngine !== undefined) {
errors.push("render.chrome replaces render.chromeEngine — remove chromeEngine");
}
+ // The end fade is a render key the deck's cut is finished with; checked
+ // here too, so a writer that validates the deck refuses a bad one.
+ errors.push(...validateEndFade(render));
const d = chrome.deck ?? {};
if (!isObj(d)) return [...errors, "render.chrome.deck must be an object"];
const w = "render.chrome.deck";
@@ -391,16 +394,30 @@ export function scheduleFrom(durs, D) {
*/
export function estimatedDuration(entry, render = {}) {
if (entry.type === "clip") {
- const hasCut = Number.isFinite(entry.cutStart) && Number.isFinite(entry.cutEnd);
- const lead = render.leadIn ?? 0.4;
- const a = hasCut ? Math.max(entry.start, entry.cutStart - lead) : entry.start;
- const b = hasCut ? Math.min(entry.end, entry.cutEnd) : entry.end;
- return Math.max(1, b - a);
+ const { from, to } = playWindow(entry, render);
+ return Math.max(1, to - from);
}
if (entry.type === "image") return Number(entry.seconds ?? 4);
return Number(entry.seconds ?? 5);
}
+/**
+ * The SOURCE seconds a clip asks to play, before silence snapping: the cut
+ * (`cutStart`/`cutEnd`) with the lead-in breath before it, clamped into the
+ * extent, when there is one; else the extent (`start`/`end`). The build
+ * snaps each end to a nearby silence, so the segment's true start is this
+ * `from` moved by up to `snapWindow` -- which is why a build records the real
+ * one beside the segment (`<id>.cut.json`).
+ */
+export function playWindow(entry, render = {}) {
+ const hasCut = Number.isFinite(entry.cutStart) && Number.isFinite(entry.cutEnd);
+ const lead = render.leadIn ?? 0.4;
+ return {
+ from: hasCut ? Math.max(entry.start, entry.cutStart - lead) : entry.start,
+ to: hasCut ? Math.min(entry.end, entry.cutEnd) : entry.end,
+ };
+}
+
/** The crossfade a build of this render block will use. */
export function transitionOf(render, { noXfade = false } = {}) {
const t = render?.transition ?? 0.5;
@@ -834,9 +851,15 @@ export function shiftedFootage(render) {
return { x: Math.round(x), y: Math.floor((H - d.height - height) / 2), width, height };
}
-/** How long each clip that carries posts is held on its last frame (entry id → seconds). */
+/**
+ * How long each clip that carries posts is held on its last frame (entry id →
+ * seconds), in WHOLE FRAMES: `tpad` clones a whole number of frames (2.5 s at
+ * 25 fps is 63, not 62.5) while `apad` pads exactly, so an unrounded hold
+ * would make a hard cut with several held clips longer than its schedule.
+ */
export function postHolds({ posts = [], entries = [], metas = [], render }) {
- const hold = resolveDeck(render).posts.hold;
+ const fps = render?.fps ?? 30;
+ const hold = Math.round(resolveDeck(render).posts.hold * fps) / fps;
const out = new Map();
if (!(hold > 0)) return out;
for (const a of attachPosts({ posts, entries, metas })) out.set(a.entryId, hold);
@@ -846,9 +869,11 @@ export function postHolds({ posts = [], entries = [], metas = [], render }) {
/**
* When the footage moves aside for a clip's posts: one move per carrying clip,
* starting as its first post appears (`at`, cut clock; `segmentAt`, the
- * segment's own clock) and easing over `shift.seconds` from the footage box
- * to `shiftedFootage`. It stays there to the end of the segment; the next
- * segment comes in at the normal box through the transition.
+ * segment's own clock, its hold included) and easing over `shift.seconds`
+ * from the footage box to `shiftedFootage`. It stays there to the end of the
+ * segment; the next segment comes in at the normal box through the
+ * transition. `segmentAt` may fall inside the hold: the build runs the move
+ * after the hold, so the frozen frame moves too.
*
* @returns {Array<{ segment, at, segmentAt, seconds, from, to }>}
*/
@@ -865,6 +890,88 @@ export function footageMoves({ posts, segments, render }) {
}
// ---------------------------------------------------------------------------
+// Cut edits made where the cut is joined, like the hold: a clip's `muteFrom`
+// and the cut's `render.endFade`. Neither touches a segment file, so
+// `--chrome-only` changes either without rebuilding a clip. Pure here: the
+// validators (umtool's writers and the build share them) and the arithmetic.
+// ---------------------------------------------------------------------------
+
+/** The fade into a `muteFrom`'s silence, in seconds: long enough not to click, short enough to keep the next word out. */
+export const MUTE_FADE = 0.04;
+
+/** `render.endFade`'s upper bound, in seconds. */
+export const END_FADE_MAX = 10;
+
+/**
+ * Why one timeline entry's `muteFrom` cannot be built, as sentences. It is in
+ * SOURCE seconds, like `start`/`end`/`cutEnd`: a number within the clip's
+ * extent. Absent (or null) is fine.
+ */
+export function validateMuteFrom(entry, where = `timeline entry ${entry?.id ?? "?"}`) {
+ const v = entry?.muteFrom;
+ if (v === undefined || v === null) return [];
+ if (entry.type !== "clip") return [`${where}.muteFrom: only a clip has sound to mute`];
+ if (typeof v !== "number" || !Number.isFinite(v)) return [`${where}.muteFrom must be a number of source seconds`];
+ if (v < entry.start || v > entry.end) {
+ return [`${where}.muteFrom ${v} is outside the clip's ${entry.start}–${entry.end}`];
+ }
+ return [];
+}
+
+/** Why `render.endFade` cannot be built, as sentences: seconds from 0 (off) to END_FADE_MAX. */
+export function validateEndFade(render) {
+ const v = render?.endFade;
+ if (v === undefined || v === null) return [];
+ if (!numIn(v, 0, END_FADE_MAX)) return [`render.endFade must be from 0 to ${END_FADE_MAX} seconds`];
+ return [];
+}
+
+/** Every `muteFrom` in the timeline and `render.endFade`, checked: the build refuses with these before it fetches. */
+export function validateCutEdits(manifest) {
+ const errors = [];
+ (manifest?.timeline ?? []).forEach((e, i) => errors.push(...validateMuteFrom(e, `timeline[${i}] (${e?.id ?? "?"})`)));
+ errors.push(...validateEndFade(manifest?.render));
+ return errors;
+}
+
+/** The end fade a render block asks for, in seconds (0 = none). */
+export const endFadeOf = (render) => (numIn(render?.endFade, 0, END_FADE_MAX) ? render.endFade : 0);
+
+/**
+ * Where a clip's `muteFrom` falls in its SEGMENT's clock, from the source
+ * second the segment really starts at.
+ *
+ * The build cuts a segment from the snapped start, and records that start
+ * beside it (`<id>.cut.json`: `{ video, start, end }`, source seconds). A
+ * record is believed when it names this clip's video and is as long as the
+ * segment (`seconds`, its probed length) to within two frames; otherwise --
+ * a segment built before records existed, or one copied without its record --
+ * the start is the unsnapped `playWindow` start, and `note` says so: snapping
+ * may have moved the true start by up to `snapWindow` seconds.
+ *
+ * @returns {{ at: number, source: "record" | "window", note?: string }}
+ * `at` in segment seconds, never below 0 (a muteFrom before the segment's
+ * start mutes it from its first sample).
+ */
+export function muteSegmentSeconds({ entry, record = null, render = {}, seconds = null }) {
+ const fps = render.fps ?? 30;
+ const ok = record && record.video === entry.video &&
+ Number.isFinite(record.start) && Number.isFinite(record.end) &&
+ (seconds == null || Math.abs(record.end - record.start - seconds) <= 2 / fps);
+ const at = (from) => Math.max(0, Math.round((entry.muteFrom - from) * 1000) / 1000);
+ if (ok) return { at: at(record.start), source: "record" };
+ const { from } = playWindow(entry, render);
+ return {
+ at: at(from),
+ source: "window",
+ note:
+ `${entry.id}: ${record ? "the cut record does not match the segment" : "no cut record beside the segment"} — ` +
+ `muteFrom measured from the unsnapped start ${from}; the real start may differ by up to ` +
+ `${render.snapWindow ?? 1.6}s. Rebuild the clip to record it.`,
+ };
+}
+
+// ---------------------------------------------------------------------------
// The render cache and the renderer command.
// ---------------------------------------------------------------------------
diff --git a/umtool/report-to-video/verify-build.mjs b/umtool/report-to-video/verify-build.mjs
@@ -153,16 +153,45 @@ export async function verifyDeck(variantDir, render, file, problems) {
export const FREEZE_TOLERANCE = 1.5;
/**
- * Each held segment's freeze is in the file: the frame at `end − hold/2` is
- * the frame at `end − hold + ε`. Compared over the picture outside the deck's
- * panel and the posts column -- both still move during a hold (the deck's
- * progress fuse burns on) -- on luma, within FREEZE_TOLERANCE of re-encoding
- * noise. A cut whose holds were dropped plays on there and differs by far more.
+ * Where a held segment's freeze can be sampled: the still span is from the
+ * later of the hold's start and the end of the footage move (the move runs
+ * after the hold, so a late one glides over the frozen frame) to the start of
+ * the outgoing dissolve (`end − D`), or the end fade on the last segment
+ * (`end − endFade`), or the segment's end. The two samples sit a frame and a
+ * half inside it. A span under three frames has nothing still to compare --
+ * at hold 0.5 under a 0.5 s crossfade the dissolve takes all of it -- and is
+ * skipped, with the reason.
+ *
+ * @returns {{ at: [number, number] } | { skip: string }}
+ */
+export function freezeSamples(segment, { fps, D = 0, last = false, endFade = 0, moveEnd = -Infinity }) {
+ const lo = Math.max(segment.end - segment.hold, moveEnd);
+ const hi = last ? segment.end - (endFade > 0 ? endFade : 0) : segment.end - D;
+ if (!(hi - lo >= 3 / fps)) {
+ return {
+ skip: `${Math.max(0, hi - lo).toFixed(3)}s of still picture between ${lo.toFixed(3)}s and ${hi.toFixed(3)}s ` +
+ `(the rest of the hold is under the ${last ? "end fade" : "dissolve"}${moveEnd > segment.end - segment.hold ? " or the move" : ""})`,
+ };
+ }
+ return { at: [lo + 1.5 / fps, hi - 1.5 / fps] };
+}
+
+/**
+ * Each held segment's freeze is in the file: two frames inside its still span
+ * (`freezeSamples`) are the same frame. Compared over the picture outside the
+ * deck's panel and the posts column -- both still move during a hold (the
+ * deck's progress fuse burns on) -- on luma, within FREEZE_TOLERANCE of
+ * re-encoding noise. A cut whose holds were dropped plays on there and differs
+ * by far more.
*/
export async function verifyHolds(file, schedule, render, problems) {
- const held = (schedule.segments ?? []).filter((s) => s.hold > 0);
+ const segs = schedule.segments ?? [];
+ const held = segs.filter((s) => s.hold > 0);
if (!held.length) return [];
const fps = Number(schedule.fps ?? render.fps);
+ const D = Number(schedule.transition ?? 0);
+ const endFade = Number(render.endFade ?? 0);
+ const moveEnd = new Map((schedule.moves ?? []).map((m) => [m.segment, m.at + m.seconds]));
const g = deckGeometry(render);
const col = postsGeometry(render);
const left = resolveDeck(render).posts.position === "top-left";
@@ -178,8 +207,14 @@ export async function verifyHolds(file, schedule, render, problems) {
};
const out = [];
for (const s of held) {
- const a = s.end - s.hold + 2 / fps;
- const b = s.end - s.hold / 2;
+ const span = freezeSamples(s, {
+ fps, D, last: s === segs.at(-1), endFade, moveEnd: moveEnd.get(s.id) ?? -Infinity,
+ });
+ if (span.skip) {
+ out.push({ segment: s.id, hold: s.hold, skipped: span.skip });
+ continue;
+ }
+ const [a, b] = span.at;
const [x, y] = await Promise.all([luma(a), luma(b)]);
let diff = 0;
if (x.length !== y.length || !x.length) diff = Infinity;
@@ -222,7 +257,9 @@ async function main() {
console.log(` posts on ${w.segment}: ${w.frames}/${w.expectedFrames} frame(s) at ${w.at.toFixed(3)}s`);
}
for (const h of res.deck.holds ?? []) {
- console.log(` hold on ${h.segment}: ${h.hold}s, frozen (${h.at.join("s ≈ ")}s, mean diff ${h.diff})`);
+ console.log(h.skipped
+ ? ` hold on ${h.segment}: ${h.hold}s, not checked — ${h.skipped}`
+ : ` hold on ${h.segment}: ${h.hold}s, frozen (${h.at.join("s ≈ ")}s, mean diff ${h.diff})`);
}
}
for (const p of res.problems) console.log(` ** ${p}`);
