commit e5a029aa9e96c9160b5eae3451414506fdff288a
parent bafe227ab86d59dc89074d4ec7b2f7022d18c0ad
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Mon, 5 Oct 2026 01:48:49 -0400
records: report-to-video README on the batched crossfade (render.xfadeBatch, REPORT_VIDEO_XFADE_BATCH, xfade-batches/); an [Unreleased] bullet
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Diffstat:
2 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/editor/CHANGELOG.md b/editor/CHANGELOG.md
@@ -1,6 +1,7 @@
# Changelog
## [Unreleased]
+- **A long report video no longer runs out of memory while its clips are crossfaded.** `build-video.mjs` used to join every segment of a cut in one ffmpeg command, which grows with the number of segments: a cut of a few hundred clips could use more memory than the machine had and be stopped. Past 24 segments the build now crossfades them in batches of consecutive segments, each into a file under `out/<variant>/xfade-batches/`, then crossfades those files together with the same transition and lays the on-screen deck, the rail and the dips over them. Every transition, the deck's schedule and the chapters land on the same frames as before, at the cost of one more video encode on such a cut. The batch size is `render.xfadeBatch` in the manifest or `REPORT_VIDEO_XFADE_BATCH` in the environment (which wins); `0` never batches. A batch file is reused while its segments, their holds and moves and the encode settings are unchanged, so a `--chrome-only` run that moves no footage redoes only the last pass.
- **Auto-download no longer tries a video the metadata scan already found members-only or private.** The scan records why it could not read a video, but only a failed download used to take a video out of the auto-download queue, so each members-only video the scan had found was still downloaded once: four yt-dlp requests, two with browser cookies, about 30 seconds each. A members-only or private answer from the scan now keeps the video out of the queue and counts it under the channel's members-only or private exclusions, and it stays in **Needs cookies** for a manual cookie run. A scan that reads the video later lifts this. A video the scan saw only as "Video unavailable" is still tried, since YouTube gives that answer when it is throttling too.
- **X post fetches stop on Drain, and an account with no posts is not searched.** Draining a `fetch-posts` job used to do nothing until gallery-dl finished its whole run. Now the timeline fetch stops at the next page boundary (at once when gallery-dl is between pages or waiting out a rate limit), the older-posts walk stops its current window's search at once and never starts the 45–120 second pause between windows, and both keep their resume point: the job ends done, not failed, and the log says "Drained; the next run resumes …". An older-posts walk is refused when nothing is archived and the last timeline fetch finished having read no posts, since it would only repeat empty searches; `"force": true` (`--force` on `archilyzer posts fetch`) walks anyway. A walk with nothing archived that finds nothing ends after two empty three-month windows instead of four, and records why; a walk that has posts keeps the year-of-empty-windows rule. Capture-posts already stopped between posts on Drain.
- **Capturing an X post that is an Article also saves the article.** An X Article (a long-form post) is archived as nothing but its link, and gallery-dl cannot read its body. When `pnpm ops capture-posts` meets a post whose archived text, or whose card on the page, links to an article, it now opens the article in the same X profile, after the same 4–10 second pause, and saves beside the post's capture: `article.json` (the title, author, date and every heading, paragraph, quote, list item, image, link and embedded post in reading order, an embedded post by its URL), `article.md` (the same as readable text), `article.png` (the whole article as shown, cut off at 16,000 pixels tall and marked `trimmed` when longer), `article.html` (the article as X served it, so it can be read again without going back to X) and the article's pictures as `article-img-1.jpg`, `article-img-2.png`, … at full size, fetched through the same browser session. `capture.json` records the article's state, title, block count and every file's size and SHA-256. `"articles": false` leaves articles alone; `"shots": false, "media": false, "articles": true` reads only the articles. An article already captured, deleted or unavailable is not opened again unless `"force": true`; one that failed is tried on the next run. If X asks to log in on the article page, the job stops there as it does for a post. The post viewer's capture panel in the editor shows the article's title with a link to `article.md`.
diff --git a/umtool/report-to-video/README.md b/umtool/report-to-video/README.md
@@ -645,7 +645,7 @@ strip whose marker vanishes for six seconds reads as a bug.
Clips also carry `section` and (auto-set) `sectionEnter`. Card styles — `title`,
`timeline`, `status`, `bullets`, `sources` — still work, but the ferret-rescue cut
uses none of them. `render` holds resolution, fps, fonts, palette and the knobs
-(`fetchPad`, `snapWindow`, `silenceRelDb`, `transition`, `slideSeconds`,
+(`fetchPad`, `snapWindow`, `silenceRelDb`, `transition`, `xfadeBatch`, `slideSeconds`,
`headerHeight`, `footerHeight`, and the optional `rail`); `provenance` holds the
sweep's scope and counts. Two further entry `type`s, `scroll` and `chart`, close a
cut off a top-level `ledger[]` — see [The claim rail](#the-claim-rail-renderrail).
@@ -1001,6 +1001,23 @@ parsing — wrapping the expression in single quotes is what protects them.
re-encode of the timeline via `xfade`/`acrossfade` — the concat demuxer can only
stream-copy hard cuts. Pass `--no-xfade` for a fast hard-cut build while
iterating; the last pass can add the transitions back.
+- **A long cut is crossfaded in batches.** One ffmpeg that opens every segment at
+ once grows with the segment count: a 223-segment 1080p cut reached 8 GB and was
+ killed, where 46 segments were fine. Past 24 segments (`render.xfadeBatch`, or
+ `REPORT_VIDEO_XFADE_BATCH` in the environment, which wins: memory is the
+ machine's; `0` never batches) the build crossfades each run of consecutive
+ segments, at most that many, into `out/<variant>/xfade-batches/batch-<key>.mov`
+ (the cut's own video encode, PCM sound), then crossfades the batch files
+ together with the same transition, laying the deck, the rail and the dips over
+ them in that last pass. A batch starts where its first segment does in one
+ pass, so every dissolve, the schedule, the chapters and the dips land on the
+ same frames; the cost is one more video encode generation on a batched cut. A
+ batch's key is its whole ffmpeg command and each input's size and modified
+ time (and its segments' holds and moves are in the command), so `--chrome-only`,
+ `--rail-only` and a rebuild that changed a few segments redo only the batches
+ those changes touch, then the last pass, and files of a stale key are
+ removed when the next batched build starts. Past 24² segments the batches grow
+ instead, so the last pass stays 24 inputs wide.
## Two cuts from one manifest (`--variant`)
@@ -1039,10 +1056,13 @@ out/
availability.json SHARED — a fact about the manifest, not about a cut
<slug>.mp4 sourced
<slug>-full.mp4 full
- sourced/{cards,segments,qr,chrome,schedule.json}
- full/{cards,segments,qr,chrome,schedule.json}
+ sourced/{cards,segments,qr,chrome,xfade-batches,schedule.json}
+ full/{cards,segments,qr,chrome,xfade-batches,schedule.json}
```
+`xfade-batches/` exists only for a cut long enough to be crossfaded in batches
+(see "Segments crossfade" above).
+
`clips-raw` is shared deliberately: `sourced`'s clips are a subset of `full`'s, so
no clip is ever fetched twice. `sourced` writes `out/<slug>.mp4` because that is
the path umtool's build probe already looks for.
