commit 8a32f70a495bd01109f209d814e6e7ddd13c30b9
parent c74737c5e5bba101d7a66a61f522d94dfb6d3f2b
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Sun, 4 Oct 2026 16:27:21 -0400
report-to-video: shoot-page is a bin and an export; README section and changelog bullet
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Diffstat:
4 files changed, 60 insertions(+), 1 deletion(-)
diff --git a/editor/CHANGELOG.md b/editor/CHANGELOG.md
@@ -1,6 +1,7 @@
# Changelog
## [Unreleased]
+- **umtool's report videos can show a highlighted sentence from a saved article.** `node umtool/report-to-video/shoot-page.mjs --page <saved page.html> --quote "<sentence>" --out <shot.png>` opens a web page saved to disk, finds the sentence in its text, highlights it and saves a PNG of the paragraph that holds it, ready to be a report manifest's `image` entry. `--batch <items.json> --out <dir>` does a list of `{ id, page, quote, context? }` at once and writes `<id>.png` for each plus a `results.json` recording each shot's crop, the matched text and the block it shot. The page is opened offline: nothing is fetched except files saved beside it, and its own scripts do not run unless `--js` is given. The sentence is found whether its quotes and apostrophes are curly or straight, across links and emphasis, and through non-breaking spaces, soft hyphens and line breaks in the page's source. A sentence that is not on the page is listed in `results.json` and on the terminal, and the run ends with an error rather than leaving it out. `--color` sets the highlight; `context` picks one occurrence of a sentence that appears more than once.
- **Capture specific X posts: a screenshot of each, and its attached media.** `pnpm ops capture-posts --json '{"slug":"<channel>","ids":["<post id>", …]}'` shoots each post as X shows it, through the connected X profile, and downloads its pictures and videos with gallery-dl, into the channel's `posts-media/<post id>/` beside a `capture.json` that records when, from which URLs, and each file's size and SHA-256. Every id must already be in the channel's posts archive; one that is not is refused by name and nothing runs. `"shots": false` or `"media": false` skips that half, and posts already captured are skipped unless `"force": true`. The job runs on the X queue with a post fetch, so the two never run at once, and waits a random 4–10 seconds before each request to X, as fetches do. A deleted post, or one behind its account's wall (protected, suspended, gone), is recorded as such in the channel's deleted-post record; a post behind a sensitive-media warning is opened and shot. If X asks to log in, or answers "Something went wrong", the job stops at that post and leaves the rest for a later run. Captures are never published: the export does not read them.
- **X posts are fetched more slowly, with random gaps.** Every read of X now waits a random 4 to 10 seconds before each request to X, where it used to page as fast as X answered, and always waits out a rate limit rather than pushing through. When fetching older posts, the pause between one three-month window and the next is a random 45 to 120 seconds instead of a fixed 15. A deep walk of an account's history takes longer; a routine fetch of new posts takes a few seconds more.
- **The MCP's search tools take `date_from` and `date_to` as `2024-10-26` as well as `20241026`, and refuse a date they cannot read.** `search_transcripts` and `enumerate_matches` used to accept only `YYYYMMDD`: any other spelling was dropped with a footer warning and the search ran with no date bound, so a whole-corpus count could be read as the bounded one. Dashed, slashed and dotted dates and ISO timestamps are now normalised, and anything else is an error and nothing is searched.
diff --git a/umtool/report-to-video/README.md b/umtool/report-to-video/README.md
@@ -472,6 +472,62 @@ cut off a top-level `ledger[]` — see [The claim rail](#the-claim-rail-renderra
under `render.chrome`'s deck layout.** The deck replaces all of it with one
persistent panel — see [The on-screen deck](#renderchrome--the-on-screen-deck).
+### Article stills: `shoot-page.mjs`
+
+An article a report cites is a receipt like a post, and `shoot-page.mjs` makes
+its still: it opens a page **saved to disk**, finds the quoted sentence in its
+text, paints it like a highlighter and screenshots the paragraph that holds it,
+at `deviceScaleFactor` 2. The PNG is then an `image` entry's `src`.
+
+```sh
+# one
+node umtool/report-to-video/shoot-page.mjs \
+ --page saved/article.html --quote "the sentence as the report quotes it" --out stills/a01.png
+# many: <dir>/<id>.png per item, and <dir>/results.json
+node umtool/report-to-video/shoot-page.mjs --batch stills.json --out stills/
+```
+
+```jsonc
+// stills.json — `page` is relative to this file
+[{ "id": "a01", "page": "saved/article.html", "quote": "…",
+ "context": "…" }] // optional: a longer stretch around the quote, to pick one occurrence
+```
+
+- **The page is loaded offline.** Every request is refused except a `file://`
+ under the page's own directory — the `<name>_files/` folder that "save page,
+ complete" writes — so a shot never touches the network and never reads a file
+ the page was not saved with. Each result lists what was `blocked`; a page that
+ saved its fonts by URL renders in fallback fonts, and that list says why.
+- **Page scripts are off** (`--js` turns them on). A saved page is already
+ rendered; its scripts, with nothing to talk to, are likelier to hide the
+ article behind an overlay than to finish drawing it.
+- **The quote is matched as text, loosely only where copying is loose:** curly and
+ straight quotes and apostrophes are one character, NBSP and every other space
+ are a space, a run of whitespace is one, an ellipsis is three dots, and soft
+ hyphens and zero-width characters are not there at all. Case and every other
+ character are exact. A match runs across element boundaries — a link, an
+ `<em>`, a `<br>` — and the end of one block and the start of the next read as a
+ space, never as nothing. Hidden (`display: none`) text is not searched.
+- **A quote found more than once** shoots the first and says so on stderr, with
+ `occurrences` in its result. Give `context` to pick another.
+- **The shot is the nearest block that holds the whole match**, plus `--padding`
+ (24 CSS px), with everything outside that block hidden for the shot so the
+ padding shows the page's ground and not the bottom of the paragraph above
+ (`--no-isolate` leaves it). A block taller than `--max-height` (1200) is trimmed
+ to that height around the quote, and says `trimmed`. The highlight
+ (`--color`, `#ffe14d`) never reflows the text: one `<mark>` per text node it
+ covers, padded vertically only.
+- **A miss is never skipped.** It is in `results.json` with its `reason`
+ (`quote not found`, `context not found`, `quote not found inside its context`,
+ `page not found`, or the load error), it is summarised on stderr, and the run
+ exits 1 after shooting every other item.
+
+Each result carries `crop` (the CSS-pixel rectangle, in document coordinates),
+`pixels` (the PNG's size), `matched` (the page's own text of the match, curly
+quotes and all), `block` (a CSS selector for the block shot) and `blocked`. The
+matcher is pure and is what `shoot-page.test.mjs` tests; the one test that
+launches Chromium runs only with `SHOOT_PAGE_BROWSER=1`.
+
### The `teaser` entry type
A season teaser's "coming soon" card: a full-frame graphic, its words the
diff --git a/umtool/report-to-video/package.json b/umtool/report-to-video/package.json
@@ -12,7 +12,8 @@
"report-resolve-windows": "./resolve-windows.mjs",
"report-check-availability": "./check-availability.mjs",
"report-verify-build": "./verify-build.mjs",
- "report-compose-chrome": "./compose-chrome.mjs"
+ "report-compose-chrome": "./compose-chrome.mjs",
+ "report-shoot-page": "./shoot-page.mjs"
},
"exports": {
"./attribution": "./attribution.mjs",
@@ -29,6 +30,7 @@
"./post-links": "./post-links.mjs",
"./render-cards": "./render-cards.mjs",
"./resolve-windows": "./resolve-windows.mjs",
+ "./shoot-page": "./shoot-page.mjs",
"./verify-build": "./verify-build.mjs"
}
}
diff --git a/umtool/report-to-video/shoot-page.mjs b/umtool/report-to-video/shoot-page.mjs
