commit 2863c1b9a83e5b7d0c840d4b3f559fdd89edd566
parent 6b8f38373580e248895f9d305eb4d5f07465494d
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Mon, 28 Sep 2026 02:02:18 -0400
plans: slice O3 as shipped — runner lows; FACTS; [Unreleased] in editor and export
The record under release-11.md's Record: the four items, the commit table,
gates (common 2,047, editor unit 85, scripts 174+1, mcp 269, editor build
55 s; e2e 61/61, cookies-mode x3 18/18, export availability-state 9/9), the
bite checks (e2e with the six source files at b568edb8: exactly the four
new/changed tests fail), the read-only count (5 videos across 2 Rumble
channels move to maybe-missing: jeralyzer 4, rekietalyzer 1), and what was
left. The /sites form's revalidate on `written` waits for O4's merge.
FACTS: the full-fetch entry, the run-log hazard's channel-page instances,
and an O3 section. Changelogs: three editor bullets, one export bullet
(reader-facing: a video whose recheck failed shows as possibly missing).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Diffstat:
4 files changed, 212 insertions(+), 0 deletions(-)
diff --git a/editor/CHANGELOG.md b/editor/CHANGELOG.md
@@ -1,5 +1,10 @@
# Changelog
+## [Unreleased]
+- **"Persist source video" or a whole-recording fetch that cannot get the source no longer marks the video's download failed.** When YouTube's subtitles came down but the source video did not, the video page said "Download failed" over a transcript that is fine. The download now keeps the subtitle pass's result and records only the failed media attempt, with yt-dlp's reason. The run itself now ends failed with that reason; it used to end done with no file, so `fetch_clip` could only say the job "finished but named no file". A partial source file is left for a retry to resume, and the run's log names it.
+- **A bucket's retry keeps its log when it empties the bucket.** On a channel's Download stage, "Download with cookies", the partial-download resume and the missing-transcript retry could lose their run log part-way: the video they fetched left the bucket, the page refreshed, and the card disappeared with the log in it. The card now stays, with its log and its button disabled, until the page is reloaded. Retry buttons elsewhere behave the same way.
+- **A release cut whose commit fails still refreshes the pages.** When the changelog's new heading was written but the commit after it failed, `pnpm ops cut-release` answered as if nothing had happened and no page showed the new heading until a reload. It now says the file was written and refreshes the changelog pages. Every refused cut's answer says whether anything was written (`untouched`), and a cut of both changelogs that stopped half-way names the one already cut as well as the failure.
+
## [0.9.4] - 2026-09-28
- **On the Dark ground the sidebar's Archilyzer mark has a thin outline.** Its slate tile now has a 1-pixel ring just outside it, following its rounded corners, in the colour of the mark's unlit lines, so the tile's edge shows against the dark page. Light and Sepia are unchanged, and so is the favicon.
diff --git a/export/CHANGELOG.md b/export/CHANGELOG.md
@@ -1,5 +1,8 @@
# Changelog
+## [Unreleased]
+- **A video whose recheck failed shows as possibly missing rather than available.** When a video drops out of its channel's listing it is marked "Missing?" until a recheck says why. A recheck that could not reach the video — a blocked request or a network error — used to clear the mark as if the video had been found. It now leaves "Missing?" in place until a recheck actually reaches the video. Needs a rebuild and deploy of every export site.
+
## [0.9.4] - 2026-09-28
- **On the Dark ground the mark's tile has a thin outline.** The header mark's ink tile is the colour of the Dark page, so only its lines showed. On Dark the tile now has a 1-pixel ring just outside it, following its rounded corners, in the colour of the mark's unlit lines. The small Archilyzer mark by the footer credit and the hub's mark get the same ring in their own slate's unlit colour. Light and Sepia are unchanged, and so are the icons. Needs a rebuild and deploy of every export site and the hub.
diff --git a/plans/FACTS.md b/plans/FACTS.md
@@ -3696,6 +3696,17 @@ box that outlives the warning stops answering `getByLabel("incomplete transcript
(`cards/PerFileTranscodeRow.tsx:39` since 3b) is NOT an instance: its `return null` depends on the file's extension against a static format
list, which a transcode cannot change, and the rows are keyed by file name.
+**On the channel page, the bucket cards** (release 11 slice O3, `86449b88`): `RetryBucketControl`
+and the Download stage's three bucket cards (`NoTranscriptList`, `PartialDownloadsList`,
+`NeedsCookiesList`) returned null on an empty bucket, and a retry EMPTIES its own bucket. What hands
+the card its emptied bucket is not only the end-of-run refresh: `recordTaskDone` →
+`requestSnapshotOnFinish` (`common/jobs/streamCommand.ts`) regenerates the snapshot after EACH video
+a batch finishes, the pulse moves, and `AutoRefresh` refreshes the page MID-RUN. Now the shape
+above: `ranHere` set in the trigger (and `onRun` to tell the card), null only while
+`ids.length === 0 && !ranHere`, `disabled={ids.length === 0}` (`RetryBucketControl.tsx:60-84`,
+`DownloadStage.tsx:357,404,454`). `cookies-mode.spec.ts:241` waits for the refreshed EMPTY card
+(`needs cookies empty`), which makes the old race certain rather than likely.
+
The persist case needed a fixture: `editor/e2e/fixtures/bin/fake-ytdlp.mjs:680-725` (at
`12d1778`) grows an app-extraction branch — the LAST branch checked, matched on the media
output template naming `source-media` (only `plan.extractionMode === "app"` emits that), which
@@ -6780,8 +6791,44 @@ S4, as shipped"; `release-10.md` "Slice L2 / L1, as shipped". Every anchor below
`transcript.json` is untouched. It rewrites `metadata.info.json` (every managed download's
prefetch does); since slice N each rewrite that changes the bytes appends to
`metadata.history.json` (`common/lib/metadataHistory.ts`). The window path never writes metadata.
+ **A forced pass that FAILS over a transcript** (release 11 slice O3) keeps the download's status
+ (the subtitle pass's `ok`) and records only the failed n: 3 attempt; `archiveSourceVideo` reads
+ it through `sourceFetchFailure` and throws, so the job ends `failed` with yt-dlp's line in the
+ poll's `error` tail — it used to end `done` with no file ("finished but named no file").
Neither path dedupes a running job: a repeated request while one runs queues a second fetch, which
is why every text says "resume with job".
- The editor must already HAVE the channel (404 `Channel "<slug>" not found` otherwise); a
public-only setup gets the "no editor configured" error naming both env vars, and the README's
`yt-dlp --download-sections` is the no-editor fallback only.
+
+### O3 — runner lows (release 11, 2026-09-28)
+
+- **An `error` probe never confirms a maybe-missing video** (`resolveMaybeMissingState`,
+ `common/lib/availability-server.ts:156-170`, `:164`). `stateFromAvailability` folds `error` (a
+ 403, a network failure, the bot check, a blocked run's one rate-limited probe) into `available`
+ because it is no evidence the video is gone; it is no evidence it is there either, so a probe
+ after the scan that came back `error` now leaves it `maybe_missing`. `needs_auth` (an age gate)
+ after the scan still confirms. The ONE caller is `buildIndex.ts:1180` (the published `state` on a
+ site's summaries pages, and the `videoState` sub-DB behind the status chart); the clean gate reads
+ `resolveEffectiveAvailability`, which the fix does not touch. `maybeMissingBuild.test.ts` pins it
+ through the real `buildIndex`. The shared transcript pages carry no `state`; the summaries (and
+ subs) pages do.
+- **The five real videos it moved (read-only count, 2026-09-28)** were all Rumble probes that got
+ `HTTP Error 410: Gone` on 2026-09-25 and were stored as `error`, just before `6d5cdbc3` taught
+ `parseUnavailableFromStderr` that 410 is `deleted`: `rekietalaw-rumble/v7e07us` and
+ `the-quartering-rumble/v4vriou v4x5o1l v4yqask v501kfc`. A "Full-check unexpected" on those two
+ channels would settle them `deleted`.
+- **`sourceFetchFailure(record)`** (`common/ytdlp/downloadOneManaged.ts:528`) is how a caller that
+ asked for the SOURCE tells whether it got it: the status for an outright failure (`failed`,
+ `failed-corrupt-source`), else a failed last attempt of kind `no-subs-fallback`. A failed
+ `keepTranscript` pass (`:1503`) leaves the status alone and logs `forceMedia: the source download
+ failed (yt-dlp exit N); the transcript on disk is untouched and the download stays <status>.`,
+ naming any `source-media.*.part` it leaves (left on purpose: a retry resumes it). `persistKept`
+ does not read it yet (it still counts a returned download as persisted).
+- **`cutReleases` always sets `untouched`** (`common/controller/cutRelease.ts`, `nothingWritten`
+ `:311`): true = no changelog was written (every refusal); false once any was, including a result
+ with **`written: true`** — a cut whose file was written and whose commit then failed (`:253`).
+ The type keeps it optional only for hand-built outcomes (`common/bin/_cli.test.ts`).
+ `describeCutFailure(outcome, target)` (`:415`) is the ops route's 400 `error`: the failure,
+ `<workspace>: `-prefixed for `all`, then `Before it, editor was already cut (## [x] - date,
+ committed <sha8>).` The route revalidates whenever `untouched !== true`.
diff --git a/plans/release-11.md b/plans/release-11.md
@@ -36,6 +36,163 @@ merges `main` once O1–O5 have landed, then does its `E2E_` rename. Then one in
## Record
+### Slice O3, as shipped — runner lows (2026-09-28)
+
+Branch `r11/runner-lows` off `main` `2162db92`, worktree `/home/user/Projects/r11-runner-lows`, one
+Opus implementer, beside O4 and O6. Four lows left by release 10: one from L2 ("New low"), one from
+slice N's review (L2) and its knock-ons, the `cookies-mode.spec:241` intermittent N's full suite
+found, and slice P's review lows L3 + L4. No settings, site or channel key; nothing on disk moves.
+`CutReleaseResult.written` and an always-set `CutReleasesOutcome.untouched` are additive.
+
+**1 — a failed confirm probe no longer clears "Missing?"** (`common/lib/availability-server.ts`).
+- `resolveMaybeMissingState` decides the published state of a video that fell out of its
+ channel's listing. A probe made after the scan that came back `public` clears the flag; so did
+ one that came back `error`, through `stateFromAvailability`'s `error` → `available` fold. An
+ `error` probe (a 403, a network failure, the bot check, a blocked run's one rate-limited probe)
+ never saw the video, so it is no evidence it is there. It now leaves the video `maybe_missing`,
+ like one never probed. `needs_auth` after the scan still confirms: an age gate is the video
+ answering.
+- **Display only, and the clean gate is untouched — proof.** `grep -rn resolveMaybeMissingState`
+ over the whole repo (not `node_modules` or `.next`) finds one caller, `buildIndex.ts:1180` (the maybe-missing overlay →
+ `stateByIndexKey` → a site's summaries pages and the `videoState` sub-DB the status chart
+ reads), plus `videoState.test.ts`. The diff to `availability-server.ts` is the one line and its
+ comment; `resolveEffectiveAvailability`, which `verifyBeforeClean` and the MtimeRecord read, is
+ not in it.
+- **The export check** is `common/controller/maybeMissingBuild.test.ts` (new), which runs the REAL
+ `buildIndex` over a temp corpus: one video per outcome of the overlay, and the `state` each gets
+ on the site's `summaries/page-0000.json` — the field the export's badge and Availability filter
+ read. `export/e2e/availability-state.spec.ts` was NOT extended: it fulfils the summaries with
+ mocked `state`s, so a new case there could not see the fix; it was re-run (9/9) as the contract
+ check that `state: "maybe_missing"` still draws "Missing?". The shared transcript pages carry no
+ `state` (found while writing the test).
+- **The real count, read-only** (`$T/o3-flip-count.ts`, `o3-flip-count.log`: reads
+ `maybe-missing.json`, `availability.json`, `download-outcome.json`, `metadata.info.json`'s
+ existence and `sites/*/site.json` through the build's own loaders; opens no LMDB, writes nothing):
+ 1,827 maybe-missing ids in 31 channels; **5 videos across 2 channels move from available to
+ maybe-missing at the next index build** — `rekietalaw-rumble/v7e07us` and
+ `the-quartering-rumble/v4vriou v4x5o1l v4yqask v501kfc`. Per site: **jeralyzer 4** ("Missing?"
+ 212 → 216), **rekietalyzer 1** (0 → 1), anilyzer, bonnellyzer, hasanalyzer and jasolyzer 0. All
+ five are Rumble probes that got `HTTP Error 410: Gone` on 2026-09-25 and were stored as `error`,
+ hours before `6d5cdbc3` taught the classifier that 410 is `deleted`; they are really deleted, and
+ "Missing?" is closer than "available". A "Full-check unexpected" on those two channels would
+ settle them as Deleted (the operator's call; nothing was run).
+
+**2 — a failed forced download keeps the subtitle pass's status** (`common/ytdlp/downloadOneManaged.ts`).
+- The attempt-3 `else` set `status = "failed"` and `lastSucceeded = false` for a `keepTranscript`
+ pass too, so a "Persist source video", `fetch_clip` `full: true` or Persist kept now whose MEDIA
+ failed put "Download failed" on a video whose transcript is fine. A `keepTranscript` failure now
+ keeps the status (the subtitle pass's `ok` / `ok-with-cookies`) and `lastSucceeded`, so the
+ archive line the subtitle pass printed is appended as on the success path; the failed n: 3
+ attempt is recorded with its error (no `failureClass`: the download did not fail). One log line:
+ `forceMedia: the source download failed (yt-dlp exit N); the transcript on disk is untouched and
+ the download stays ok. Left for a retry to resume: source-media.f137.mp4.part (<size>).`
+- **Knock-on 1, fixed: the job reports it.** `sourceFetchFailure(record)` (exported) says whether a
+ download asked for the source got it: the status for an outright failure, the last attempt for a
+ failed forced pass. `archiveSourceVideo` (`videoActions.ts`, both "Persist source video" and
+ `fetchFullSourceAction`) throws `The source video was not downloaded: <yt-dlp's line>` after its
+ revalidate, so the job ends `failed` and the poll's `error` tail carries the line; the MCP renders
+ "Editor job … failed" with it instead of "finished but named no file". A download that failed
+ outright ends the job `failed` too (before: `done`).
+- **Knock-on 2, left on purpose: the `.part`.** yt-dlp resumes a partial container on a retry, and
+ deleting one that is several GB makes a retry after a network blip start over — more requests
+ against the source, which the pacing rules want fewer of. Nothing else surfaces a
+ `source-media.*.part` (the Partial downloads bucket matches `audio.<ext>.part` only), so the log
+ line names it and its size instead.
+
+**3 — the `cookies-mode.spec:241` intermittent** (`DownloadStage.tsx`, `RetryBucketControl.tsx`).
+- The mechanism is wider than the end-of-run refresh: `recordTaskDone` → `requestSnapshotOnFinish`
+ regenerates the snapshot after EACH video a batch finishes, the pulse moves and `AutoRefresh`
+ refreshes the page while the run still streams. The refreshed `NeedsCookiesList` got an empty
+ bucket and returned null, and so did the `RetryBucketControl` in it — two unmounts, either
+ enough to lose the log.
+- The FACTS shape, in both: `ranHere` set inside the trigger (`RetryBucketControl` also calls a new
+ optional `onRun` so its card knows), null only while `ids.length === 0 && !ranHere`, and
+ `disabled={ids.length === 0}` so the surviving panel is not a second Run button. Applied to all
+ three bucket cards on the Download stage (`NoTranscriptList`, `PartialDownloadsList`,
+ `NeedsCookiesList`, same hazard) and to `RetryBucketControl` itself, so the Diagnostics buckets'
+ retry controls keep their log too. A reload drops an empty card as before (the reload check at
+ the end of the same test still passes); labels and test ids unchanged.
+- **Made deterministic.** The spec now waits for the refreshed, EMPTY card (`needs cookies empty`,
+ which renders only inside a card that survived the refresh) and asserts the log's last line and a
+ disabled `Download with cookies (0)` in it. On the old code this fails every time, at that line
+ (below).
+
+**4 — cut-release review lows L3 + L4** (`common/controller/cutRelease.ts`, `api/ops/cut-release`).
+- **L3:** `applyCut`'s commit failure returns `{ok: false, workspace, error, written: true}` — the
+ file on disk has its new heading. The route revalidates whenever anything was written.
+- **L4:** `cutReleases` sets `untouched` on EVERY outcome (true = no changelog written, every refusal
+ included; false once one was, a `written` failure included); the type keeps it optional only so
+ the CLI's hand-built display tests compile, and absent reads as false. `describeCutFailure`
+ builds the route's 400 `error`: the failure (prefixed `<workspace>: ` for `all`), then `Before
+ it, editor was already cut (## [0.9.5] - 2026-09-28, committed 1a2b3c4d).` Every 400 from the
+ writer carries `untouched`. An `all` whose FIRST write fails wrote nothing and is `untouched: true`, so the CLI
+ prints "not cut — all cuts both or neither, and nothing was written" for the export — true.
+ The input refusals before the writer (a bad workspace, version, date or key) are the ops door's
+ 400s and carry no `untouched`, as before.
+- The CLI (`common/bin/release.ts`) is unchanged: its per-line output already named the earlier cut
+ and says "failed — Cut release X, but the commit failed" for L3.
+- **The `/sites` form** (`editor/app/sites/lib/cutReleaseAction.ts`) does not yet revalidate on
+ `written: true`: it is under `editor/app/sites/**`, which slice O4 owns tonight, so that one-line
+ change waits for O4's merge (WAITING_FOR_O4).
+
+| sha | what |
+|---|---|
+| `877403db` | 1: `resolveMaybeMissingState` — `error` after the scan stays `maybe_missing`; `videoState.test.ts` +2, `maybeMissingBuild.test.ts` (new, 1) |
+| `8c72265c` | 2: the `keepTranscript` failure keeps the status; `sourceFetchFailure`; `archiveSourceVideo` fails the job; `forceMedia.test.ts` +2; the fake's `.fake-ytdlp-media-fail` knob; `fetch-window.spec.ts` +1, `persist-youtube-handling.spec.ts` +1 |
+| `86449b88` | 3: `ranHere` / `onRun` / disabled in `RetryBucketControl` and the three Download-stage bucket cards; `cookies-mode.spec.ts:241` made deterministic |
+| `edec55ca` | 4: `written`, always-set `untouched`, `describeCutFailure`, the route; `cutRelease.test.ts` +4 (1 changed); `ops-cut-release.spec.ts` pins `untouched` |
+| _this_ | `plans:` this record, FACTS (three amendments, one new section), the `[Unreleased]` bullets |
+
+**Gates**, from the worktree root; logs `o3-*.log` in `$T`.
+- **tsc** (`pnpm -r --no-bail --workspace-concurrency=1 exec tsc --noEmit`) clean before each code
+ commit: `o3-tsc1` (item 1), `o3-tsc3` (items 2 + 3, item 4 stashed), `o3-tsc4` (item 4).
+- **common 2,047/2,047** (release 10's 2,038 + 9: videoState +2, maybeMissingBuild +1, forceMedia
+ +2, cutRelease +4); **editor unit 85/85**; **`test:scripts` 174 + 1 skip of 175**; **mcp
+ 269/269** (`o3-units1.log`, on `edec55ca`).
+- **Editor build** `pnpm --filter editor exec next build` ok, 55 s (`o3-build1.log`; the route table
+ lists `ƒ /api/ops/cut-release`). **Export build not run**: nothing under `export/` changed but
+ its CHANGELOG.
+- **e2e** (queued, detached, `export/public` linked per path from the primary with `sw.js` a plain
+ copy, no dangling links; the lock was free each time):
+
+ | run | specs | passed | failed | time |
+ |---|---|---|---|---|
+ | A | `cookies-mode maybe-missing persist-youtube-handling fetch-window saved-videos cut-release ops-cut-release retry-bucket partial-downloads-bucket auto-subs-replace bulk-actions no-subs-fallback storage-locations` (`o3-e2e-specs.txt`: the prompt's seven, the four that drive a retry-bucket control, and the two that drive the source-media branch) | **61** | **0** | 3.9 min |
+ | B | `cookies-mode.spec.ts --repeat-each=3` | **18** | **0** | 1.8 min |
+ | C | EXPORT `availability-state.spec.ts` | **9** | **0** | 30 s |
+
+ `archives-off.spec.ts` draws no state badge, so it was not added. The primary's
+ `export/public/sw.js` was not written (md5 `355d0d21…`, mtime 00:51:32, before and after).
+- **Numbers tool:** `o3-flip-count.ts` (item 1), above.
+
+**They bite.**
+- Unit, item 1 (`o3-bite1.log`): with the `error` line reverted, `maybeMissingBuild.test.ts` fails
+ and `videoState.test.ts`'s "error, probed AFTER the scan" fails; 15 others pass.
+- Unit, items 2 + 4 (`o3-bite2.log`): with the `keepTranscript` branch removed, forceMedia's new
+ behaviour test fails (the `sourceFetchFailure` table test is of a new function); with `written`
+ and the always-set `untouched` reverted, 5 of 20 cutRelease tests fail (the four new and the
+ changed half-way test).
+- **e2e** (`o3-e2e-bite.log`): the six source files at their `2162db92` versions, the new specs and
+ fake kept; `fetch-window persist-youtube-handling cookies-mode ops-cut-release`: **16 passed, 4
+ failed, 2.5 min**, and the four are exactly the new and changed tests —
+ `cookies-mode.spec.ts:241` at `getByLabel("needs cookies empty")` (the card is gone),
+ `fetch-window.spec.ts:465` (`done`, not `failed`), `ops-cut-release.spec.ts:73`
+ (`untouched` undefined), `persist-youtube-handling.spec.ts:182` (no failure line; the old run
+ marked the download failed). Restored with `git checkout HEAD --`.
+
+**Found and left.**
+- **`persistKept` still counts a returned download as persisted** (pre-existing, N's record). It
+ could read `sourceFetchFailure`; not this slice's file.
+- **L3/L4 have no HTTP e2e for the half-way cases.** The fixture server's changelogs are gitignored
+ copies, so a commit there fails only after the dirty-tree guard has run git in the WORKTREE, and a
+ failed write needs a read-only directory the server's path cannot be pointed at. The controller's
+ tests drive both in temp repos (a refusing pre-commit hook; a `0555` directory); the spec pins
+ `untouched: true` on a refusal.
+- **A used retry control on the Diagnostics buckets** now stays (disabled, with its log) after its
+ run empties the bucket, until a reload. Intended, and the same rule; noted because it is visible.
+- **The five 410 videos** are published as available today and will read "Missing?" after the next
+ build; a Full-check unexpected would make them Deleted.
+
## Rollout
Nothing is rolled out tonight. The morning runbook lists what is owed: the :3001 editor restart,
