Archilyzer · Source

archilyzer

Archilyzer
git clone https://archilyzer.pages.dev/source/archilyzer.git
Log | Files | Refs | README | LICENSE

commit 561f53b905699f77443c0bfd0af7778e05393959
parent 1485e6cf1cc6f43793aa0f36d8df7cf8592ab455
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date:   Fri, 25 Sep 2026 15:29:20 -0400

plans: release 8 rollout record — 660957f7 live on :3001 (video titles + one shared auto-queue state); STATE + FACTS

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Diffstat:
Mplans/FACTS.md | 87+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Mplans/STATE.md | 66++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------
Mplans/release-8.md | 223+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
3 files changed, 367 insertions(+), 9 deletions(-)

diff --git a/plans/FACTS.md b/plans/FACTS.md @@ -6323,3 +6323,90 @@ Line numbers are `plans/FACTS.md` lines at `e172749b`, before this record's in-p stale generated file left by an old `next dev`, naming routes that moved. 0 source errors; worktrees without that file are clean. Clear it with `rm -rf editor/.next/dev` when no dev server runs. + +## Release 8 — slices V, S (verified 2026-09-25, `main` @ `0e72ef73`) + +- **A channel's video titles come from `common/controller/videoTitles.ts`.** + `readChannelVideoTitles(paths, slug, ids)` returns `Map<id, {title, source}>`. It tries three + sources in order, and the first hit wins: + 1. **`index`**: the LMDB `sums` sub-DB, reached by a key-only walk of the channel's `byChannel` + range. It is opened `readOnly` with `compression: true`, and an index title equal to the id is + skipped as `summarize()`'s fallback. + 2. **`scan`**: `channels/<slug>/metadata-scan.json`, through `loadMetadataScan`. + 3. **`metadata`**: `data/<id>/metadata.info.json`. It reads a 16 KB head plus a regex, and does + the full parse only when the head has no title. + + **Source 3 is memoized per channel** in `globalThis.__yttVideoTitleMemo__`, a + `Map<dataDir, {dataDirMtimeMs, titles}>`, **keyed by the `data/` dir's mtime**: one `stat` per + render, and a changed mtime (a dir added or removed) drops the channel's entry. + `MEMO_MAX_CHANNELS = 64`, evicted least recently used. **Untitled ids are not memoized**, so a + dir still downloading picks up its title later. A `metadata.info.json` rewritten inside an + existing dir does not refresh the memo, which is accepted. `resetVideoTitleMemo()` is called by + `editor/app/api/test/invalidate-cache/route.ts`. Nothing here creates `data/<id>/`. + + Why the memo: every Rumble dir name misses the index, whose key is the embed id while the dir is + the URL slug. `the-quartering-rumble` (8,049 dirs) cost 5,024 ms cold / 224 ms warm on every + render before the memo, and 5,331 ms first then ~6 ms after. +- **The list and search contract** (`editor/app/channels/[slug]/videos/page.tsx`, + `components/VideoListPane.tsx`, `lib/videoRows.ts`). `VideoRow.title?` is filled once per + render over data-dir ids ∪ `undownloadedIds`. A titled row shows the title, with the id on a + muted mono line under it. An untitled row shows the id alone. **The row labels are unchanged: + `open <id>`, the checkbox `select <id>`, and the search box's label `search videos`.** Its + placeholder is "Search title or id…". `matchesVideoQuery(row, q)` (`lib/videoRows.ts`: id OR + title, case-insensitive) is the ONE predicate, shared by the client filter and the server's + `?q=` prev/next ordering. Rows stay sorted by id. +- **The video page falls back to the scan.** `editor/app/channels/[slug]/videos/[id]/page.tsx` + reads `readVideoMetadataForDisplay(paths, slug, id)`, which returns `{title?, description?, + webpageUrl?, uploader?, uploadDate?, duration?, source: "metadata"|"scan"|"none"}` with + `metadata.info.json` first. When `source` is `scan`, the header shows the scan's title, date and + duration, plus an italic note, "from the listing scan — not downloaded" + (`aria-label="metadata source"`). A collapsed `<details>` **Description** appears when either + source has one. `generateMetadata` falls back title → id. +- **ONE `AutoQueueState` object per process, for real.** The holder is + `common/jobs/autoQueueState.ts` on `globalThis.__yttAutoQueueState__`. + `sharedAutoQueueState(paths)` caches the **in-flight read, per state file**, so the four lanes + booting concurrently all await one `readAutoQueueState` and share the object. A failed read is + not cached. `common/controller/autoRunner.ts` re-exports it, and `jobs/` still imports nothing + from `controller/`. `liveAutoQueueState(paths)` returns the live object, or the in-flight one, + without starting a read. Before this, each lane held a private copy, and `writeAutoQueueState` + (which writes all four lanes) let any lane's persist revert another's. That is the live + 2026-09-25 13:33:37 revert of a 6 h deferral and `fails: 13`, which recurred at ~14:18. +- **`recordDownloadBackoff` (`common/jobs/downloadBackoff.ts`) writes through the live object.** + This is the 429 cooldown recorded by a manual Sync, a metadata scan or the video page. When a + runner is live, it merges the disk entry in (the later `until` wins, and the entry is copied + whole: a known race can lower `fails`), escalates on the live object and writes through it. + With no runner live, it is a disk round trip. `platformCooldownRemainingMs` and the status + builder's `computeLeafPending` prefer the live object. +- **The e2e reset clears two more globals.** `api/test/invalidate-cache` now sets + `__yttAutoQueueState__ = undefined` beside `__yttAutoRunner__`, and calls + `resetVideoTitleMemo()` (`__yttVideoTitleMemo__`). A new process memo or holder must be added + there, or it leaks across specs. +- **A QUEUED job that calls `revalidatePath` ends `failed` after doing its work** (the invariant). + The editor's job bodies and completion hooks call `revalidatePath`: + `editor/app/channels/[slug]/pipelineActions.ts:220-224` in `runPipelineAction` (sync, + download-missing, download-missing-subs, retry-bucket), the metadata scan's `afterRun` + (`:364-367`), and the `onDone` callbacks passed to `runOperationChannelJob` (the hook exists + because common cannot import `next/cache`, `common/controller/operationJobs.ts:23`). A job that + starts inside the submitting request (an empty platform queue) has a request store. A job that + was queued is started later by the queue's drain loop, which has none, and Next throws `[error] + Invariant: static generation store missing in revalidatePath /channels/<slug>`. The work is + done and the files are written, but the status is `failed`, `pnpm ops … --wait` exits 1, and + nothing is revalidated. First seen live: sync `01M3CY7BN3DNN12T93X9GZM48X` (teamrcn), + 2026-09-25 15:21. Fix owed: a store-tolerant revalidate. +- **A cancelled metadata scan keeps what it scanned.** `channels/<slug>/metadata-scan.json` is + written incrementally, and a later scan resumes the remainder (paramount-tactical: cancelled at + 377/1,382, and the store holds 377). +- **The download filter settles scanned non-matches with no request.** `runYtdlp` + (`common/ytdlp/runYtdlp.ts`, `settledByTitleFilterIds`) reads the scan store once per batch and + drops every id the channel's `downloadFilter` already rejects, before the per-video metadata + prefetch. Its log line is `Settled by this channel's download filter: <n> skipped (already + decided).` Live: 376 of 377 scanned on paramount-tactical with `include: "Quartering"`. Unscanned + ids still pay a prefetch each. So **scan first, then download-missing**, on a filtered channel. +- **The `/jobs` stall label is age-without-tasks, not age-without-progress.** + `common/views/jobRows.ts` marks a live record `possibly-stalled` when it has no `tasks` and has + run past `STUCK_AGE_MS` (10 min). A metadata scan registers no tasks, so a healthy scan at ~10 + videos/min (`--sleep-requests 1` plus per-video requests) reads `STUCK · POSSIBLY-STALLED` after + 10 minutes. It is soft: surfaced, never auto-healed. +- **Restart leftovers.** Jobs queued when the editor stops keep `status: "queued"` in their meta + forever (no re-queue, no mark). A just-drained job also reads `queued` (`startedAt: null`) for a + while after its log starts. Both are cosmetic. diff --git a/plans/STATE.md b/plans/STATE.md @@ -3,12 +3,66 @@ The working memory for the local-AI derived-corpus work. Rewritten at the end of every session, before context is cleared. See [`README.md`](README.md) for the protocol. -**Live on :3001 (2026-09-25, 13:11):** `bb3dbb4c` (releases 6 + 7 together), `BUILD_ID` -`6kMVr9Gn2093S9tp_pgqT`; umtool on :3050 rebuilt and restarted, `BUILD_ID` `5gN2_DKycn5pOV7fNMVos`. -**The hub and the homepage are deployed for the first time:** https://archilyzer-hub.pages.dev -(Pages project `archilyzer-hub`, five members) and https://archilyzer.pages.dev (project -`archilyzer`, 200 — no longer 522; its hero links the hub). See the -[release-7 rollout record](release-7.md#rollout-2026-09-25-afternoon--bb3dbb4c-live-on-3001-releases-6--7-together). +**Live on :3001 (2026-09-25, 14:43):** `0e72ef73` (release 8, on top of release 7's `bb3dbb4c`), +`BUILD_ID` `9pC6zELQSzjdkIPDNsVox`. umtool on :3050 was rebuilt without a code change and restarted +with it, `BUILD_ID` `a9YAe_dwhuM6DiCPzIwMD`. Still live from release 7: +- https://archilyzer-hub.pages.dev (five members); +- https://archilyzer.pages.dev (200, its hero links the hub); +- https://rekietalyzer.pages.dev (rebuilt 13:00). + +See the +[release-8 rollout record](release-8.md#rollout-2026-09-25-late-afternoon--0e72ef73-live-on-3001-second-restart-of-the-day). + +**Last updated:** 2026-09-25 (late afternoon). **Release 8 is live, the day's second restart.** +- Release 8 (`bb3dbb4c` → `0e72ef73`) has two slices, each Opus-reviewed: SHIP AFTER FIXES, then + SHIP. + - **V, video titles** (`one-core/r8-video-titles` → `faa92c51`), the operator's 13:40 ask. The + channel video list shows and searches titles: index → scan store → `metadata.info.json`, + memoized per channel by the `data/` mtime after the review measured Rumble lists at 5 s per + render. The video page shows scan-store metadata for an undownloaded video. + - **S, one shared auto-queue state** (`one-core/r8-state-share` → `0e72ef73`). This is the root + cause of the 13:33 `state.json` revert, which recurred at ~14:18: four lanes booted + concurrently and each held a private copy. The in-flight read is now cached, and a manual + Sync/scan 429 cooldown writes through the live object. + - Record: [`release-8.md`](release-8.md). +- Gate on `0e72ef73`: tsc 0, common 1,810, editor unit 75, scripts 161 + 1 skip, `next build` + ok, the union of both slices' specs 86/86. No numbers changed. +- Rollout. md5 80 → 80, with one explained difference: `paramount-tactical/config.json`, the + operator's `downloadFilter.include: "Quartering"` plus a `lastFullDownloadAt` stamp. The + deferral and `fails: 13` survived the restart. Smoke `SMOKE_FAIL=1` with both flags explained: + a moving jobs list, and a correct deferral where the old check expected `[]`. Titles are proved + live on `/channels/paramount-tactical/videos`. +- **Second-restart proof:** no `legal-mindset` re-scan on either of today's boots. +- **Paramount Tactical.** The boot-dispatched metadata scan ran at about 10/min with 0 429s. It + was cancelled at 377/1,382 on the operator's ask, and its store kept the 377. download-missing + then settled **376** non-matching videos by the filter with no request and is downloading the + rest. `pnpm ops keep-videos` ran live for the first time: 1 match (`NV1QqS9NOiU`), marked. +- **The owed `teamrcn` sync ran** and moved exactly `lastSyncedAt` + `lastFullSweepAt`, but it + reads `failed`: `Invariant: static generation store missing in revalidatePath`. A QUEUED job's + own `revalidatePath` runs outside any request. +- Operator questions answered today: the 429s are not concurrent downloads, and + `downloadFilter.include` has a UI (release 7). + +**Open:** +- **the queued-job `revalidatePath` fix**, small. A `safeRevalidate` in the editor, or drop the + job-body revalidations for `requestChannelSnapshot`. Until then, an ops job that queues reads + `failed` after doing its work. The running download-missing `01M3CY7FHQNMTA376HA724BKK7` is + expected to end that way; +- **the keep-videos re-run** after download-missing finishes (dry run first, `match: + "TheQuartering|Quartering"`). Optionally, the rest of the Paramount Tactical scan (~1,000) for + list titles; +- the `/jobs` stall label misreads a slow scan (`STUCK · POSSIBLY-STALLED` at 10/min); stale + `queued` metas after a restart; the smoke's deferred check should become "well-formed"; +- S's optional merge nit (merge `until` and `fails` separately); V's memo nits and L3; +- the evening pacing watch: `grep -h 'deferred 6h' transcripts/.jobs/*.log`; +- the 410 re-read (release 7 step 11), not observed; +- candidate: YouTube `--sleep-requests` in `common/ytdlp/platformArgs.mjs`, not applied; +- worktrees to remove: `one-core-r8-{video-titles,state-share}`, `one-core-r7-{pacing,cli,keep}`, + `one-core-r5-{exports,rumble}`, `one-core-r6-followups`, `one-core-phase-4-s1`; +- `thequartering-X`'s transcripts manifest goes live at the next jeralyzer build + deploy. + +**Next:** the `revalidatePath` fix (a small slice), then Phase 4 slice 3: config + docs, +`doctor`, `run`, `mcp`, and `PUBLISH.md`. **Last updated:** 2026-09-25 (afternoon). **Release 7 is live, with release 6 riding along.** - Release 7 (`0032ed8a` → `bb3dbb4c`), three slices, each Opus-reviewed: **Y** pacing diff --git a/plans/release-8.md b/plans/release-8.md @@ -1,7 +1,7 @@ # Release 8 — video titles (+ follow-ups) -`main` at `bb3dbb4c`, release 7 live 2026-09-25; this release starts with the operator's video-titles -ask. Rules: `plans/tools/implementer-rules.md`. Record file: this file. +`main` at `bb3dbb4c`, release 7 live 2026-09-25 13:11; release 8 followed the same afternoon, starting with +the operator's video-titles ask. Rules: `plans/tools/implementer-rules.md`. Record file: this file. ## Record @@ -179,4 +179,221 @@ format is unchanged. persist its old object to the same fixture file until it notices its job is gone. That is a cross-spec clobber, not a two-lanes-in-one-process problem, and this slice did not introduce it. -## Rollout +## Rollout 2026-09-25 (late afternoon) — `0e72ef73` live on :3001 (second restart of the day) + +The second editor restart of the day. Release 7 (`bb3dbb4c`) had gone live at 13:11. Two things +made a release 8 that same afternoon: +- **The operator's ask**, at 13:40: "the individual video view should show video metadata like + the title, and `/videos` should show titles in the list when available and allow searching by + them." A read-only Explore pass scoped it. The video page already showed the title from + `metadata.info.json`, but showed a bare id for an undownloaded video. There is no global + `/videos`, and the per-channel list `channels/<slug>/videos` showed ids and searched ids only. + That became slice V. +- **The 13:33:37 state revert.** The live `state.json` lost the 13:20 deferral and `fails: 13` + (release 7's rollout, "And then the disk lost it"). The trace (`$T/q-state-revert.md`) found + that `sharedAutoQueueState` cached the resolved value, not the in-flight read. The four lanes + booted concurrently, each held a private copy, and any persist overwrote the whole file. That + became slice S. **It happened again at ~14:18:** the state read `fails: 12` again, with no + deferral, before S was live. + +**Merge.** `main` moved `b558a7b8` (plans: release 7, the 13:33 revert traced) → `faa92c51` (merge +V, tip `4122471e`, off `bb3dbb4c`) → **`0e72ef73`** (merge S, tip `89a7d6ab`, off `eff6a3a7`). +`0e72ef73` is the sha that went live. S's merge conflicted only in `editor/CHANGELOG.md`, and both +sides were kept. S's record section was pasted in above this heading in the merge commit. +`git diff --stat b558a7b8 0e72ef73` shows **17 files, +1,439 / −104**. umtool and the lockfile +are unchanged. + +**Reviews** (Opus, read-only). Both slices came back SHIP AFTER FIXES and were SHIP on re-review. +- **V** (`$T/r8-review-v.md`). **M1** was the Rumble list. Every Rumble dir misses the index, so + every row fell through to a `metadata.info.json` read on **every** render. Every row is a + `<Link ?video=>` on a force-dynamic page, so every click re-rendered. On + `the-quartering-rumble`, with **8,049** dirs, that measured **5,024 ms cold / 224 ms warm per + render**. The fix, `4e8bb285`, memoizes titles per channel, keyed by the `data/` dir mtime: LRU + 64, untitled ids not memoized, cleared by invalidate-cache. Measured live read-only: **5,331 ms + for the first render, then 5.5 ms and 6.5 ms**. L1 (comment placement) and L2 (a test) were fixed in + `41facfe0`, and `4122471e` records the re-gate. +- **S** (`$T/r8-review-s.md`). **Item 3 was required in the slice.** `recordDownloadBackoff` + (`common/jobs/downloadBackoff.ts:31-43`) wrote straight to disk, so any other lane's persist + erased a 429 cooldown recorded by a manual Sync, a metadata scan or the video page. The fix, + `4cfc7491`, moves the holder and `sharedAutoQueueState` to `common/jobs/autoQueueState.ts` on + `__yttAutoQueueState__` and re-exports them, and adds `liveAutoQueueState(paths)`. + `recordDownloadBackoff` now merges disk into the live object (the later `until` wins) and + writes through it. `platformCooldownRemainingMs` and the status builder prefer the live object, + and invalidate-cache clears the global. It adds 4 tests, 3 of which fail on the old code. + `89a7d6ab` is the CHANGELOG clause. One optional low item remains: merge `until` and `fails` + separately. + +**Gate on `0e72ef73`** (worktree `one-core-r8-state-share` detached at the merge, +`$T/r8-gate.log`, 14:1x–14:25): +- tsc **0 errors**. +- common **1,810/1,810** = 1,795 + V's 7 + S's 8. +- Editor unit **75/75** = 72 + 3 (`videoRows.test.ts`). +- test:scripts **161 + 1 skip**. +- `next build` ok. +- EDITOR e2e over the union of both slices' specs (`$T/r8-gate-specs.txt`: `video-titles + video-page video-filter-combine channel-embedded-video auto-queue lane-runner pacing queues + rumble-sweep ops-api`): **86 passed, 0 failed, 4.9 min**. +- **Numbers: none.** No schema key changed. Both slices only read or reshape. + +**Baseline and builds.** +- md5 `before` (14:25) and `pre-restart` (14:43): **80 files** each (`r8-md5.sh`, the same set + as release 7). `state.json`'s download keys were already `picks, platformBackoff, runtime, + videoDeferrals`. +- Builds (`r8-builds.log`, 14:25:49). Editor: exit 0, **37 s**, `BUILD_ID` + `6kMVr9Gn2093S9tp_pgqT` → **`9pC6zELQSzjdkIPDNsVox`**, and client-s3 still resolves into + `node_modules/.pnpm`. +- **umtool was rebuilt too, although its code did not change.** The cloned build script builds + both: exit 0, 16 s, `5gN2_DKycn5pOV7fNMVos` → **`a9YAe_dwhuM6DiCPzIwMD`**. So umtool was + **restarted as well**, because a rebuilt `.next` under a running server serves a stale chunk + set. + +**The operator's call.** The operator's download-missing on `paramount-tactical` was running +(61 → 91 of 1,473 dirs). They were asked, and chose **"Restart now"**, with a re-queue after +boot. Jobs at the restart (`r8-jobs-before-restart.json`): +- the auto-transcribe and auto-download runners; +- download-missing `01M3CSDWSS148ZW7PWG7EBRNGC`, running; +- metadata scans `01M3CT2H88CP3EPBXR4MS2N8GY` + `01M3CV5W6WG8QJW09G2F0N1V8A` and sync + `01M3CXGDYWWHPA4F7GDYZJPG8S`, queued. + +**Restart** (`r8-restart.log`, 14:43). Editor: 1232277 / 1232307 killed, `/` 200 after **3 s**. +umtool: 1232396 / 1232411 killed, `/` 200 after **3 s**. `/tags` 200. ZodError 0, one +"Ready in", umtool errors 0. + +**After boot.** +- **md5 `after-boot`: 80 files, one differs**, `channels/paramount-tactical/config.json`. During + the afternoon the operator had set `downloadFilter.include: "Quartering"` through the form, and + the cancelled download-missing stamped `lastFullDownloadAt`. A diff against the 13:30 copy shows + exactly those two keys. Boot wrote nothing. +- **The state survived.** `state.json` after boot held `platformBackoff.youtube {fails: 13, until + 13:52:55}` and `videoDeferrals {_60iFE_FBPQ}`. The download lane's last persist lived through + the restart, and with S live it cannot be clobbered again. +- **Stale metas.** The cancelled download-missing reads `cancelled`. The three queued jobs still + read `queued` in their meta files: a restart neither re-queues nor marks them. This is cosmetic. +- **New jobs.** A NEW metadata scan for `paramount-tactical`, `01M3CY1ZXR9EMTA42DR1FN9BKP`, + started at boot by itself (the scan auto-dispatch). The parent then queued the owed sync + `teamrcn`, `01M3CY7BN3DNN12T93X9GZM48X` (release 7 step 10), and then re-queued download-missing, + `01M3CY7FHQNMTA376HA724BKK7`. The order was deliberate, so the 7-video sync would not wait behind + hours of downloads. + +**The second-restart proof** (release 7 step 12, the scan-error refresh). The old build's +auto-download runner log (`01M3BAAN4K…`) has **1** `legal-mindset` line. The 13:11 boot's +(`01M3CRRX37…`) and the 14:43 boot's (`01M3CY1YFH…`) have **0**. +`channels/legal-mindset/metadata-scan.json`'s mtime is still 2026-09-24 23:52, so the +members-only ids were not re-scanned today. **Proved.** + +**Smoke** (`r8-smoke.out`, 14:44:22–14:48:09). `SMOKE_FAIL=1`, with **both flags explained**: +- **Seven of the eight pairs were identical.** `/api/jobs/active` and `/api/view/activeJobs` + differed by 500 B, then by 1 B on a recheck, while refresh-reports were finishing. That is a + moving list, not a view difference; release 7's smoke ran idle. +- **`autoQueueStatus.download.deferred`** = `[{videoId: _60iFE_FBPQ, channelSlug: + paramount-tactical-videos, untilMs: …}]`. The check, carried over from release 7, expected + `[]`, and that expectation is what is stale. A deferral exists, and it is correct. +- **Everything else passed.** 404s, presets, and an unchanged pulse rev. Pages 200 (`/` 4 s). + `/operations/download` 200 with one `Rate-limit cooldown` region (youtube cooling, one deferred + video). `/sites` has Build hub + Deploy hub. umtool `/` 200 on `a9YAe_dwhuM6DiCPzIwMD`. + +**Live title proof.** +- `/channels/paramount-tactical/videos` 200: 630,962 B, 42 `open <id>` rows. The RSC payload + carries real titles for the downloaded rows, for example `"title":"What is Really Going on With + The Strait of Hormuz ft. Colonel Rob Ma…"` and `"Complete Guide To Precision Scope Mounting - + Part 1 of 2"`. +- `/channels/paramount-tactical/videos/1p-YxSNbmsc` renders `<h2>FIRST SU*C*DE Bomber TIED to + IRAN</h2>`. +- The scan-only rows showed bare ids until the running scan reached them. + +**Paramount Tactical: scan → cancel → download-missing.** +- **The scan** (`01M3CY1ZXR9EMTA42DR1FN9BKP`). 1,382 listed videos, `--sleep-requests 1`, about + **10 videos/min**, **0 429s**. 11 members-only errors ("Join this channel") were recorded and + passed over. +- **Cancelled at 377.** At 15:20 the operator asked to stop it so downloads could start, since + the Quartering collaborations are recent. It was cancelled through the `/jobs` row's Cancel + button (`r8-cancel-scan.mjs`) at **377 / 1,382**. The scan store + `channels/paramount-tactical/metadata-scan.json` is written incrementally, so it keeps its **377 + entries**, and a later scan resumes the remainder. +- **The stall label.** While the scan progressed at 10/min, its `/jobs` row read `STUCK · + POSSIBLY-STALLED`. The stall heuristic misjudges a slow but alive scan (below). +- **download-missing** (`01M3CY7FHQNMTA376HA724BKK7`) started as soon as the scan was cancelled: + `Prefilter: 1382 missing … 91 already complete (of 1473)`. Then **`Settled by this channel's + download filter: 376 skipped (already decided)`**. The 377 scanned entries let the operator's + `downloadFilter.include: "Quartering"` settle 376 non-matching videos with **no extra + request**. Then `running the full set (1006 item(s))`: per-video prefetch, matches downloaded + (the first was `DLOM_ARCHIVE youtube NV1QqS9NOiU`), and `Sleeping 30s before next download`. + Its meta still read `queued` while its log was 49 lines long. The status flips late, which is + cosmetic. + +**The owed sync (release 7 step 10)**, `teamrcn`, job `01M3CY7BN3DNN12T93X9GZM48X`, ran at 15:21 +after 35 min in the queue. +- **The work was done.** Full sweep, availability check (1 dir, 0 attempted), backfill wrote 0. + `teamrcn/config.json` moved on exactly `lastSyncedAt` (2026-05-18 → 2026-09-25T19:21:15.754Z) + and a new `lastFullSweepAt`. No other file changed (md5). +- **The status reads `failed`.** Its last line is `[error] Invariant: static generation store + missing in revalidatePath /channels/teamrcn`. The work stands, but the status is wrong, the ops + exit code (1) is wrong, and the pages are not revalidated. +- **Where it throws.** A sync submitted through the action runs `runPipelineAction`'s job body, + and that body calls `revalidatePath` itself once `runYtdlp` returns + (`editor/app/channels/[slug]/pipelineActions.ts:220-224`). Release 7's + step 10 calls this the `onDone` hook. For a sync it is the job body's own revalidation. The + same idea is also written as an explicit hook: the metadata scan's `afterRun` + (`pipelineActions.ts:364-367`) and the `onDone` of `runOperationChannelJob` + (`common/controller/operationJobs.ts:23` explains that it exists because common cannot import + `next/cache`). +- **Why it throws.** When the job was QUEUED, the body runs from the platform queue's drain loop, + where Next has no request store. A job that starts inside the ops request (an empty queue) never + hits it, which covers every earlier ops sync, including release 5's. Only one live job log in + the last 200 carries the invariant. +- **Expected next.** The re-queued download-missing `01M3CY7FHQNMTA376HA724BKK7` runs the same + body and also started from the drain loop. Expect it to end `failed` on the same line once its + 1,006 items are through, with its downloads intact. +- **Fix owed** (small): make the job-completion revalidations tolerate a missing store (below). + +**keep-videos, live** (slice K's first real run, 15:23, `r8-keep-videos-run2.json`): + + pnpm ops keep-videos --json '{"slug":"paramount-tactical","match":"TheQuartering|Quartering","note":"keep-videos: TheQuartering collaboration (operator ask 2026-09-25)"}' + +- **Result.** `considered 468` (91 downloaded + 377 scanned), **`matched 1`**: `NV1QqS9NOiU`, + "Charlie Kirk Conspiracies, Internet Sleuths & The Outrage Machine | Th…". That is the first + video download-missing archived under the filter. It was `marked 1` on the first run and + `alreadyKept 1` on the idempotent second. +- **The marker.** `data/NV1QqS9NOiU/do-not-clean.json` = + `{"setAt":"2026-09-25T19:23:55.075Z","note":"keep-videos: TheQuartering collaboration (operator + ask 2026-09-25)"}`. +- **What was not matched.** `notDownloaded []`: no scanned but undownloaded match, which is + consistent with the filter settling 376 as non-matching. `unscanned 1002`, `noMetadata 3`. +- **Before the scan.** An earlier dry run at 14:0x read `considered 24, matched 0, unscanned + 1449`. +- **Owed:** re-run keep-videos after download-missing finishes, because each match it downloads + gets its marker only then. This is the morning step. + +**Found and left.** +- **The `/jobs` stall label misreads a slow scan.** A metadata scan advancing at 10 videos/min + read `STUCK · POSSIBLY-STALLED` (`possibly-stalled`, `common/views/jobRowView.ts`, + `common/views/jobRows.ts`). The heuristic needs a notion of "progress moved recently". +- **Stale `queued` metas after a restart.** Jobs queued when the server went down keep `queued` + in their meta. A freshly drained job also reads `queued` for a while (`startedAt: null`). Both + are cosmetic. The release-7 waiter's sync attempt died with the restart + (`r7-scan-sync2.log`: "sync exit=1 attempt=1 575s"). +- **The smoke's `DEFERRED_OK` check** must become "the list is well-formed", not "empty". +- **V's memo notes.** + - A `metadata.info.json` rewritten inside an existing dir does not refresh the memo. Titles do + not change, so this is accepted. + - A new dir drops the whole channel's memo. The reviewer's optional follow-up is to keep the + entry and read only the new ids. + - **L3:** the scan note can say "not downloaded" beside a dir whose metadata file is missing or + corrupt. This is rare. +- **S's merge rule.** It copies the whole disk entry when its `until` is later. A brief race can + therefore lower `fails` or revive a just-cleared entry. Optional fix: merge `until` and `fails` + separately (max of each). +- **The queued-job `revalidatePath` fix, owed.** Proposed shape: one editor-side `safeRevalidate` + that calls `revalidatePath` and treats the missing-store invariant as a no-op, used by every job + body, `afterRun` and `onDone` that revalidates. The alternative is to drop those revalidations in + favour of the file-based `requestChannelSnapshot`, since the pages are force-dynamic and the + snapshot scheduler already refreshes on completion. Until then, an ops-submitted job that + queues reports `failed` (and `pnpm ops … --wait` exits 1) after doing its work. +- **YouTube `--sleep-requests`** (release 7's finding) is still not applied in + `common/ytdlp/platformArgs.mjs`. Phase 4 slice 3 is still next. +- **Worktrees to remove:** `one-core-r7-pacing`, `one-core-r7-cli`, `one-core-r7-keep`, + `one-core-r8-video-titles`, `one-core-r8-state-share`, plus the older `one-core-r5-exports`, + `one-core-r5-rumble`, `one-core-r6-followups` and `one-core-phase-4-s1`. Removing shifts port + blocks. +- **keep-videos re-run owed** after download-missing finishes. Optionally, a + `pnpm ops metadata-scan` on `paramount-tactical` finishes the remaining ~1,000 for list titles.