Archilyzer · Source

archilyzer

Archilyzer
git clone https://archilyzer.pages.dev/source/archilyzer.git
Log | Files | Refs | README | LICENSE

commit e2424587e9bf1b79b822207f673837268942fc5e
parent 93155dc703fcb4ba5caf216148fdfa0b06ab96c9
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date:   Fri, 25 Sep 2026 13:42:46 -0400

plans: release 7 rollout record — 93155dc7 live on :3001 (releases 6 + 7), hub + homepage first deploys; STATE + FACTS

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Diffstat:
MRUNNING_IN_DOCKER.md | 2+-
Mplans/FACTS.md | 80+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Mplans/STATE.md | 53+++++++++++++++++++++++++++++++++++++++++++++++------
Mplans/release-7.md | 191++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
4 files changed, 318 insertions(+), 8 deletions(-)

diff --git a/RUNNING_IN_DOCKER.md b/RUNNING_IN_DOCKER.md @@ -249,7 +249,7 @@ pnpm ops channel-config --json '{"slug":"the-quartering","patch":{"downloadFil pnpm ops channel-priority --json '{"slugs":["the-quartering"],"operation":"download","tier":"paused"}' pnpm ops lane --json '{"lane":"download","held":true}' pnpm ops refresh-report --json '{"all":true}' -pnpm ops keep-videos --json '{"slug":"paramount-tactical-videos","match":"TheQuartering","dryRun":true}' +pnpm ops keep-videos --json '{"slug":"paramount-tactical","match":"TheQuartering","dryRun":true}' pnpm ops get channel the-quartering pnpm ops list # every action name ``` diff --git a/plans/FACTS.md b/plans/FACTS.md @@ -6243,3 +6243,83 @@ Line numbers are `plans/FACTS.md` lines at `e172749b`, before this record's in-p (`channelGroups.ts:118`, missing = last); channel `order` is parsed (`siteSchema.ts:154`) and read by nothing. So a "no-op" site save can move bytes; `jq -S` plus the `order` fills is the expected diff, not a bug. + +## Release 7 — slices Y, C, K (verified 2026-09-25, `main` @ `bb3dbb4c`) + +- **A rate-limited video is deferred, per video, for 6 h.** `common/jobs/platformBackoff.ts`: + `VideoDeferral {until, channelSlug}`, `VIDEO_RATE_LIMIT_DEFER_MS` (6 h), `deferVideo`, + `isVideoDeferred`, `pruneDeferred`, `coerceVideoDeferrals`. The map is `videoDeferrals` on every + lane of `transcripts/.auto-queue/state.json`, **beside `platformBackoff`, persisted** (a restart + honours it; an older build drops the key on its next write). The one seam is + `common/jobs/unitOutcome.ts` `applyUnitOutcome` — `rate_limit` backs the platform off AND defers + the id; `network` backs off only; success clears the backoff. Its line, grep-able in + `transcripts/.jobs/*.log`: `Auto-download: <pf> rate_limit — backing off <s>s (attempt <n>). + <id> deferred 6h; next video after cooldown.` (the `network` line, and the pre-release-7 + `rate_limit` line, end `<id> will retry after cooldown.`). The download pick drops deferred ids; when that leaves nothing the idle reason + is **`deferred`** (`AutoRunnerIdleReason`, `dispatch.ts` `idleReasonText`, `activeJobs.ts` + `autoIdleNote` — both exhaustive). **Manual Sync and download-missing ignore deferrals**; a manual + 429 records a platform cooldown only. `/api/view/autoQueueStatus` carries `<lane>.deferred: + [{videoId, channelSlug, untilMs}]`; `/operations/download` draws `role="region"` "Rate-limit + cooldown" with lists "Platforms in cooldown" / "Deferred videos". The runner never merges + deferrals from disk mid-run (only it writes them). The pick filter is by video id, so a channel + rename leaves a stale but harmless `channelSlug` until expiry. +- **An identical metadata-scan error refreshes its `at`** once it is + `METADATA_SCAN_ERROR_COOLDOWN_MS` old (`common/controller/metadataScanStore.ts` + `upsertMetadataScan`), so members-only ids are no longer re-scanned at every runner start. +- **YouTube is ONE serialized queue.** Every job kind with `queueKeyStrategy: "platform"` (sync, + metadata-scan, download-missing, retry-bucket, fetch-window, auto-download-unit, …) lands on + `platform:youtube` (`common/lib/queueKeys.ts:69-73` `downloadQueueKey`), and the registry submits + every job with `concurrency: 1` (`common/jobs/registry.ts:189-211`). Two YouTube downloads never + overlap (0 overlaps in 36 h of job metas, 2026-09-25). The exceptions are dormant: + `check-kept-deleted` runs on `channel:<slug>`, and the backfill re-acquire runs on the + `backfill` queue (off while `backfill.allowRedownload` is false). YouTube has **no** + `--sleep-requests` in `common/ytdlp/platformArgs.mjs` (Rumble has `1`); every retained YouTube + 429 is a per-video subtitle fetch. +- **`archilyzer` is the one command line**: `common/bin/archilyzer.ts`, a table over + `common/bin/_cli.ts` (`resolveCommand` longest-path, `argumentProblem` refuses an unknown flag + before `run`) and `_parseFlags.ts` `parseArgv` (skips pnpm 11's forwarded `--`). Run as + `pnpm --filter yt-dlp-transcript-common exec tsx bin/archilyzer.ts <cmd>` (or `tsx + ../common/bin/archilyzer.ts` from `export/` / `homepage/`). Commands: `index`; `compose + site [<id>] | hub | homepage`; `build site [<id>] [--nodata] [--skip-archives] | all + [--skip-archives] | hub | homepage`; `deploy site [<id>] [--preview <b>] | hub [--preview <b>] | + homepage`; `sync tick`; `settings example [--check]`. Exit 0 ok, 1 failed/refused, 2 usage. + Not yet: `doctor`, `run`, `mcp` (Phase 4 slice 3). The ten bins export `main(opts)` and + auto-run through `runIfEntryPoint`, so `tsx bin/x.ts` still works. +- **`export/package.json` `build` IS the CLI** (`tsx ../common/bin/archilyzer.ts build site`, + `SITE_ID` from env); `build:hub` → `… build hub`, `deploy` → `… deploy site` (refuses without a + project). **`prebuild` and `build:nodata` are gone** — the data phase is an explicit step of + `buildSiteSteps` (`common/publish/build.ts`: `build:data` unless skipData, `compose:site`, `next + build`). `docker/build-site.sh` / `publish-site.sh` call the CLI. `common/publish/build.ts` has + the named entry points `buildSite`, `deploySite`, `buildAll`, `composeHub`, `buildHub`, + `deployHub`, `composeHomepage`, `buildHomepage`, `deployHomepage`; the editor's jobs call them. +- **The hub builds and deploys.** `buildHub` = compose-hub + `INSTANCE_MODE=hub next build` into + **the shared `export/out`** (it removes `public/site.json` first; compose-site removes + `public/hub-sites.json`). `deployHub` reads `homepage.json` `cloudflareProject` — live value + **`archilyzer-hub`**, `siteUrl` `https://archilyzer-hub.pages.dev` — and `hubProjectProblem` + refuses the literal `archilyzer` (`HOMEPAGE_PAGES_PROJECT`, the homepage's project). Deploys + re-check the bundle inside the job: `builtHubProblem` (`common/lib/builtExport.ts`; a `site.json` + means a site bundle even with a stale `hub-sites.json`) / `builtSiteProblem`. Entry points: + `/sites` Hub section (`HubBuildButtons.tsx`: **Build hub**, **Deploy after build**, **Deploy + hub**), `pnpm ops build-hub` / `deploy-hub`, the CLI. **A Pages project must exist before its + first deploy:** wrangler offers to create one only on a TTY, and a job or the CLI spawns it with + piped stdin, so it fails fast. A production `deploy hub` takes its branch from the git checkout; + `deploy homepage` passes `--branch main` to `archilyzer`. +- **Posts-only channels publish their manifest.** `common/bin/compose-site.ts` + `reconcileChannelTree` copies a manifest-only tree under `MANIFEST_ONLY_SIGNATURE` instead of + removing it; `corpus.json` still advertises `manifests.transcripts` unconditionally, spec 4. +- **`pnpm ops keep-videos`** `{slug, match, fields?, note?, dryRun?}` + (`common/controller/keepVideosMatching.ts`, `editor/app/api/ops/keep-videos/route.ts`) sets the + do-not-clean marker on every video whose title + description matches — the download filter's + own matcher (`compileDownloadFilter` / `classifyAgainstFilter`, ReDoS guard first). It never + creates `data/<id>/`: matches with no dir come back in `notDownloaded`, ids with no text in + `unscanned`. One-shot, idempotent; re-run after new downloads. Media guard first. +- **e2e isolation: `export/out` belongs to the checkout, not the fixture.** `resolveOutDir` = + `paths.exportDir/out`, so whatever the last `e2e:2origin` / hub build left there is what an + editor spec's deploy route sees. `ops-api.spec.ts`'s deploy-hub "not the hub" sub-case therefore + runs only when `../export/out/hub-sites.json` is absent or a `site.json` is present, else it + records `subcase-not-exercisable` (`211d4666`) — before that guard it started a real deploy job. + Any spec that can reach a deploy must check the build dir before calling with a valid project. +- **tsc in the primary checkout reports errors in `editor/.next/dev/types/validator.ts`** — a + stale generated file left by an old `next dev`, naming routes that moved. 0 source errors; + worktrees without that file are clean. Clear it with `rm -rf editor/.next/dev` when no dev + server runs. diff --git a/plans/STATE.md b/plans/STATE.md @@ -3,12 +3,53 @@ The working memory for the local-AI derived-corpus work. Rewritten at the end of every session, before context is cleared. See [`README.md`](README.md) for the protocol. -**Live on :3001 (2026-09-24, 23:39):** `93dcb532` (releases 4 + 5 together), `BUILD_ID` -`FKE60BTWpUiUa94WCSxTO`, next-server 888107. **Release 6 (follow-ups `4d97049f` + Phase 4 slice 1 -`cda35622`) is on `main` but NOT rolled out** — the next release's prelude rolls it out, after -`pnpm install --frozen-lockfile` in the primary (umtool→common link, aws-sdk now under common). The -one-sync md5 sweep owed since release 3 is DONE (`rekietalaw-rumble`, 4 Rumble videos archived — the -first since August). See the [release-5 rollout record](release-5.md#rollout-2026-09-24-night--93dcb532-live-on-3001-releases-4--5-together). +**Live on :3001 (2026-09-25, 13:11):** `bb3dbb4c` (releases 6 + 7 together), `BUILD_ID` +`6kMVr9Gn2093S9tp_pgqT`; umtool on :3050 rebuilt and restarted, `BUILD_ID` `5gN2_DKycn5pOV7fNMVos`. +**The hub and the homepage are deployed for the first time:** https://archilyzer-hub.pages.dev +(Pages project `archilyzer-hub`, five members) and https://archilyzer.pages.dev (project +`archilyzer`, 200 — no longer 522; its hero links the hub). See the +[release-7 rollout record](release-7.md#rollout-2026-09-25-afternoon--bb3dbb4c-live-on-3001-releases-6--7-together). + +**Last updated:** 2026-09-25 (afternoon). **Release 7 is live, with release 6 riding along.** +- Release 7 (`0032ed8a` → `bb3dbb4c`), three slices, each Opus-reviewed: **Y** pacing + (`one-core/r7-pacing` → `2497d20b`): a rate-limited video is deferred 6 h (`videoDeferrals`, + persisted beside `platformBackoff`) so the lane moves on after the cooldown; an identical + metadata-scan error refreshes its `at` once a cooldown old. **C** the CLI (`one-core/r7-cli` → + `3049be43`): `common/bin/archilyzer.ts`, named publish entry points, export's `build` = the CLI + (`build:nodata` / `prebuild` gone), docker scripts on the CLI, hub build + deploy (`/sites`, + `pnpm ops`, CLI), `build|deploy homepage`, the posts-only 404 fixed in the composer. **K** + `pnpm ops keep-videos` (`one-core/r7-keep` → `ac2c4aee`), the operator's mid-rollout ask. Plus + `211d4666`, a test-only guard (below). Record: [`release-7.md`](release-7.md). +- Final suites on `3049be43`: editor 626 passed / 5 failed / 12 skipped (53.1 min, machine in + swap) — four load timeouts and one REAL isolation bug: `ops-api.spec.ts` deploy-hub started a + deploy job of the hub a 2origin run had left in `export/out` (nothing reached Cloudflare), fixed + `211d4666`; the five specs on `bb3dbb4c`: 55/55. Export 192/192, hub 8/8, 2origin 3/3. +- Rollout: install no-op; numbers 1,353 paths / 3,859 lines (corpus growth); md5 80 → 80 + identical across the restart; smoke `SMOKE_FAIL=0`; a Rekietalyzer `skipData` build through + `pnpm ops` (27 s, no `prebuild`, no data phase). The old server had already run C's new + `build site` script live at 13:00 (a Rekietalyzer build-deploy, 81 files). +- **Pacing proved live within 10 minutes of boot:** the old build had looped 12× on + `paramount-tactical-videos/_60iFE_FBPQ`; the new runner's first 429 logged `… (attempt 13). + _60iFE_FBPQ deferred 6h; next video after cooldown.` +- Operator questions answered: the 429s are NOT concurrent downloads (one serialized + `platform:youtube` queue, 0 overlaps in 36 h; all 26 are subtitle fetches, multiplied by the old + re-pick loop); `downloadFilter.include` has a UI (Configure form). + +**Open:** +- the owed `teamrcn` sync (step 10) and the Paramount Tactical metadata scan, queued behind the + youtube cooldown; then `pnpm ops keep-videos {"slug":"paramount-tactical","match":"TheQuartering"}` + (dry run first) — the channel was renamed from `paramount-tactical-videos` at ~13:29; +- the evening watch: `grep -h 'deferred 6h' transcripts/.jobs/*.log`; the scan-error refresh + proof needs a second restart within 24 h; +- the 410 re-read (step 11), not observed today; +- candidate: YouTube `--sleep-requests` in `common/ytdlp/platformArgs.mjs` — not applied; +- worktrees to remove (`one-core-r7-{pacing,cli,keep}`, `one-core-r5-*`, `one-core-r6-followups`, + `one-core-phase-4-s1`); +- `thequartering-X`'s transcripts manifest goes live at the next jeralyzer build + deploy. + +**Next:** Phase 4 slice 3 (config + docs, `doctor`, `run`, `mcp`; `PUBLISH.md` absorbing +`DEPLOY_DOCKER.md` + `DEPLOY_CLOUDFLARE.md`, and the two stale `archilyzer.pages.dev` hub hints C +left: `SettingsForm.tsx:48`, the `homepage.ts:31` comment). **Last updated:** 2026-09-25 (early morning). **Release 5 is live; release 6 is merged.** - Release 5 (`f4da04a9` → `93dcb532`): slice R (`one-core/r5-rumble` → `3adaea9b`): one yt-dlp arg diff --git a/plans/release-7.md b/plans/release-7.md @@ -609,4 +609,193 @@ the session's permission classifier**, so the branch is still on `3049be43`. `gi - **Commit trailers** name `Claude Opus 5.5 (1M context)`, as the release-6 and release-7 implementers did. -## Rollout +## Rollout 2026-09-25 (afternoon) — `bb3dbb4c` live on :3001 (releases 6 + 7 together) + +ONE editor restart and ONE umtool restart, at the end of release 7, as the plan said: release 6 +changed umtool (4 files), so this rollout rebuilt and restarted both. The live editor had served +`93dcb532` (releases 4 + 5, `BUILD_ID` `FKE60BTWpUiUa94WCSxTO`) since the release-5 night. `main` +moved during the rollout: `0032ed8a` → `6ee1d336` (this plan) → `2497d20b` (merge Y, tip +`e60ac2f4`) → `7b79a945` (plans nit) → `3049be43` (merge C, tip `dbe35c51`) → `211d4666` (the +deploy-hub e2e guard, below) → `ac2c4aee` (merge K, tip `374bdae7`, the operator's mid-rollout +ask) → **`bb3dbb4c`** (K's two review doc nits) — the sha that went live. Every slice had an Opus +read-only review: Y and C were SHIP AFTER FIXES then SHIP on re-review (Y's fixes `8094615d`, +`74c24b1f`, `e60ac2f4`; C's `bc2d9fb6`, `d1ba90e9`, `dbe35c51`); K was SHIP AFTER FIXES with two +doc nits, applied by the parent in `bb3dbb4c`. + +**Step 1 — final suites on `3049be43`** (worktree `one-core-r7-cli` detached at the merge sha, +ports 3601/3611/3610; logs `final-e2e-r7-{editor,export,hub,2origin}.log`). EDITOR: **626 passed, +5 failed, 12 skipped, 53.1 min**. Slow because the machine sat at 14/15 GB RAM plus 8–14 GB of +swap: the LIVE editor's parakeet transcription worker held 2.3 GB. Four of the five failures were +load timeouts — `channel-storage.spec.ts:79` (ECONNRESET from the fixture server's +apiRequestContext), `site-scope.spec.ts:61`, `sites-crud.spec.ts:148` ("Saved" not visible in +5 s), `social-channel.spec.ts:115` (30 s timeout). **The fifth was real, and it was test +isolation:** `ops-api.spec.ts:964` "deploy-hub refuses … a bundle that is not the hub" expected +400 and got 200. `export/out` is the checkout's own build dir (`resolveOutDir` = +`paths.exportDir/out`, not fixture-controlled), and C's own `e2e:2origin` run had built a hub +into it, so `builtHubProblem` found a hub, the route accepted, and **a deploy job of that hub to +`archilyzer-hub`, preview `hub-check`, was started**. Nothing reached Cloudflare — `wrangler pages +deployment list --project-name archilyzer-hub` was empty afterwards. Fix **`211d4666`** (test +only): the sub-case runs only when `../export/out/hub-sites.json` is absent or a `site.json` is +present (mirroring `builtHubProblem`: a site's `site.json` wins); otherwise the test records a +`subcase-not-exercisable` annotation and makes NO call with a valid project. EXPORT **192 passed, +6.5 min**. HUB **8 passed, 15.7 s**. 2ORIGIN (`TWO_ORIGIN_REBUILD=1`) **3 passed, 34.3 s**; the +worktree's `export/public` entries that compose-hub writes (`hub-sites.json`, `site.json`, +`corpus.json`, `llms.txt`, `robots.txt`, `sitemap.xml`, `_headers`) were swapped for real copies +before the run and relinked after, so the suites did not touch the primary's files (their +12:59/13:04 mtimes came from the live editor's own Rekietalyzer build, below). **Rerun of the +five failed specs on `bb3dbb4c`** (`r7-rerun-editor.log`: `channel-storage ops-api site-scope +sites-crud social-channel`): **55 passed, 0 failed, 3.8 min**, including slice K's new ops-api +test and the guarded deploy-hub sub-case. tsc on `bb3dbb4c` in the primary (`r7-tsc.out`): 8 +errors, **all in the stale generated `editor/.next/dev/types/validator.ts`**, 0 in source; the +worktrees, which have no such file, were clean. + +**Step 2 — install.** `pnpm install --frozen-lockfile` in the primary: "Already up to date", +601 ms (`r7-install.log`). The three facts held: `editor/node_modules/@aws-sdk` absent; +`common/node_modules/@aws-sdk/{client-s3,lib-storage}` present; +`umtool/report-to-video/node_modules/yt-dlp-transcript-common` → `../../../common`. The OLD +live build's `editor/.next/node_modules/@aws-sdk/client-s3-*` resolved into +`node_modules/.pnpm/@aws-sdk+client-s3@3.1080.0`. + +**Step 3 — numbers on the live corpus, new code** (`r7-numbers.log`). Settings: +`SETTINGS_RT_OK` **1,353 scalar paths**, diff empty. Files: 77 `unknown keys: []` lines, 0 +non-empty, no `WRITE THREW`, **3,859 lines** (`r7-files-numbers.txt`). The plan expected +release 5's 3,839; the +20 predates the rollout — C's worktree run before the merge already gave +3,859 over the frozen copy. The corpus grew; no key changed in 6 or 7. + +**Step 4 — md5 baseline** (`r7-md5.sh`: `settings.json` + 6 `site.json` + every channel +`config.json` + `sites/_homepage/homepage.json`). `before` at 12:03: **79 files**. +`pre-restart` at 13:10: **80 files** — the operator added `paramount-tactical-videos` during the +day. `state.json` `.download|keys` before: `["picks","platformBackoff","runtime"]` on all four +lanes. + +**The reshaped build path ran live before the restart.** At 13:00 the OLD live editor +(`93dcb532`) ran job `01M3CQ95WY2DSH5Q2SS2417K1T`, a Rekietalyzer build-deploy. It spawned +`pnpm run build` in `export/`, and that script on disk was already C's `tsx +../common/bin/archilyzer.ts build site`. It compiled ("✓ Compiled successfully in 9.8s"), +uploaded 81 files and deployed https://51e1101b.rekietalyzer.pages.dev — a real-world pass of +the new build path under the old server. It is also what regenerated the primary's +`export/public` (Rekietalyzer staging, mtime 12:59:44), which decided step 8's site. + +**Step 5 — builds** (`r7-builds.log`, 13:09:37, detached while the old server served). Editor: +exit 0, **38 s**, `BUILD_ID` `FKE60BTWpUiUa94WCSxTO` → **`6kMVr9Gn2093S9tp_pgqT`**; `readlink -f +editor/.next/node_modules/@aws-sdk/client-s3-*` → +`node_modules/.pnpm/@aws-sdk+client-s3@3.1080.0/node_modules/@aws-sdk/client-s3`. umtool (cwd +`umtool/`): exit 0, **15 s**, `JKtTfoVrUbTosNENO3GMj` → **`5gN2_DKycn5pOV7fNMVos`**. + +**Step 6 — ONE restart** (`r7-restart.log`, 13:11). Running at the time: only the two runners +(auto-transcribe `01M3BAAN42XQ7YJP3F3KDM60DF`, auto-download `01M3BAAN4K2733K0KYTM5BVZSF`) and a +transcription of `omnibased/9yXc8WhVZZo`, which was cut and re-picked after boot. Editor: TERM +888092 (pnpm) + 888107 (next-server, cwd `editor/`), `setsid nohup pnpm run start -H 0.0.0.0`, +`/` 200 after **2 s**, "Ready in 175ms" (`editor-start-r7.log`), `/tags` 200. umtool: TERM 5863 + +5881, its `start` on :3050, `/` 200 after **3 s** (`umtool-start-r7.log`). + +**Step 7 — after boot.** md5 `after-boot`: **80 files, identical** to pre-restart — boot +rewrote nothing. `ZodError` 0; umtool errors 0. `state.json`: every lane now carries +`videoDeferrals` (`{}`) beside `picks` / `platformBackoff` / `runtime` — the expected change +outside the md5 baseline. `/api/view/autoQueueStatus` `.download.deferred == []`, +`.transcription.deferred == []`. + +**Step 8 — smoke** (`r7-smoke.out`, 13:13:48 → 13:16:28, **`SMOKE_FAIL=0`**). All eight +`/api/*` ↔ `/api/view/*` pairs identical with the live fields stripped (`autoQueueStatus` +511,533 B); `/api/view/bogus` and `/api/view/invalidate-cache` 404; presets 200; `?rev=` +`changed=false` on both paths; ten pages 200 (`/` 4 s, `/channels`, `?sort=size`, `/jobs` 3 s, +`/operations/diarization` 110 s, `/settings`, `/storage`, `/tags`, `/channels/FearAnd`, the video +page). `/operations/download` 200 with **one** `Rate-limit cooldown` region — the plan said none +"while nothing cools", and youtube WAS cooling: the persisted state carried `fails: 12` from the +old build's loop (step 12), so the region is correct. `/sites` 200 with `Build hub` and `Deploy +hub`; umtool `/` 200 on the new `BUILD_ID`. **Site build:** `pnpm ops build-site --json +'{"siteId":"rekietalyzer","skipData":true,"skipArchives":true}' --wait` +(`r7-ops-build-site.log`): **exit 0, 27 s, compiled, 0 `prebuild` lines, 0 `build:data` +lines** — the SDK resolves from common and the build is compose + `next build` with no data +phase. **Rekietalyzer, not the plan's anilyzer:** a `skipData` build composes over whatever is +staged, and the primary's `export/public` held the 13:00 Rekietalyzer staging; anilyzer over it +would have been a wrong bundle. + +**Step 9 — hub and homepage, the first deploys ever.** The Pages project `archilyzer-hub` did +not exist; the parent created it beforehand (`wrangler pages project create archilyzer-hub +--production-branch main`, ~11:5x), because wrangler never gets a TTY from a job or the CLI and +fails fast on a missing project (C's record). `archilyzer` existed with no deployment — hence +the 522. **Form** (`r7-hub-form.mjs`, Playwright on the live `/sites` Hub form): +`cloudflareProject` `archilyzer` → `archilyzer-hub`, `siteUrl` blank → +`https://archilyzer-hub.pages.dev`, "Saved."; both persisted across a reload; md5 `after-form`: +only `homepage.json` moved. **Build:** `pnpm ops build-hub --json '{}' --wait` +(`r7-ops-build-hub.log`): exit 0, **41 s**, `export/out/hub-sites.json` with **5 members**, +`corpus.json` present. **Deploy:** `pnpm ops deploy-hub --json '{}' --wait` +(`r7-ops-deploy-hub.log`): exit 0, **750 s** ("Uploaded 548 files (726.96 sec)" — the first +upload of every file), https://f73a7c21.archilyzer-hub.pages.dev. **Public:** +`https://archilyzer-hub.pages.dev/hub-sites.json` 200 (579 B: anilyzer, bonnellyzer, hasanalyzer, +jeralyzer, rekietalyzer), `/corpus.json` 200 (spec 4, 5 members), `/llms.txt` 200, `/` 200. +**Browser proof** (`h-hub-check.mjs`, `h-hub-report.md`, `h-hub-search.png`, `h-hub-modal.png`): +the shelf shows all 5 members badged "Member"; "Quartering" (metadata scope) hit only 1 site, so +"lawsuit" was used — **5 results spanning 4 origins** (jeralyzer, anilyzer, rekietalyzer, +bonnellyzer); the first result's transcript modal opened; the three export controls of +`export/e2e-hub/transcript-downloads.spec.ts` ("Mark both clip start and end first", "Download +this transcript as a file", "Copy this transcript as Markdown (for AI)") each count **0**; the +share link is present. **MCP proof** (`h-mcp-list-sources.mjs`, the package's own smoke-harness +pattern): the server spawned with `TRANSCRIPT_HUB_URL=https://archilyzer-hub.pages.dev` logged +`default corpus: hub:https://archilyzer-hub.pages.dev`, and `list_sources` listed the 5 members +as `remote:<url>` handles. **Homepage:** `pnpm --filter yt-dlp-transcript-common exec tsx +bin/archilyzer.ts build homepage` exit 0, **41 s** (`r7-cli-build-homepage.log`); `… deploy +homepage` exit 0, **33 s** → https://6508d288.archilyzer.pages.dev (`r7-cli-deploy-homepage.log`). +Public: `https://archilyzer.pages.dev/` **200 (was 522)**, `/docs/install/` 200, +`/downloads/archilyzer-source.tar.gz` 200 (2,541,513 B); the hero shows "Search all archives" → +`https://archilyzer-hub.pages.dev`. + +**Step 10 — the owed sync + md5.** `teamrcn` (7 videos, the smallest YouTube video channel) is +queued behind the youtube cooldown by `r7-scan-sync2.sh` (log `r7-scan-sync2.log`), after the +Paramount Tactical metadata scan; `config.json` before: `r7-teamrcn-config.before.json`. +_(sync result: pending — filled in by the parent)_ + +**Step 11 — 410 re-read: not observed today.** No availability check reached +`rekietalaw-rumble/v7e07us` or the four quartering ids. Left to their next check, as the plan +says (observe, do not force). + +**Step 12 — pacing, proved live the same afternoon.** The persisted state at boot carried +`download.platformBackoff.youtube = {fails: 12}` from the OLD build's loop on +`paramount-tactical-videos/_60iFE_FBPQ`: the auto-download lane had started on the operator's new +1,473-video channel, the subtitle fetch 429'd, and the old runner re-picked the same video after +every cooldown — the plan's Short, again. The old runner's line: + + Auto-download: youtube rate_limit — backing off 1658s (attempt 12). _60iFE_FBPQ will retry after cooldown. + +The cooldown lapsed at 13:20:12; the NEW runner re-picked `_60iFE_FBPQ` (not yet deferred — the +old build never wrote a deferral), it 429'd, and the log reads: + + Auto-download: youtube rate_limit — backing off 1949s (attempt 13). _60iFE_FBPQ deferred 6h; next video after cooldown. + +`state.json` then held `download.videoDeferrals = {"_60iFE_FBPQ": {"until": <+6 h>, +"channelSlug": "paramount-tactical-videos"}}` and `platformBackoff.youtube = {fails: 13, until: +13:52:55}`. The next pick after this cooldown is a different video. The manual `metadata-scan` +and `sync` submitted during the cooldown were refused with the platform-cooldown sentence +(`r7-scan-sync.log`) — unchanged behaviour, as the plan says. **(iii), the evening watch, is still +owed:** `grep -h 'deferred 6h' transcripts/.jobs/*.log` — each id at most once per 6 h, `attempt +N` climbing only across distinct ids. The scan-error refresh proof (a second restart within 24 h +logs no `legal-mindset` re-scan) is not provable today: there was one restart. + +**Found and left.** +- **The operator asked: "more 429s than ever — are there multiple YouTube downloads at once?"** + No (`q-429-report.md`). Every YouTube-touching job kind lands on ONE `platform:youtube` queue, + submitted with `concurrency: 1` (`common/jobs/registry.ts:189-211`, `common/lib/queueKeys.ts:69-73`); + 122 YouTube jobs in the retained 36 h, **zero overlaps**. All 26 real 429s are subtitle fetches, + on `quarteringvlogs` (20) and paramount-tactical, and the old re-pick loop multiplied them — + which is exactly what slice Y removes. Gap found: YouTube has no `--sleep-requests` in + `common/ytdlp/platformArgs.mjs` (Rumble has `1`) — a candidate follow-up, **not applied**. + Dormant concurrency gaps: `check-kept-deleted` runs on channel keys, not the platform queue; + the backfill re-acquire runs on its own queue (disabled, `backfill.allowRedownload: false`). +- **The operator asked: "does `downloadFilter.include` have a UI?"** Yes — the channel Configure + form's "Download filter: include (regex)" field (`editor/app/channels/components/ChannelForm.tsx:702`). +- **The channel was renamed** `paramount-tactical-videos` → `paramount-tactical` by the operator + at ~13:29 (8 data dirs at that time). The live deferral keeps the old slug — harmless: it + expires in 6 h and the pick filter is by video id. `RUNNING_IN_DOCKER.md`'s keep-videos example + named the old slug in `bb3dbb4c`; this record's commit corrects it. +- **Slice K's live run is pending the scan.** `pnpm ops keep-videos` on `paramount-tactical` + (`match: "TheQuartering"`) needs the metadata scan first (0 of 1,472 scanned at 12:52); the scan + is queued in `r7-scan-sync2.sh` ahead of the teamrcn sync. +- **Worktrees to remove:** `one-core-r7-pacing`, `one-core-r7-cli` (detached), `one-core-r7-keep`, + plus the older `one-core-r5-*`, `one-core-r6-followups`, `one-core-phase-4-s1`. `pnpm wt list`, + then `git worktree remove <path>`; removing shifts port blocks. +- **The `.next/dev/types` tsc noise in the primary** is a stale generated file, not source; `rm + -rf editor/.next/dev` clears it (as Y's re-gate did in its worktree). Not done here — the live + `.next` is the running server's. +- The e2e isolation lesson is in FACTS: **`export/out` belongs to the checkout, not the fixture**, + so a spec that can reach a deploy must check what is built before it calls with a valid project.