# Release 19 — agents run the archive (ops, CLI and MCP; machine safety; the operator's guide) Written 2026-10-09 from the "next batch" plan (releases 19–22), against `r18/integration` `4cffda3f` (release 18 complete, `main` `e921f82f` merged in). Rules: `plans/tools/implementer-rules.md` — three parallel Opus implementers, one per track, each in its own worktree branched from `4cffda3f`; the parent merges each track into `r19/integration` `--no-ff` on a clean tree, then fast-forwards `main`; records state rulings never reasons; counts-only privacy greps before every merge; implementers' commits carry their OWN model's `Co-Authored-By` + the session line; never boot a second editor against the real corpus; the homepage build gate is `build:nodata`; memory gate (`free -m` available ≥ 6 GB) before any build, render or e2e, every `next build` under a 5 GB `systemd-run` scope. Record file: this file (shape of `plans/release-18.md`). Release 20 (the data model) is [`release-20.md`](release-20.md). ## Context - Five working sessions (Super Hasanalyzer, Super Jeralyzer, Super Jasolyzer, candace owens corpus analysis, unified media aggregation) do corpus and content work and hold no unmerged code. Every gap they report is repo-wide tooling: wrappers that find `WORKER_TOKEN`, `joblog.sh`/`watchjobs.sh` job watchers, hand-run archive.org imports, hard-linking another site's `export/out` aside, two OOMs from concurrent heavy work. - Precedents to reuse: `editor/app/api/ops/_lib.ts` (`ops`, `opsAuth`, field readers, `jobResponse`), route unit tests (`editor/app/api/ops/{fetch-windows,transcribe,rename-channel}/route.test.ts`), `scripts/archilyzer-ops.mjs` (`ACTIONS`, `GETTERS`, `--wait`), the views (`common/views/names.ts`, `/api/view/*`), `mcp/src/fetchClip.ts`. - Tests: the editor e2e is 130 specs / ~709 tests / ~40 min, serial, behind a machine-wide lock; the editor has 29 unit-test files. No CI, no root `test` / `typecheck`; the editor's `lint` has no config. - **Precondition (Wave 0, release 18's rollout):** `main` fast-forwarded to `r18/integration`, the release 18 rollout run, the `r18-*` worktrees pruned. Every `editor/app/api/ops` or `archilyzer-ops.mjs` change merges after it. ## Rulings (operator, 2026-10-09; do not re-open) - **The MCP gains archival writes through the editor** — the `fetch_clip` pattern: the MCP server asks the editor (editor URL + `WORKER_TOKEN`), the editor writes. Settings, storage and deletes stay CLI/ops-only. - **Three parallel tracks**: A owns the e2e slot (code over ops/CLI/MCP, sequential), B is machine safety and tooling (no editor e2e; one umtool run), C is docs and plans (no e2e). - **Release 18 lands first**, by a session that may touch the primary checkout. - **e2e budget:** one editor/umtool/export e2e run at a time machine-wide; the full editor suite only at a release's end. Every slice carries its class: `[none]` docs/plans only, `[unit]` unit tests are its gate, `[spec]` one or two named specs, `[suite]` the full suite. - Polite scraping applies to every import and listing: one stream, real gaps, back off on 429. ## Slices ### Track A — release 19 (owner: the Track A implementer; sequential; owns the e2e slot) | slice | class | what | after | |---|---|---|---| | **A1** `pnpm ops` finds its token | `[unit]` | read `WORKER_TOKEN` / `ARCHILYZER_EDITOR_URL` from `editor/.env` when unset; `--wait` polls a real authed route instead of the `/api/jobs/active` rewrite | Wave 0 | | **A2** jobs over ops | `[unit]` + `[spec ops-api]` | `get jobs [--active\|--failed\|--kind\|--slug]`, `get job [--tail]`, `job cancel\|retry\|retry-failed\|drain\|promote\|force-release `, `--wait` on many ids with queue position; wraps `editor/app/jobs/actions.ts` | A1 | | **A2b** ops routes to unit tests | `[unit]` | the `ops-api.spec` route cases move to `route.test.ts` beside each route (moved from B6 on 2026-10-09: Track A owns the ops routes) | A2 | | **A3** read side | `[unit]` | `get settings\|storage\|sites\|workers\|auto-queue\|scheduler\|cleanup ` over the existing view builders/readers; `/api/auto-queue/control` gated behind `opsAuth` (unauthenticated today) | A2 | | **A4** archival writes | `[unit]` | `settings` patch (through `saveSettings` + the schema); `lane` start/stop/drain for every lane incl. `publish`; `clear-platform-hold`; `workers enable\|disable`; per-video `transcribe-one\|delete-file\|do-not-clean`; cleanup buckets; `relocate {dryRun}`; `archilyzer storage report` (tierable bytes per channel) | A3 | | **A5** fetch queue hygiene | `[unit]` | the editor dedupes identical fetch-window requests (same slug/id/window → the existing job); `fetch-via-editor.mjs` waits without a timeout, printing queue position; small fetch-window jobs get their own queue key / priority instead of waiting behind a multi-hour `persist-videos` | A4 | | **A6** imports | `[unit]` | `import-archive-org {items:[…] \| query}` with the polite inter-item gap, a `dryRun` that flags restricted (401) items, skip-if-held; `get remote-listing ` for Odysee/BitChute (paced, one stream) diffed against held videos | A5 | | **A7** cues without an index | `[unit]` | `pnpm ops build-cues --slug [--ids]` (normalize → `transcript.cues.json`); MCP `get_transcript` falls back to the VTT | A6 | | **A8** publish edges | `[unit]` | `POST publish {verb:"build"}` takes `allowMissingMedia`; `pnpm ops lane` covers `publish`; `archilyzer publish build --out ` for a private build served elsewhere (cheap on release 18's bundles) | A7 | | **A9** MCP archival writes | `[unit]` | on the `fetch_clip` pattern: `get_job` (status + log tail), `enqueue` for sync / download-missing ids / transcribe-bucket / fetch-posts / import-video (each → the existing ops route), `channel_coverage` (held videos by date, gaps), notes `list\|read\|reply` (umtool `/api/notes`, `UMTOOL_URL`); instructions in `mcp/src/instructions.ts`; the README tool table | A8 | | release end | `[spec ops-api]` + `[suite]` | one `ops-api.spec` run, then the full editor suite once | A1–A9 | ### Track B — release 19 (owner: the Track B implementer; no editor e2e) | slice | class | what | after | |---|---|---|---| | **B1** heavy-work gate | `[unit]` | `scripts/queue-lock.mjs` generalized into `pnpm heavy -- `: one heavy slot machine-wide + a ≥ 6 GB free-memory floor; used by e2e, `next build` (publish stages, `build-site.sh`), `build-video.mjs` renders; a render may hold the transcription lane for its duration (ops `lane`); documented in WORKTREES.md + AGENTS.md | — | | **B2** report-to-video robustness | `[unit]` | prune `*-frames` after the final mux (`--keep-frames` keeps them); `--chrome-only` builds missing segments; a manifest lint before render (teaser > 34 chars, image src relative to the manifest, a QR legible at 720p). Track B's, after B5; the Candace session reviews it before merge (ruled 2026-10-09) | B5 | | **B3** report CLI | `[unit]` | `archilyzer reports check [--reports …]` (compose without a build); `reports verify-quotes ` (quote vs cue span, the en-orig check); `reports attach-video` encoding to fit the 24 MiB compose limit | — | | **B4** ops transcribe | `[unit]` | `durationMs` excludes queue wait; a priority for short one-file jobs over long auto jobs | Wave 0 | | **B5** umtool small debts | `[unit]` → `[spec umtool]` | per-worktree umtool e2e ports (`portFor`); `mix.spec` order dependence; `umtool window` writes edit notes; a selection spanning two blocks gets a Note button; the CLI finds `SITES_DIR` from the repo, not the cwd; umtool's /sites heading becomes "Articles" (naming hazard vs the editor's /sites). ONE umtool e2e run | B1 | | **B6** test economy | `[unit]` | pure/API e2e moves to unit: `audio-check-classifier.spec` → common; `view-route`, `worker-unit`, `media-file-abort`. Root `pnpm test` and `pnpm typecheck`; the editor gets an eslint config or loses its broken `lint` script | — | ### Track C — docs and plans (owner: the Track C implementer; no e2e) | slice | class | what | |---|---|---| | **C1** plans hygiene | `[none]` | STATE "Now"; PLAN.md's release index; this file and `release-20.md`; the "landed without a plan" record ([`landed-2026-10.md`](landed-2026-10.md)); one-core Phase 4, `editor-operations-ia.md`, `stats-cache-key.md` statuses | | **C2** OPERATING.md + `archilyzer docs cli [--check]` | `[unit]` | recipes (add a channel; sync and download; import from archive.org / Odysee / BitChute / Wayback; fetch and capture posts; transcribe; tag; prepare and export reports; publish) over `pnpm ops`, `archilyzer` and the MCP; a generated command reference from `common/bin/_cli.ts` `usage()` and `scripts/archilyzer-ops.mjs` `usage()` on the `env-docs.ts` / `file-schemas-docs.ts` pattern; linked from AGENTS.md, README, CONTRIBUTING (its hand-picked commands replaced by the link). Regenerated after Track A's actions land | | **C3** doc fixes | `[none]` | `mcp/README.md` lists every tool; README's docs table adds AGENTS, SETTINGS, SITE, CHANNEL, REPORT, CITATIONS; RUNNING_IN_DOCKER's ops list points at the generated reference | | **C4** homepage docs | `[unit]` | `homepage/content/docs/operate.md` gains the CLI/ops/MCP overview; gates `build:nodata` + `pnpm --filter homepage test` | | **C5** FACTS refresh | `[none]` | verified facts for main's last ~70 commits (notes store, articles kind registry, ops transcribe word timings, fetch-windows pacing, metadata refresh) | ### File ownership | track | owns | |---|---| | A | `editor/app/api/ops/**` (except B4's transcribe route), `scripts/archilyzer-ops.mjs` + its test, `mcp/src/**`, `editor/app/jobs/actions.ts`, `common/controller/fetchWindows.ts`, `fetch-via-editor.mjs`, the `ops-api.spec` cases A2b moves | | B | `scripts/queue-lock.mjs` and the heavy gate, `umtool/**` (`umtool/report-to-video/*` for B2 only, the Candace session reviewing before merge), the e2e specs B6 moves, the report CLI, `common/controller/transcribeFile.ts`, `editor/app/api/ops/transcribe/**` | | C | `plans/**` (except the records others append), root `*.md` docs, `homepage/content/docs/**`, `mcp/README.md`, the `archilyzer docs cli` generator | | shared, append-only | `editor/CHANGELOG.md`, `AGENTS.md`, `common/bin/_cli.ts` (rows only) | ### Graph and merge order ``` Wave 0 (release 18 lands) ──► A1 ──► A2 ──► A2b ──► … ──► A9 ──► release-end suite (Track A is sequential: A1–A5, then A6–A9) └─► B4 now: B1 ──► B5 ──► B2 (Candace session review); B3; B6 (B6 touches no release-18 file) now: C1, C3 ──► C2 ──► C4, C5; C2 regenerated after A's actions land ``` - **Merge order:** Wave 0 before any `editor/app/api/ops` or `archilyzer-ops.mjs` change (they conflict with release 18). C1, C3 and B6 touch no release-18 file and may merge first. Track A merges slice by slice in its own order; B and C merge whenever a slice is green. The parent regenerates the command reference (C2) after each Track A merge that adds an ops action or CLI row. - **Do not touch:** `umtool/report-to-video/*` except B2, which the Candace session reviews before merge; `transcripts/**` except through the writers; the working sessions' `~/reports/*` workspaces. ## Verification - Every Track A slice: route unit tests (the `route.test.ts` pattern) + `scripts/archilyzer-ops.test.mjs`; MCP tools via `pnpm --filter yt-dlp-transcript-mcp test` and one live `/ask`-style session against the editor; one `ops-api.spec` run per release; the full editor suite once at release end. - B1: a unit test of the lock + a manual two-build collision showing the second waits. B6: the moved tests pass and the editor suite shrinks by the moved count. B5: one umtool e2e run. - C2: `archilyzer docs cli --check` in the gates beside `docs env --check` and `docs files --check`; every command a recipe names exists (grepped against the generated reference). C4: homepage `build:nodata` + unit. - Every track: `pnpm -r --no-bail --workspace-concurrency=1 exec tsc --noEmit` clean per commit; counts-only privacy greps (home paths in added lines 0, no `transcripts/` content). ## Record (Each slice adds a "#### Slice , as shipped" section under its track, in merge order.) ### Track A Branch `worktree-agent-a103bef2ee4055362` (worktree `.claude/worktrees/agent-a103bef2ee4055362`, ports editor 4101, test 4111, export 4110) off `4cffda3f`, one Opus implementer, A1–A5 in order, one commit each; `r19/integration` merged in before this record. Scratch files `a-*` in the job's `tmp`. | commit | slice | one line | |---|---|---| | `988b282c` | A1 | `pnpm ops` reads `WORKER_TOKEN` / `ARCHILYZER_EDITOR_URL` from `editor/.env(.local)`; `--wait` asks `GET /api/ops/job/` | | `5f86b10a` | A2 | `GET /api/ops/jobs`, `POST /api/ops/job {verb, ids}`; `get job`, `get jobs`, `job `, `job wait` | | `e065bd2d` | A2b | 19 of `ops-api.spec`'s request/response tests become route unit tests | | `30111658` | A3 | the read side (`get settings|storage|sites|workers|auto-queue|scheduler|cleanup`); `/api/auto-queue/control` behind the token | | `5f442885` | A4 | archival writes (settings patch, the publish lane, platform holds, workers, one video, cleanup, relocate dryRun); `archilyzer storage report` | | `c261b64a` | A5 | clip windows on `clips:`; an in-flight window answers its job | | `e53f3169` | — | `r19/integration` merged in | | `cde0e759` | A2b | the test corpus helper's paths opt out of tracing (`next-build-trace.test.mjs`) | #### Slice A1, as shipped — `pnpm ops` finds its token - `scripts/archilyzer-ops.mjs` `loadEditorEnv()` fills `WORKER_TOKEN` and `ARCHILYZER_EDITOR_URL` when unset — a variable already set, even to `""`, wins — from `editor/.env.local` then `editor/.env` of the checkout the SCRIPT is in (`REPO_ROOT` from `import.meta.url`, never the cwd), then of the main worktree when it runs in a linked one (`mainWorktreeOf`: `.git` file → `gitdir` → `commondir`, read off disk, no git process). Only those two keys are taken; the files also hold deploy credentials. A 401/503 prints `tokenHint`: where the token came from, never its value. `parseDotenv` reads `KEY=value`, `export`, quotes, comments. - **What `--wait` polled and why it was a rewrite.** After three failed log polls it asked `/api/jobs/active`, which since one-core phase 3 slice 2 is a `next.config` rewrite onto `/api/view/activeJobs` — kept at its old path for the pages and pinned widgets that poll it — and that view builds the whole live payload (channel stats, the disk gate, every runner's status) to answer "is job X still there". It now asks `GET /api/ops/job/` (behind the token: one registry record or `.meta.json` sidecar). An `archived` log status (an id the registry forgot) resolves to the sidecar's terminal status instead of exiting 1 for a job that ended `done`; a 404 `{ok:false}` is "gone". While a job waits, `followJob` prints `queued — position N on ` once per change. - `GET /api/ops/job/[?tail=N]` (`editor/app/api/ops/_jobs.ts`): the `getJobEntry` row without `logPath`, `queue: {key, position, queued, head}` while queued or running, the runner's `progress`; `tail` (≤ 500) is `common/jobs/listJobs.ts` `readLogTail` — the last N lines read from the end (256 KiB window, a partial first line dropped). #### Slice A2, as shipped — jobs over ops - `GET /api/ops/jobs` — `?active` (registry queued/running, queue order, each with its place), `?failed`, `?kind`, `?slug`, `?limit` (50, ≤ 500); a filtered history list reads the newest 2000 jobs (`JOBS_SCAN`); `active` with `failed` is a 400; unknown query keys are a 400. - `POST /api/ops/job {verb, ids}` — `cancel | drain | promote | force-release | retry` per id through `editor/app/jobs/actions.ts` (the row buttons' own actions), each answered in `results`; one refused id makes the answer `ok: false` (400, naming it) without stopping the others. `retry-failed` (no ids) is Retry all; `retryAllFailedAction` now also returns `jobIds` and cancels the new jobs' unread streams. `jobIds` = the jobs a retry started, which `--wait` follows. - CLI: `get job [--tail [N]]`, `get jobs [--active|--failed] [--kind] [--slug] [--limit]`, `job `, `job wait ` (no request; `followAll`, the `--wait` loop factored out). A read's flag on the wrong noun or on an action is refused, not ignored. - `ops-api.spec` gains "jobs over ops": two `/api/test/stuck-job` holders on one queue — the live list, one job's place and head, `?tail`, a promote refused ("already at the front"), a cancel of one known and one unknown id, `force-release` of the head. The success paths revalidate `/jobs`, which only a running server can. #### Slice A2b, as shipped — the ops-api spec's request/response tests are unit tests - 19 of `ops-api.spec`'s 33 tests (32 at base + A2's) exercised only a route's request and response; they moved, assertion for assertion, to `editor/app/api/ops/_door.test.ts` (the 401, the 503, unknown keys, the traversing slug on fourteen routes) and route tests for retry-bucket, tag-videos, persist-videos, build-site (+ build-deploy), deploy-site (+ build-deploy), deploy-hub, deploy-homepage and publish; fetch-posts and capture-posts gain one each. **ops-api.spec: 33 → 14 tests.** What stays needs the running server: writes that revalidate, jobs that run, the publish queue's runs, the jobs verbs. - `editor/app/api/ops/_testCorpus.ts` (imported only by tests): copies the named e2e fixture into a temp dir, sets the e2e test server's environment (fake yt-dlp, gallery-dl, ffmpeg/ffprobe, whisper, chough, parakeet, diarize, claude, findmnt, udisksctl, wrangler, next; the fake Cloudflare token; `ARCHILYZER_BRANCH=main`; `E2E_LIVE_CHECK=skip`) and `holdPublishQueue()`s as e2e did, so a refusal that regresses can only queue a fake. - The 503 no longer needs `/api/test/worker-token`: the unit process unsets its own `WORKER_TOKEN`. That test route now has no e2e caller (left in place). #### Slice A3, as shipped — the read side; the lane control route needs the token - `GET /api/ops/settings[?key]` (getSettings, migrated and defaulted), `storage` (`buildStorage`), `sites` (`listSites`: id, title, `siteUrl`, Pages project, audience, `isListedSite`, search, publish policy, channels), `workers` (`buildWorkersPayload`), `auto-queue` (`buildAutoQueueStatusPayload`), `scheduler` (`buildSchedulerStatusPayload`), `cleanup/` (new `loadCleanupRow` — `loadCleanupSummary`'s row for one channel, id lists as counts). One door, `editor/app/api/ops/_read.ts` `readRoute` (token, unknown query keys refused) and `redactSecrets` (any string under a key naming a token/secret/password/api key/credential → `""`; a remote worker's `token` is the one that exists today). CLI `get settings []`, `get storage|sites|workers|auto-queue| scheduler`, `get cleanup `. - **`/api/auto-queue/control` answers `opsAuth`.** Its one UI caller — `RunnerOperationView`'s Start/Drain/Stop on /operations — calls `startAutoQueueAction` / `stopAutoQueueAction` / new `drainAutoQueueAction` instead: `opsAuth` has no exemption for the UI (the UI never called an ops route; it uses server actions, which carry Next's origin check), and a page holds no token. The six e2e specs that post to the route (auto-queue, auto-subs-replace, disk-space, lane-runner, pacing, rate-limit) send the test token. #### Slice A4, as shipped — archival writes - `POST /api/ops/settings {patch}` → `saveSettings` (one-level merge). Refused before anything is written (`settings/patch.ts`): an unknown top-level key; **a key another writer owns** — `channelPriority` (compiled with the lanes' trees in one write), `autoQueue`, `workers`, `storage` — naming that writer; and a value the schema would not keep as sent (it coerces, never throws), every leaf of the patch compared with the schema's reading of the merged settings and each difference named ("minFreeDiskGB: sent -5, would be saved as "). Answers `{changed, value}` read back, redacted. - `lane` takes `publish`: `enabled` through `savePublishSettingsAction` with the stored fields, `held` through the pause gate, Start/Drain/Stop the publish page's actions; the ingest lanes' drain goes through `drainAutoQueueAction`. - `clear-platform-hold {platform}` (`clearPlatformHoldAction`, answers its sentence); `POST /api/ops/workers {op: enable|disable|drain, ids}` (the live pool, per id); `transcribe-one {slug, id, file?}` (`transcribeOneAction` with a file, else the video page's `whisperVideoAction`, which fetches audio through the managed path when there is none; the file name is checked as one path segment); `delete-file {slug, id, file}` (`deleteVideoFileAction` — `removeMediaFile`, refused while the media drive is not answering); `do-not-clean {slug, id, keep?}` (set, not toggled); `POST /api/ops/cleanup {slug, sweep: transcribed|extra-formats|wrong-format|failed-transcriptions}` (the channel page's four actions, as jobs); `relocate {dryRun}` (`previewRelocationAction` per slug — which, as on the panel, tiers a never-tiered channel in place first; a slug with no channel is the bulk path's "channel not found"). - `archilyzer storage report […] [--json]` (`common/bin/storage-report.ts`): per channel the tierable media, text and clip bytes off its report (`totalMediaBytes` is `isTierable`'s share) and where its media is — corpus disk, a location (`config.mediaDir` under its root, longest match), a custom path, or legacy — with totals per place; a missing figure is unknown (`—` / `null`), never 0; social channels left out; no drive touched, no editor needed. #### Slice A5, as shipped — fetch queue hygiene - **Its own queue.** `common/lib/queueKeys.ts` `clipWindowQueueKey`: a window job runs on `clips:` (from the platform queue it would have used, so `platform:vimeo.com` → `clips:vimeo.com`) — single (`fetchWindowAction`) and batch (`fetchWindowsAction`). A queue runs one job at a time and a tier orders only the waiting ones, so priority alone could not get a window past a running multi-hour persist; the queue key can. **Ruling recorded:** one window may now run beside one download on the same platform; windows stay one at a time per platform with the batches' `clip-window:` gap, and the platform's hold, cooldown and 429 backoff are shared with the downloads. - **Deduped.** `common/jobs/windowJobs.ts` `windowInFlight`: a queued or running `fetch-window` whose window contains the asked one, or a `fetch-windows` batch with such an item — same channel, video and height cap — read off the job's replay spec. `/api/media/fetch-window` answers that job (202, `existing: true`, its window and file); a batch lists such windows in `inFlight` and queues nothing for them, and the ops route adds their jobs to `jobIds` so `--wait` follows the windows asked for. Retry (`stream: true`) is not deduplicated. - **Not done: `fetch-via-editor.mjs`.** It is `umtool/report-to-video/fetch-via-editor.mjs` — under the directory no track touches — so its 10-minute poll timeout and its wait output are unchanged. The editor half covers its retry loop (a re-run after its timeout gets the job already fetching), and `pnpm ops job wait ` waits with no timeout, printing the queue position. #### Slices A6–A9, as shipped (2026-10-10, overnight Track A) Branch `r20/a6-a9` (worktree `r20-a6-a9`, ports editor 4001, test 4011, export 4010) off `r20/integration` `5b690d2e`, one Opus implementer, A6–A9 in order; `r20/integration` (release 20 D1) merged in before A9. Scratch files `a-*` in the job's `tmp`. | commit | slice | one line | |---|---|---| | `cf0bb7a6` | A6 | `import-archive-org {items \| query}`: one job over many items, held records skipped, RESTRICTED flagged, 401/403 storms stop; `remote-listing` / `get remote-listing ` for Odysee/BitChute | | `f83d17b1` | — | a `usage()` paragraph for the twelve actions that had none (C2's found-and-left) | | `890b3355` | A7 | `build-cues {slug, ids?, force?}`; `GET /api/ops/transcript` / `get transcript `; the MCP's `get_transcript` falls back to the editor | | `1b08ccc0` | A8 | `publish {verb: "build", allowMissingMedia}`; `archilyzer publish build [--allow-missing-media] [--out ]` | | `79b62feb` | — | `r20/integration` merged in (release 20 D1's `recordedDate.ts`) | | `d9318dbe` | A9 | MCP `get_job`, `enqueue`, `channel_coverage`, `notes`; `GET /api/ops/coverage` / `get coverage `; instructions, README, ENVIRONMENT | #### Slice A6, as shipped — imports and remote listings - **`import-archive-org` takes three shapes** (route, `importArchiveOrgAction`): `{item, files | match}` as before; `{items: ["" | {item, files? | match?}, …]}` (at most 500; a bare id takes the top-level `match`, else every media original, each one record); `{query, limit?}` — archive.org's advanced search (`archiveOrgSearchUrl`: `q`, `fl[]=identifier,title`, `sort[]=identifier asc`, `rows` ≤ 500, one request through the polite client's chain, gap and backoff), its first `limit` (100) items. Every item is validated before any job; a stray shape is refused by name. - **One job, the politeness shared** (`runArchiveOrgBatchImport`, `controller/archiveOrgImport.ts`): the download gap (`archiveOrgGapMs`) before every download after the first across the whole batch, the consecutive-failure count, the rate-limit stop (the rest `notReached`; a re-run resumes), plus **an inter-item gap** (`ARCHIVE_ORG_ITEM_GAP_SECONDS` = 3 s, up to half again at random) before each item's metadata request, on top of the client's 2 s. An item archive.org has no record of is `missing` and passed over. `runArchiveOrgImport` (one item) is the batch of one, with its old result shape plus `restricted`. - **Held is skipped**: `isHeldArchiveOrgRecord` = `destinationExists` (what every download path asks) **or** a saved-video pointer (`isSavedVideo`) — a record whose media lives in the saved-container tier is never fetched again. - **Restricted** (`archiveOrgRestriction`): an item carrying `access-restricted-item`, else each file marked `private` — both answer a download with 401/403. A dry run lists each file as `held`, `RESTRICTED` or `would get`; an import skips restricted files. A download answered 401/403 (`isRefusalError`: the client's "archive.org answered HTTP 40[13]") marks the rest of its item restricted and is not a failure streak; **three such items in a row stop the job** (`refusedStorm`) and back the platform off as a rate limit does. The log ends with `summary: {dryRun, items, planned, imported, held, restricted: [{identifier, files}], failed, missing, notReached, stopped?}`. A dry run is refused, as an import is, while archive.org is held or cooling down (it asks archive.org for metadata and searches). - **`remote-listing {slug}`** (`controller/remoteListing.ts`, job kind `remote-listing`: platform queue, `needsText`, not ingest): an Odysee or BitChute channel's own URL, one `fetchFlatPlaylistUrls` read (the platform's paced args) on `platform:

` — one stream with every other request there — refused while the platform is held or cooling down, after the platform's import floor (`waitForPlatformGap`), noting the floor after; an `EnumerationIncompleteError` (429) backs the platform off. **Nothing is written** (no roster merge, no maybe-missing). The log ends with `@@remote-listing {slug, platform, url, listedAt, listed, unparsed, held, notHeld: [{id, url}], heldNotListed: [id]}`; `pnpm ops get remote-listing ` posts it, waits and prints that JSON on stdout (the transcribe result-marker path). Any other platform is refused, naming its sync. #### Slice A7, as shipped — cues without an index - **`build-cues {slug, ids?, force?, queueKey?}`** → the digest card's `normalizeChannelAction` (now `{ids, force}`; `normalizeAllTranscripts` gains `videoIds` and `force`), on the channel's queue; every id must be held — a stray is refused, named, before any job. - **`GET /api/ops/transcript?id[&slug]`** (`controller/videoCues.ts`, `pnpm ops get transcript [--slug]`): one video's cues off disk, nothing written — a fresh `transcript.cues.json` (`isCuesJsonFresh`), else what normalize would write (`buildNormalizedTranscript`, factored out of `normalizeTranscript`, which now calls it), else the English VTT alone when there is no metadata. Without `slug` the channel holding `data//` is found; two holders are a 409 naming them. Text-guarded (503 when the drive is not answering). - **The MCP's `get_transcript`**, for a video its archive does not hold and no `track` asked, asks that route through the configured editor (`mcp/src/editorOps.ts`, the fetch_clip env and timeout) and renders the cues headed `NOT IN THE ARCHIVE — read from the local editor's disk`, with no moment links. No editor, or a 404, keeps "video not found". #### Slice A8, as shipped — publish edges - `POST /api/ops/publish {verb: "build", allowMissingMedia: true}` carries the flag through `TargetAsk.build` → the plan step → the stage request (`--allow-missing-media`); the docker runner refuses it (its builds do not take it). - **`pnpm ops lane` covers `publish`** — shipped in A4 (`lane` takes `publish`); nothing added. - **`archilyzer publish build [--allow-missing-media] [--out

]`** (`common/bin/publish.ts` `layOutBundle`): after the build — or the no-op of a fresh one — the bundle is laid out in `` with **hard links** where the filesystem allows (a build installs a new bundle and never edits one in place), else copies; symlinks kept. `` is emptied first (its contents) and refused **before any build** where a local deploy's destination would be (`localDestProblem`); `--out` with `all` is refused. Not a deploy: nothing recorded. #### Slice A9, as shipped — MCP archival writes - On the fetch_clip pattern, one existing route each (`mcp/src/archivalTools.ts`): **`get_job`** (`GET /api/ops/job/?tail=N`: status, times, exit code, queue place and head, the log tail); **`enqueue`** (`kind`: `sync` (`full`), `download-missing`, `retry-bucket` (`bucket`, `ids` — the way to download chosen videos), `transcribe-bucket` (`ids`), `fetch-posts` (`full`, `older`), `import-video` (`url`); a key a kind does not take is refused before any request; answers the job id and how to follow it); **`channel_coverage`** (`GET /api/ops/coverage`); **`notes`** (umtool via `UMTOOL_URL`: `list` from `/api/browse/decisions`, `read` from `/api/notes/context`). - **`GET /api/ops/coverage?slug[&gapDays][&from][&to][&list]`** (`controller/channelCoverage.ts`, `pnpm ops get coverage `): every held video dated off disk with `coverageDate()` — the recorded date when the channel has a `recordedDate` title rule and the title yields one, else the upload date — a small `metadata.info.json` parsed whole, a big one read at both ends (16 KB head for `title`, 8 KB tail for `upload_date`); `undated` named, never guessed; per year and month; gaps longer than `gapDays` (30). Text-guarded; nothing written. - **Ruling recorded: a notes reply is not written by the MCP.** umtool's `/api/notes` stamps every write as the operator's (its own rule: no way to claim to be an agent there), so `notes` `reply` answers with the `umtool notes reply "" [--resolve]` command instead of posting. - Settings, storage and deletes are not reachable from the MCP (ruled). `mcp/src/instructions.ts` gains the archive-work step (both plans; only when asked; report the job, do not wait it out) and both heads say the MCP changes nothing itself; README tool table; `UMTOOL_URL`, `WORKER_TOKEN`, `ARCHILYZER_EDITOR_URL` name their MCP readers (ENVIRONMENT.md regenerated). `channel-config`'s help names `recordedDateTitlePattern`. #### Track A gates (A1–A5, on the merged tip) - `pnpm -r --no-bail --workspace-concurrency=1 exec tsc --noEmit` — clean before every commit and on the merged tip (the A2/A2b/A3 commits under the 12:35 throttle ran the editor package's tsc alone; A1, A4, A5 and the tip ran the whole workspace). - **editor unit** 205/205 (159 at base: +5 A1, +4 A2, +19 A2b, +7 A3, +9 A4, +2 A5). **test:scripts** 693: 689 pass, 3 skip, 1 fail — `next-build-trace.test.mjs`'s cwd-path check on the new test helper, fixed by `cde0e759` (the rerun's one failure is `queue-lock.test.mjs` "prints a banner naming the holder", run while another session's e2e held the machine lock; it passed in the gate run). `archilyzer-ops.test.mjs` 46 → 59. **common** 3473: 3469 pass, 4 fail — `fetchPosts.test.ts` and `relocateDir.test.ts` pass alone (load); `autoRunner.test.ts`'s two ("computeLeafPending takes its channel listing from `shared`", "a channel with a move marker projects no work…") fail the same way on an extracted `4cffda3f` tree: not this branch's. +8 here (listJobs 1, windowJobs 3, storage-report 4). **mcp** 292/292. - **`pnpm --filter editor exec next build`** (5 GB scope) — ok. - **e2e** (detached, queued, memory-gated; `export/public` linked to the composed fixture as `r18-integration`'s is — the primary's no longer has `summaries/`, so the export server never answered and the first launch timed out waiting for it): `ops-api jobs fetch-window auto-queue lane-runner disk-space pacing rate-limit auto-subs-replace` — **65 passed, 5 failed, 19.0 min**, at load average ~26–32. ops-api (14/14), fetch-window (9/9), lane-runner, pacing and auto-subs-replace passed. The five failures are 30-s test timeouts before any code this branch changed ran: auto-queue's "UI: build a policy…" (the Start click its own comment calls a race against the poll — it lost, the button went disabled under it), "Drain completes when … parked" (timed out polling for the manual whisper job, before the Drain click), disk-space's "prevents a download…" (the channel page's Download click), jobs' "kicks off the index update" (the index stage still running at 30 s), rate-limit's "an overdue hold…" (`ECONNRESET` from the dev server on a control POST; the same POST with the token passed in the other specs). **Recheck of the five alone** (load ~22, 2.0 min): **4 passed, 1 failed** — the UI Start and the UI Drain (the two buttons A3 moved to server actions), disk-space and rate-limit pass; jobs' index test hit its 30-s test timeout again while its own page snapshot shows the stage's log ending `[stage] update-index _index: Done` — the index stage, which this branch does not touch, finishing late under load. **Tonight, A6–A9 (on `d9318dbe`):** - `pnpm -r --no-bail --workspace-concurrency=1 exec tsc --noEmit` — clean before every commit (A6 94 s, A7, A8, A9). - **common** 3672/3672 (+15 in the files this track added or extended: archiveOrgImport 7 → 12, remoteListing 4, channelCoverage 3, publish-out 3; `_cli`'s publish-row case updated). **editor unit** 237/237 (+9 here: import-archive-org 1, remote-listing 2, build-cues 3, publish +1, coverage 2). **test:scripts** 732: 729 pass, 3 skip (`archilyzer-ops.test.mjs` 59 → 63). **mcp** 303/303 (298 → 303: editorOps 5, archivalTools 5, protocol's tool list). `docs env|files|cli --check` 0. - **e2e** `ops-api.spec` alone, start mode (first build here 41 s, after a 1 min 37 s wait for the heavy slot): **13 passed, 27.7 s**. - Not run (the brief): the full editor suite (the orchestrator's); an editor build of its own beyond the e2e start build; a live `/ask`-style MCP session against the live editor. ### Track B Branch `worktree-agent-a8b654c51bf472562` off `4cffda3f` (r18/integration with main merged), one Opus implementer, its own worktree under `.claude/worktrees/`. Scratch files `b-*` in the job's `tmp`. Slices in the order shipped: B6, B1, B4, B3, B5. **B2 did not ship** (below). No editor e2e was run, as planned. #### Slice B6, as shipped — test economy - **Pure and route-handler e2e moved to unit tests; the editor suite shrinks by 19** (`playwright test --list`, which boots no server: 733 tests in 132 files → 714 in 130). `audio-check-classifier.spec` (6) → `common/ytdlp/ffmpegStreamClassify.test.ts`. `worker-unit.spec` (2) → `editor/app/api/worker/unit/route.test.ts`: the four handlers in-process, a temp corpus, an ollama stub. `view-route.spec` 12 → 1; the rest → `editor/app/api/view/[name]/route.test.ts`: the rewrite table in `next.config.ts` read as data (each old path a rewrite to its view, every view one; the test sets a global `__dirname` for the config, which uses it), the dispatcher's 404 for unknown names and for every `/api/test/*` directory, and `/api/widget/presets` untouched. The test that stays e2e is the query string surviving a rewrite (`/api/pulse?rev=`): only a running Next shows that. - **`media-file-abort.spec` stays e2e.** Cancelled bodies run against the handler in-process pass even with `Readable.toWeb` or a naive enqueue-after-cancel wrapper (both tried), so the race happens in the server's response pipeline, and only the HTTP round trip tests it. Added beside it: `files/[name]/route.test.ts` (a range, a suffix range, a bad range returns the whole file, traversal is a 400, a missing file a 404) and `common/lib/safeStreamController.test.ts` (a raw controller throws `ERR_INVALID_STATE` once it is closed; the guard goes quiet). - **Root `pnpm test`** runs every package's unit suite and then `test:scripts`, and is non-zero if either fails. **Root `pnpm typecheck`** runs the tsc sweep. Both pass `--no-sort`. Under pnpm 11, `--no-bail` alone still SKIPS every dependent of a failed package: a red `common` ran no editor, export, homepage or mcp test, and no `tsc` in them. **The documented gate `pnpm -r --no-bail --workspace-concurrency=1 exec tsc --noEmit` has the same hole.** Not changed here: it is gate text in the plans and the rules (Track C's). `pnpm typecheck` is the corrected spelling. - **The editor's `lint` script is removed.** The editor had no eslint config. With export's config, eslint finds 35 errors in 25 editor files (17 `react-hooks/set-state-in-effect`, 7 `react/no-unescaped-entities`, 4 `react-hooks/purity`, 4 `react-hooks/refs`, 3 `@next/next/no-html-link-for-pages`). Each fix would change how a UI file behaves. #### Slice B1, as shipped — the heavy slot (`pnpm heavy`) - `scripts/queue-lock.mjs` adds a second machine-global lock, `/heavy-queue.lock`, and a memory floor. Once the slot is held, the run waits until `/proc/meminfo` MemAvailable ≥ `HEAVY_MIN_FREE_MB` (6000). The slot is taken first and the floor checked second, so no one slips in while the holder waits for memory. `pnpm heavy -- ` is `queue-lock.mjs --heavy`, and pnpm's own `--` is accepted. - **Decision: e2e takes the heavy slot too, FIRST, then the e2e queue.** With one order everywhere, nesting cannot deadlock. `HEAVY_HELD` lets a heavy command inside one pass through, as `QUEUE_LOCK_HELD` does. `E2E_QUEUE=0` skips both locks and keeps the floor. The e2e queue's own banner, semantics and bypasses are unchanged. Covered entry points: every package's `e2e` scripts (the CLI) and `run-sharded-e2e.mjs` (`withQueue`). - **The publish stages' real `next build`** (a site, the hub, the homepage) runs through the slot (`build.ts` `heavyGated`). The e2e fake `EXPORT_NEXT_BIN` does not. A heavy run forwards SIGTERM/SIGHUP to its command, so a stage's Cancel, which signals the wrapper, still stops the build. **`docker/build-site.sh` is not edited.** Inside a docker-runner container the gate runs through the same code (`nextBuildStep`). The slot there is the container's own, and the floor reads the host's meminfo, so a fan-out slows down when the host runs low but is not serialised. - **Renders:** documented as `pnpm heavy -- node umtool/report-to-video/build-video.mjs …`. The in-file wiring was B2 (not shipped). **"A render holds the transcription lane"** is a documented recipe (WORKTREES.md) with `pnpm ops lane` hold/release, which exists. It is not automated. - A waiter is told who it waits behind and what they are running. A machine whose MemTotal is under the floor runs with a note. With no usable `flock`, the gate warns and enforces the floor only. Bypasses: `HEAVY=0`, `HEAVY_MIN_FREE_MB`, `HEAVY_TIMEOUT`. Test seams: `HEAVY_LOCK_FILE`, `HEAVY_MEMINFO_FILE`, `HEAVY_POLL_MS`. ENVIRONMENT.md is regenerated (`docs env`), and WORKTREES.md and AGENTS.md each get a section. - **Manual collision on the real lock:** a second `pnpm heavy` printed `waiting for the heavy slot — held by agent-a8b… (…, pid …) for 4s: sleep 6` and ran when the first finished. The floor seen live: while the parent's suite was up, `pnpm heavy -- echo` waited at 2.8 GB available. #### Slice B4, as shipped — `ops transcribe`: the engine's time, and a short file first - `durationMs` now runs from the moment a worker takes the job (`onWorker`, the last attempt's) to the transcript. A new `waitedMs` is the queue time. The job's log gives both. - A cut of ≤ 15 min of audio (`URGENT_MAX_AUDIO_SEC`, measured from the cut WAV's size) acquires a worker in the pool's existing `urgent` tier. It goes ahead of parked manual batches as well as the lane (which was already `background`). A longer file keeps `foreground`. The tier orders waiters only, so nothing running is interrupted. `transcribeWithWorker` takes `tier`. #### Slice B3, as shipped — `reports check`, `verify-quotes`, `attach-video` - `archilyzer reports check [--reports a,b] [--allow-missing-media]` is `resolveSiteReports` with nothing written. It exits 1 and prints compose's own problem lines. `--reports` covers drafts that are not in site.json. - `archilyzer reports verify-quotes [--json]` runs compose's quote check on each citation and prints the best track plus the `en-orig` track's score. It reports `en-orig-drift` when a served `en` track matches the quote and en-orig does not. The span check is now ONE function, `checkSpanQuote`, which compose also calls. - `archilyzer reports attach-video