Archilyzer · Source

archilyzer

Archilyzer
git clone https://archilyzer.pages.dev/source/archilyzer.git
Log | Files | Refs | README | LICENSE

commit 25db381f5bf3533f0a68d29ad9272fc1668a31e5
parent bf3b864dc40c72489b01aefa9d8d25e9b9a3c56b
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date:   Mon, 24 Aug 2026 23:07:00 -0400

plans: STATE/FACTS catch up to the arbiter, workers and executors; track the relocate design

STATE.md said "last updated 2026-08-12" and had no entry for the 20
commits from 08-19 to 08-24 — recency ordering, the arbiter and lane
guards, the pipelines band, /channels, worker tags/slots, LLM fan-out,
unit executors, the sweep scope console. README.md's context-clear
protocol was skipped twice. The new header block is the catch-up,
pointing at commit bodies for the measurements rather than paraphrasing
them; the 08-12 narrative stays under a dated heading. The Phase 9 row
now says attribution-text IS on (11,337 videos, ~194k chunk calls,
1 done), which 5215b91 surfaced.

FACTS.md gains the facts a cold agent would re-derive: band.ts's
directive-free rule and its guard test, the `${operation}\0${id}` claim
key, the arbiter's empty queueKey and armed-sweep refusal, the five
never-summed SweepKindCounts populations (and the tsc-vs-node-test gap
that let main go red), workerMatches, the /api/worker/unit door guard,
the 14.5% dir-name/metadata-id divergence, and that editor has no lint
config.

relocate-channel-media.md was written 2026-08-24 and never added.
Designed, not started; prerequisite is mounting sdb1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Diffstat:
Mplans/FACTS.md | 54++++++++++++++++++++++++++++++++++++++++++++++++++++++
Mplans/STATE.md | 100+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----
Aplans/relocate-channel-media.md | 234+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 384 insertions(+), 4 deletions(-)

diff --git a/plans/FACTS.md b/plans/FACTS.md @@ -2032,3 +2032,57 @@ ms/video** (773-video channel), i.e. 0.7x–1.2x the `readVideoFiles` the snapsh given `createDownloadProgressParser()` by `common/jobs/taskHooks.ts`, which matches none of it. The zero-samples line prints a BARE index (`window 2 decoded zero samples`) with no `/N`, so it correctly matches nothing. + +## Verified 2026-08-24 — the arbiter, the band, the workers + +Facts a cold agent would re-derive expensively, from the 08-19 → 08-24 commits that landed +with no STATE.md entry. Commit bodies (`git log -1 --format=%b`) carry the measurements. + +**`band.ts` must carry no directive and no value import** (`editor/app/components/pipelines/buildBands.test.ts:333` pins both). `FlowStation` +is a server component and CALLS `bandSentence()`; a `"use client"` on the module makes every +channel page 500 at request time — and `pnpm build` passes, because the route is force-dynamic +and nothing prerenders it. `StateBand.tsx` exports only components for the same reason. + +**The auto-queue claim key is `${operation}\0${id}`** (`common/jobs/autoQueuePolicy.ts:366`). The same video is +legitimately pending for digest AND diarization; an id-keyed set lets whichever leaf runs first +steal the other's work. Every pre-existing tree names no operation, so its keys are `"\0"+id` +and the dedup is byte-identical to before. + +**The arbiter runs on `queueKey: ""`** (`common/controller/arbiter.ts:395`) — a long-lived job that WAITS on jobs needing +the real keys must not hold one or it deadlocks against its own work. It reserves by +`lane.queueKey`, never invents one, dispatches one unit per lane per pass, one job per CHANNEL +(77,000 per-video jobs would evict the registry's 100 records), and **refuses to start beside an +armed sweep** (`common/controller/arbiter.ts:205`) because two dispatchers on one lane start the same channel twice. + +**`SweepKindCounts` carries five populations that are never summed** (`common/lib/sweepPlan.ts:37`): `reachable`, +`missingInput`, `blocked`, `deferred`, plus `eligible` / `present` as `number | null` — null +poisons a sum deliberately, because a partial denominator smaller than its numerator is a worse +lie than "unknowable". Reachable and needs-media differ by ~91× on this corpus. + +**A test fixture of that type must carry all five** — `common/controller/sweepPreview.test.ts:258` had three literals with the +old four fields after `dee7500` widened the type; `tsc` failed on `main` while `node --test` +stayed 821/821, because the test runner does not typecheck. Run `tsc` per package before +calling a session green. + +**Worker tags route work through one rule** (`common/lib/workers.ts:136`): untagged matches everything, tagged +matches on intersection, a requirement lists acceptable qualifications. The pool's +`eligible / pickFree / hasEligibleWorker` and the sweep console's "also runs on …" line both go +through it, so display and routing cannot disagree. An `llm` worker never matches a +requirement-less lease, so transcription can never land on a bare `ollama serve`. + +**`/api/worker/unit` refuses anything that is not a backfill kind** (`common/controller/workerServer.ts:232`, `editor/app/api/worker/unit/route.ts:41`) — +download politeness and the transcription pool stay single-scheduler. A unit's `inputs` are +ordered with `transcript.cues.json` LAST so the executor's scratch materialization passes the +`isCuesJsonFresh` mtime gate. `applyResult` runs on the PRIMARY through the guarded writers; +only `diarization.json` is copied verbatim. + +**A directory name is not a metadata id on ~14.5% of this corpus** (recency, `efea56f`): +`the-quartering-rumble/data/v1007ay` holds `vxe1ae`. Every scheduling path hands +`buildRecencyKeys` DIRECTORY names, and the transcript index is keyed by `summary.id`, so a +corpus-wide flat map lets channel A's dir name inherit channel B's date. The lookup is scoped +through `datesBySlug` when `owner` is supplied; every scheduling path supplies it. (Same trap +already recorded for `community-notes/v2fkbw7` → `v2cywen` under Phase 2.) + +**`editor`'s `lint` script has no config** — `eslint` v9 finds no `eslint.config.*` in +`editor/` or the root; the root `lint` delegates to `export`. `tsc` + e2e are the editor's +verification, not lint. diff --git a/plans/STATE.md b/plans/STATE.md @@ -3,9 +3,101 @@ The working memory for the local-AI derived-corpus work. Rewritten at the end of every session, before context is cleared. See [`README.md`](README.md) for the protocol. -**Last updated:** 2026-08-12 — **the digest bake-off can now measure QUALITY, not just -defects; and the sweep was running 8× slower than its own projection.** Uncommitted on -`main`. See [`bakeoff/speakers-round1-notes.md`](bakeoff/speakers-round1-notes.md) and +**Last updated:** 2026-08-24 — **`main` is green again, `feat/channel-groups` is merged, and +this file caught up with two sessions it had no entry for.** The 2026-08-12 session below +landed in `7d32438` on 08-18; everything from 08-19 to 08-24 — recency ordering, the +arbiter, lane guards, the pipelines band, worker tags and slots, LLM fan-out, unit +executors, the sweep scope console — shipped on `main` with no STATE.md entry. The +context-clear protocol in [`README.md`](README.md) was skipped twice; this block is the +catch-up. Commit bodies are the primary record and carry the measurements — read them +with `git log -1 --format=%b <sha>` rather than trusting a paraphrase here. + +## What landed 2026-08-18 → 08-24 + +- **Chapter-boundary quality metric** (`7d32438`) — the 08-12 narrative below, committed. + `lib/boundaryScore.ts`, `--score-existing`, and the READ-RECALL-NOT-PRECISION finding. +- **Recency ordering, four commits** (`74f8e51` → `efea56f` → `72745fe` → `bddae39`). A + per-runner `order` (listed | newest | oldest) sorts within a rule; `digest.recencyOrder` + and `backfill.order` do the same inside a batch; `reach` ("channel" | "corpus") extends it + to the CHANNEL plan. The date comes from a layered key: a key-only scan of `index.mdb`'s + `byChannel` (~300 ms for 78,583 videos, but only videos WITH a transcript — 122 of 870 + pending), then an 8 KB tail read of `metadata.info.json` (868/870, 0.19 ms each, + memoized), then playlist position marked `estimated`. **Directory name ≠ metadata id on + ~14.5% of this corpus** (`the-quartering-rumble/data/v1007ay` holds `vxe1ae`), so the + index lookup is scoped to the owning channel or channel A's dir name inherits channel + B's date. Measured: 11,329 candidates ordered in 118 ms; the sweep re-plans in ~1.5 s. + `/auto-queue` became a dispatcher board saying what it would run next and why. +- **Operation leaves, lane guards, the arbiter** (`3299a53`, `09d4598`, `ab68635`) — + [`unified-operations-model.md`](unified-operations-model.md) steps **2, 3, 4 DONE; 1, 5, + 6 open**. (That file's "designed, not built" header is stale by three steps; its step + list is current.) `AutoQueueMatch.operation` beside `bucket`, claim key + `${operation}\0${id}` so digest and diarization cannot steal each other's work; + `common/controller/laneGuards.ts` holds digest's six rules as a preflight (throws) and a + per-pull gate (holds, never stops); `common/controller/arbiter.ts` runs on `queueKey ""`, + one unit per lane per pass, one job per channel, re-plans every pass, and REFUSES beside + an armed sweep. Not persisted across restart — that is step 6's. +- **The console and the band** (`1f2b7c2`, `43173c8`, `7fa975d`) — `/auto-queue` is one + comparison rail over four pipelines; `components/pipelines/` holds the five-population + band (reachable / needs-media / blocked / deferred / present — never summed, ~91× apart on + this corpus). `band.ts` must stay directive-free and value-import-free: a server component + calls its pure readings, and `"use client"` on it 500s every channel page at request + time while `pnpm build` passes — caught by e2e, now pinned by a unit test. +- **/channels, `group`, `costBasis`, and a word that was three things** (`4d57a5e`, + `a5c4e48`, `07227ea`) — six bands per channel row; on the live corpus every row draws the + same stripe and that uniformity IS the finding. Operations declare `group` and + `costBasis`. **The fact that hid: `attribution-text` is ON, reachable on 11,337 videos of + one channel, its unit is the transcript CHUNK (~194,000 model calls corpus-wide), and it + has completed ONE video** — it read as a quiet row because "11,337 reachable" looks the + same whether the unit is an audio pass or a per-chunk call. "Backfill" no longer names a + figure anywhere; it is a queue key, listed with its members only where the shared pause + and sweep live. +- **Workers: tags route, slots multiply, endpoints fan out, executors run units** + (`1ff2256`, `549c77b`, `2fcd08f`, `16004ac`, `eee3a07`). `Worker.tags` is consulted + (`common/lib/workers.ts`); a remote is N slots, blank N filled from its own + `/api/worker/health` through `controller/remoteCapacity.ts`; `WorkerKind "llm"` is a box + running nothing but `ollama serve`, leased per call with only `baseUrl` swapped (outside + the freshness identity → zero churn), verified against `/api/tags` for the exact model + tag; every backfill kind declares `inputs / outputs / applyResult` and + `/api/worker/unit` runs one unit on a corpus-less box (`controller/remoteUnit.ts`, + `workerServer.startWorkerUnit`), refusing anything that is not a backfill kind. Executor + deployment is in RUNNING_IN_DOCKER.md. `worker-unit.spec.ts` 6/6. +- **Sweep scope console** (`dee7500`, `215d4e3`) — pick operations and channels before + arming; scope is written by the ARM action only; a read-only "also runs on …" line per + row derived through the same `workerMatches` the pool grants by. +- **Channel groups on /channels** (`66ec9eb`, `421a13c`, merged 2026-08-24 from the + `feat/channel-groups` worktree, which had sat uncommitted since 08-11). One table + sectioned by the site's authored groups, each header carrying that group's slice of the + pipeline as stations whose figure IS the button. Rebased over `4d57a5e`: the branch's row + component now renders main's `PipelineCell` columns, sections join to rows by slug, and + the header's `colSpan` is `8 + columns.length` rather than a literal. + +**Housekeeping, 2026-08-24.** `common` had failed `tsc` since `dee7500` (three test +fixtures missing the widened `SweepKindCounts` fields; the node runner does not typecheck, +so 821/821 hid it) — fixed in `6ea5ea2`. The `flexsearch-spike` worktree +(`feat/client-flexsearch-index`, last touched 06-27, superseded by +`common/components/searchIndex.worker.ts`) was **abandoned at `b4dee06`** — reflog only. +74 already-merged local branches deleted; the empty May stash dropped. +[`relocate-channel-media.md`](relocate-channel-media.md) is now tracked: designed and +verified against the tree, **not started**, and its real prerequisite is operational — +`sdb1` (1.8 T platter) is not mounted. **Not covered here:** `umtool` (27 commits on 08-18, +a separate workspace package) is documented in `umtool/docs` and AGENTS.md. + +**Recommended next** (supersedes the list under "Pre-sweep decisions", which still +describes the reasoning): + +1. **GPU yield on a quiet box** — still the gate on arming the sweep, still unmeasured. +2. **Unified-ops step 1**: collapse `noDigest` into `snapshot.backfill.digest` — the step + Phase C skipped, and now the arbiter can act on the result. +3. **Relocate `omnimirror`'s media** once the platter is mounted (131 GB off a 94%-full SSD). +4. **Phase 6 Ollama `/ask`** — genuinely independent; a good parallel task. +5. Decide `attribution-text`'s fate on that one channel: ~194,000 chunk-level calls is a + sweep-sized commitment that was never priced. + +--- + +## Session 2026-08-12 — landed in `7d32438` on 2026-08-18 + +See [`bakeoff/speakers-round1-notes.md`](bakeoff/speakers-round1-notes.md) and [`digest-context-review.md`](digest-context-review.md). **EVERY METRIC THE BAKE-OFF HAD WAS A DEFECT COUNTER.** `zeroYieldRate`, @@ -289,7 +381,7 @@ again — server modules hot-reload under `next dev`, but a job in flight holds | 6 · Ollama `/ask` provider | not started | **No dependencies.** | | 7 · Tags + chat highlights | not started | Tag generation is now VERIFIED WORKING (it had never run once — 0 of 109 sidecars). 11 videos, 0 warnings, metrics in `digest-validate.ts`, e2e in `digest.spec.ts`. Nothing consumes it yet. Known gap for this phase: only 2.4% cross-video tag reuse, and `antisemitism`/`anti-semitism` are two tags — the vocabulary does not converge, so a consumer needs normalization. | | 8 · Visibility policy | not started | | -| 9 · Attribution + quote filtering | **capture + both attribution lanes landed, nothing armed** | Capture (`diarization.json`) landed 2026-08-07 with the cleanup guard. Attribution followed the same day: `attribution.json`, both lanes (`attribution-diarized` ≈ 1 call/video, `attribution-text` ≈ 1 call/chunk), registered as the second and third **backfill kinds** — so the lane, the share, the sweep and all four indicators came for free. Both lanes are **OFF by default under an off-by-default master switch**; nothing runs until a pilot prices it. Still not started: `attributionMs` through the build, viewer badges, per-channel export counts, and quote filtering (which also depends on Phase 8). PLAN.md's bespoke "upgrade job" is **not needed** — `attribution-diarized` reports a text-only record as `missing`, so the existing lane queues the upgrade, and media re-acquisition is already `controller/backfillReacquire.ts`. Spike results in `plans/diarization-spike-results.md`. | +| 9 · Attribution + quote filtering | **capture + both attribution lanes landed, nothing armed** | Capture (`diarization.json`) landed 2026-08-07 with the cleanup guard. Attribution followed the same day: `attribution.json`, both lanes (`attribution-diarized` ≈ 1 call/video, `attribution-text` ≈ 1 call/chunk), registered as the second and third **backfill kinds** — so the lane, the share, the sweep and all four indicators came for free. Both lanes are **OFF by default under an off-by-default master switch**; nothing runs until a pilot prices it. **Update 2026-08-22 (`07227ea`):** `attribution-text` IS on, reachable on 11,337 videos of one channel (~194,000 chunk-level calls), 1 completed; `costBasis` now prints the unit beside the backlog wherever it is armed. Still not started: `attributionMs` through the build, viewer badges, per-channel export counts, and quote filtering (which also depends on Phase 8). PLAN.md's bespoke "upgrade job" is **not needed** — `attribution-diarized` reports a text-only record as `missing`, so the existing lane queues the upgrade, and media re-acquisition is already `controller/backfillReacquire.ts`. Spike results in `plans/diarization-spike-results.md`. | | 10 · Lead with the derived corpus | not started | | | 11a · Review queue | **minimum landed** | Total failures now persist warnings (they used to persist none), `buckets.digestWarnings` + a `digest_warnings` filter + an /actionable section, and duplicate-cluster confirm/reject. Approve/dismiss state deferred — needs new persistence. | | 11b · Viewer feedback | not started | | diff --git a/plans/relocate-channel-media.md b/plans/relocate-channel-media.md @@ -0,0 +1,234 @@ +# Relocate a channel's media to another drive + +**Ask:** designate a channel's downloaded videos (`transcripts/channels/<slug>/data/`) to live +in another directory or drive, so a disk-heavy, mostly-redundant channel like `omnimirror` +(131 GB of 396 GB in `channels/`, 718 mp3s) moves off the SSD onto the larger platter drive. + +**Box today:** `/home` (nvme, 1.5 T) is at **94 %, 87 G free**. The platters `sda1` (466 G, +ext4) and `sdb1` (1.8 T, ext4) are present but **not mounted** (`lsblk`). Mounting one is a +prerequisite of the rollout, not of the code. + +--- + +## What the tree says (verified 2026-08-24) + +| Fact | Anchor | +|---|---| +| There is **no** `channelDir()`/`videoDir()`/`dataDir()` helper. `"data"` is joined inline at **~74 non-test call sites**, most with only `(paths, slug)` in scope — no config. | `common/ytdlp/runYtdlp.ts:563,1048,1331,1445`, `downloadOneManaged.ts:435…`, `controller/channelSnapshot.ts:555`, `channels.ts:156,175,230`, `buildIndex.ts:300,1113`, `autoRunner.ts:936`, `editor/app/channels/[slug]/videos/page.tsx:137`, … | +| yt-dlp output templates are **relative** (`-o data/%(id)s/…`) and the download dir is set purely by `cwd: channelDir`. No `-P`/`--paths`. | `runYtdlp.ts:256-262, 339, 1653`; `downloadOneManaged.ts:312` | +| `archive`, `playlist`, `config.json`, `snapshot.json`, `roster.json`, `failed-transcriptions` all live in the channel dir, **not** in `data/`. The archive is appended by the app, not by `--download-archive`. | `downloadOneManaged.ts:417` | +| **Zero** symlink-aware code (`lstat`/`readlink`/`realpath`/`symlink`) in `common/` and `editor/`. Every reader goes through `path.join(channelDir, "data", …)` and would follow a symlink transparently. | grep | +| Channel listing filters `isDirectory()` on `channelsDir` entries — a symlinked **channel dir** would vanish. Only `data/` may be a link. | `channels.ts:285`, `buildIndex.ts:284`, `site.ts:380` | +| Snapshot swallows a missing `data/` as "no videos": `readdir(dataDir).catch(() => [])`. An **unmounted** drive would therefore read as *everything undownloaded*. | `channelSnapshot.ts:576` | +| `deleteChannel` is `rm(dir, {recursive})` — on a symlinked `data/` that removes the **link**, orphaning the media on the other drive. | `channels.ts:398-405` | +| `renameChannel` renames the channel dir (a symlink inside moves fine) and already has an EXDEV message for the saved-video store. | `renameChannel.ts:80-97` | +| LMDB index stores ids + mtimes, **no paths**. A move that preserves mtimes (`rsync -a`) needs no reindex. | `buildIndex.ts:181,462` | +| Low-disk gate is hard-wired to `paths.transcriptsDir`; `checkDiskSpaceFor(dir)` exists but nobody passes a dir; the hysteresis latch is one global. | `common/lib/diskSpace.ts:57-61, 207` | +| `export/` never reads `data/`; `buildIndex` reads captions/metadata only. | agent survey §7 | +| **Precedent for exactly this shape:** the saved-video store — `ChannelConfig.savedVideosDir` per-channel override, `savedVideoRoot()`/`savedVideoDir()` resolver, `moveFileCrossDevice()` (rename → EXDEV → copy+atomic rename), rsync mirror job with `execa` + `cancelSignal`, real-rsync unit tests. | `channelConfig.ts:96-101`, `savedVideo.ts:52-61`, `savedVideo-server.ts:66-82`, `controller/backupSavedVideos.ts`, `backupSavedVideos.test.ts:30` | +| Job launch template: `runManagedFunction({kind, queueKey, paths, fn(onLog, signal)})`; a kind is one entry in `JOB_KINDS`. | `editor/app/saved-videos/backupActions.ts`, `common/jobs/jobKinds.ts` | +| Rename refuses while the channel has running/queued jobs — the guard to copy. | `editor/app/channels/actions.ts:463-473` | +| `/cleanup` and channel stats are per-file `stat` through the data path — unaffected by a link. | `channelSnapshot.ts:647`, `editor/app/cleanup/lib/loadCleanup.ts:31` | + +--- + +## Decision: symlink is the mechanism, `config.json` is the record, a guard makes it safe + +Two ways to do this: + +**A. Resolver migration** — add `ChannelConfig.dataDir` and a `channelDataDir(paths, slug, config)` +helper, then rewrite all ~74 call sites (and add `-P`/absolute `-o` for yt-dlp). Correct in +principle, but most sites don't have the config in hand, the runners touch production data +unattended, and **one missed site silently reads an empty dir** — the auto-runner would then +"re-download" 131 GB onto the SSD. High blast radius for a feature that moves bytes. + +**B. Symlink** — the job copies `data/` to the other drive, verifies, swaps in +`channels/<slug>/data -> <root>/<slug>/data`, and records the target in `config.json`. +Every existing reader, yt-dlp's `cwd`-relative writes, the LMDB index and the export build +keep working with **no call-site changes**, because the on-disk contract +(`channelDir/data/<id>/…`) is preserved. The known hazards are three and are each addressed +below: an unmounted drive (guard), `deleteChannel` orphaning the target (fix), and the disk +gate measuring the wrong volume (thread the dir). + +**Go with B.** It is also exactly how an operator would do it by hand today, minus the +verification, the guard and the UI. A can be layered on later if a second mechanism is ever +needed (a networked store, say); the config field designed here is the same one A would use. + +### Layout + +- Target: `<root>/<slug>/data/` — `<root>` is any directory the operator picks + (e.g. `/mnt/platter/archilyzer-media`); the `<slug>/data` suffix is fixed so one root can + hold many channels and the shape mirrors the saved-video store (`<root>/<slug>/<videoId>`). +- `channels/<slug>/data` becomes an **absolute** symlink to that target. +- `config.json` gains `dataDir: "<root>/<slug>/data"` (the link target, stored resolved). + Blank/absent = in place. Written **only by the relocate job on success** (same rule as the + sweep console: the record is the output of the action, never a free-text field that can + drift from disk). +- Marker while a move is in flight: `channels/<slug>/.relocating.json` + `{ target, direction: "out"|"back", startedAt, phase }`. Removed on completion. Its presence + means "media is in transition" to every guard below, and lets an interrupted job resume. + +--- + +## Changes + +### 1. `common/lib/channelMedia.ts` (new) — the one place that knows about relocation + +```ts +export type ChannelMediaLocation = { + dataDir: string; // always channelDir/data — the path readers use + relocated: boolean; // config.dataDir set + target?: string; // config.dataDir + status: "in-place" | "ok" | "unreachable" | "in-transition" | "inconsistent"; + detail?: string; // "target /mnt/… does not exist (drive not mounted?)", … +}; +export async function inspectChannelMedia(paths, slug, config): Promise<ChannelMediaLocation> +export async function assertChannelMediaReachable(paths, slug, config): Promise<void> // throws ChannelMediaUnreachableError +export function relocatedDataDir(root: string, slug: string): string // <root>/<slug>/data +``` + +`inspect` = `lstat(data)` (is it a link? does that agree with `config.dataDir`?) + `stat(target)` +(exists, is a directory) + `.relocating.json` presence. Because the link points at a *deep* +path, an unmounted root gives `ENOENT` — an empty mountpoint dir is not mistaken for the +media. Pure and cheap: two stats. Unit-tested with tmp dirs. + +### 2. Guard the readers that would otherwise see "nothing downloaded" + +- `channelSnapshot.ts:576` — before the `readdir(dataDir)`, if `config.dataDir` is set and + `inspect().status !== "ok"`, **throw `ChannelMediaUnreachableError`** instead of computing a + snapshot over an empty dir. The scheduler already treats a failed refresh as "keep the last + snapshot.json" (`snapshotScheduler.ts:97,205`); verify that path logs the reason. +- `autoRunner.ts` — skip a channel whose media is not `ok` (log once per state change), so + unattended download/transcribe/clean never runs against a dangling link. +- `undownloadedVideos.ts:28` and the per-channel manual actions + (`pipelineActions.ts`, `whisperActions.ts`, `persistActions.ts`, `bulkVideoActions.ts`, + `videoActions.ts`) — call `assertChannelMediaReachable` at the top; the error becomes the + action's `{ ok: false, error }`. +- Channel page + dashboard: a banner/badge from `inspect()` — "Media relocated to `<root>` + (131 GB) · reachable" / "**unreachable** — mount the drive". Cheap, render-safe (two stats). + +### 3. `common/controller/relocateChannelMedia.ts` (new) — the move, both directions + +`relocateChannelMedia({ paths, slug, direction: "out", root, onLog, signal })`: + +1. **Preflight** (all throw with an operator-readable message): channel exists and is not + social; no `.relocating.json` from a *different* target; `root` exists, is a directory, + is writable; target `<root>/<slug>/data` is absent **or** a partial from an earlier aborted + run (resumable); free space on `root` ≥ bytes in `data/` + `resumeMarginGB` (walk with + `stat`, like the snapshot does; `getFreeBytes(root)` from `diskSpace.ts`); source `data/` + is a real directory (not already a link — moving root→root is `direction:"out"` from the + current *target*, handled by resolving the real source first). +2. **Write the marker**, phase `copy`. +3. **Copy**: `rsync -a --partial --info=progress2 <src>/ <target>/` via `execa(paths.rsyncBin, …, { cancelSignal: signal })`, + exactly as `backupSavedVideos.ts:78-90`. `-a` preserves mtimes → no LMDB reindex. Abort + leaves the source untouched and a resumable target. +4. **Verify**: `rsync -a --dry-run --itemize-changes` must print nothing; then file count and + byte sum must match on both sides. Any drift → fail, source untouched, marker stays + (rerun resumes at step 3). +5. **Swap** (phase `swap`): `rename(data, data.relocated-<ts>)` (same device, atomic) → + `symlink(target, data)` → `writeChannelConfig({ ...config, dataDir: target })`. +6. **Reclaim** (phase `reclaim`): `rm -r data.relocated-<ts>`; remove marker; log bytes + moved and free space on the source volume now. + +`direction: "back"`: copy `target/` → `channels/<slug>/data.incoming/`, verify, `unlink(data)`, +`rename(data.incoming, data)`, clear `dataDir`, `rm -r target`, remove marker. Same code, +reversed endpoints. + +**Crash recovery** is the marker's `phase`: a rerun with a matching marker re-verifies and +continues from `swap`/`reclaim`; `inspect()` reports `in-transition` until it finishes. The +sibling `data.relocated-*` / `data.incoming` names are deliberate so nothing else ever +mistakes them for the live dir. + +### 4. Job kind + action + +- `common/jobs/jobKinds.ts`: `"relocate-channel-media"` — label "Relocate channel media", + `drainable: false`, `replayable: false`, `queueKeyStrategy: "custom"`. Queue key = + `channelQueueKey(slug)` (`common/lib/queueKeys.ts:34`) so it serializes with the channel's + own jobs. +- `editor/app/channels/[slug]/storageActions.ts` (new, `"use server"`): + - `previewRelocationAction(slug, root)` → `{ bytesToMove, freeOnRoot, freeOnSource, sameDevice, existingPartial, error? }` (`statfs` both sides; `sameDevice` via `stat().dev` — a warning, not a block). + - `relocateChannelMediaAction(slug, root)` / `moveChannelMediaBackAction(slug)` — the + rename-style active-jobs guard (`actions.ts:463-473`), then `runManagedFunction`. +- `deleteChannel` (`channels.ts:398`): if `config.dataDir` is set, `rm -r` the target first + (and the `<root>/<slug>` parent if it is then empty), then the channel dir. Refuse when + a `.relocating.json` marker is present. +- `renameChannel.ts`: after the channel-dir rename, if `dataDir` matches + `<root>/<oldSlug>/data`, rename `<root>/<oldSlug>` → `<root>/<newSlug>`, re-create the + symlink, rewrite `dataDir`; roll back the dir rename on failure (same rollback shape as + `:94`). A target that doesn't follow the convention is left alone (still works — the link is + absolute). + +### 5. Disk gate measures the right volume + +`diskGate(paths, settings, { mode, dir? })` — `dir` defaults to `paths.transcriptsDir`. Thread +the channel's `data` path from the callers that write media for a known channel: +`runYtdlp.ts:876`, `downloadOneManaged.ts` (root is in scope), `autoRunner.ts:654`, +`backfillReacquire.ts:79`, `pipelineActions.ts:48`, `fixIncompleteTranscript.ts:78`, +`videoActions.ts:266,301`. The latch (`gateLatched`, `diskSpace.ts`) becomes a +`Map<dir, boolean>` so a full SSD does not pause downloads that land on the platter and vice +versa. `persistKept`/saved-video callers keep their current dir. Tests in `diskSpace.test.ts` +extend the existing hysteresis cases with two dirs. + +### 6. UI + +- **Channel page** (`editor/app/channels/[slug]/components/`): a "Storage" panel — + location (in place / target path), status from `inspect()`, media bytes (from the loaded + snapshot, no new walk), free space on the current volume; a "Move media to…" form + (root path input → live preview from `previewRelocationAction` → confirm button that + starts the job), and "Move back in place" when relocated. Disabled with a reason while the + channel has jobs or a marker is present. +- **`ChannelForm.tsx`** (`:641-647`, next to "Saved-video store dir"): a read-only line + "Media location: <path> — change from the Storage panel". Not an input (see Layout). +- **Channels list / dashboard**: a small "relocated" badge; red when unreachable. +- **`/cleanup`**: nothing to change; the ledger already reads snapshots. Optionally show the + channel's volume next to its reclaimable bytes. + +### 7. Docs + +- `AGENTS.md` corpus table: `data/` may be a symlink to `config.dataDir`; never symlink a + whole channel dir (`isDirectory()` filters); the marker file name. +- `RUNNING_IN_DOCKER.md`: the corpus is a named volume, so a relocated target must be + bind-mounted into the container **at the same absolute path** (e.g. + `- /mnt/platter/archilyzer-media:/mnt/platter/archilyzer-media`), otherwise the guard reports + the channel unreachable (by design — it does not fall back to empty). +- `WORKTREES.md` / shard notes: syncing `channels/` with `rsync -a` copies the link as a link; + use `--copy-links` (or sync the root too) when a shard must carry the media. Executors that + need no corpus are unaffected. +- `editor/CHANGELOG.md` entry. + +--- + +## Tests + +- **Unit (`common/`, vitest, real rsync like `backupSavedVideos.test.ts`)**: + `channelMedia.test.ts` (in-place / ok / dangling target / marker present / link disagrees + with config); `relocateChannelMedia.test.ts` — out: link + config + mtimes preserved + + source gone; abort mid-copy (abort the signal from an `onLog` hook) leaves source intact and + target resumable; rerun completes; verify-failure keeps source; back: restores a real dir + and clears config; `deleteChannel` removes the target; `renameChannel` re-points a + convention-shaped target and rolls back on failure; `channelSnapshot` throws on unreachable + rather than writing an empty snapshot; `diskGate` per-dir latch. +- **e2e (one spec, `editor/e2e/channel-storage.spec.ts`)**: relocate the fixture channel to a + tmp root from the Storage panel, wait for the job, assert the badge, the videos list still + lists videos and a transcript file route still serves; move back. Runs under the global e2e + queue (`E2E_QUEUE`) — expect to wait. +- `pnpm exec tsc --noEmit` in `common/` and `editor/`; `pnpm -C common test`. +- **Offline dry-run on the real corpus** (no second editor — `instrumentation.ts` arms the + runners): a `tsx` script calling `previewRelocation`/`inspectChannelMedia` for `omnimirror` + against a mounted root, per the offline-controller-run rule. + +## Rollout for `omnimirror` + +1. Mount `sdb1` (1.8 T ext4 platter) at e.g. `/mnt/platter` with `nofail` in fstab; create + `/mnt/platter/archilyzer-media` owned by `user`. +2. Editor → `omnimirror` → Storage → Move media to `/mnt/platter/archilyzer-media`. Preview + should show ~131 GB to move; the copy is one platter write, ~15–25 min. The channel's jobs + are blocked for the duration; everything else keeps running. +3. Afterwards `/home` gains ~131 GB; the channel's next sync downloads straight onto the + platter through the link, and the disk gate watches that volume for it. + +## Out of scope (deliberately) + +- Per-video or per-file relocation (the saved-video store already covers "keep the big + container elsewhere"); a named global list of storage roots (one drive today — a free-text + root plus preview is enough); making shards symlink-aware; network stores.