# Re-acquired audio is handed to auto-transcribe, not deleted under it ## Context **Verified read-only against the tree at `da339a4` (clean) on 2026-08-27.** The previous slice (`fed4b01`, `ca42f7b`) made the backfill re-acquire land `audio.mp3` on an ASR-only `handling: "youtube"` video, diarize, then delete the audio in the `finally` at `backfillBatch.ts:684-693` (`buildCleanup`, `backfillReacquire.ts:257-290`; the only veto is `isDoNotClean`). Meanwhile `autoQueue.transcription` (enabled, `replaceAutoSubs: true`, strict tree ending in an `all` leaf) draws from the snapshot bucket `downloadedAutoSubsOnly` = ASR VTT **and audio present** (`channelSnapshot.ts:987-999`), so the moment any unit finishing on that channel regenerates the snapshot (1 s debounce, `autoRunner.ts:871`), the runner can start parakeet on the very file the backfill is about to unlink. **There is no per-video lock anywhere** — the registry serializes on `queueKey` only (`registry.ts:78-80`). Parakeet re-opens the audio per 480 s window (`scripts/parakeet-stitch.mjs:350`); on the unlink its next `sliceWav` fails, no `transcript.json` is written, and `transcribeOneFromQueue.ts:164-166` appends the id to `failed-transcriptions`. Only `TranscribeError` with `failureClass === "no-audio"` is a skip (`:156-163`). The manual per-channel batch honours that list permanently (`whisperBatch.ts:101-107`); the auto runner passes no `failedSet` and would retry. Two facts that shape the design. **Nothing automatic deletes audio after a transcription** — `cleanAudioFromTranscribed` is a manual channel button (`whisperActions.ts:415`) — so audio kept for the runner persists exactly like any downloaded audio, enters `transcribedWithAudio` after transcription, and waits for the operator's Clean-audio sweep; the steady-state cost is bounded by the existing UI and by the disk floor. And **the operator wants the "whisper wins" outcome**: re-acquired audio on an ASR-only video is an opportunity for a real transcript, not a hazard. So the fix is a deterministic, policy-driven *hand-off* at cleanup time, an in-flight veto as defense in depth, a transcription side that treats vanished audio as a skip, and attribution freshness that knows which transcript the names were made from. **Decided (operator):** attribution only. The digest lane is equally blind to a transcript replacement (`contextHash` hashes the channel's `digest-context.md`, not the transcript; `DigestProvenance` records no transcript source) — recorded in FACTS as a follow-up, not fixed here, because every ASR→whisper replacement would then re-queue a local LLM digest. **Three things the exploration found that the design must respect:** 1. `replaceAutoSubs: false` alone is not a refusal: a leaf with `match.bucket === "downloadedAutoSubsOnly"` draws it regardless (`autoQueuePolicy.ts:147-149, :180-184`). The rule is "no leaf covering this channel draws that bucket", where a bucket-less leaf draws `defaultBucketsForPolicy(kind, policy)`. 2. The remote transcribe branch wraps ENOENT as `"transport"` (`remoteTranscribe.ts:111, :126-133`), so a vanished-audio check at the local rethrow alone would still let the remote path blacklist the video. 3. Case (11)'s seeded VTT (`backfill.spec.ts:840-842`) carries no ASR fingerprints; provenance falls back to `metadata.info.json`, which the fake yt-dlp's re-acquire rewrites **without** `automatic_captions` (`fake-ytdlp.mjs:116-123`). The hand-off e2e must seed an ASR-shaped VTT (`ASR_VTT`, `auto-subs-replace.spec.ts:35-48`) so the 4 KB sniff decides. Also fixed in passing, same hunk: the `"failed"` outcome returns a real partial-file cleanup (`backfillReacquire.ts:196-208`) that the batch `finally` never calls — it only calls `cleanup()` for `"fetched"`. ## Step 0 — the plan on disk Write this file verbatim to `plans/reacquire-handoff.md` and commit it alone: `plans: the re-acquire hand-off planned`. ## Order: three commits 1. **The keep decision + in-flight veto + vanished-audio skip**, with units. 2. **Attribution transcript identity**, with units. 3. **e2e + wording + docs + memory.** tsc in all six packages + `pnpm -C common test` + editor units after each code commit. e2e once after commit 3, **detached** (memory `e2e-run-detached`): `backfill.spec.ts auto-subs-replace.spec.ts auto-queue.spec.ts diarization.spec.ts`. Edit nothing while it runs. If port 3011 is held, use the offset block (`PORT=3111 EXPORT_PORT=3110 OLLAMA_STUB_PORT=11535`) — do not kill anything. ## Commit 1 — `backfill: re-acquired audio is handed to auto-transcribe, never deleted under it` ### `common/jobs/autoQueuePolicy.ts` - `:307` `function matchesChannel` → **export** it; widen its parameter to `Pick`. - After `defaultBucketsForPolicy` (`:223`) add a pure helper: ```ts // Would this runner kind, under this policy, draw `bucket` for this channel? // A leaf naming an operation draws no bucket; a leaf naming a bucket draws // only that one; a bucket-less leaf draws defaultBucketsForPolicy — which is // where replaceAutoSubs enters, and the only place it does. export function policyDrawsBucket( kind: "transcription" | "download", policy: Pick, channel: Pick, bucket: string, ): boolean ``` Body: `flattenLeaves(policy.root).some(l => !l.match.operation && matchesChannel(l.match, channel) && (l.match.bucket ? l.match.bucket === bucket : defaultBucketsForPolicy(kind, policy).includes(bucket)))`. ### `common/controller/digestYield.ts` — the in-flight helper After `transcriptionActivity` (`:144`), in the file's own idiom (pure decision + fail-open wrapper, `:89-144`): ```ts export function evaluateTranscribingVideo( jobs: ReadonlyArray>, videoId: string, ): boolean // some job with status "running" has a task { kind: "transcribe", id: videoId } export function isTranscribingVideo(videoId: string): boolean // getRegistry().list() in try/catch → false ``` Registry tasks cover BOTH the auto runner and the manual whisper-all batch (`transcribeOne.ts:351-357` → `taskHooks.ts:60` → `registry.addTask`); `getAutoRunnerStatus` sees only the runner, so it is not used. Fail open to `false`: "not transcribing" just falls through to the policy decision, which is the primary mechanism. (Inline transcribe inside a download, `downloadOneManaged.ts:1031-1038`, passes no tracker and is invisible — accepted.) ### `common/controller/backfillReacquire.ts` - **Header `:38-43`** — rewrite as "the two exceptions to (4)": do-not-clean (unchanged) and THE HAND-OFF: kept when `settings.autoQueue.transcription` would pick this video from `downloadedAutoSubsOnly` (with a fresh disk check at the resume mark) or when a transcription task is already running on it. Say what happens next: the runner's own transcription leaves the audio in `transcribedWithAudio` for the operator's Clean-audio sweep, like any other download. - Imports: `getFreeBytes, evaluateDiskGate` (`../lib/diskSpace`; keep `diskGate` for `:107`), `readVideoFiles` (`../lib/videoStatus`), `isAutoSubsOnly` (`../lib/subtitleProvenance`), `detectPlatform` + `type Platform`, `policyDrawsBucket, type AutoQueuePolicy` (`../jobs/autoQueuePolicy`), `isTranscribingVideo` (`./digestYield`). - Replace the `cleanup: () => Promise` shape in `ReacquireOutcome` (`:61-72`) and `NOTHING_TO_CLEAN` (`:74`): ```ts export type KeepReason = "do-not-clean" | "in-flight" | "hand-off"; export type RefuseReason = | "no-audio" | "not-auto-subs-only" | "policy-off" | "policy-snoozed" | "no-leaf" | "disk-low"; export type KeepDecision = | { keep: true; reason: KeepReason } | { keep: false; reason: RefuseReason }; export type CleanupOutcome = | { status: "removed" } | { status: "nothing-added" } | { status: "kept"; reason: KeepReason }; ``` - **The pure decision, exported**, before `buildCleanup` (`:249`): ```ts export type KeepInput = { doNotClean: boolean; transcribing: boolean; // isTranscribingVideo(videoId) hasAudio: boolean; // files.audioFiles.length > 0, RE-READ after the fetch autoSubsOnly: boolean; // isAutoSubsOnly(videoDir, files) policy: Pick; channel: { slug: string; platform: Platform | null }; disk: { freeBytes: number; minFreeDiskGB: number; resumeMarginGB: number }; }; export function decideKeep(input: KeepInput): KeepDecision ``` Order: `doNotClean` → `transcribing` → `!hasAudio` → `!autoSubsOnly` → `!policy.enabled` → `policy.snoozeUntil != null` (a lapsed snooze is already normalized to `null` by the sanitizer, `autoQueuePolicy.ts:582-586`) → `!policyDrawsBucket("transcription", policy, channel, "downloadedAutoSubsOnly")` → disk → `{ keep: true, reason: "hand-off" }`. **Disk bar: `evaluateDiskGate({ ...input.disk, latched: true }).ok`** — i.e. free ≥ `resumeBytes` (floor + margin); a disabled gate (`minFreeDiskGB: 0`) never refuses. Why the resume mark: a hand-off is a download the backfill was about to give back, and the download runner itself resumes only at `resumeBytes` (`autoRunner.ts:663-668`) — the backfill must never keep audio the runner would refuse to fetch. `latched: true` on the **pure core** reuses the hysteresis math without touching the module latch; never call `diskGate` in enforce mode from a keep decision (`diskSpace.ts:231` mutates shared state). - `buildCleanup(videoDir, before, videoId, log, ctx: { paths; channelSlug; platform })` now returns `() => Promise`. Replace `:273-279`: if nothing added → `nothing-added`; else build `KeepInput` with thin I/O — `isDoNotClean`, `isTranscribingVideo`, **`readVideoFiles(videoDir, { checkUntranscribable: true })` re-read here** (the batch's `Candidate` carries no `files`, and they would be stale by the fetch anyway — same "re-checked HERE" discipline as `:96-102`), `isAutoSubsOnly`, `getSettings()` for `autoQueue.transcription` / `minFreeDiskGB` / `resumeMarginGB`, `getFreeBytes(ctx.paths.transcriptsDir)` — and call `decideKeep`. On keep, log exactly one of: - `Keeping re-acquired media for ${videoId}: marked "do not clean" (${added}).` (unchanged) - `Keeping re-acquired media for ${videoId}: a transcription is running on it (${added}).` - `Keeping re-acquired media for ${videoId}: handed to auto-transcribe, which will replace the auto-captions (${added}).` and return `{ status: "kept", reason }`. Otherwise the existing rm loop and `Removed…` line, return `{ status: "removed" }`. Call sites `:198, :207, :212` pass `{ paths: opts.paths, channelSlug: opts.channelSlug, platform: detectPlatform(config.url) }`. ### `common/controller/backfillBatch.ts` - `BackfillBatchResult` (`:134-140`): add `reacquireHandedOff: number` — counts `hand-off` **and** `in-flight` (both mean "left to the transcription lane"; the log line already distinguishes them). Init at `:261-263`. - The `finally` (`:689-692`): ```ts if (reacquired?.status === "fetched" || reacquired?.status === "failed") { const out = await reacquired.cleanup(); // the failed path's partial-file cleanup now runs if (reacquired.status === "fetched") { if (out.status === "removed") result.reacquireCleaned++; else if (out.status === "kept" && out.reason !== "do-not-clean") result.reacquireHandedOff++; } } ``` Update the comment at `:685-688`. (`present` / `disk-floor` / `gone` / config-unreadable return `NOTHING_TO_CLEAN` — harmless.) - Reconciliation (`:829-836`): same condition; text → `Re-acquired N file(s), removed M, handed H to auto-transcribe. Any other difference is media kept because its video is marked "do not clean" — if that is not what you expect, check the disk.` ### `common/controller/operationJobs.ts:132-134` Inside the reacquired clause append `(batch.reacquireHandedOff > 0 ? `, ${…} handed to auto-transcribe` : "")`. ### `common/controller/transcribeOne.ts` — vanished audio is a skip - After `resolveAudioFile` (`:45`): `async function throwIfAudioVanished(videoDir, audio, videoId)` — if `!(await pathExists(path.join(videoDir, audio)))`, throw `new TranscribeError(`audio ${audio} for ${videoId} vanished mid-run (removed by another lane) — skipped, not failed`, "no-audio")`. Reuses the existing skip branch; no new class. - Local catch (`:183-197`): after the `pauseRequested` return, `await throwIfAudioVanished` before `throw err`. Also before the "produced no output" throw (`:206`). - Remote branch (`:128-137`): wrap `transcribeViaRemote` in try/catch → `throwIfAudioVanished` then rethrow (finding 2 above). - Parakeet resume cache is untouched: `/.audio.mp3.parakeet/` matches none of `buildCleanup`'s predicates, and cached windows are re-validated against duration/segment when the same audio is re-fetched (`parakeet-stitch.mjs:288-297`). Say so in a comment. ### Units (commit 1) - `common/controller/backfillReacquire.test.ts` — `decideKeep`, pure, one per reason plus the two disk edges: do-not-clean keeps before any policy is consulted; a running transcription keeps whatever the policy says; hand-off when ASR-only + enabled policy + covering leaf + disk above the resume mark; refuse without audio, or not ASR-only; refuse when the runner is off or snoozed; refuse when no leaf covering the channel draws the bucket; **an explicit leaf bucket hands off with `replaceAutoSubs` off**; free between threshold and resume ⇒ `disk-low`; `minFreeDiskGB: 0` ⇒ keep. Never mutates its input. - `common/jobs/autoQueuePolicy.test.ts` — `policyDrawsBucket`: all/channel/platform leaves, explicit bucket vs default union, operation leaves never. - `common/controller/digestYield.test.ts` — `evaluateTranscribingVideo`: only a running job's `transcribe` task for this id counts (download task with the same id, finished job, other id ⇒ false). - `common/controller/transcribeOne.test.ts` (new): tmp dir + an executable shell script as the worker binary (the repo's own fake-bin convention, not a module mock): `rm -f audio.mp3; exit 1` ⇒ rejects with `failureClass === "no-audio"`; plain `exit 1` ⇒ not `no-audio`. **If driving `transcribeOne` locally needs more than ~40 lines of scaffolding (paths, worker config, app registry), drop it, cover by inspection, and say so in the report.** Commit body: the collision mechanics (no lock, 1 s regen, parakeet re-slice, the `failed-transcriptions` append), the two exceptions to guard (4), the disk-bar choice, and the dead `"failed"` cleanup now called. ## Commit 2 — `attribution: freshness knows which transcript the names were made from` ### `common/lib/attribution.ts` - `export type AttributionTranscriptSource = "whisper" | "vtt";` — narrower than `NormalizedTranscript.source` because cues.json's `source` is `picked.kind` (`normalizeTranscript.ts:133`), never `live_chat`. `export function transcriptSourceOf(source: string | null | undefined): AttributionTranscriptSource | undefined` — identity for the two literals, `undefined` otherwise ("unknown ⇒ do not assert"). - `AttributionProvenance` (after `:111`): `transcriptSource?: AttributionTranscriptSource;` with a comment in the `diarizationGeneratedAt` voice: a whisper transcript replacing the ASR captions rewrites the text the names were found in. - `AttributionFreshnessTarget` (after `:185`): `transcriptSource?:` — absent = do not compare. - `isAttributionFresh` (after `:250`): `if (target.transcriptSource !== undefined && p.transcriptSource !== undefined && p.transcriptSource !== target.transcriptSource) return false;` — **the record-carries-field rule** (the inverse of the `diarizationGeneratedAt` guard), so no record on disk is invalidated by the field's arrival. Comment it next to `sameVersion`'s rationale (`:215-223`). - `attributionTarget()` unchanged; callers splice like the diarized lane does. ### `common/controller/attributeOne.ts` `const src = transcriptSourceOf(transcript.source)` from the object already in hand (`:141`); spread `...(src ? { transcriptSource: src } : {})` into `fullTarget` (`:166-171`) and into the written provenance (`:283-296`). ### `common/lib/operations.ts` Local `transcriptSourceTarget(files: VideoFiles)` → `{ transcriptSource }` from `pickIndexTranscript(files)?.kind` (`videoStatus.ts:210-214`, **zero I/O** — `state()` runs per video per job start; the cost bar is `channelSnapshot.ts:590-597`), `{}` when `null`. Spread it into the target at the text lane (`:862`) and beside `diarizationGeneratedAt` in the diarized lane (`:944-947`). `attributionTarget.ts`, `attributionStatus.ts`, `attribution-server.ts` validator: no change. ### Units (`common/lib/attribution.test.ts`, after `:165`, the `:143-165` three-assertion shape) our own transcript replacing the auto-captions invalidates names made from them (vtt record vs whisper target ⇒ stale; whisper vs whisper ⇒ fresh); a record written before `transcriptSource` existed is not invalidated by it; a target that does not know the source does not compare it (`attributionTarget(CURRENT, "text-only").transcriptSource === undefined`); `transcriptSourceOf` maps `live_chat`/null to undefined. ## Commit 3 — `backfill: the hand-off is pinned end to end, and the docs say so` ### `editor/e2e/backfill.spec.ts` - `seedSubtitleChannel(videoId, vtt = SEEDED_VTT)` (`:844`); add `ASR_VTT` (copy of `auto-subs-replace.spec.ts:35-48`) and extract `ytAudioFiles(id)` from the inline filter at `:910-915`. - **Case (12)** after `:945`: `re-acquired audio is handed to auto-transcribe when the policy would replace the auto-captions`. Settings: `{ ...backfillSettings({ backfill: { allowRedownload: true } }), autoQueue: { transcription: { enabled: true, maxWorkers: 1, replaceAutoSubs: true, root: { id: "root", mode: "strict", children: [{ id: "leaf-all", match: { type: "all" } }] } }, download: {} } }` — `sanitizePolicy` (`autoQueuePolicy.ts:588`) accepts it; `auto-queue.spec.ts:98-100` is the precedent. No workers, and the runner is **not** started (`startAutoRunnersIfEnabled` runs only at boot, `editor/instrumentation.ts:83-86`; the invalidate-cache route starts nothing) — the case is deterministic. `seedSubtitleChannel(VID, ASR_VTT)`, `generateReport`, click "Run speaker work". Assert: `diarization.json` lands (poll 60 s); the output contains `handed to auto-transcribe`; the summary line reads `… 0 cleaned up, 1 handed to auto-transcribe`; `ytAudioFiles(VID)` non-empty after the job; then poll `${YT_ROOT}/snapshot.json` (job-end regen, `operationJobs.ts:138`) until `buckets.downloadedAutoSubsOnly` contains `VID`. - **Case (12b)**: same seed, no `autoQueue` policy ⇒ output contains `Removed re-acquired media`, not `handed to`, `ytAudioFiles(VID)` → `[]` — today's behaviour pinned through the new code path. - Do **not** drive the runner end to end here: once the audio is in `downloadedAutoSubsOnly` the rest is exactly the lane `auto-subs-replace.spec.ts:165` already walks, and a backfill-vs-runner race test would be timing-bound. ### Wording — one sentence each - `common/lib/settings.ts:329-330` (`allowRedownload` doc): after "unless the video is marked do-not-clean" — "or unless the auto-transcribe policy would replace its auto-captions (`replaceAutoSubs`, or a leaf on `downloadedAutoSubsOnly`), in which case the audio is kept for that runner." - `editor/app/operations/components/settings/LaneSettingsForm.tsx:109-111`: same content, operator-facing. - `editor/app/channels/[slug]/components/stages/SpeakersStage.tsx:166`: "…fetch audio (even on a subtitle-only channel) and then delete it — or keep it for auto-transcribe when that policy would replace the auto-captions — bounded by the free-disk floor." ### Docs and memory - `plans/FACTS.md` — new section `## Verified 2026-08-27 — the re-acquire / auto-transcribe hand-off`: no per-video lock (`registry.ts:78-80`); the `finally` (`backfillBatch.ts:684-693`) and the bucket rule (`channelSnapshot.ts:997`); regen timing (`autoRunner.ts:871`, 1 s debounce; backfill regenerates once at job end); parakeet re-slice (`:350`) vs whisper single open; the `no-audio` skip and the `failed-transcriptions` append; the manual batch honours the list, the auto runner does not; remote ENOENT ⇒ `"transport"`; the dead `"failed"` cleanup; nothing deletes audio after a transcription (`whisperActions.ts:415` only); the disk-bar choice; **the sweep has no inter-download sleep and is channel-major** (`backfillSweep.ts:55, 335-368`); **digest lane equally blind** (`contextHash` = `hashDigestContext(note)`, `digestContext-server.ts:47-60`) — follow-up, same field, same rule, operator's LLM-cost call. - `plans/STATE.md` — "Last updated" prepend; runbook item 2 step 3 (`:175-179`): drop the "30 s / ≥ 23 days" claim, say channel-major newest-first and no inter-download sleep; step 4 (`:180-186`) → RESOLVED by commit 1 (sha): hand-off + in-flight veto + vanished-audio skip; note the operator no longer needs `replaceAutoSubs: false` for the corpus run. - `editor/CHANGELOG.md` [Unreleased], first bullet, house voice: re-acquired audio on a subtitle-downloading channel is now kept for auto-transcribe when that policy would replace the auto-captions (and still deleted otherwise, bounded by the disk floor); a video whose audio is removed mid-transcription is skipped, not marked failed; speaker attribution goes stale when the transcript it named speakers in is replaced (only for records written from now on). - Memory: new `reacquire-handoff.md` (+ `MEMORY.md` line); amend the `slice-3-chosen-next` index line: runbook step 4 resolved. ## Verification 1. After commits 1 and 2: `pnpm -C exec tsc --noEmit` for `common editor export homepage umtool mcp`; `pnpm -C common test` (835 → +N); editor units `pnpm -C editor exec tsx --test "app/**/*.test.ts"` (85 → ±0). 2. Grep gates after commit 1: `grep -rn "decideKeep\|policyDrawsBucket\|isTranscribingVideo" common` → definitions, call sites, tests; `grep -n "cleanup()" common/controller/backfillBatch.ts` shows the fetched|failed condition. 3. e2e once, detached, after commit 3: `backfill.spec.ts auto-subs-replace.spec.ts auto-queue.spec.ts diarization.spec.ts`. 4. **No editor boot against `transcripts/`, nothing written under it, no census re-run.** 5. Manual (optional, on the e2e fixture, `PORT=3021 pnpm dev:test`): the speakers stage sentence mentions keeping audio for auto-transcribe; kill the server, remove `editor/test-transcripts` and `editor/test-settings.json`, `git status` clean. ## Out of scope - Digest-lane transcript identity (recorded as a follow-up in FACTS). - Automatic audio deletion after auto-transcribe — handed-off audio waits for the operator's Clean-audio sweep like every other download; changing that is its own policy. - A per-video lock in the registry; the hand-off makes the race moot for the auto runner and the in-flight veto covers the manual paths. - Driving the auto runner inside the backfill e2e. - Re-arming the sweep (operator's, after this lands). ## Handoff — the cadence On approval, Fable does not implement (memory `plan-then-opus-implements`). It spawns one `general-purpose` agent, `model: "opus"`, with: the plan path (after step 0, which the agent commits), the fish-shell caveats (commit via `git commit -F `; quote `[slug]` paths and globs; `cd` persists; `for … end` is fish syntax in a fish shell but the Bash tool may hand commands to bash — use POSIX `for … do … done`), never boot against `transcripts/`, e2e detached, tmp files under `$CLAUDE_JOB_DIR/tmp`, and the report contract: commit shas with one line each; exact tsc/test outputs; e2e pass/fail per spec with any retry; every divergence from the plan and why; anything undone. Fable reviews on return (`git log --oneline da339a4..`, the hunks in `backfillReacquire.ts`, `backfillBatch.ts`, `transcribeOne.ts`, `attribution.ts`, `operations.ts` and the new spec; re-runs grep gates + `pnpm -C common test` + editor units, not e2e), sends fixes via SendMessage, and reports.