commit 42576a316d6907a3234d2ebedd8379a176697dd7
parent 54df7aca3f424b2d6be0c03acc357d8d6d077631
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Mon, 31 Aug 2026 12:55:35 -0400
plans: the transcode operation is removed, and the docs say so
The IA doc's Transcode row said the band question waited on the snapshot writer
recording a transcode population. It is closed by deletion instead, and the row
says which census settled it. The slice-5 "what stayed" list marks its two
leftovers fixed.
STATE gets the shipped item and stops recommending the band. FACTS gets the
census verbatim, the three layers the one name covered and which was deleted,
the resolveAudioFile fallback that made the operation moot, the bucket rename
and why no regen, the catalog going 8 → 7 and where that is pinned, the worker
tag being on /workers and not /settings, and the duplicates-page merge hazard.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Diffstat:
4 files changed, 99 insertions(+), 8 deletions(-)
diff --git a/editor/CHANGELOG.md b/editor/CHANGELOG.md
@@ -1,6 +1,7 @@
# Changelog
## [Unreleased]
+- **The transcode operation is gone — it never fired.** A channel page had a *Transcode* stage, `/operations/transcode` had a "no console here" panel, `/cleanup` offered "Clear failed transcodings", and the video list drew a third status dot — all for a re-encode step built against two failures that never happened in production: in 68 channels, no snapshot has ever listed a video as missing its target format, no `failed-transcodings` file has ever held an id, and only four channels even met the stage's gate. Transcription never needed it — a video whose audio is in another format transcribes from that file. What stayed is everything that was never the operation's: the download path still re-encodes what it extracts itself, the video page still offers **Transcode audio.\<ext\> → \<fmt\>** per file, and both audio-format sweeps on the Cleanup stage and `/cleanup` are unchanged (gated on the channel having an `audioFormat`, which is what they compare against). The snapshot bucket behind the sweep is `wrongFormatAudio` now — its operator-facing name — and old reports keep their stray key until their next refresh. A `?stage=transcode` bookmark opens the channel overview. **Nothing on disk changes.** Also: the Pool's running-jobs list names the eight kinds its buttons enqueue, and the site's Search aliases tab no longer carries a "no site selected" branch that could not run.
- **A site has tabs, and the family has one page.** Charts, Search aliases, Deploy, Build and Homepage were five sidebar entries beside *Sites*, three of them reading the site from a `?site=` parameter the sidebar picker had to seed, one of them (Build) about no site at all, and one (Homepage) about the family's own hub. A site is one thing now: **`/sites/<id>` is Settings · Charts · Search aliases · Publish**, the site named in the path, the picker following it (and Dashboard and Channels following the picker). **`/sites` is the family page**: the list, then *Release notes*, *Build all sites* with the Basic/Docker mode, the *Hub*, and the *Pool* — the corpus-wide index, stats, sidecar and archive jobs — folded under a disclosure. Search aliases keep both sections on the site's tab: the global dictionary and the site's overrides. Every button, label and log is unchanged; "Select a specific site from the sidebar" is gone because a site's page always has one. The five routes redirect — a `?site=<id>` bookmark lands on that site's tab (the query rides along), `?site=__all__` and the bare routes on `/sites`; a bookmark to a deleted site 404s there exactly as `/sites/<id>` does. The Sites group is one entry; the nav is **eleven**, the IA doc's end state. **Nothing on disk changes.**
- **Jobs is one list.** `/jobs`, `/jobs/active` and `/jobs/queue` were three pages over three shapes — the directory listing, the registry's running work with its progress bars, and the scheduler's slots with their stuck reasons — and the same job was drawn three ways or hidden by one page's filter. `/jobs` is one table now: **one row per job**, the live head first (running, then queued in queue order with *next in line* / *2nd in line*, then anything that finished in the last half-minute) and the paged history below it, refreshed as before. A running row carries its per-task progress bars, its ETA and its *Transcripts: n / m* inside the Status cell; a queued row its Promote / ↑ / ↓; a stuck slot its `stuck · reason` badge, the last line of its log and **Force-release**, with **Reap stuck** on the health line above (active queues · running · queued · stuck · workers) and the lane strip above that. The kind/status/search filters apply to every row, live ones included. **The scheduler's drift check rides the live payload now**: every surface that draws it — this page, the dashboard, the monitor widget, `/api/jobs/active` — frees a running slot whose record is finished or gone the moment it sees one, after drawing it once. The channel, video, build and operation pages' *Active jobs* cards come off the same builder, so they show the progress bars they used to drop. `/jobs/active` and `/jobs/queue` redirect; the sidebar's *Active* entry and its running-count badge are gone (*Jobs* still counts running + queued); `/api/jobs/active` never moved. **Nothing on disk changes.**
- **Sync is an operation on the board, and the schedule is its page.** The pipelines rail's first row is **Sync** — channels on a cadence, due now, overdue, in the same four state words every other row uses — because a corpus that has stopped noticing new videos is not idle, it is broken, and until now that fact sat in a panel of its own below the rail. Its page is **`/operations/sync`**, which is the schedule that used to be at `/scheduler`: the per-channel cadence table, the bulk retune bar, *Run scheduler now* and the recent ticks, every control unchanged — with **every Sync scheduler setting below it**, the fieldset that used to be on **Settings**, saved by one button. `/scheduler` redirects, so bookmarks land; the API paths and the `pnpm sync:tick` cron client never moved; the sidebar's separate *Schedule* entry is gone, and searching the command palette for "schedule" or "cadence" finds Operations. The two **Storage chores** that ride the same heartbeat — the keep-latest deletion check and the saved-video backup — are named as such on the console, in a tick's own summary line and on **Saved videos**: they are not sync, they just share the one timer the editor has. **Transcription workers are configured on Workers now**, directly under the live list, with their own *Save workers* — so seeing a worker and changing it are no longer two pages. **Settings** keeps the machine and the site, and points at both. The reason a lane is not working is now one table rather than two. **Nothing on disk changes; no setting is renamed.**
diff --git a/plans/FACTS.md b/plans/FACTS.md
@@ -2226,6 +2226,8 @@ costs is its COST BASIS (`costBasis`), and no surface calls that a unit: the `Fi
prop in `OperationRail.tsx` / `SweepLane.tsx` was a population label and is `population`;
prose saying "unit" for cost says "cost basis". `DurationUnit` is a third, unrelated meaning.
+**SUPERSEDED 2026-08-30 — removed, see the section at the end of this file.**
+
**`transcode` is registered** (`EXTERNAL_OPERATIONS`, after `download`): group `media`,
`lane: { queueKey: TRANSCRIPTION_QUEUE, contendsFor: "cpu" }`, `dependsOn: ["download"]`,
`dispatch: "external"`, no `runner`, and the first `appliesTo(config)` —
@@ -3102,3 +3104,83 @@ unique: none of the ~15 buttons added to `/sites` contains "delete".
`<details>` hides**, so anything inside the Pool disclosure or "Individual steps" needs the
click first — which is why `e2e/helpers.ts` grew `buildIndex(page)` (goto `/sites`, click
"Pool jobs", press, assert "Done") and now imports `expect` as a value.
+
+
+## Verified 2026-08-30 — transcode: what it was, what stayed
+
+Read-only `jq` over `transcripts/channels/*/{snapshot,config}.json` plus `ls` on the
+`failed-transcodings` files. Nothing booted, nothing written.
+
+| fact | figure |
+|---|---|
+| channels / snapshots | **68 / 68** — every snapshot carries a `buckets.untranscoded` key |
+| `generatedAt` | 44 on 2026-08-27, 19 on 08-30, 5 on 08-31 (UTC) |
+| `buckets.untranscoded` non-empty | **0 of 68** |
+| `buckets.multipleAudioFormats` non-empty | **0 of 68** |
+| `cleanupBytes.foreignAudio > 0` | **0 of 68** |
+| `failed-transcodings` files on disk | **2** (`cornbreadman` mtime Jun 29, `the-quartering-rumble` mtime May 16), **both 0 bytes** |
+| channels passing `handling === "transcribe" && !!audioFormat` | **4**: cornbreadman, hasanabi, omnimirror, the-quartering-rumble (all mp3) |
+| `keepLatest > 0` (forces `extractionMode: "app"`, `persistencePlan.ts:92-94`) | **2**: nuxanor-kick 1, quarteringvlogs 5 |
+
+**Transcode was three layers wearing one name, and only the middle one was deleted.**
+
+1. **Download-time extraction.** The app runs ffmpeg itself (`transcodeAudio`,
+ `common/controller/transcode.ts`) when `extractionMode === "app"` — **two callers**,
+ `common/ytdlp/audioCheckedDownload.ts:44,768` and
+ `common/ytdlp/downloadOneManaged.ts:30,224` (`finalizeAppExtraction`) — and `keepLatest > 0`
+ forces that mode (`persistencePlan.ts:92-94`). **Live in production, KEPT.**
+2. **The transcode OPERATION.** Built by `70ed00f` (2026-05-19, "transcode failure handling")
+ and half of `bb380c6` (2026-06-18): a channel-page Transcode stage, a `failed-transcodings`
+ ledger, four job kinds, `buckets.untranscoded`, a "transcoded" dot and failed filter on the
+ video list, the catalog entry carrying the registry's only `appliesTo`, and
+ `/operations/transcode`. **DELETED 2026-08-30.**
+3. **Wrong-format audio cleanup** — the OTHER half of `bb380c6`: `multipleAudioFormats` /
+ `foreignAudio`, both sweeps and their bulk actions. **KEPT**, re-gated on
+ `!!config.audioFormat` (`wrongFormatApplies`, `CleanupStage.tsx`).
+
+**It could not have mattered.** `common/controller/transcribeOne.ts:22-31` `resolveAudioFile`
+falls back to ANY real audio file when `audio.<fmt>` is absent; strict matching is opt-in per
+manual batch and defaults false (`whisperBatch.ts:73,99`). A video whose audio is in another
+format transcribes from that file.
+
+**`transcode-audio` was never a registered job kind.** Its only spellings were
+`videos/[id]/videoActions.ts:84` (the KEPT per-file action) and the stage map in
+`stageStatus.ts`. The map entry left with the stage; the job still appears in
+`RunningJobsList` and now marks no stage as running, which is right — there is no stage.
+
+**The bucket is `wrongFormatAudio`, and there is no regen.** The population is unchanged (the
+predicate in `channelSnapshot.ts` is untouched): audio on disk, none of it in the channel's
+`audioFormat` — the orphan half of what "Remove wrong-format audio" deletes. No snapshot
+schema version exists and every bucket field is optional by design, so an old snapshot keeps a
+stray `untranscoded` key and reads `wrongFormatAudio` as `[]` until its next regen. An offline
+rewriter would race `snapshotScheduler`. Production has 0 ids in the bucket, so nothing
+rendered moves. The five e2e fixtures still spelling `untranscoded: []` are untyped JSON and
+are left alone: `normalizeBuckets` picks only keys it knows, exactly as it ignores production's
+stray key.
+
+**The "runner first" order in `pauseLaneFor` survives without its example.** It is a property
+of the catalog's SHAPE, not of transcode: an external entry sharing a runner's queue with no
+runner of its own must answer `null`, or a queue-key map alone would hand it that runner's
+pause. Same rewording in `operations.ts`, `operations.test.ts`, `operations/[id]/page.tsx`,
+`operations/lanes.ts` and `OperationDetail.tsx` (whose `NoConsoleView` stays as the
+fallthrough).
+
+**The catalog is SEVEN.** Pinned at `pauseGates.test.ts` (`ids.length === 7`), stated in
+`pauseGates.ts` ("all seven ids"), and listed in `operations.test.ts` ("the catalog covers
+every operation"). `jobKinds.test.ts` uses `clear-failed-transcriptions` as its label-less
+replayable example now.
+
+**The worker tag lives on `/workers`, not `/settings`** — correcting the last line of the
+2026-08-26 section above. `workers/page.tsx:21-28` builds `knownTags` from
+`operationCatalog().filter(o => o.scope === "video")`, so the "transcode" tag disappeared with
+the catalog entry and needed no edit.
+
+**`EXTERNAL_BAND_IDS` is `["download", "transcription"]` permanently** (`buildBands.ts:163`,
+pinned by `buildBands.test.ts`). The band question is closed by deletion, not by a
+snapshot-shape change.
+
+**Merge hazard.** The `duplicates-page` worktree edits `common/jobs/jobKinds.ts` (a +30 hunk
+around `:251-290`) while this change deleted four entries lower in the same file, adds
+`/duplicates` to `nav.ts` where main's `nav.test.ts` pins eleven, and still carries files main
+deleted (`build/page.tsx` with its own `BUILD_KINDS`, `actionable/page.tsx`). Their merge —
+recorded so nobody is surprised.
diff --git a/plans/STATE.md b/plans/STATE.md
@@ -3,7 +3,9 @@
The working memory for the local-AI derived-corpus work. Rewritten at the end of every
session, before context is cleared. See [`README.md`](README.md) for the protocol.
-**Last updated:** 2026-08-30 — **editor IA slice 5 shipped** (`140212a` → `df9b839`):
+**Last updated:** 2026-08-31 — **the transcode operation is removed** (`904f1a9` → `f717a36`): it never fired in production. The census over 68 channels found the `untranscoded` bucket empty in every one, both `failed-transcodings` files 0 bytes, and only four channels even passing the stage's gate — and `resolveAudioFile` falls back to any real audio file anyway, so transcription never needed it. Gone: four controllers, four job kinds, the channel stage and station, the catalog entry and with it `appliesTo`/`operationApplies` (catalog 8 → 7). Kept: `transcodeAudio` and the download path, the per-file *Transcode …* rows, both wrong-format sweeps — whose bucket is `wrongFormatAudio` now. The slice-5 leftovers rode first (`measure-nav.mjs`, `BUILD_KINDS`, `EditorAliasesClient`). Plan: [`editor-transcode-removed.md`](editor-transcode-removed.md).
+
+**Previously:** 2026-08-30 — **editor IA slice 5 shipped** (`140212a` → `df9b839`):
**a site has tabs, and the family has one page.** Charts, Search aliases, Deploy, Build and
Homepage were five sidebar entries beside *Sites*, three reading the site from a `?site=`
param the picker had to seed, one about no site at all, one about the family's own hub.
@@ -324,11 +326,14 @@ nothing renders.
IA slice 5 seams". A site has tabs and the family page has the globals; the five satellite
routes 307. **The nav is eleven** — the IA doc's end state — and the IA doc's UI slices are
complete.
+14. ~~**The transcode operation**~~ — **REMOVED 2026-08-30**, `904f1a9` → `f717a36`. Plan:
+ [`editor-transcode-removed.md`](editor-transcode-removed.md); census in FACTS "Verified
+ 2026-08-30 — transcode: what it was, what stayed". The band question is closed by
+ deletion; `EXTERNAL_BAND_IDS` is two for good.
-**Recommended next (editor IA).** The UI slices are done; what is left is not UI. The
-candidates are the **transcode band** (`EXTERNAL_BAND_IDS` to three, which needs the snapshot
-writer to record a transcode population per channel — a snapshot-shape change, so it is its
-own plan), **Phase 6**, and **slice 9**, which is gated on the sweep running in production.
+**Recommended next (editor IA).** The UI slices are done and the transcode band is no longer a
+candidate. What is left is **Phase 6** and **slice 9**, which is gated on the sweep running in
+production.
---
@@ -461,6 +466,7 @@ this entry is the decisions.
is the no-console panel slice 2 built. **Its band is not registered**: a snapshot carries
no `handling`/`audioFormat`, so a pure band cannot tell "never transcodes" from
"finished" — that waits for the snapshot writer to record it (unified-ops step 1).
+ **Removed 2026-08-30** — see item 14 above.
**Decisions worth keeping:**
- **The station label is derived, not "Speakers".** `channel-stage-selection.spec.ts` pins
diff --git a/plans/editor-operations-ia.md b/plans/editor-operations-ia.md
@@ -57,7 +57,7 @@ and concludes the model was wrong.
| **Sync** | **SHIPPED 2026-08-29** (slice 8a+8b). An operation, but channel-scoped and cadence-triggered: `scope: "video" \| "channel"` and `trigger: "backlog" \| "cadence"` are on `OperationDescriptor`, `SYNC_OPERATION` is first in `operationCatalog()`, and `/operations/sync` is its page. Two corrections to the wording above. **(1) "Keeps its own runner" could not be `runner`** — that field is typed `AutoQueueKind` and the heartbeat is not one of them, so the page chooses the cadence console off `trigger` and `runner` stays `undefined` (`operations.test.ts` pins it, with the reason). **(2) The board row is composed, not folded into a band** — `OperationBand`'s five populations are videos and sync's figures are channels, so `buildOperationBands` gains no sync band and `OperationRail` draws a `SyncRailRow` first in the same `<ul>`. The keep-latest checks and the saved-video backup that ride the same heartbeat (`editor/app/scheduler/runTick.ts`) are **Storage chores**, not sync, and are labelled as such on the console, in the skip reasons and on Saved videos. |
| **Build / deploy** | **Not** operations. Per-site, no per-video state, no lane. They are the Site's *publish* verb → `/sites/[siteId]`. |
| **Cleanup / saved-videos / relocate** | **Not** operations: they consume outputs rather than producing derived artifacts. Third noun, **Storage**, filed under Machine. The channel `cleanup` stage stays where it is. |
-| **Transcode** | **Registered 2026-08-26** (the vocabulary pass, commit 3): an external descriptor, group `media`, `lane: { TRANSCRIPTION_QUEUE, cpu }`, `dependsOn: ["download"]`, no runner, and the registry's first `appliesTo(config)` — `handling === "transcribe" && !!audioFormat`, exposed as `operationApplies(id, config)` and replacing the three verbatim copies of that gate. `/operations/transcode` is slice 2's no-console panel. **Its BAND waits**: `buildOperationBands` is snapshot-only and pure, and `ChannelSnapshot` carries no `handling`/`audioFormat`, so a band could not tell "never transcodes" from "finished". The honest route is the snapshot writer recording a transcode population per channel — a snapshot-shape change. **It was filed "with unified-ops step 1" and that coupling is void as of 2026-08-26**: both were thought to need one regeneration of the snapshots, and step 1 shipped needing none. This is now its own change and its own plan — the writer recording a transcode population (`appliesTo(config)` plus a kept-media denominator beside `buckets.untranscoded`) so `EXTERNAL_BAND_IDS` can go to three. It stays two until that lands. |
+| **Transcode** | **REMOVED 2026-08-30 — never fired in production; the band question is closed.** The census that settled it (read-only `jq` over 68 channels, 2026-08-30): `buckets.untranscoded` non-empty in **0 of 68**, `multipleAudioFormats` non-empty in 0 of 68, `cleanupBytes.foreignAudio > 0` in 0 of 68, both `failed-transcodings` files on disk **0 bytes**, and only **4** channels passing the gate `handling === "transcribe" && !!audioFormat`. It could not have mattered either: `transcribeOne`'s `resolveAudioFile` falls back to any real audio file when `audio.<fmt>` is absent. **Deleted**: the four controllers, the four job kinds, the channel page's Transcode stage and station, `buckets.untranscoded` as a to-do list, the video list's "transcoded" dot and failed filter, the catalog entry and with it `appliesTo`/`operationApplies` (the entry was the only taker) — catalog 8 → 7. **Kept**: `transcodeAudio` and the download path (`extractionMode: "app"`, two callers), the per-file *Transcode <file> → <fmt>* rows on the video page, and both wrong-format-audio sweeps, re-gated on `!!config.audioFormat`; the bucket is `wrongFormatAudio` now. **`EXTERNAL_BAND_IDS` stays `["download", "transcription"]` permanently** — there is no third band to add. Plan: `plans/editor-transcode-removed.md`; census in FACTS.md, "Verified 2026-08-30 — transcode: what it was, what stayed". |
| **Social channels** | A channel whose operation set is `{fetch-posts}`. An explicit **non-goal** here: leave the short-circuit at `editor/app/channels/[slug]/page.tsx:156` alone. |
| **The channel stage list** | Half-derived and that is correct. The middle (`download … backfill`) derives from `OPERATION_GROUP_ORDER`; the bookends (`configure`, `playlist` / `cleanup`, `diagnostics`, `danger`) are **channel chores**, not operations, and stay hand-listed. Say so in the code so the next reader does not "finish" the derivation. |
| **Digest's two lanes** | A **registry defect**, not a noun problem. `common/controller/arbiter.ts` special-cased `DIGEST_KIND_ID` because digest runs on its own queue. **Resolved in slice 2** by giving digest its own `laneFor(settings)` and resolving every operation through `laneForOperation` — but NOT by "giving every kind a `laneFor` defaulting to `lane`": `laneFor` is optional and its PRESENCE is what `backfillBatch.ts` keys the GPU idle-only rule off, so a default would enrol every operation in that rule (FACTS.md, "laneFor? is OPTIONAL"). Add one only where the lane genuinely varies. |
@@ -766,7 +766,8 @@ instead.** The census is in FACTS.md ("Verified 2026-08-30 — editor IA slice 5
pages already did per request; it was `force-dynamic` already.
14. **`BUILD_KINDS` is wrong in both directions** — six kinds, missing the three live-chat
kinds the buttons enqueue and including `build-export`, which none does. Moved verbatim
- with a comment saying exactly that; **not fixed**.
+ with a comment saying exactly that; **not fixed** — fixed 2026-08-30, first commit of the
+ transcode removal.
15. **`SettingsForm`'s "The mode can also be toggled on the Deploy page"** points at Sites.
16. **A stale `?site=<deleted>` bookmark 404s at `/sites/<id>/charts`** exactly as `/sites/<id>`
does, where `/charts?site=bogus` used to fall back to the lone/all site. The picker never
@@ -775,7 +776,8 @@ instead.** The census is in FACTS.md ("Verified 2026-08-30 — editor IA slice 5
**What stayed.** `seedsSiteParam` and its reason (seeding on `/sites*` clobbered an in-flight
push to `/sites/<id>`). `BUILD_KINDS`' mismatch (14). `EditorAliasesClient`'s
`siteId: string | null` and its now-unreachable global-only branch — a `git mv` plus one import
-line is the whole diff, and narrowing it was not this slice. The charts preview's
+line is the whole diff, and narrowing it was not this slice. Both — fixed 2026-08-30, first
+commit of the transcode removal. The charts preview's
last-composed data (11). The hub deploy script's hardcoded project name (10).
`RunningJobsList` as the Pool's renderer. `/api/*` — nothing there belonged to any of the five,
and the charts preview's `/stats` rewrite is absolute and unaffected. `deleteSiteAction` still
