commit 18e0407b7ad06de5ae93627150e43e82b2273ee3 parent e12f3f621dd99fa546611a68679f99cc3e98e9a8 Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st> Date: Tue, 25 Aug 2026 23:46:04 -0400 editor: /auto-queue becomes /operations, and the nav gets four groups The backend found its organizing noun — the operation — and the UI had not. Counted rather than estimated: 28 pages behind a 19-link nav whose "Pool" group held ten unrelated tools, four channel tables, four job lists, three renderings of lane state, and four different answers to "what needs doing". plans/editor-operations-ia.md is the UI half of unified-operations-model.md, written down whole — the four nouns (Corpus, Operations, Sites, Machine), the seven places "operation as the noun" honestly breaks and what to do about each, the nav end state, the reconciliations with unified-ops steps 5/6, PLAN.md Phase 11a and relocate-channel-media §6, and nine slices, each shippable, each deleting something. Slice 1, here: `git mv app/auto-queue app/operations`. /operations is the board (the rail, the arbiter, a sync row); /operations/<id> is ONE operation, resolved against operationCatalog() so an unknown id 404s and a NEW registry entry gets a page with no route work. /auto-queue redirects. The lane <select> is deleted. With one lane per page there is no switch to lose policy-edit state across, which was the only reason the non-selected lanes stayed mounted. The runner section's ~1,200-line structural e2e contract survived the move verbatim; RunnerOperationView.tsx states every clause of it. Decisions worth keeping: - The sweep scope is NOT pre-ticked to the operation being viewed. The scope is written at arm time, so a per-page default would arm a narrower sweep than the operator sees. - A speaker operation's page draws its OWN band — the shared panel could only say "this lane covers several operations" — and says in a sentence that the lane, sweep and pause are shared. A per-operation page is not a per-operation switch. - The arbiter is the BOARD's control, not a lane's: it decides BETWEEN lanes. - Retired routes redirect, never 404, and the nav cap is umtool's rule, quoted in nav.ts so two consoles do not drift into two nav philosophies. Verified: tsc clean, common 821/821, auto-queue.spec.ts + navigation.spec.ts 29/29. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Diffstat:
53 files changed, 1801 insertions(+), 1070 deletions(-)
diff --git a/PLAN.md b/PLAN.md @@ -516,12 +516,24 @@ Without this, a corpus-wide backfill is a one-shot gamble on prompt quality. **`feedback`** for viewer-submitted items and **`review`** for triage state. Do not add a fourth meaning of "report". -**11a — review queue (land before backfilling).** Extend the existing `/actionable` page -rather than building a parallel one: its `SectionConfig` array with counters in -`loadActionable.ts` is the designed extension point. New sections: digests needing review -(driven by the `warnings` array Phase 1 persists), proposed channel context awaiting -promotion, uncertain attribution, viewer feedback, and **duplicate clusters awaiting -confirmation**. +**11a — review queue (land before backfilling).** **REVISED 2026-08-25 by +[`plans/editor-operations-ia.md`](plans/editor-operations-ia.md), which dissolves +`/actionable` in its slice 4.** The instruction here used to be "extend `/actionable` rather +than building a parallel one", and the reasoning behind it still holds — do not build a +parallel page — but the destination moved: `/actionable` was one of four different answers to +"what needs doing", and the operation is the noun those answers are folding into. So the same +sections, filed by what they are about: + +- **Per-operation review** — digests needing review (driven by the `warnings` array Phase 1 + persists) and uncertain attribution — becomes the *attention* section of + `/operations/<id>`, beside that operation's own lane, sweep and band. +- **Corpus review** — duplicate clusters awaiting confirmation, viewer feedback, and proposed + channel context awaiting promotion — becomes a small `/review` page under Corpus. These are + judgements about the ARCHIVE, not about one operation's output. + +The `SectionConfig` array with its counters in `loadActionable.ts` is still the designed +extension point and still the thing to extend; it **moves with the sections** rather than +disappearing (the widget payload reads `loadActionable.ts` directly and must keep working). **Duplicate clusters awaiting confirmation** *(added by the duplicate-detection work — UI only, the data and the write path already exist).* A `title-duration` cluster is a suspect: diff --git a/editor/CHANGELOG.md b/editor/CHANGELOG.md @@ -1,6 +1,8 @@ # Changelog ## [Unreleased] +- **The auto-queue console is `/operations`: a board over every operation, and a page for each one.** The old page put a rail across the top saying what all four pipelines were doing, and then a **Lane dropdown** decided which one you could act on. A dropdown is not an address — you could not link a colleague to the download lane, Back did not return to it, and the two panes it switched between had to stay mounted so an unsaved policy edit would survive the switch. The rail's rows are links now: `/operations` is the board (every pipeline, one line each, plus the arbiter and a sync row), and `/operations/<id>` is one operation in full — the runner's rules and Start/Drain/Stop for **Download** and **Transcription**, the sweep, the pause and the itinerary for **Digest** and each **Speaker** operation. Two things follow from one page per operation: a speaker operation's own band is drawn on its own page (the shared panel could only say "this lane covers several operations", because summing an audio pass and a per-chunk model call gives a figure in no unit), and the page says in a sentence that the lane, the sweep and the pause it shows are **shared** with the other speaker operations — a per-operation page is not a per-operation switch. **`/auto-queue` redirects**, so every bookmark still lands; the API paths under `/api/auto-queue/*` never moved. The sweep scope is deliberately **not** pre-ticked to the operation you are looking at: the scope is written at arm time, and a helpful default would arm a narrower sweep than the operator can see. +- **The sidebar is four groups — Corpus, Operations, Sites, Machine — instead of three, and "Pool" is gone.** Nineteen links in a group called *Pool* that held ten unrelated tools was the shape of the problem, not a labelling accident: the editor has four nouns, and the nav is where that is either true or not. *Monitor* leaves the nav entirely — the widget is a **projection** of the board's own payload drawn for a second screen, and a projection is not a place; it is reached from the **Pipeline** band on the dashboard, which is the thing it mirrors. Nothing was deleted: the interim entries (Actionable, Schedule, Charts, Search aliases, Deploy, Build, Homepage) sit in the group that will absorb them, and the rule for what comes next is umtool's, quoted where it can be read: fold under an entry rather than add beside it, and retire a route with a redirect rather than a 404. The full model, and the eight slices still ahead of this one, are in `plans/editor-operations-ia.md`. - **`/channels` is sectioned by the groups a site files its channels into, and each section can run the pipeline over its own shelf.** Every site already sorts its channels into authored groups — on Jeralyzer they are *Archives* ("Channels that archive Jeremy's content, out of his reach"), *Guest Appearances* and *Extended Universe* — and the editor showed none of it: it scoped the rows to the active site, threw the grouping away, and rendered one flat table. Those descriptions were written on the Sites page, stored, and rendered **nowhere**. The table is now one table sectioned by group — columns stay locked across every section, because comparing transcript counts between groups is the point — with each group's name, its description, and an *off by default* note when visitors don't get that group preselected. Sorting a column sorts within each group; a **Group by section** tick flattens it again without leaving the site. Under *all sites* the table stays flat, since groups only partition one site's channels and there is no single grouping across the pool. - **Each group header carries that group's slice of the pipeline, and every stage is labelled with the work it would actually do.** `Sync ── Download 143 ── Transcribe 27 ── Digest 1,204 ── Backfill 88`: the channel page's transit line, collapsed to the five runnable stages, where each station's figure IS the button — you cannot press a stage without the number it will act on being the label you pressed. "Re-sync the archives" and "get the guest appearances transcribed" used to mean clicking through channels one at a time. The five figures are never added together, and each is read through the same snapshot reader the channel page uses, so a group total cannot disagree with the channel it came from. A station whose work is provably finished stops being a button and becomes quiet text (*Downloaded*, *Transcribed*); a station with no eligible channel is disabled and says why — a `youtube`-handling channel never runs whisper, so counting it would inflate a figure on a button that would skip it. A figure that includes a channel which has never reported is marked with a trailing `+`, because it is a floor and not a total, and where **no** channel has reported it reads `—` rather than `0`. A switched-off lane reads **off**, never `0`: a snapshot's counts outlive the feature being switched off, and an empty work list because something is disabled is not the same as being finished. Download, Transcribe, Digest and Backfill confirm first — at group scale these are GPU-days. The fan-out queues the existing per-channel job once per channel, so they serialize on the lane they already shared: no new concurrency, and a channel with nothing to do is skipped rather than given a no-op job. - **The pool-wide button now says what it does: Sync every channel.** It was labelled *Sync all* / *Full sweep all* on a page that has had a site scope for some time, so it read as "all of these" when it has always swept the entire channel pool regardless of what is on screen. Its behaviour is unchanged; a group's own **Sync** is the one that respects the visible scope. A group sweep also deliberately does *not* stamp the "last full sync" marker the monitor widget reads — a group is not the pool, and it must not claim to be. diff --git a/editor/app/api/auto-queue/status/route.ts b/editor/app/api/auto-queue/status/route.ts @@ -1,5 +1,5 @@ import { NextResponse } from "next/server"; -import { buildAutoQueueStatusPayload } from "../../../auto-queue/status"; +import { buildAutoQueueStatusPayload } from "../../../operations/status"; export const dynamic = "force-dynamic"; diff --git a/editor/app/auto-queue/actions.ts b/editor/app/auto-queue/actions.ts @@ -1,252 +0,0 @@ -"use server"; - -import { revalidatePath } from "next/cache"; -import { - getSettings, - writeSettings, - type SiteSettings, -} from "yt-dlp-transcript-common/lib/settings"; -import { - startAutoRunner, - stopAutoRunner, -} from "yt-dlp-transcript-common/controller/autoRunner"; -import type { AutoQueueKind } from "yt-dlp-transcript-common/jobs/autoQueueState"; -import { - isGroup, - type AutoQueueGroup, - type AutoQueueNode, - type AutoQueueOrder, - type AutoQueueReach, -} from "yt-dlp-transcript-common/jobs/autoQueuePolicy"; - -export type SaveResult = { ok: true } | { ok: false; error: string }; - -// Persist one runner kind's policy and bring its runner up/down to match, -// WITHOUT a server restart. writeSettings sanitizes the tree (sanitizeAutoQueue), -// so a slightly-off client payload is coerced rather than trusted. Enabling -// starts the runner immediately; disabling is picked up by the running loop on -// its next iteration (getSettings reads from disk), so it stops on its own. -export async function saveAutoQueueAction( - kind: AutoQueueKind, - input: { - enabled: boolean; - maxWorkers: number | null; - // Opt in to the lowest-priority replace-auto-captions lane (default false). - replaceAutoSubs: boolean; - // Ordering WITHIN each rule (newest / oldest upload first, or listed order). - order: AutoQueueOrder; - root: AutoQueueGroup; - }, -): Promise<SaveResult> { - const current = getSettings(); - // NOTE: this object lists every persisted policy field EXPLICITLY, so a field - // added to AutoQueuePolicy and forgotten here is silently dropped on every - // save rather than failing loudly. `snoozeUntil` is deliberately carried over - // from `current` instead of taken from the form: the snooze is set by its own - // action, and a policy save (e.g. reordering rules) must not cancel it. - const next: SiteSettings = { - ...current, - autoQueue: { - ...current.autoQueue, - [kind]: { - enabled: input.enabled, - maxWorkers: input.maxWorkers, - replaceAutoSubs: input.replaceAutoSubs === true, - order: input.order, - snoozeUntil: current.autoQueue[kind].snoozeUntil ?? null, - root: input.root, - }, - }, - }; - try { - await writeSettings(next); - } catch (e) { - return { ok: false, error: (e as Error).message }; - } - if (input.enabled) { - await startAutoRunner(kind); - } - revalidatePath("/auto-queue"); - return { ok: true }; -} - -// Explicit start/stop for the Start/Stop buttons (also reachable as -// /api/auto-queue/control for e2e). Start is a no-op when the policy is disabled. -export async function startAutoQueueAction( - kind: AutoQueueKind, -): Promise<SaveResult> { - const jobId = await startAutoRunner(kind); - if (jobId === null && !getSettings().autoQueue[kind].enabled) { - return { ok: false, error: "Enable the policy before starting the runner." }; - } - revalidatePath("/auto-queue"); - return { ok: true }; -} - -export async function stopAutoQueueAction( - kind: AutoQueueKind, -): Promise<SaveResult> { - stopAutoRunner(kind); - revalidatePath("/auto-queue"); - return { ok: true }; -} - -// Idle a runner until `untilMs` (epoch ms) without stopping it, or wake it now -// with null. Deliberately NOT part of saveAutoQueueAction: snoozing is a -// one-click operational act, and routing it through the policy form would mean a -// pending tree edit had to be saved (or discarded) to snooze. The runner re-reads -// settings every iteration, so this takes effect on the next tick — and because -// it lives in settings.json rather than runner memory, it survives a restart. -export async function snoozeAutoQueueAction( - kind: AutoQueueKind, - untilMs: number | null, -): Promise<SaveResult> { - const current = getSettings(); - const next: SiteSettings = { - ...current, - autoQueue: { - ...current.autoQueue, - [kind]: { ...current.autoQueue[kind], snoozeUntil: untilMs }, - }, - }; - try { - await writeSettings(next); - } catch (e) { - return { ok: false, error: (e as Error).message }; - } - revalidatePath("/auto-queue"); - return { ok: true }; -} - -// Recursively drop every leaf that matches this channel, so re-prioritizing the -// same channel doesn't accumulate duplicate leaves (a group emptied of children -// is kept — sanitizeAutoQueue tolerates it, and removing it could orphan a -// group the operator configured). Returns a fresh tree. -function stripChannelLeaves(node: AutoQueueNode, slug: string): AutoQueueNode { - if (!isGroup(node)) return node; - const children = node.children - .filter( - (c) => - isGroup(c) || - !(c.match.type === "channel" && c.match.value === slug), - ) - .map((c) => stripChannelLeaves(c, slug)); - return { ...node, children }; -} - -// "Add to top of auto-queue": prepend a channel leaf at the HEAD of the download -// policy's strict root (first child = highest priority), enable the download -// policy, and start the runner if it isn't up. Directly uses the existing -// policy-tree engine — no engine change. Idempotent: any prior leaf for this -// channel is stripped first so the head stays the single owner (first-match-wins -// in buildPendingByLeaf). -export async function prioritizeChannelDownloadAction( - slug: string, -): Promise<SaveResult> { - const trimmed = (slug ?? "").trim(); - if (!trimmed) return { ok: false, error: "No channel slug supplied." }; - const current = getSettings(); - const download = current.autoQueue.download; - const stripped = stripChannelLeaves(download.root, trimmed) as AutoQueueGroup; - const nextRoot: AutoQueueGroup = { - ...stripped, - children: [ - { id: `prioritize-${trimmed}`, match: { type: "channel", value: trimmed } }, - ...stripped.children, - ], - }; - const next: SiteSettings = { - ...current, - autoQueue: { - ...current.autoQueue, - download: { ...download, enabled: true, root: nextRoot }, - }, - }; - try { - await writeSettings(next); - } catch (e) { - return { ok: false, error: (e as Error).message }; - } - await startAutoRunner("download"); - revalidatePath("/auto-queue"); - return { ok: true }; -} - -// Order and reach for the two SWEEP-fed lanes. -// -// A separate action from saveAutoQueueAction because these lanes have no policy -// tree to save alongside: their order lives in settings.digest / settings. -// backfill, not in settings.autoQueue. Routing them through the policy form -// would mean an operator could not change an order without also committing a -// pending tree edit for a different lane. -// -// SPREAD-AND-OVERRIDE, never a rebuilt literal. settings.digest carries -// `sweepEnabled` and its scope, and settings.backfill carries both plus -// `allowRedownload`; a literal here that omitted one would disarm a multi-week -// sweep, or write media to a 97%-full disk, on an unrelated save. That exact -// bug is documented in the settings form's own digest block. -// -// It takes effect on the sweep's NEXT PASS — both sweeps re-read the setting per -// pass rather than at launch — and on the next per-channel batch, which resolves -// the order itself. Nothing needs restarting. -export async function saveLaneOrderAction( - lane: "digest" | "backfill", - input: { order: AutoQueueOrder; reach: AutoQueueReach }, -): Promise<SaveResult> { - const current = getSettings(); - const next: SiteSettings = - lane === "digest" - ? { - ...current, - digest: { - ...current.digest, - recencyOrder: input.order, - recencyReach: input.reach, - }, - } - : { - ...current, - backfill: { - ...current.backfill, - order: input.order, - reach: input.reach, - }, - }; - try { - await writeSettings(next); - } catch (e) { - return { ok: false, error: (e as Error).message }; - } - revalidatePath("/auto-queue"); - return { ok: true }; -} - -// The arbiter: start / stop the unified dispatcher. -// -// Kept out of saveLaneOrderAction and out of the policy form: starting a -// dispatcher is an operational act, not a setting, and routing it through a -// form would mean a pending tree edit had to be saved or discarded first. -// -// It REFUSES rather than silently no-opping when a sweep is armed, and the -// refusal carries the reason — two dispatchers on one lane would start the same -// channel twice, and an operator who clicks Start and sees nothing happen -// deserves to be told which switch is in the way. -export async function startArbiterAction(): Promise<SaveResult> { - const { startArbiter } = await import( - "yt-dlp-transcript-common/controller/arbiter" - ); - const result = await startArbiter(); - revalidatePath("/auto-queue"); - return result.jobId - ? { ok: true } - : { ok: false, error: result.error ?? "The arbiter could not be started." }; -} - -export async function stopArbiterAction(): Promise<SaveResult> { - const { stopArbiter } = await import( - "yt-dlp-transcript-common/controller/arbiter" - ); - stopArbiter(); - revalidatePath("/auto-queue"); - return { ok: true }; -} diff --git a/editor/app/auto-queue/components/AutoQueueView.tsx b/editor/app/auto-queue/components/AutoQueueView.tsx @@ -1,464 +0,0 @@ -"use client"; - -import { useCallback, useEffect, useId, useRef, useState } from "react"; -import type { AutoQueueKind } from "yt-dlp-transcript-common/jobs/autoQueueState"; -import type { - AutoQueueStatusPayload, - AutoQueueKindStatus, - PlatformCooldownView, -} from "../status"; -import { ArbiterBar } from "./ArbiterBar"; -import { InFlightList } from "./InFlightList"; -import { LaneHeader } from "./LaneHeader"; -import { NextUp } from "./NextUp"; -import { OperationRail, type RailLaneState } from "./OperationRail"; -import { PolicyTreeEditor } from "./PolicyTreeEditor"; -import { SnoozeControl } from "./SnoozeControl"; -import { SweepLane } from "./SweepLane"; -import { - type Channel, - SELECT_CLASS, - formatClock, - formatCooldown, - idleReasonText, - leafOrder, -} from "./dispatch"; -import { deriveLaneState } from "../../components/lanes/laneState"; -import type { AutoQueueLanesPayload, SweepLaneStatus } from "../lanes"; - -// THE CONSOLE. One grammar for four pipelines. -// -// It reads top to bottom as: every lane at a glance (the rail), then ONE lane in -// full (chosen by a dropdown). That split is the whole redesign. Before it, the -// page stacked two identical runner panels and had no room for the other two -// pipelines at all — so "the digest lane is idle because transcription is -// hogging the GPU" was a fact you could only assemble by visiting three pages. -// -// THE RAIL IS THE SWITCHER'S CONTEXT, not a summary of it. You never have to -// switch lanes to learn that THIS lane is idle because ANOTHER one is, which on -// this corpus is the normal case: attribution-diarized is 99.9% blocked behind -// diarization, diarization is 99.2% media-gone. -// -// STRUCTURAL CONTRACT, load-bearing for ~1,200 lines of e2e — every clause below -// survives this redesign unchanged: -// * each runner kind is a literal <section> containing an <h2> named exactly -// "Auto-transcribe" / "Auto-download", with the policy editor and the -// Start/Drain/Stop buttons inside it; -// * NOTHING inside a kind section is itself a <section> — the suite scopes with -// locator("section", { has: heading }), and a nested one would match two -// ancestors, failing every scoped lookup on strict mode; -// * the rail names pipelines in plain text, never as headings, for the same -// reason; -// * policy controls stay native <select> / <input type=checkbox>; -// * role="status" stays reserved for "Saved."; -// * data-hydrated stays on the section. -// -// THE NON-SELECTED LANES STAY MOUNTED, hidden rather than unmounted. Two reasons, -// and the second is the load-bearing one: a runner lane holds live policy-edit -// state that unmounting would silently discard, and the page-wide option counts -// the suite asserts must keep resolving to the same number. Auto-transcribe is -// the default selection, which is where every existing UI spec already scopes. - -type LaneId = AutoQueueKind | "digest" | "backfill"; - -// The two RUNNER lanes have fixed names; the two sweep-fed ones are named after -// the operations they hold, which the server resolves and puts on the payload. -// A hardcoded "Backfill" here would be a fourth copy of a list that already -// drifted once. -function laneOptions( - lanes: AutoQueueLanesPayload | null, -): { id: LaneId; label: string }[] { - return [ - { id: "transcription", label: "Auto-transcribe" }, - { id: "download", label: "Auto-download" }, - { id: "digest", label: lanes?.digest.label ?? "Digest" }, - { id: "backfill", label: lanes?.backfill.label ?? "Derived data" }, - ]; -} - -export function AutoQueueView({ - initial, - channels, - platforms, - bucketsByKind, -}: { - initial: AutoQueueStatusPayload; - channels: Channel[]; - platforms: string[]; - bucketsByKind: Record<AutoQueueKind, string[]>; -}) { - const [data, setData] = useState<AutoQueueStatusPayload>(initial); - const [lane, setLane] = useState<LaneId>("transcription"); - const mounted = useRef(true); - const now = useNow(); - const switcherId = useId(); - - const refresh = useCallback(async () => { - try { - const res = await fetch("/api/auto-queue/status", { cache: "no-store" }); - if (!res.ok) return; - const next = (await res.json()) as AutoQueueStatusPayload; - if (mounted.current) setData(next); - } catch { - /* transient; next poll retries */ - } - }, []); - - useEffect(() => { - mounted.current = true; - const id = setInterval(refresh, 3000); - return () => { - mounted.current = false; - clearInterval(id); - }; - }, [refresh]); - - // The rail is keyed by OPERATION id; the switcher by LANE. They are not the - // same space — "backfill" is one lane covering several operations — so the - // highlight resolves through this rather than comparing the two directly. - const highlightedOperation = - lane === "transcription" ? "transcription" : lane === "download" ? "download" : lane; - - return ( - <div className="flex flex-col gap-6"> - <OperationRail - bands={data.lanes.bands} - states={railStates(data)} - selectedId={highlightedOperation} - /> - - <ArbiterBar arbiter={data.lanes.arbiter} onRefresh={refresh} /> - - <div className="flex flex-wrap items-center gap-2"> - <label - htmlFor={switcherId} - className="font-mono text-xs uppercase tracking-[0.14em] text-muted-foreground" - > - Lane - </label> - {/* A native <select>, NOT the kit's Radix one. Radix portals its options - and unmounts them while closed, which would take the page-wide option - counts the suite asserts to zero. Tabs are out for the same class of - reason — they unmount their panels. */} - <select - id={switcherId} - className={SELECT_CLASS} - value={lane} - onChange={(e) => setLane(e.target.value as LaneId)} - > - {laneOptions(data.lanes).map((o) => ( - <option key={o.id} value={o.id}> - {o.label} - </option> - ))} - </select> - </div> - - <KindLane - kind="transcription" - title="Auto-transcribe" - status={data.transcription} - channels={channels} - platforms={platforms} - buckets={bucketsByKind.transcription} - now={now} - hidden={lane !== "transcription"} - onRefresh={refresh} - /> - <KindLane - kind="download" - title="Auto-download" - status={data.download} - channels={channels} - platforms={platforms} - buckets={bucketsByKind.download} - now={now} - hidden={lane !== "download"} - onRefresh={refresh} - /> - {/* `hidden` ALONE IS NOT ENOUGH HERE. Tailwind's preflight sets - [hidden]{display:none} at low specificity, and a `flex` utility in the - utilities layer beats it — so a section carrying both would stay - visible while claiming to be hidden. The class is switched too, and - the attribute kept for semantics. */} - <section - data-lane="digest" - hidden={lane !== "digest"} - className={lane === "digest" ? "flex flex-col gap-4" : "hidden"} - > - <SweepLane - lane={data.lanes.digest} - band={data.lanes.bands.find((b) => b.id === "digest") ?? null} - onRefresh={refresh} - /> - </section> - <section - data-lane="backfill" - hidden={lane !== "backfill"} - className={lane === "backfill" ? "flex flex-col gap-4" : "hidden"} - > - <SweepLane - lane={data.lanes.backfill} - // Backfill is ONE lane over SEVERAL operations, and their counts are - // in different units (attribution-text is ~1 model call per transcript - // chunk; diarization is one audio pass per video). There is no single - // band for it, and inventing one by summing is the exact mistake - // laneKindEntriesOf exists to make unspellable. - band={null} - onRefresh={refresh} - /> - </section> - </div> - ); -} - -// operation id -> the state of the lane that would dispatch it. -// -// SEVERAL OPERATIONS SHARE ONE LANE, and that is the fact worth seeing: every -// backfill kind runs on one queue, so a diarization row reading "Holding" and -// an attribution row reading "Holding" are one gate, not two. The rail draws -// them separately because their WORK COUNTS are in different units, and maps -// them onto the same lane state because their DISPATCH is not. -// -// Any operation the console has no lane for reads "Off" rather than "Idle" — -// see deriveLaneState: an idle-looking lane claims "all caught up" where the -// truth is "nothing would run this". -function railStates(data: AutoQueueStatusPayload): Record<string, RailLaneState> { - const out: Record<string, RailLaneState> = {}; - - for (const status of [data.transcription, data.download]) { - const running = status.runner.running; - const inFlight = status.runner.inFlight.length; - out[status.kind] = { - // A runner has no sweep; its feed is the job queue, so `activeCount` is - // what tells running from idle. A STOPPED runner is "Off", not "Idle" — - // it will never pick anything up, which "Idle" does not say. - state: !running - ? "unavailable" - : deriveLaneState({ gateHeld: false, activeCount: inFlight }), - note: running ? idleReasonText(status.runner.idleReason, status.kind) : null, - }; - } - - const sweepState = (sweep: SweepLaneStatus): RailLaneState => ({ - state: deriveLaneState({ - available: sweep.available, - gateHeld: sweep.gateHeld, - feedRunning: sweep.sweeping, - activeCount: sweep.inFlight.length, - }), - note: !sweep.available - ? "no operation switched on" - : sweep.gateHeld - ? sweep.sweeping - ? "sweep armed, lane paused" - : "lane paused" - : sweep.sweeping || sweep.inFlight.length > 0 - ? null - : "no sweep armed", - }); - - out.digest = sweepState(data.lanes.digest); - // Every OTHER catalog operation is dispatched by the one backfill lane, so it - // takes that lane's state — including the ones with no band of their own. - const backfill = sweepState(data.lanes.backfill); - for (const band of data.lanes.bands) { - if (out[band.id]) continue; - out[band.id] = backfill; - } - return out; -} - -function KindLane({ - kind, - title, - status, - channels, - platforms, - buckets, - now, - hidden, - onRefresh, -}: { - kind: AutoQueueKind; - title: string; - status: AutoQueueKindStatus; - channels: Channel[]; - platforms: string[]; - buckets: string[]; - now: number | null; - hidden: boolean; - onRefresh: () => Promise<void>; -}) { - const [busy, setBusy] = useState(false); - - const control = useCallback( - async (action: "start" | "stop" | "drain") => { - setBusy(true); - try { - await fetch("/api/auto-queue/control", { - method: "POST", - headers: { "content-type": "application/json" }, - body: JSON.stringify({ kind, action }), - }); - await onRefresh(); - } finally { - setBusy(false); - } - }, - [kind, onRefresh], - ); - - return ( - // data-hydrated is a TESTING AFFORDANCE, and a deliberate one. A React state - // update that lands before hydration is silently discarded — no error, no - // request, nothing — and this repo has lost days to that failure mode more - // than once. `now` is null until the client mount effect runs, so it is an - // exact "React is live on this subtree" signal that already exists; exposing - // it lets a test wait for the page rather than race it. - <section - // See the note at the digest section: the `hidden` attribute needs the - // class switched with it, or a `flex` utility overrides it. - className={hidden ? "hidden" : "flex flex-col gap-4"} - hidden={hidden} - // MEASURED, and it contradicts the plan this redesign came from: keeping - // a lane MOUNTED does NOT keep it addressable. A heading inside a - // display:none subtree is out of the accessibility tree, so - // locator("section", { has: getByRole("heading", …) }) resolves to ZERO - // elements for a hidden lane — and `toBeHidden()` passes vacuously on an - // element that does not resolve at all. - // - // That is correct behaviour (a screen reader must not see the hidden - // pane either), so the fix is an addressing hook rather than a rendering - // change. What mounting genuinely preserves is the lane's React state — - // an unsaved policy edit survives a switch away and back. - data-lane={kind} - data-hydrated={now !== null ? "true" : undefined} - > - <LaneHeader - title={title} - status={status} - now={now} - busy={busy} - onControl={control} - /> - - {kind === "transcription" && ( - <p className="text-xs text-muted-foreground"> - Auto-transcribe runs at background priority: a manually-triggered - transcription preempts it for the next free worker — the in-flight auto - transcription finishes (drains), then the manual one runs, then auto - resumes. - </p> - )} - - {status.cooldowns.length > 0 && ( - <CooldownStrip cooldowns={status.cooldowns} now={now} /> - )} - - <SnoozeControl - kind={kind} - title={title} - snoozeUntil={status.policy.snoozeUntil ?? null} - now={now} - onChanged={onRefresh} - /> - - <NextUp status={status} channels={channels} /> - <InFlightList status={status} channels={channels} now={now} /> - - <PolicyTreeEditor - kind={kind} - status={status} - channels={channels} - platforms={platforms} - buckets={buckets} - /> - - <RecentPicks status={status} /> - </section> - ); -} - -// The pick log, now cross-referenced against the ladder by ORDINAL rather than -// by the leafLabel() id lookup this page used to carry — the rung numbers are -// right there, so "rule 2" is a pointer you can follow with your eye. -function RecentPicks({ status }: { status: AutoQueueKindStatus }) { - const leaves = leafOrder(status.policy.root); - return ( - <div className="flex flex-col gap-1"> - <p className="font-mono text-xs uppercase tracking-[0.14em] text-muted-foreground"> - Recent picks - </p> - {status.picks.length === 0 ? ( - <p className="text-sm text-muted-foreground">Nothing picked yet.</p> - ) : ( - <ul className="flex flex-col gap-0.5 text-sm"> - {status.picks.slice(0, 15).map((p, i) => { - const at = leaves.findIndex((l) => l.id === p.leafId); - return ( - <li - key={`${p.at}-${i}`} - className="flex flex-wrap gap-x-3 text-muted-foreground" - > - <span className="tabular-nums">{formatClock(p.at)}</span> - <span className="font-mono text-foreground"> - {p.channelSlug}/{p.videoId} - </span> - <span className="text-xs"> - {at >= 0 ? `rule ${at + 1}` : p.leafId} - </span> - </li> - ); - })} - </ul> - )} - </div> - ); -} - -// Wall-clock that re-renders each second, starting null so SSR and the first -// client render agree (no hydration mismatch from Date.now()). -function useNow(): number | null { - const [now, setNow] = useState<number | null>(null); - useEffect(() => { - setNow(Date.now()); - const id = setInterval(() => setNow(Date.now()), 1000); - return () => clearInterval(id); - }, []); - return now; -} - -// Platforms paused by a rate-limit/network backoff. A manual Sync on one of -// these is refused until it lapses; the runner skips it meanwhile. -function CooldownStrip({ - cooldowns, - now, -}: { - cooldowns: PlatformCooldownView[]; - now: number | null; -}) { - return ( - <div className="flex flex-col gap-1 rounded-md border border-warning/30 bg-warning-soft px-3 py-2 text-sm"> - <span className="font-medium text-warning"> - Rate-limit cooldown — auto-download and manual sync are paused on: - </span> - <ul className="flex flex-wrap gap-x-4 gap-y-1 text-warning"> - {cooldowns.map((c) => { - const secs = - now === null - ? null - : Math.max(0, Math.ceil((c.untilMs - now) / 1000)); - return ( - <li key={c.platform} className="tabular-nums"> - <span className="font-mono">{c.platform}</span> - {secs !== null && ( - <> — {formatCooldown(secs)} left (attempt {c.fails})</> - )} - </li> - ); - })} - </ul> - </div> - ); -} diff --git a/editor/app/auto-queue/components/OperationRail.tsx b/editor/app/auto-queue/components/OperationRail.tsx @@ -1,178 +0,0 @@ -"use client"; - -import { - bandCoverage, - bandDenominator, - type OperationBand, -} from "../../components/pipelines/band"; -import { - BandLegend, - StateBand, -} from "../../components/pipelines/StateBand"; -import type { LaneState } from "../../components/lanes/laneState"; -import { LANE_DOT, LANE_TEXT, LANE_WORD } from "../../components/lanes/laneState"; - -// What a rail row says about the lane that would do this operation's work, in -// the vocabulary deriveLaneState already established: Running / Holding / Idle / -// Off. "Holding" is the state this codebase already named for "sweep armed, -// gate shut", and it is precisely what a crowded card was failing to say. -export type RailLaneState = { - state: LaneState; - // Why it is not working, in words — idleReasonText for a runner, the gate for - // a sweep. Null when it IS working, or when there is nothing to explain. - note: string | null; -}; - -// THE COMPARISON RAIL: every pipeline, one line each, always present. -// -// It is the lane switcher's context, not decoration. On this corpus the normal -// case is that a lane is idle BECAUSE another one is — attribution-diarized is -// 99.9% blocked behind diarization, diarization is 99.2% media-gone — and -// without the rail you have to switch lanes to discover that, one lane at a -// time, which is exactly the question a console should answer without being -// asked. -// -// EXACTLY ONE SATURATED COLOUR ON THE PAGE: `reachable`. Everything else is -// texture on neutral. Glancing down four bands shows you WHERE THE COLOUR IS, -// which is "where work can happen right now" — and that is the whole boldness -// budget, in agreement with LaneCard's own note that a third tinted background -// would be the accessory to remove. -// -// FILL PATTERN FIRST, HUE SECOND. Not a stylistic whim: report-to-video's claim -// rail established here BY MEASUREMENT that no four-colour palette clears -// all-pairs colour-blindness. Pattern (solid / hatched / dotted / hollow) -// survives CVD, greyscale, and a dim laptop at 2am, which is when this console -// is actually read. -// -// EACH BAND IS NORMALISED TO ITS OWN `eligible`, stated on its own row. Digest's -// 79,681 is genuinely a different population from diarization's 78,019, and -// quietly sharing one denominator to make the bars comparable would be a lie -// about what is being compared. - -export function OperationRail({ - bands, - states, - selectedId, -}: { - bands: OperationBand[]; - // operation id -> the state of the lane that dispatches it. Several - // operations can share one lane (every backfill kind runs on one queue), and - // that is exactly the fact the rail exists to make visible. - states: Record<string, RailLaneState>; - // The lane the console is focused on, highlighted so the rail and the - // dropdown are visibly the same control. - selectedId: string | null; -}) { - return ( - <div className="rounded-lg border border-border bg-card"> - <p className="border-b border-border px-4 py-2 font-mono text-xs uppercase tracking-[0.14em] text-muted-foreground"> - Pipelines - </p> - <ul className="divide-y divide-border"> - {bands.map((band) => ( - <RailRow - key={band.id} - band={band} - lane={states[band.id] ?? { state: "unavailable", note: null }} - selected={band.id === selectedId} - /> - ))} - </ul> - <BandLegend className="border-t border-border px-4 py-2" /> - </div> - ); -} - -function RailRow({ - band, - lane, - selected, -}: { - band: OperationBand; - lane: RailLaneState; - selected: boolean; -}) { - const denominator = bandDenominator(band); - const coverage = bandCoverage(band); - - return ( - <li - data-operation={band.id} - className={`flex flex-wrap items-center gap-x-4 gap-y-1 px-4 py-2.5 text-sm ${ - selected ? "bg-muted/40" : "" - }`} - > - <span - className={`flex min-w-36 shrink-0 items-center gap-2 ${ - selected ? "font-medium text-foreground" : "text-muted-foreground" - }`} - > - <span - aria-hidden="true" - className={`size-2 shrink-0 rounded-full ${LANE_DOT[lane.state]}`} - /> - {band.label} - </span> - <StateBand band={band} size="rail" /> - {/* PLAIN TEXT, never a heading — the suite scopes whole tests with - locator("section", { has: heading "Auto-transcribe" }), and a second - element with that accessible name would make every one ambiguous. */} - <span className={`shrink-0 text-xs ${LANE_TEXT[lane.state]}`}> - {LANE_WORD[lane.state]} - {lane.note ? ( - <span className="text-muted-foreground">: {lane.note}</span> - ) : null} - </span> - <span className="ml-auto flex shrink-0 items-baseline gap-3 text-xs text-muted-foreground"> - {/* THE FIGURES ARE STATED SEPARATELY AND NEVER SUMMED. On the measured - corpus reachable and needs-media are 91x apart; one "remaining" - number would say the same thing about a finished lane and a lane - that cannot start. */} - <Figure n={band.reachable} unit="reachable" tone="text-foreground" /> - {band.blocked > 0 && <Figure n={band.blocked} unit="blocked" />} - {band.missingInput > 0 && ( - <Figure n={band.missingInput} unit="no media" /> - )} - {band.deferred > 0 && <Figure n={band.deferred} unit="held" />} - <span className="tabular-nums"> - {coverage === null - ? "coverage unknown" - : `${(coverage * 100).toFixed(coverage < 0.1 ? 2 : 0)}% of ${denominator?.toLocaleString()}`} - </span> - {/* WHAT THIS BACKLOG COSTS, only where there IS one and only for an - operation this system actually dispatches. The sentence the console - was missing: attribution-text sits at 77,000-odd reachable on a lane - whose unit is the transcript CHUNK — the order of 194,000 model - calls — and read exactly like diarization's 647 audio passes. - Stated flat. A "this is a lot" threshold would be a magic number the - next operation gets wrong, and what is affordable is not this page's - call to make. */} - {band.dispatched && band.reachable > 0 && band.costBasis && ( - <span - aria-label={`${band.label} cost`} - className="italic text-muted-foreground/80" - > - {band.costBasis} - </span> - )} - </span> - </li> - ); -} - -function Figure({ - n, - unit, - tone = "", -}: { - n: number; - unit: string; - tone?: string; -}) { - return ( - <span> - <span className={`tabular-nums ${tone}`}>{n.toLocaleString()}</span>{" "} - {unit} - </span> - ); -} diff --git a/editor/app/auto-queue/page.tsx b/editor/app/auto-queue/page.tsx @@ -1,57 +0,0 @@ -import type { Metadata } from "next"; -import Link from "next/link"; -import { getPaths } from "yt-dlp-transcript-common/lib/paths"; -import { listChannelConfigs } from "yt-dlp-transcript-common/controller/channels"; -import { PLATFORM_VALUES } from "yt-dlp-transcript-common/lib/platform"; -import { selectableBucketsForKind } from "yt-dlp-transcript-common/jobs/autoQueuePolicy"; -import { buildAutoQueueStatusPayload } from "./status"; -import { AutoQueueView } from "./components/AutoQueueView"; -import { HowPriorityWorks } from "./components/HowPriorityWorks"; - -export const dynamic = "force-dynamic"; - -export const metadata: Metadata = { title: "Auto-queue" }; - -// Buckets each runner can draw from — derived from the policy engine's single -// source of truth (selectableBucketsForKind) so the picker can't drift from the -// runner. Includes the opt-in auto-caption buckets: a leaf that names one gets -// per-channel opt-in without flipping the runner-wide switch. -const BUCKETS_BY_KIND = { - transcription: [...selectableBucketsForKind("transcription")], - download: [...selectableBucketsForKind("download")], -}; - -export default async function AutoQueuePage() { - const [initial, channels] = await Promise.all([ - buildAutoQueueStatusPayload(), - listChannelConfigs(getPaths()), - ]); - const channelOptions = channels.map((c) => ({ - slug: c.slug, - name: c.config.name ?? null, - })); - - return ( - <div className="flex flex-col gap-4"> - <div className="flex items-center justify-between"> - <h1 className="font-display text-2xl font-semibold tracking-tight"> - Auto-queue - </h1> - <Link href="/scheduler" className="text-sm underline"> - Sync schedule - </Link> - </div> - <p className="text-sm text-muted-foreground"> - Decide, continuously and unattended, which video across every channel - gets worked on next. - </p> - <HowPriorityWorks /> - <AutoQueueView - initial={initial} - channels={channelOptions} - platforms={[...PLATFORM_VALUES]} - bucketsByKind={BUCKETS_BY_KIND} - /> - </div> - ); -} diff --git a/editor/app/channels/[slug]/components/flow/FlowStation.tsx b/editor/app/channels/[slug]/components/flow/FlowStation.tsx @@ -83,7 +83,7 @@ export function StationFoot({ station }: { station: Station }) { // three unrelated pipelines. Its members are three different // populations in two different units, and the only honest picture is // three bands — the same instrument, at the smallest of its three - // scales, so a reader coming from /channels or /auto-queue already + // scales, so a reader coming from /channels or /operations already // knows what the fills mean. station.operations.map((op) => ( <StationPipeline diff --git a/editor/app/channels/[slug]/components/stages/TranscribeStage.tsx b/editor/app/channels/[slug]/components/stages/TranscribeStage.tsx @@ -130,8 +130,8 @@ function AutoSubsSection({ speech recognition (no punctuation, rolling duplicate cues,{" "} <code>[Music]</code> filler). Replacing them costs an audio download plus a transcription each, so nothing happens automatically unless the{" "} - <a href="/auto-queue" className="underline"> - auto-queue + <a href="/operations/transcription" className="underline"> + transcription operation </a>{" "} opts in. Human-written captions are never listed here. The old VTT is kept as a backup; purge it from the Cleanup stage when you no longer diff --git a/editor/app/channels/[slug]/lib/channelFlow.ts b/editor/app/channels/[slug]/lib/channelFlow.ts @@ -69,7 +69,7 @@ export type FlowStationId = | "backfill"; // One pipeline drawn under a station. The band is the same instrument the -// /channels strip and the /auto-queue rail draw, at station scale — which is +// /channels strip and the /operations rail draw, at station scale — which is // what makes a figure here and a figure there impossible to disagree. export type StationOperation = { id: string; diff --git a/editor/app/channels/actions.ts b/editor/app/channels/actions.ts @@ -42,7 +42,7 @@ import { import { queueForSlugs, type QueueOutcome } from "./lib/queueForSlugs"; import { storePlaylistAction, syncAction } from "./[slug]/pipelineActions"; import { fetchPostsAction } from "./[slug]/socialActions"; -import { prioritizeChannelDownloadAction } from "../auto-queue/actions"; +import { prioritizeChannelDownloadAction } from "../operations/actions"; export type ActionResult = { error: string } | undefined; diff --git a/editor/app/components/dashboard/ChannelsTable.tsx b/editor/app/components/dashboard/ChannelsTable.tsx @@ -3,7 +3,7 @@ import Link from "next/link"; import { useState, useTransition } from "react"; import { syncAction } from "../../channels/[slug]/pipelineActions"; -import { prioritizeChannelDownloadAction } from "../../auto-queue/actions"; +import { prioritizeChannelDownloadAction } from "../../operations/actions"; import { InlineActionButton } from "../../actionable/components/InlineActionButton"; import { fmtTime } from "../../widget/lib/relativeTime"; import type { DashboardChannel } from "./types"; diff --git a/editor/app/components/dashboard/PipelineBand.tsx b/editor/app/components/dashboard/PipelineBand.tsx @@ -68,9 +68,22 @@ export function PipelineBand({ className="flex flex-col gap-3 rounded-md border border-border bg-card px-4 py-4" > <div className="flex flex-wrap items-center justify-between gap-3"> - <h2 className="font-mono text-xs uppercase tracking-[0.14em] text-muted-foreground"> - Pipeline - </h2> + <div className="flex items-baseline gap-3"> + <h2 className="font-mono text-xs uppercase tracking-[0.14em] text-muted-foreground"> + Pipeline + </h2> + {/* THE MONITOR WIDGET IS REACHED FROM THE THING IT MIRRORS, and no + longer from the primary nav. It is a PROJECTION of this band's own + payload drawn for a second screen — the same lanes, the same + figures — and a projection is not a place. See + editor/app/lib/nav.ts and plans/editor-operations-ia.md. */} + <Link + href="/widget/builder" + className="font-mono text-xs uppercase tracking-[0.14em] text-muted-foreground underline underline-offset-4 hover:text-foreground" + > + Monitor ↗ + </Link> + </div> <div className="flex flex-wrap items-center gap-x-4 gap-y-1 text-sm tabular-nums"> <Instrument dotClass={running > 0 ? "bg-info animate-pulse motion-reduce:animate-none" : "bg-muted-foreground/40"} diff --git a/editor/app/components/pipelines/StateBand.tsx b/editor/app/components/pipelines/StateBand.tsx @@ -9,7 +9,7 @@ import { // THE INSTRUMENT. One component, three scales, one vocabulary. // -// The comparison rail built for /auto-queue answers "what can this pipeline do +// The comparison rail built for /operations answers "what can this pipeline do // right now" for the whole corpus. Shrunk, the same instrument answers it per // channel; shrunk again, per station on a channel page. Nothing new is invented // here — the reason the three surfaces read as one system is that they are diff --git a/editor/app/components/pipelines/band.ts b/editor/app/components/pipelines/band.ts @@ -10,9 +10,9 @@ // So StateBand and every client surface import THIS, and the builder stays // server-side. Nothing here may import anything but types. // -// LIVES UNDER components/pipelines/ RATHER THAN auto-queue/, because three +// LIVES UNDER components/pipelines/ RATHER THAN operations/, because three // surfaces at three scales now render the same five fills: the corpus rail on -// /auto-queue, the per-channel strip on /channels, and the station foot on a +// /operations, the per-channel strip on /channels, and the station foot on a // channel page. That they are literally one model, one component and one // palette is the reason the three read as one instrument rather than three // charts that happen to rhyme. @@ -212,7 +212,7 @@ export function bandHeadline(band: OperationBand): string { // ── A BAND FOR THE OPERATIONS A SWEEP HAS IN SCOPE ────────────────────────── // -// The plan on /auto-queue draws one strip per channel, and the strip has to +// The plan on /operations draws one strip per channel, and the strip has to // re-draw when an operation is ticked off — in the browser, with no round trip. // So the fold lives here, on the directive-free side, beside the type. // diff --git a/editor/app/lib/nav.ts b/editor/app/lib/nav.ts @@ -10,7 +10,6 @@ import { LayoutDashboard, ListChecks, ListPlus, - Monitor, Regex, Rocket, ScrollText, @@ -26,6 +25,23 @@ import { // (editor/app/components/CommandPalette.tsx) so they never drift — the old // palette hard-coded a stale 5-item subset. Icons are used by the palette and // the compact sidebar; `badgeKey` drives the live count pills / changelog dot. +// +// FOUR GROUPS, ONE NOUN EACH: the Corpus (what is on disk), the Operations that +// derive things from it, the Sites that publish it, and the Machine that runs +// it. That is the model in plans/editor-operations-ia.md, and the nav is where +// it is either true or not — a group called "Pool" holding ten unrelated tools +// was the shape of the problem, not a labelling accident. +// +// THE CAP IS THE RULE UMTOOL ALREADY WROTE, and it is quoted here rather than +// re-derived so the two consoles do not drift into two nav philosophies. +// umtool/components/AppNav.tsx: "SEVEN visible entries, and the cap is still +// NINE … The next tool goes UNDER one of these, not beside them — which is +// exactly what happened to the four judging piles and the sources page: they +// are the `song ▸` group now, at the same URLs they always had." +// +// So: FOLD, DO NOT ADD, and every retired route REDIRECTS rather than 404s +// (see editor/next.config.ts). The entries marked "interim" below are the ones +// later slices fold into the page above them; the end state is eleven. export type NavBadgeKey = "jobs" | "running" | "changelog" | "cleanable"; @@ -45,40 +61,65 @@ export type NavGroup = { export const NAV_GROUPS: NavGroup[] = [ { - label: "Site", + label: "Corpus", links: [ - { href: "/", label: "Dashboard", icon: LayoutDashboard, keywords: "home overview" }, + { href: "/", label: "Dashboard", icon: LayoutDashboard, keywords: "home overview monitor widget pipeline" }, { href: "/channels", label: "Channels", icon: Tv }, + ], + }, + { + label: "Operations", + links: [ + { + href: "/operations", + label: "Operations", + icon: ListPlus, + keywords: "auto queue runner policy lane sweep pipeline digest diarization attribution download transcription", + }, + // Interim, until slice 4 moves its per-operation sections onto + // /operations/<id> and the rest onto /cleanup, Sites and the dashboard. + { href: "/actionable", label: "Actionable", icon: TriangleAlert, keywords: "needs attention todo" }, + // Interim, until slice 8 makes this /operations/sync. + { href: "/scheduler", label: "Schedule", icon: CalendarClock, keywords: "sync cron cadence" }, + ], + }, + { + label: "Sites", + links: [ + { href: "/sites", label: "Sites", icon: Globe }, + // Interim: slice 5 folds these four into /sites/[siteId]. They are all + // per-site facts already; only their routes say otherwise. { href: "/charts", label: "Charts", icon: ChartColumnBig, keywords: "stats graphs" }, { href: "/aliases", label: "Search aliases", icon: Regex, keywords: "synonyms suggestions regex search terms" }, { href: "/deploy", label: "Deploy", icon: Rocket, keywords: "publish release" }, + { href: "/build", label: "Build", icon: Hammer, keywords: "static export" }, + { href: "/homepage", label: "Homepage", icon: House, keywords: "hub" }, ], }, { - label: "Pool", + label: "Machine", links: [ { href: "/jobs", label: "Jobs", icon: ListChecks, badgeKey: "jobs", keywords: "queue tasks" }, { href: "/jobs/active", label: "Active", icon: Activity, badgeKey: "running", keywords: "monitor running live" }, { href: "/workers", label: "Workers", icon: Cpu, keywords: "transcription whisper gpu" }, - { href: "/widget/builder", label: "Monitor", icon: Monitor, keywords: "widget embed" }, - { href: "/scheduler", label: "Schedule", icon: CalendarClock, keywords: "sync cron cadence" }, - { href: "/auto-queue", label: "Auto-queue", icon: ListPlus, keywords: "auto runner policy" }, - { href: "/build", label: "Build", icon: Hammer, keywords: "static export" }, - { href: "/actionable", label: "Actionable", icon: TriangleAlert, keywords: "needs attention todo" }, { href: "/cleanup", label: "Cleanup", icon: Trash2, badgeKey: "cleanable", keywords: "clean reclaim disk space prune audio" }, { href: "/saved-videos", label: "Saved videos", icon: Bookmark, keywords: "backup" }, - ], - }, - { - label: "Manage", - links: [ - { href: "/sites", label: "Sites", icon: Globe }, - { href: "/homepage", label: "Homepage", icon: House, keywords: "hub" }, { href: "/settings", label: "Settings", icon: Settings, keywords: "config preferences" }, { href: "/changelog", label: "Changelog", icon: ScrollText, badgeKey: "changelog", keywords: "release notes" }, ], }, ]; +// /widget/builder IS NOT HERE, deliberately. The monitor widget is a PROJECTION +// of the board's payload — the same lanes, the same figures, drawn for a second +// screen — and a projection is not a place. It is reached from the pipeline +// band on the dashboard, which is the thing it mirrors. +// +// THAT COSTS IT THE COMMAND PALETTE, and the cost is stated rather than papered +// over: the palette renders NAV_GROUPS and has no way to type a URL, so leaving +// the nav removes it from the palette too. "monitor" and "widget" are keywords +// on Dashboard instead, which is where the link now lives — the palette takes +// you to the page that has it, not to a dead end. + // Flat list of every nav link (palette convenience). export const NAV_LINKS: NavLink[] = NAV_GROUPS.flatMap((g) => g.links); diff --git a/editor/app/operations/[id]/page.tsx b/editor/app/operations/[id]/page.tsx @@ -0,0 +1,122 @@ +import type { Metadata } from "next"; +import { notFound } from "next/navigation"; +import { getPaths } from "yt-dlp-transcript-common/lib/paths"; +import { listChannelConfigs } from "yt-dlp-transcript-common/controller/channels"; +import { PLATFORM_VALUES } from "yt-dlp-transcript-common/lib/platform"; +import { selectableBucketsForKind } from "yt-dlp-transcript-common/jobs/autoQueuePolicy"; +import { operationCatalog } from "yt-dlp-transcript-common/lib/backfillKinds"; +import type { AutoQueueKind } from "yt-dlp-transcript-common/jobs/autoQueueState"; +import { getRegistry } from "yt-dlp-transcript-common/jobs/registry"; +import { buildAutoQueueStatusPayload } from "../status"; +import { OperationDetail } from "../components/OperationDetail"; + +export const dynamic = "force-dynamic"; + +// Buckets each runner can draw from — derived from the policy engine's single +// source of truth (selectableBucketsForKind) so the picker can't drift from the +// runner. Includes the opt-in auto-caption buckets: a leaf that names one gets +// per-channel opt-in without flipping the runner-wide switch. +const BUCKETS_BY_KIND = { + transcription: [...selectableBucketsForKind("transcription")], + download: [...selectableBucketsForKind("download")], +}; + +// The job kinds that ARE this operation, for the running-jobs list. Only the +// sweep-fed operations need one: a runner lane lists its own units in flight, +// and a second list beside it would be the pile this redesign is undoing. +// +// Keyed by LANE rather than by operation, because that is the truth: one job +// on the shared queue is doing whichever kinds the sweep armed. +const JOB_KINDS_BY_LANE: Record<string, readonly string[]> = { + digest: [ + "digest-sweep", + "digest-channel-local", + "digest-channel-remote", + "digest-share-cluster", + ], + backfill: ["backfill-sweep", "backfill-channel", "diarize-channel"], +}; + +function descriptorFor(id: string) { + return operationCatalog().find((o) => o.id === id) ?? null; +} + +export async function generateMetadata({ + params, +}: { + params: Promise<{ id: string }>; +}): Promise<Metadata> { + const { id } = await params; + const op = descriptorFor(id); + return { title: op ? op.label : "Operation" }; +} + +export default async function OperationPage({ + params, +}: { + params: Promise<{ id: string }>; +}) { + const { id } = await params; + // THE CATALOG IS THE ROUTE TABLE. An operation is a registry entry, so an id + // the registry does not know is a 404 and not an empty console — and an + // operation ADDED to the registry gets this page with no route work at all. + const op = descriptorFor(id); + if (!op) notFound(); + + const runnerKind: AutoQueueKind | null = + id === "transcription" || id === "download" ? id : null; + + const [initial, channels] = await Promise.all([ + buildAutoQueueStatusPayload(), + listChannelConfigs(getPaths()), + ]); + const channelOptions = channels.map((c) => ({ + slug: c.slug, + name: c.config.name ?? null, + })); + + const laneJobKinds = runnerKind + ? [] + : (JOB_KINDS_BY_LANE[id === "digest" ? "digest" : "backfill"] ?? []); + const activeJobs = + laneJobKinds.length === 0 + ? [] + : getRegistry() + .list() + .filter( + (j) => + laneJobKinds.includes(j.kind) && + (j.status === "running" || j.status === "queued"), + ) + .map((j) => ({ + id: j.id, + kind: j.kind, + status: j.status as "queued" | "running", + queueKey: j.queueKey, + channelSlug: j.channelSlug, + videoId: j.videoId, + })); + + return ( + <div className="flex flex-col gap-4"> + <div className="flex flex-wrap items-baseline justify-between gap-2"> + <h1 className="font-display text-2xl font-semibold tracking-tight"> + {op.label} + </h1> + <p className="font-mono text-xs uppercase tracking-[0.14em] text-muted-foreground"> + {op.costBasis} + </p> + </div> + <p className="text-sm text-muted-foreground">{op.hint}</p> + <OperationDetail + id={id} + initial={initial} + channels={channelOptions} + platforms={[...PLATFORM_VALUES]} + bucketsByKind={BUCKETS_BY_KIND} + activeJobs={activeJobs} + runnerKind={runnerKind} + /> + </div> + ); +} diff --git a/editor/app/operations/actions.ts b/editor/app/operations/actions.ts @@ -0,0 +1,265 @@ +"use server"; + +import { revalidatePath } from "next/cache"; +import { + getSettings, + writeSettings, + type SiteSettings, +} from "yt-dlp-transcript-common/lib/settings"; +import { + startAutoRunner, + stopAutoRunner, +} from "yt-dlp-transcript-common/controller/autoRunner"; +import type { AutoQueueKind } from "yt-dlp-transcript-common/jobs/autoQueueState"; +import { + isGroup, + type AutoQueueGroup, + type AutoQueueNode, + type AutoQueueOrder, + type AutoQueueReach, +} from "yt-dlp-transcript-common/jobs/autoQueuePolicy"; + +// EVERY OPERATIONS SURFACE, not just the board. The console is a board plus one +// page per operation, and every action here changes something both of them +// draw — the second argument makes the dynamic segment revalidate as a route +// rather than as one literal path, which is the only way to reach +// /operations/digest and /operations/diarization without naming them. +// +// The old single revalidatePath("/auto-queue") outlived its route: that path is +// a redirect now, and revalidating it refreshes nothing. +function revalidateOperations(): void { + revalidatePath("/operations"); + revalidatePath("/operations/[id]", "page"); +} + +export type SaveResult = { ok: true } | { ok: false; error: string }; + +// Persist one runner kind's policy and bring its runner up/down to match, +// WITHOUT a server restart. writeSettings sanitizes the tree (sanitizeAutoQueue), +// so a slightly-off client payload is coerced rather than trusted. Enabling +// starts the runner immediately; disabling is picked up by the running loop on +// its next iteration (getSettings reads from disk), so it stops on its own. +export async function saveAutoQueueAction( + kind: AutoQueueKind, + input: { + enabled: boolean; + maxWorkers: number | null; + // Opt in to the lowest-priority replace-auto-captions lane (default false). + replaceAutoSubs: boolean; + // Ordering WITHIN each rule (newest / oldest upload first, or listed order). + order: AutoQueueOrder; + root: AutoQueueGroup; + }, +): Promise<SaveResult> { + const current = getSettings(); + // NOTE: this object lists every persisted policy field EXPLICITLY, so a field + // added to AutoQueuePolicy and forgotten here is silently dropped on every + // save rather than failing loudly. `snoozeUntil` is deliberately carried over + // from `current` instead of taken from the form: the snooze is set by its own + // action, and a policy save (e.g. reordering rules) must not cancel it. + const next: SiteSettings = { + ...current, + autoQueue: { + ...current.autoQueue, + [kind]: { + enabled: input.enabled, + maxWorkers: input.maxWorkers, + replaceAutoSubs: input.replaceAutoSubs === true, + order: input.order, + snoozeUntil: current.autoQueue[kind].snoozeUntil ?? null, + root: input.root, + }, + }, + }; + try { + await writeSettings(next); + } catch (e) { + return { ok: false, error: (e as Error).message }; + } + if (input.enabled) { + await startAutoRunner(kind); + } + revalidateOperations(); + return { ok: true }; +} + +// Explicit start/stop for the Start/Stop buttons (also reachable as +// /api/auto-queue/control for e2e). Start is a no-op when the policy is disabled. +export async function startAutoQueueAction( + kind: AutoQueueKind, +): Promise<SaveResult> { + const jobId = await startAutoRunner(kind); + if (jobId === null && !getSettings().autoQueue[kind].enabled) { + return { ok: false, error: "Enable the policy before starting the runner." }; + } + revalidateOperations(); + return { ok: true }; +} + +export async function stopAutoQueueAction( + kind: AutoQueueKind, +): Promise<SaveResult> { + stopAutoRunner(kind); + revalidateOperations(); + return { ok: true }; +} + +// Idle a runner until `untilMs` (epoch ms) without stopping it, or wake it now +// with null. Deliberately NOT part of saveAutoQueueAction: snoozing is a +// one-click operational act, and routing it through the policy form would mean a +// pending tree edit had to be saved (or discarded) to snooze. The runner re-reads +// settings every iteration, so this takes effect on the next tick — and because +// it lives in settings.json rather than runner memory, it survives a restart. +export async function snoozeAutoQueueAction( + kind: AutoQueueKind, + untilMs: number | null, +): Promise<SaveResult> { + const current = getSettings(); + const next: SiteSettings = { + ...current, + autoQueue: { + ...current.autoQueue, + [kind]: { ...current.autoQueue[kind], snoozeUntil: untilMs }, + }, + }; + try { + await writeSettings(next); + } catch (e) { + return { ok: false, error: (e as Error).message }; + } + revalidateOperations(); + return { ok: true }; +} + +// Recursively drop every leaf that matches this channel, so re-prioritizing the +// same channel doesn't accumulate duplicate leaves (a group emptied of children +// is kept — sanitizeAutoQueue tolerates it, and removing it could orphan a +// group the operator configured). Returns a fresh tree. +function stripChannelLeaves(node: AutoQueueNode, slug: string): AutoQueueNode { + if (!isGroup(node)) return node; + const children = node.children + .filter( + (c) => + isGroup(c) || + !(c.match.type === "channel" && c.match.value === slug), + ) + .map((c) => stripChannelLeaves(c, slug)); + return { ...node, children }; +} + +// "Add to top of auto-queue": prepend a channel leaf at the HEAD of the download +// policy's strict root (first child = highest priority), enable the download +// policy, and start the runner if it isn't up. Directly uses the existing +// policy-tree engine — no engine change. Idempotent: any prior leaf for this +// channel is stripped first so the head stays the single owner (first-match-wins +// in buildPendingByLeaf). +export async function prioritizeChannelDownloadAction( + slug: string, +): Promise<SaveResult> { + const trimmed = (slug ?? "").trim(); + if (!trimmed) return { ok: false, error: "No channel slug supplied." }; + const current = getSettings(); + const download = current.autoQueue.download; + const stripped = stripChannelLeaves(download.root, trimmed) as AutoQueueGroup; + const nextRoot: AutoQueueGroup = { + ...stripped, + children: [ + { id: `prioritize-${trimmed}`, match: { type: "channel", value: trimmed } }, + ...stripped.children, + ], + }; + const next: SiteSettings = { + ...current, + autoQueue: { + ...current.autoQueue, + download: { ...download, enabled: true, root: nextRoot }, + }, + }; + try { + await writeSettings(next); + } catch (e) { + return { ok: false, error: (e as Error).message }; + } + await startAutoRunner("download"); + revalidateOperations(); + return { ok: true }; +} + +// Order and reach for the two SWEEP-fed lanes. +// +// A separate action from saveAutoQueueAction because these lanes have no policy +// tree to save alongside: their order lives in settings.digest / settings. +// backfill, not in settings.autoQueue. Routing them through the policy form +// would mean an operator could not change an order without also committing a +// pending tree edit for a different lane. +// +// SPREAD-AND-OVERRIDE, never a rebuilt literal. settings.digest carries +// `sweepEnabled` and its scope, and settings.backfill carries both plus +// `allowRedownload`; a literal here that omitted one would disarm a multi-week +// sweep, or write media to a 97%-full disk, on an unrelated save. That exact +// bug is documented in the settings form's own digest block. +// +// It takes effect on the sweep's NEXT PASS — both sweeps re-read the setting per +// pass rather than at launch — and on the next per-channel batch, which resolves +// the order itself. Nothing needs restarting. +export async function saveLaneOrderAction( + lane: "digest" | "backfill", + input: { order: AutoQueueOrder; reach: AutoQueueReach }, +): Promise<SaveResult> { + const current = getSettings(); + const next: SiteSettings = + lane === "digest" + ? { + ...current, + digest: { + ...current.digest, + recencyOrder: input.order, + recencyReach: input.reach, + }, + } + : { + ...current, + backfill: { + ...current.backfill, + order: input.order, + reach: input.reach, + }, + }; + try { + await writeSettings(next); + } catch (e) { + return { ok: false, error: (e as Error).message }; + } + revalidateOperations(); + return { ok: true }; +} + +// The arbiter: start / stop the unified dispatcher. +// +// Kept out of saveLaneOrderAction and out of the policy form: starting a +// dispatcher is an operational act, not a setting, and routing it through a +// form would mean a pending tree edit had to be saved or discarded first. +// +// It REFUSES rather than silently no-opping when a sweep is armed, and the +// refusal carries the reason — two dispatchers on one lane would start the same +// channel twice, and an operator who clicks Start and sees nothing happen +// deserves to be told which switch is in the way. +export async function startArbiterAction(): Promise<SaveResult> { + const { startArbiter } = await import( + "yt-dlp-transcript-common/controller/arbiter" + ); + const result = await startArbiter(); + revalidateOperations(); + return result.jobId + ? { ok: true } + : { ok: false, error: result.error ?? "The arbiter could not be started." }; +} + +export async function stopArbiterAction(): Promise<SaveResult> { + const { stopArbiter } = await import( + "yt-dlp-transcript-common/controller/arbiter" + ); + stopArbiter(); + revalidateOperations(); + return { ok: true }; +} diff --git a/editor/app/auto-queue/components/ArbiterBar.tsx b/editor/app/operations/components/ArbiterBar.tsx diff --git a/editor/app/auto-queue/components/ClaimLadder.tsx b/editor/app/operations/components/ClaimLadder.tsx diff --git a/editor/app/auto-queue/components/HowPriorityWorks.tsx b/editor/app/operations/components/HowPriorityWorks.tsx diff --git a/editor/app/auto-queue/components/InFlightList.tsx b/editor/app/operations/components/InFlightList.tsx diff --git a/editor/app/auto-queue/components/LadderRung.tsx b/editor/app/operations/components/LadderRung.tsx diff --git a/editor/app/auto-queue/components/LaneHeader.tsx b/editor/app/operations/components/LaneHeader.tsx diff --git a/editor/app/auto-queue/components/NextUp.tsx b/editor/app/operations/components/NextUp.tsx diff --git a/editor/app/operations/components/OperationDetail.tsx b/editor/app/operations/components/OperationDetail.tsx @@ -0,0 +1,164 @@ +"use client"; + +import type { AutoQueueKind } from "yt-dlp-transcript-common/jobs/autoQueueState"; +import type { AutoQueueStatusPayload } from "../status"; +import { HowPriorityWorks } from "./HowPriorityWorks"; +import { OperationRail } from "./OperationRail"; +import { RunnerOperationView } from "./RunnerOperationView"; +import { SweepLane } from "./SweepLane"; +import { railStates, sweepLaneIdFor } from "./railStates"; +import { useHydrated, useOperationsStatus } from "./useOperationsStatus"; +import { type Channel } from "./dispatch"; +import { StateBand } from "../../components/pipelines/StateBand"; +import { + RunningJobsList, + type RunningJobsListItem, +} from "../../jobs/components/RunningJobsList"; + +// ONE OPERATION, IN FULL. +// +// THE RAIL IS STILL AT THE TOP, and that is not decoration either: on this +// corpus the normal case is that a lane is idle BECAUSE another one is, so the +// answer to "why is this operation not moving" is usually on a different row. +// It was the switcher's context when the switcher was a <select>; it is the +// same context now that the switcher is a set of links. +export function OperationDetail({ + id, + initial, + channels, + platforms, + bucketsByKind, + activeJobs, + runnerKind, +}: { + id: string; + initial: AutoQueueStatusPayload; + channels: Channel[]; + platforms: string[]; + bucketsByKind: Record<AutoQueueKind, string[]>; + // Jobs on this operation's queues, SSR-rendered. Empty renders nothing at + // all (RunningJobsList returns null), so a quiet page carries no second + // in-flight list beside the lane's own. + activeJobs: RunningJobsListItem[]; + // Set for the two operations a RUNNER dispatches; null for everything the + // registry sweeps. Resolved on the server against operationCatalog() so this + // component never re-decides which is which. + runnerKind: AutoQueueKind | null; +}) { + const { data, refresh } = useOperationsStatus(initial); + + return ( + <div className="flex flex-col gap-6"> + <OperationRail + bands={data.lanes.bands} + states={railStates(data)} + selectedId={id} + /> + + {runnerKind ? ( + <> + <HowPriorityWorks /> + <RunnerOperationView + kind={runnerKind} + title={runnerKind === "transcription" ? "Auto-transcribe" : "Auto-download"} + status={data[runnerKind]} + channels={channels} + platforms={platforms} + buckets={bucketsByKind[runnerKind]} + onRefresh={refresh} + /> + </> + ) : ( + <SweepOperationView + id={id} + data={data} + activeJobs={activeJobs} + onRefresh={refresh} + /> + )} + </div> + ); +} + +// A REGISTRY OPERATION: digest, or one of the speaker kinds. +// +// The lane below is SHARED — every kind on BACKFILL_QUEUE is dispatched by one +// sweep and held by one gate — and the panel says so rather than letting a +// per-operation page imply a per-operation switch. What IS this operation's +// alone is the band: its own five populations, its own denominator, in its own +// unit. That is the figure the shared panel could not draw before, because a +// lane holding three operations has no single total that means anything. +function SweepOperationView({ + id, + data, + activeJobs, + onRefresh, +}: { + id: string; + data: AutoQueueStatusPayload; + activeJobs: RunningJobsListItem[]; + onRefresh: () => Promise<void>; +}) { + const hydrated = useHydrated(); + const laneId = sweepLaneIdFor(id); + const lane = data.lanes[laneId]; + const band = data.lanes.bands.find((b) => b.id === id) ?? null; + // The other operations this lane would dispatch, named. Not a count: "shares + // a lane with 2 others" is exactly the sentence that made "Backfill" mean + // three different costs. + const siblings = lane.operations.filter((op) => op.id !== id); + // In the CATALOG but not on the lane: the operation is registered and + // switched off. Worth a sentence, because everything else on this page — an + // available lane, an armable sweep, a plan — would otherwise read as "ready". + const switchedOn = lane.operations.some((op) => op.id === id); + + return ( + <section + // TWO HOOKS, because there are two facts. data-lane names the LANE this + // panel controls — what the e2e suite scopes the sweep and pause buttons + // by, and deliberately the SAME value on every speaker operation's page, + // since they are one lane. data-operation names which operation the page + // is about, which the lane cannot say. data-hydrated is the same testing + // affordance the runner sections carry; see RunnerOperationView. + data-lane={laneId} + data-operation={id} + data-hydrated={hydrated ? "true" : undefined} + className="flex flex-col gap-4" + > + {band && ( + <div className="flex flex-col gap-2 rounded-lg border border-border bg-card px-4 py-3"> + <StateBand band={band} size="rail" /> + </div> + )} + + {!switchedOn && ( + <p className="text-sm text-warning"> + This operation is switched off in Settings, so nothing dispatches it — + not the sweep below, and not the arbiter. That is not the same as + being finished, which is what an empty lane would otherwise imply. + </p> + )} + + {siblings.length > 0 && ( + <p className="text-sm text-muted-foreground"> + This operation shares one lane, one sweep and one pause with{" "} + {joinLabels(siblings.map((s) => s.label))}. Arming the sweep below + runs whatever is ticked in its scope, not just this one — and the + counts are in different units, so read the scope before arming it. + </p> + )} + + <RunningJobsList jobs={activeJobs} /> + + <SweepLane lane={lane} band={band} onRefresh={onRefresh} /> + </section> + ); +} + +// "A", "A and B", "A, B and C". Written out because the sentence it lands in is +// the one that keeps an operator from reading a per-operation page as a +// per-operation switch. +function joinLabels(labels: string[]): string { + if (labels.length <= 1) return labels[0] ?? ""; + return `${labels.slice(0, -1).join(", ")} and ${labels[labels.length - 1]}`; +} diff --git a/editor/app/operations/components/OperationRail.tsx b/editor/app/operations/components/OperationRail.tsx @@ -0,0 +1,186 @@ +"use client"; + +import Link from "next/link"; +import { + bandCoverage, + bandDenominator, + type OperationBand, +} from "../../components/pipelines/band"; +import { + BandLegend, + StateBand, +} from "../../components/pipelines/StateBand"; +import type { LaneState } from "../../components/lanes/laneState"; +import { LANE_DOT, LANE_TEXT, LANE_WORD } from "../../components/lanes/laneState"; + +// What a rail row says about the lane that would do this operation's work, in +// the vocabulary deriveLaneState already established: Running / Holding / Idle / +// Off. "Holding" is the state this codebase already named for "sweep armed, +// gate shut", and it is precisely what a crowded card was failing to say. +export type RailLaneState = { + state: LaneState; + // Why it is not working, in words — idleReasonText for a runner, the gate for + // a sweep. Null when it IS working, or when there is nothing to explain. + note: string | null; +}; + +// THE COMPARISON RAIL: every pipeline, one line each, always present. +// +// It IS the switcher — each row's name links to that operation's page — and it +// is the context for whatever is below it, not decoration. On this corpus the +// normal case is that a lane is idle BECAUSE another one is — +// attribution-diarized is 99.9% blocked behind diarization, diarization is +// 99.2% media-gone — and without the rail you would have to open one operation +// at a time to discover that, which is exactly the question a console should +// answer without being asked. +// +// EXACTLY ONE SATURATED COLOUR ON THE PAGE: `reachable`. Everything else is +// texture on neutral. Glancing down four bands shows you WHERE THE COLOUR IS, +// which is "where work can happen right now" — and that is the whole boldness +// budget, in agreement with LaneCard's own note that a third tinted background +// would be the accessory to remove. +// +// FILL PATTERN FIRST, HUE SECOND. Not a stylistic whim: report-to-video's claim +// rail established here BY MEASUREMENT that no four-colour palette clears +// all-pairs colour-blindness. Pattern (solid / hatched / dotted / hollow) +// survives CVD, greyscale, and a dim laptop at 2am, which is when this console +// is actually read. +// +// EACH BAND IS NORMALISED TO ITS OWN `eligible`, stated on its own row. Digest's +// 79,681 is genuinely a different population from diarization's 78,019, and +// quietly sharing one denominator to make the bars comparable would be a lie +// about what is being compared. + +export function OperationRail({ + bands, + states, + selectedId, +}: { + bands: OperationBand[]; + // operation id -> the state of the lane that dispatches it. Several + // operations can share one lane (every backfill kind runs on one queue), and + // that is exactly the fact the rail exists to make visible. + states: Record<string, RailLaneState>; + // The operation this page is about, highlighted so the rail says where you + // are. Null on the board, which is about all of them. + selectedId: string | null; +}) { + return ( + <div className="rounded-lg border border-border bg-card"> + <p className="border-b border-border px-4 py-2 font-mono text-xs uppercase tracking-[0.14em] text-muted-foreground"> + Pipelines + </p> + <ul className="divide-y divide-border"> + {bands.map((band) => ( + <RailRow + key={band.id} + band={band} + lane={states[band.id] ?? { state: "unavailable", note: null }} + selected={band.id === selectedId} + /> + ))} + </ul> + <BandLegend className="border-t border-border px-4 py-2" /> + </div> + ); +} + +function RailRow({ + band, + lane, + selected, +}: { + band: OperationBand; + lane: RailLaneState; + selected: boolean; +}) { + const denominator = bandDenominator(band); + const coverage = bandCoverage(band); + + return ( + <li + data-operation={band.id} + className={`flex flex-wrap items-center gap-x-4 gap-y-1 px-4 py-2.5 text-sm ${ + selected ? "bg-muted/40" : "" + }`} + > + {/* THE ROW'S NAME IS THE LINK, not the whole row. A link wrapping the + row would take its accessible name from every figure in it — + "Digest 1,204 reachable 79,681 …" — which is a name that changes on + every poll and cannot be addressed. The dot is aria-hidden, so the + link is named exactly the operation. */} + <Link + href={`/operations/${band.id}`} + className={`flex min-w-36 shrink-0 items-center gap-2 hover:underline hover:underline-offset-2 ${ + selected ? "font-medium text-foreground" : "text-muted-foreground" + }`} + > + <span + aria-hidden="true" + className={`size-2 shrink-0 rounded-full ${LANE_DOT[lane.state]}`} + /> + {band.label} + </Link> + <StateBand band={band} size="rail" /> + {/* PLAIN TEXT, never a heading — the suite scopes whole tests with + locator("section", { has: heading "Auto-transcribe" }), and a second + element with that accessible name would make every one ambiguous. */} + <span className={`shrink-0 text-xs ${LANE_TEXT[lane.state]}`}> + {LANE_WORD[lane.state]} + {lane.note ? ( + <span className="text-muted-foreground">: {lane.note}</span> + ) : null} + </span> + <span className="ml-auto flex shrink-0 items-baseline gap-3 text-xs text-muted-foreground"> + {/* THE FIGURES ARE STATED SEPARATELY AND NEVER SUMMED. On the measured + corpus reachable and needs-media are 91x apart; one "remaining" + number would say the same thing about a finished lane and a lane + that cannot start. */} + <Figure n={band.reachable} unit="reachable" tone="text-foreground" /> + {band.blocked > 0 && <Figure n={band.blocked} unit="blocked" />} + {band.missingInput > 0 && ( + <Figure n={band.missingInput} unit="no media" /> + )} + {band.deferred > 0 && <Figure n={band.deferred} unit="held" />} + <span className="tabular-nums"> + {coverage === null + ? "coverage unknown" + : `${(coverage * 100).toFixed(coverage < 0.1 ? 2 : 0)}% of ${denominator?.toLocaleString()}`} + </span> + {/* WHAT THIS BACKLOG COSTS, only where there IS one and only for an + operation this system actually dispatches. The sentence the console + was missing: attribution-text sits at 77,000-odd reachable on a lane + whose unit is the transcript CHUNK — the order of 194,000 model + calls — and read exactly like diarization's 647 audio passes. + Stated flat. A "this is a lot" threshold would be a magic number the + next operation gets wrong, and what is affordable is not this page's + call to make. */} + {band.dispatched && band.reachable > 0 && band.costBasis && ( + <span + aria-label={`${band.label} cost`} + className="italic text-muted-foreground/80" + > + {band.costBasis} + </span> + )} + </span> + </li> + ); +} + +function Figure({ + n, + unit, + tone = "", +}: { + n: number; + unit: string; + tone?: string; +}) { + return ( + <span> + <span className={`tabular-nums ${tone}`}>{n.toLocaleString()}</span>{" "} + {unit} + </span> + ); +} diff --git a/editor/app/operations/components/OperationsBoard.tsx b/editor/app/operations/components/OperationsBoard.tsx @@ -0,0 +1,130 @@ +"use client"; + +import Link from "next/link"; +import type { AutoQueueStatusPayload } from "../status"; +import type { SyncRowView } from "../syncRow"; +import { ArbiterBar } from "./ArbiterBar"; +import { OperationRail } from "./OperationRail"; +import { railStates } from "./railStates"; +import { useHydrated, useOperationsStatus } from "./useOperationsStatus"; +import { LANE_DOT, LANE_TEXT, LANE_WORD } from "../../components/lanes/laneState"; + +// THE BOARD: every operation this install runs, one line each, and the +// dispatcher above them. It is the whole of the page — there is nothing here +// that is about ONE operation, because that is what /operations/<id> is for. +// +// This is what the lane <select> used to hide. The console before it stacked +// two identical runner panels and had no room for the other pipelines at all, +// so "the digest lane is idle because transcription is hogging the GPU" was a +// fact you could only assemble by visiting three pages. Then it was one page +// with a dropdown, which said the same thing in one place but made "open the +// download lane" an action with no address. Now the rail's rows are links: the +// glance and the drill-down are the same control. +export function OperationsBoard({ + initial, + sync, +}: { + initial: AutoQueueStatusPayload; + sync: SyncRowView; +}) { + const { data, refresh } = useOperationsStatus(initial); + const hydrated = useHydrated(); + + return ( + // data-board is the board's addressing hook, and data-hydrated the signal + // that React is live on it — the arbiter's buttons below are inert until it + // is. See useHydrated. + <div + data-board="operations" + data-hydrated={hydrated ? "true" : undefined} + className="flex flex-col gap-6" + > + <OperationRail + bands={data.lanes.bands} + states={railStates(data)} + selectedId={null} + /> + + <ArbiterBar arbiter={data.lanes.arbiter} onRefresh={refresh} /> + + <SyncRow sync={sync} /> + </div> + ); +} + +// SYNC IS AN OPERATION TOO, and the board would be lying by omission without +// it: a corpus that has stopped noticing new videos is not idle, it is broken, +// and every figure on the rail above would still read "all caught up". +// +// It is a row rather than a rail entry because it is the one operation that is +// CHANNEL-scoped and CADENCE-triggered rather than per-video and backlog-fed, +// so it has no reachable/blocked band to draw. It gets a real rail row when the +// descriptor gains `scope` and `trigger` and /scheduler becomes /operations/sync +// — see plans/editor-operations-ia.md, slices 1 and 8. +// +// RENDERED FROM THE SSR PAYLOAD, not polled. A cadence measured in minutes does +// not need a 3-second poll, and a second poll on this page would buy a spinner +// nobody asked for. +function SyncRow({ sync }: { sync: SyncRowView }) { + // The same four words the rail uses, and they mean the same things here. + // "Holding" is the honest one for a channel that is due and has not been + // picked up: the scheduler is not stopped and it is not working — the next + // tick is what moves it. + const state = !sync.enabled + ? "unavailable" + : sync.running > 0 + ? "running" + : sync.due > 0 + ? "holding" + : "idle"; + const note = !sync.enabled + ? "the scheduler is off — new videos are only noticed by a manual sync" + : sync.heartbeatSeconds === 0 + ? "no internal timer; ticks come from an external `pnpm sync:tick`" + : sync.due > 0 + ? "picked up on the next tick" + : null; + return ( + <div className="rounded-lg border border-border bg-card"> + <p className="border-b border-border px-4 py-2 font-mono text-xs uppercase tracking-[0.14em] text-muted-foreground"> + Sync + </p> + <div className="flex flex-wrap items-center gap-x-4 gap-y-1 px-4 py-2.5 text-sm"> + <Link + href="/scheduler" + className="flex min-w-36 shrink-0 items-center gap-2 text-muted-foreground hover:underline hover:underline-offset-2" + > + <span + aria-hidden="true" + className={`size-2 shrink-0 rounded-full ${LANE_DOT[state]}`} + /> + Sync schedule + </Link> + <span className={`shrink-0 text-xs ${LANE_TEXT[state]}`}> + {LANE_WORD[state]} + {note && <span className="text-muted-foreground">: {note}</span>} + </span> + <span className="ml-auto flex shrink-0 items-baseline gap-3 text-xs text-muted-foreground"> + <span> + <span className="tabular-nums text-foreground"> + {sync.eligible.toLocaleString()} + </span>{" "} + channels on a cadence + </span> + {sync.due > 0 && ( + <span> + <span className="tabular-nums">{sync.due.toLocaleString()}</span> due + now + </span> + )} + {sync.overdue > 0 && ( + <span className="text-warning"> + <span className="tabular-nums">{sync.overdue.toLocaleString()}</span>{" "} + overdue + </span> + )} + </span> + </div> + </div> + ); +} diff --git a/editor/app/auto-queue/components/OrderReach.tsx b/editor/app/operations/components/OrderReach.tsx diff --git a/editor/app/auto-queue/components/PolicyTreeEditor.tsx b/editor/app/operations/components/PolicyTreeEditor.tsx diff --git a/editor/app/operations/components/RunnerOperationView.tsx b/editor/app/operations/components/RunnerOperationView.tsx @@ -0,0 +1,211 @@ +"use client"; + +import { useCallback, useEffect, useState } from "react"; +import type { AutoQueueKind } from "yt-dlp-transcript-common/jobs/autoQueueState"; +import type { AutoQueueKindStatus, PlatformCooldownView } from "../status"; +import { InFlightList } from "./InFlightList"; +import { LaneHeader } from "./LaneHeader"; +import { NextUp } from "./NextUp"; +import { PolicyTreeEditor } from "./PolicyTreeEditor"; +import { SnoozeControl } from "./SnoozeControl"; +import { type Channel, formatClock, formatCooldown, leafOrder } from "./dispatch"; + +// ONE RUNNER OPERATION, IN FULL — download or transcription. +// +// Lifted VERBATIM out of AutoQueueView's KindLane when the console became one +// page per operation. The only thing that changed is what is around it: there +// is no lane <select> any more, so there is no `hidden` prop, no second lane +// mounted beside this one, and no reason to keep an unmounted pane's React +// state alive. +// +// STRUCTURAL CONTRACT, load-bearing for ~1,200 lines of e2e. Every clause below +// survived the move and must keep surviving: +// * this is a literal <section> containing an <h2> named exactly +// "Auto-transcribe" / "Auto-download", with the policy editor and the +// Start/Drain/Stop buttons inside it; +// * NOTHING inside it is itself a <section> — the suite scopes with +// locator("section", { has: heading }), and a nested one would match two +// ancestors, failing every scoped lookup on strict mode; +// * the rail above names pipelines in plain text, never as headings, for the +// same reason; +// * policy controls stay native <select> / <input type=checkbox> — Radix +// portals its options and unmounts them while closed, which would take the +// page-wide option counts the suite asserts to zero; +// * role="status" stays reserved for "Saved."; +// * data-hydrated stays on the section. +export function RunnerOperationView({ + kind, + title, + status, + channels, + platforms, + buckets, + onRefresh, +}: { + kind: AutoQueueKind; + title: string; + status: AutoQueueKindStatus; + channels: Channel[]; + platforms: string[]; + buckets: string[]; + onRefresh: () => Promise<void>; +}) { + const [busy, setBusy] = useState(false); + const now = useNow(); + + const control = useCallback( + async (action: "start" | "stop" | "drain") => { + setBusy(true); + try { + await fetch("/api/auto-queue/control", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ kind, action }), + }); + await onRefresh(); + } finally { + setBusy(false); + } + }, + [kind, onRefresh], + ); + + return ( + // data-hydrated is a TESTING AFFORDANCE, and a deliberate one. A React state + // update that lands before hydration is silently discarded — no error, no + // request, nothing — and this repo has lost days to that failure mode more + // than once. `now` is null until the client mount effect runs, so it is an + // exact "React is live on this subtree" signal that already exists; exposing + // it lets a test wait for the page rather than race it. + <section + className="flex flex-col gap-4" + data-lane={kind} + data-hydrated={now !== null ? "true" : undefined} + > + <LaneHeader + title={title} + status={status} + now={now} + busy={busy} + onControl={control} + /> + + {kind === "transcription" && ( + <p className="text-xs text-muted-foreground"> + Auto-transcribe runs at background priority: a manually-triggered + transcription preempts it for the next free worker — the in-flight auto + transcription finishes (drains), then the manual one runs, then auto + resumes. + </p> + )} + + {status.cooldowns.length > 0 && ( + <CooldownStrip cooldowns={status.cooldowns} now={now} /> + )} + + <SnoozeControl + kind={kind} + title={title} + snoozeUntil={status.policy.snoozeUntil ?? null} + now={now} + onChanged={onRefresh} + /> + + <NextUp status={status} channels={channels} /> + <InFlightList status={status} channels={channels} now={now} /> + + <PolicyTreeEditor + kind={kind} + status={status} + channels={channels} + platforms={platforms} + buckets={buckets} + /> + + <RecentPicks status={status} /> + </section> + ); +} + +// The pick log, cross-referenced against the ladder by ORDINAL rather than by a +// leafLabel() id lookup — the rung numbers are right there, so "rule 2" is a +// pointer you can follow with your eye. +function RecentPicks({ status }: { status: AutoQueueKindStatus }) { + const leaves = leafOrder(status.policy.root); + return ( + <div className="flex flex-col gap-1"> + <p className="font-mono text-xs uppercase tracking-[0.14em] text-muted-foreground"> + Recent picks + </p> + {status.picks.length === 0 ? ( + <p className="text-sm text-muted-foreground">Nothing picked yet.</p> + ) : ( + <ul className="flex flex-col gap-0.5 text-sm"> + {status.picks.slice(0, 15).map((p, i) => { + const at = leaves.findIndex((l) => l.id === p.leafId); + return ( + <li + key={`${p.at}-${i}`} + className="flex flex-wrap gap-x-3 text-muted-foreground" + > + <span className="tabular-nums">{formatClock(p.at)}</span> + <span className="font-mono text-foreground"> + {p.channelSlug}/{p.videoId} + </span> + <span className="text-xs"> + {at >= 0 ? `rule ${at + 1}` : p.leafId} + </span> + </li> + ); + })} + </ul> + )} + </div> + ); +} + +// Wall-clock that re-renders each second, starting null so SSR and the first +// client render agree (no hydration mismatch from Date.now()). +export function useNow(): number | null { + const [now, setNow] = useState<number | null>(null); + useEffect(() => { + setNow(Date.now()); + const id = setInterval(() => setNow(Date.now()), 1000); + return () => clearInterval(id); + }, []); + return now; +} + +// Platforms paused by a rate-limit/network backoff. A manual Sync on one of +// these is refused until it lapses; the runner skips it meanwhile. +function CooldownStrip({ + cooldowns, + now, +}: { + cooldowns: PlatformCooldownView[]; + now: number | null; +}) { + return ( + <div className="flex flex-col gap-1 rounded-md border border-warning/30 bg-warning-soft px-3 py-2 text-sm"> + <span className="font-medium text-warning"> + Rate-limit cooldown — auto-download and manual sync are paused on: + </span> + <ul className="flex flex-wrap gap-x-4 gap-y-1 text-warning"> + {cooldowns.map((c) => { + const secs = + now === null + ? null + : Math.max(0, Math.ceil((c.untilMs - now) / 1000)); + return ( + <li key={c.platform} className="tabular-nums"> + <span className="font-mono">{c.platform}</span> + {secs !== null && ( + <> — {formatCooldown(secs)} left (attempt {c.fails})</> + )} + </li> + ); + })} + </ul> + </div> + ); +} diff --git a/editor/app/auto-queue/components/SaveBar.tsx b/editor/app/operations/components/SaveBar.tsx diff --git a/editor/app/auto-queue/components/SnoozeControl.tsx b/editor/app/operations/components/SnoozeControl.tsx diff --git a/editor/app/auto-queue/components/SweepLane.tsx b/editor/app/operations/components/SweepLane.tsx diff --git a/editor/app/auto-queue/components/SweepPlan.tsx b/editor/app/operations/components/SweepPlan.tsx diff --git a/editor/app/auto-queue/components/SweepScope.tsx b/editor/app/operations/components/SweepScope.tsx diff --git a/editor/app/auto-queue/components/dispatch.ts b/editor/app/operations/components/dispatch.ts diff --git a/editor/app/operations/components/railStates.ts b/editor/app/operations/components/railStates.ts @@ -0,0 +1,76 @@ +import type { AutoQueueStatusPayload } from "../status"; +import type { SweepLaneStatus } from "../lanes"; +import { deriveLaneState } from "../../components/lanes/laneState"; +import type { RailLaneState } from "./OperationRail"; +import { idleReasonText } from "./dispatch"; + +// operation id -> the state of the lane that would dispatch it. +// +// SEVERAL OPERATIONS SHARE ONE LANE, and that is the fact worth seeing: every +// backfill kind runs on one queue, so a diarization row reading "Holding" and +// an attribution row reading "Holding" are one gate, not two. The rail draws +// them separately because their WORK COUNTS are in different units, and maps +// them onto the same lane state because their DISPATCH is not. +// +// Any operation the console has no lane for reads "Off" rather than "Idle" — +// see deriveLaneState: an idle-looking lane claims "all caught up" where the +// truth is "nothing would run this". +// +// LIVES IN ITS OWN MODULE because the board and every operation page draw the +// same rail, and two copies of this fold would let two pages disagree about +// which lane is holding. +export function railStates( + data: AutoQueueStatusPayload, +): Record<string, RailLaneState> { + const out: Record<string, RailLaneState> = {}; + + for (const status of [data.transcription, data.download]) { + const running = status.runner.running; + const inFlight = status.runner.inFlight.length; + out[status.kind] = { + // A runner has no sweep; its feed is the job queue, so `activeCount` is + // what tells running from idle. A STOPPED runner is "Off", not "Idle" — + // it will never pick anything up, which "Idle" does not say. + state: !running + ? "unavailable" + : deriveLaneState({ gateHeld: false, activeCount: inFlight }), + note: running ? idleReasonText(status.runner.idleReason, status.kind) : null, + }; + } + + const sweepState = (sweep: SweepLaneStatus): RailLaneState => ({ + state: deriveLaneState({ + available: sweep.available, + gateHeld: sweep.gateHeld, + feedRunning: sweep.sweeping, + activeCount: sweep.inFlight.length, + }), + note: !sweep.available + ? "no operation switched on" + : sweep.gateHeld + ? sweep.sweeping + ? "sweep armed, lane paused" + : "lane paused" + : sweep.sweeping || sweep.inFlight.length > 0 + ? null + : "no sweep armed", + }); + + out.digest = sweepState(data.lanes.digest); + // Every OTHER catalog operation is dispatched by the one backfill lane, so it + // takes that lane's state — including the ones with no band of their own. + const backfill = sweepState(data.lanes.backfill); + for (const band of data.lanes.bands) { + if (out[band.id]) continue; + out[band.id] = backfill; + } + return out; +} + +// The lane that dispatches an operation, in the two ids the payload is keyed +// by. Digest has its own queue; everything else the registry dispatches rides +// the shared backfill lane. Stated once, because the page that renders a lane +// and the page that names it must not answer this differently. +export function sweepLaneIdFor(operationId: string): "digest" | "backfill" { + return operationId === "digest" ? "digest" : "backfill"; +} diff --git a/editor/app/operations/components/useOperationsStatus.ts b/editor/app/operations/components/useOperationsStatus.ts @@ -0,0 +1,58 @@ +"use client"; + +import { useCallback, useEffect, useRef, useState } from "react"; +import type { AutoQueueStatusPayload } from "../status"; + +// THE ONE POLL. Every operations surface — the board and each operation page — +// reads the same payload from the same endpoint on the same 3-second cadence, +// seeded by SSR so the first paint is not empty. +// +// ONE PAYLOAD, NOT ONE PER PANEL: the rail's whole purpose is that the lanes +// are read TOGETHER, and two polls would let the rail and the lane below it +// disagree about the same moment. +// +// The endpoint keeps its /api/auto-queue/* path. It is addressed directly by +// three specs and by nothing user-facing, so renaming it would be churn with a +// test bill and no reader. +export function useOperationsStatus(initial: AutoQueueStatusPayload): { + data: AutoQueueStatusPayload; + refresh: () => Promise<void>; +} { + const [data, setData] = useState<AutoQueueStatusPayload>(initial); + const mounted = useRef(true); + + const refresh = useCallback(async () => { + try { + const res = await fetch("/api/auto-queue/status", { cache: "no-store" }); + if (!res.ok) return; + const next = (await res.json()) as AutoQueueStatusPayload; + if (mounted.current) setData(next); + } catch { + /* transient; next poll retries */ + } + }, []); + + useEffect(() => { + mounted.current = true; + const id = setInterval(refresh, 3000); + return () => { + mounted.current = false; + clearInterval(id); + }; + }, [refresh]); + + return { data, refresh }; +} + +// "React is live on this subtree" — the same signal `now !== null` gives the +// runner sections, for the surfaces that have no clock of their own. +// +// A TESTING AFFORDANCE, and a deliberate one: a React state update that lands +// before hydration is silently discarded — no error, no request, nothing — and +// this repo has lost days to that failure mode more than once. Exposing it lets +// a test wait for the page rather than race it. +export function useHydrated(): boolean { + const [hydrated, setHydrated] = useState(false); + useEffect(() => setHydrated(true), []); + return hydrated; +} diff --git a/editor/app/auto-queue/lanes.ts b/editor/app/operations/lanes.ts diff --git a/editor/app/operations/page.tsx b/editor/app/operations/page.tsx @@ -0,0 +1,38 @@ +import type { Metadata } from "next"; +import { buildAutoQueueStatusPayload } from "./status"; +import { buildSyncRow } from "./syncRow"; +import { OperationsBoard } from "./components/OperationsBoard"; + +export const dynamic = "force-dynamic"; + +export const metadata: Metadata = { title: "Operations" }; + +// THE BOARD. Every operation this install runs, and the dispatcher above them. +// +// One page per operation is the whole redesign (plans/editor-operations-ia.md). +// Before it this route was one page with a lane <select>: the rail said what +// every pipeline was doing, and then a dropdown decided which one you could act +// on. A dropdown is not an address — you could not link to the download lane, +// the browser Back button did not return to it, and the two panes it switched +// between had to stay mounted to keep their edits alive. The rail's rows are +// links now, and each destination is a page. +export default async function OperationsPage() { + const [initial, sync] = await Promise.all([ + buildAutoQueueStatusPayload(), + buildSyncRow(), + ]); + + return ( + <div className="flex flex-col gap-4"> + <h1 className="font-display text-2xl font-semibold tracking-tight"> + Operations + </h1> + <p className="text-sm text-muted-foreground"> + Every operation that turns this archive into a derived corpus, and what + each one is doing right now. Open one to set its rules, arm its sweep or + hold its lane. + </p> + <OperationsBoard initial={initial} sync={sync} /> + </div> + ); +} diff --git a/editor/app/auto-queue/status.ts b/editor/app/operations/status.ts diff --git a/editor/app/operations/syncRow.ts b/editor/app/operations/syncRow.ts @@ -0,0 +1,43 @@ +import { getRegistry } from "yt-dlp-transcript-common/jobs/registry"; +import { buildSchedulerStatusPayload } from "../scheduler/status"; + +// THE SYNC ROW, folded down to what the board can act on. The full +// per-channel schedule stays on /scheduler (which becomes /operations/sync in +// slice 8 — see plans/editor-operations-ia.md). +export type SyncRowView = { + enabled: boolean; + // Channels the scheduler would sync on a cadence — not the channel count. + eligible: number; + // Eligible and past due right now. + due: number; + // Past due AND flagged overdue by the scheduler's own rule, which is the + // stronger statement: a channel that keeps missing its window. + overdue: number; + // Sync jobs running or queued this instant, from the registry rather than + // from the schedule — a manual "Sync all" is sync work too, and a board that + // only counted the scheduler's own picks would read "Idle" while the box was + // saturated. + running: number; + // 0 = no internal timer; ticks come from an external `pnpm sync:tick`. + heartbeatSeconds: number; +}; + +export async function buildSyncRow(): Promise<SyncRowView> { + const payload = await buildSchedulerStatusPayload(); + const eligible = payload.channels.filter((c) => c.autoSyncEligible); + return { + enabled: payload.scheduler.enabled, + eligible: eligible.length, + due: eligible.filter( + (c) => c.nextDueAt !== null && c.nextDueAt <= payload.now, + ).length, + overdue: eligible.filter((c) => c.overdue).length, + running: getRegistry() + .list() + .filter( + (j) => + j.kind === "sync" && (j.status === "running" || j.status === "queued"), + ).length, + heartbeatSeconds: payload.heartbeatSeconds, + }; +} diff --git a/editor/app/settings/components/SettingsForm.tsx b/editor/app/settings/components/SettingsForm.tsx @@ -920,10 +920,10 @@ export function SettingsForm({ initial, apps, digestApps, workerTags }: Props) { scope saved separately would be resurrected as a corpus-wide run by the next restart — so it lives with the plan it produces, on{" "} <Link - href="/auto-queue" + href="/operations" className="underline underline-offset-2 hover:text-foreground" > - Auto-queue + Operations </Link> , where you can read the itinerary before committing to it. </p> diff --git a/editor/e2e/auto-queue.spec.ts b/editor/e2e/auto-queue.spec.ts @@ -370,9 +370,13 @@ test("UI: build a policy in the editor, save it, and start the runner", async ({ }, }); - await page.goto("/auto-queue"); - await expect(page.getByRole("heading", { name: "Auto-queue" })).toBeVisible(); - // Scope to the Auto-transcribe section (the page also has an Auto-download one). + await page.goto("/operations/transcription"); + await expect( + page.getByRole("heading", { name: "Transcription", exact: true }), + ).toBeVisible(); + // Scope to the Auto-transcribe section. There is exactly one runner section + // per page now — the lane <select> is gone — but the scoping stays, because + // it is what keeps every locator in this suite unambiguous. const section = page.locator("section", { has: page.getByRole("heading", { name: "Auto-transcribe" }), }); @@ -440,7 +444,7 @@ test("UI: Move to top jumps a rule to the front of its siblings", async ({ }, }); - await page.goto("/auto-queue"); + await page.goto("/operations/transcription"); const section = page.locator("section", { has: page.getByRole("heading", { name: "Auto-transcribe" }), }); @@ -590,7 +594,7 @@ test("Active Jobs: Drain and Cancel buttons work on the runner job", async ({ await expect.poll(() => runnerRunning(request), { timeout: 15_000 }).toBe(false); }); -test("Auto-queue page: the Drain button stops the runner gracefully", async ({ +test("Transcription page: the Drain button stops the runner gracefully", async ({ page, request, }) => { @@ -599,7 +603,7 @@ test("Auto-queue page: the Drain button stops the runner gracefully", async ({ await writeSettings(IDLE_SETTINGS(ALPHA_ROOT)); await startRunner(request); - await page.goto("/auto-queue"); + await page.goto("/operations/transcription"); const section = page.locator("section", { has: page.getByRole("heading", { name: "Auto-transcribe" }), }); @@ -677,7 +681,7 @@ test("Drain completes when an auto-transcribe unit is parked behind a busy worke // 3) Drain the runner. The parked a1 unit must unblock (skip) so the runner // finalizes instead of hanging. - await page.goto("/auto-queue"); + await page.goto("/operations/transcription"); const section = page.locator("section", { has: page.getByRole("heading", { name: "Auto-transcribe" }), }); @@ -733,7 +737,7 @@ test("Drain completes (and lets the unit finish) when an auto-transcribe unit is // 2) Drain the runner. Pre-fix this hung forever (event-loop starvation); // post-fix the loop paces on a real timer, the running unit finishes, and // the runner finalizes well within the timeout. - await page.goto("/auto-queue"); + await page.goto("/operations/transcription"); const section = page.locator("section", { has: page.getByRole("heading", { name: "Auto-transcribe" }), }); @@ -828,7 +832,7 @@ test("UI: the order select round-trips through save", async ({ page }) => { }, }); - await page.goto("/auto-queue"); + await page.goto("/operations/transcription"); const section = page.locator("section", { has: page.getByRole("heading", { name: "Auto-transcribe" }), }); @@ -982,7 +986,7 @@ test("a running runner says why it is idle", async ({ page, request }) => { ) .toBe("no-pending"); - await page.goto("/auto-queue"); + await page.goto("/operations/transcription"); const section = page.locator("section", { has: page.getByRole("heading", { name: "Auto-transcribe" }), }); @@ -1038,7 +1042,7 @@ test("snooze idles the runner without stopping it, and Wake now resumes", async // Waking is a live settings change, not a restart: the same runner picks the // work up on its next iteration. - await page.goto("/auto-queue"); + await page.goto("/operations/transcription"); const section = page.locator("section", { has: page.getByRole("heading", { name: "Auto-transcribe" }), }); @@ -1054,9 +1058,14 @@ test("snooze idles the runner without stopping it, and Wake now resumes", async expect((await getStatus(request)).transcription.runner.running).toBe(true); }); -// --- The console: the rail, the switcher, and the two sweep lanes ----------- +// --- The console: the board, the rail, and the two sweep lanes ------------- +// +// ONE PAGE PER OPERATION. The rail used to be a summary above a <select> that +// chose which lane you could act on; it is the switcher itself now, and each +// row's NAME is the link. That is what these three tests pin: the board lists +// every pipeline, the rows go somewhere, and the old address still lands. -test("UI: the rail carries every pipeline, and the switcher focuses one", async ({ +test("UI: the board carries every pipeline, and each row links to it", async ({ page, }) => { await resetData(null); @@ -1070,41 +1079,103 @@ test("UI: the rail carries every pipeline, and the switcher focuses one", async autoQueue: transcriptionAutoQueue(ALPHA_ROOT), }); - await page.goto("/auto-queue"); - const transcribe = page.locator("section", { - has: page.getByRole("heading", { name: "Auto-transcribe" }), - }); - await awaitHydration(transcribe); + await page.goto("/operations"); + await expect( + page.getByRole("heading", { name: "Operations", exact: true }), + ).toBeVisible(); + await awaitHydration(page.locator('[data-board="operations"]')); - // The rail is always present and carries a row per pipeline, whichever lane - // is focused — that is the whole reason it exists. + // The rail carries a row per pipeline — the two the runners dispatch, the one + // digest owns, and every speaker operation that is switched on. It is the + // CATALOG view: what this install can do, not what it is doing. const rail = page.locator("li[data-operation]"); await expect(rail.filter({ hasText: "Download" }).first()).toBeVisible(); await expect(rail.filter({ hasText: "Transcription" }).first()).toBeVisible(); await expect(rail.filter({ hasText: "Digest" }).first()).toBeVisible(); - // Auto-transcribe is the default focus. Auto-download stays MOUNTED — which - // is what preserves its unsaved policy edits across a switch — but a hidden - // subtree is out of the accessibility tree, so it is addressed by data-lane - // rather than by its heading. A heading-scoped locator resolves to ZERO here, - // which is correct: a screen reader must not see the hidden pane either. - const download = page.locator('section[data-lane="download"]'); + // THE ROW'S NAME IS THE LINK, and its accessible name is the operation alone + // — a link wrapping the whole row would be named by figures that change on + // every poll. + await expect( + rail.getByRole("link", { name: "Transcription", exact: true }), + ).toHaveAttribute("href", "/operations/transcription"); + await rail.getByRole("link", { name: "Digest", exact: true }).click(); + await expect(page).toHaveURL(/\/operations\/digest$/); + + // NOTHING ON THE BOARD IS A RUNNER PANEL. The board is about every operation; + // one operation in full is what /operations/<id> is for. + await page.goto("/operations"); + await expect(page.locator("section[data-lane]")).toHaveCount(0); +}); + +test("UI: /auto-queue still lands, and an unknown operation is a 404", async ({ + page, +}) => { + await resetData(null); + await makeChannel("alpha", ["a1"]); + await writeSettings({ + adminTitle: "Test Admin", + maxTranscriptPageBytes: 8388608, + sleepBetweenDownloadsSeconds: 0, + minFreeDiskGB: 0, + workers: ONE_WORKER, + autoQueue: transcriptionAutoQueue(ALPHA_ROOT), + }); + + // A RETIRED ROUTE REDIRECTS, NEVER 404s — the nav rule this repo shares with + // umtool. Every bookmark, every link in a commit message, every note in + // plans/ that says /auto-queue still lands on the console. + await page.goto("/auto-queue"); + await expect(page).toHaveURL(/\/operations$/); + await expect( + page.getByRole("heading", { name: "Operations", exact: true }), + ).toBeVisible(); + + // The catalog IS the route table, so an id the registry does not know is a + // 404 rather than an empty console. + const res = await page.goto("/operations/not-an-operation"); + expect(res?.status()).toBe(404); +}); + +test("UI: each operation page carries its own lane, and the rail as context", async ({ + page, +}) => { + await resetData(null); + await makeChannel("alpha", ["a1", "a2"]); + await writeSettings({ + adminTitle: "Test Admin", + maxTranscriptPageBytes: 8388608, + sleepBetweenDownloadsSeconds: 0, + minFreeDiskGB: 0, + workers: ONE_WORKER, + autoQueue: transcriptionAutoQueue(ALPHA_ROOT), + }); + + // ONE RUNNER SECTION PER PAGE. Auto-download is not hidden here, it is simply + // not on this page — which is what deleting the switcher bought: no second + // pane to keep mounted, and no policy edit to lose across a switch. + await page.goto("/operations/transcription"); + const transcribe = page.locator("section", { + has: page.getByRole("heading", { name: "Auto-transcribe" }), + }); + await awaitHydration(transcribe); await expect(transcribe).toBeVisible(); - await expect(download).toHaveCount(1); - await expect(download).toBeHidden(); + await expect(page.locator('section[data-lane="download"]')).toHaveCount(0); + // The rail is still above it — on this corpus a lane is usually idle BECAUSE + // another one is, so the context travels with the operation. + await expect(page.locator("li[data-operation]").first()).toBeVisible(); + + await page.goto("/operations/download"); await expect( page.locator("section", { has: page.getByRole("heading", { name: "Auto-download" }), }), - ).toHaveCount(0); - - await page.getByLabel("Lane", { exact: true }).selectOption("download"); - await expect(download).toBeVisible(); - await expect(transcribe).toBeHidden(); + ).toBeVisible(); - // The digest lane is a real lane on this page now, with both of its controls. - await page.getByLabel("Lane", { exact: true }).selectOption("digest"); + // The digest lane is a page of its own, with both of its controls. + await page.goto("/operations/digest"); const digest = page.locator('section[data-lane="digest"]'); + await awaitHydration(digest); await expect(digest.getByRole("heading", { name: "Digest" })).toBeVisible(); await expect( digest.getByRole("button", { name: "Start Digest sweep" }), @@ -1130,16 +1201,12 @@ test("UI: Reach is disabled until an order is chosen, and it persists", async ({ autoQueue: transcriptionAutoQueue(ALPHA_ROOT), }); - await page.goto("/auto-queue"); - await awaitHydration( - page.locator("section", { - has: page.getByRole("heading", { name: "Auto-transcribe" }), - }), - ); - await page.getByLabel("Lane", { exact: true }).selectOption("digest"); + await page.goto("/operations/digest"); + await awaitHydration(page.locator('section[data-lane="digest"]')); - // Scoped to the lane: every lane on the page carries its own Order/Reach - // pair, and getByLabel does not filter by visibility. + // Scoped to the lane. One lane per page now, but the scoping stays: Order and + // Reach are a pair every lane carries, and an unscoped getByLabel would be + // ambiguous the moment a second one shares a page again. const digest = page.locator('section[data-lane="digest"]'); const order = digest.getByLabel("Order", { exact: true }); const reach = digest.getByLabel("Reach", { exact: true }); @@ -1225,12 +1292,12 @@ test("the arbiter refuses to start beside an armed sweep, and says which one", a digest: { sweepEnabled: true }, }); - await page.goto("/auto-queue"); - await awaitHydration( - page.locator("section", { - has: page.getByRole("heading", { name: "Auto-transcribe" }), - }), - ); + await page.goto("/operations"); + // THE ARBITER IS THE BOARD'S CONTROL, not a lane's: it is the thing that + // decides BETWEEN lanes, so it sits on /operations beside the rail. The board + // carries its own hydration signal — its buttons are inert until React is + // live on it, exactly as a runner section's are. + await awaitHydration(page.locator('[data-board="operations"]')); const start = page.getByRole("button", { name: "Start the arbiter" }); await expect(start).toBeDisabled(); // Two dispatchers on one lane would start the same channel twice, so the @@ -1252,12 +1319,12 @@ test("the arbiter will not start with no rule naming an operation", async ({ autoQueue: transcriptionAutoQueue(ALPHA_ROOT), }); - await page.goto("/auto-queue"); - await awaitHydration( - page.locator("section", { - has: page.getByRole("heading", { name: "Auto-transcribe" }), - }), - ); + await page.goto("/operations"); + // THE ARBITER IS THE BOARD'S CONTROL, not a lane's: it is the thing that + // decides BETWEEN lanes, so it sits on /operations beside the rail. The board + // carries its own hydration signal — its buttons are inert until React is + // live on it, exactly as a runner section's are. + await awaitHydration(page.locator('[data-board="operations"]')); // It would come up, find nothing to dispatch and stop. Saying so before the // click is the difference between a disabled button and a mystery. await expect( @@ -1286,12 +1353,12 @@ test("the arbiter starts, appears as a job, and stops", async ({ page }) => { }), }); - await page.goto("/auto-queue"); - await awaitHydration( - page.locator("section", { - has: page.getByRole("heading", { name: "Auto-transcribe" }), - }), - ); + await page.goto("/operations"); + // THE ARBITER IS THE BOARD'S CONTROL, not a lane's: it is the thing that + // decides BETWEEN lanes, so it sits on /operations beside the rail. The board + // carries its own hydration signal — its buttons are inert until React is + // live on it, exactly as a runner section's are. + await awaitHydration(page.locator('[data-board="operations"]')); await page.getByRole("button", { name: "Start the arbiter" }).click(); const stop = page.getByRole("button", { name: "Stop the arbiter" }); await expect(stop).toBeVisible({ timeout: 15_000 }); diff --git a/editor/e2e/auto-subs-replace.spec.ts b/editor/e2e/auto-subs-replace.spec.ts @@ -397,7 +397,7 @@ test("the auto-queue opt-in is off by default and persists when enabled", async await resetData(); await seedChannel([{ id: "asrvid0002", captions: "asr", audio: true }]); - await page.goto("/auto-queue"); + await page.goto("/operations/transcription"); const optIn = page.getByLabel( "replace YouTube auto-captions for auto-transcription", ); diff --git a/editor/e2e/backfill.spec.ts b/editor/e2e/backfill.spec.ts @@ -464,23 +464,20 @@ test("a sweep can be scoped to one operation from the console, and the scope is }, }); - await page.goto("/auto-queue"); + // ONE OPERATION, ONE PAGE. Diarization's page renders the lane it is + // dispatched by — the SHARED backfill lane, which is also attribution's — so + // the scope control below is the same one, reached by an address instead of a + // dropdown. + // + // The poll this replaced re-selected the lane until it took: a selectOption + // before hydration sets the DOM value and fires nothing at all — no state + // change, no error, no clue. There is no <select> to lose a change on now, so + // the wait is the ordinary one: React is live on the section. + await page.goto("/operations/diarization"); const lane = page.locator('section[data-lane="backfill"]'); - // RE-SELECTED UNTIL IT TAKES. The lane switcher only swaps panes once React - // has attached; a selectOption before hydration sets the DOM value and fires - // nothing at all — no state change, no error, no clue. Same class of failure - // as the pre-hydration lost click this suite has been bitten by before. - await expect - .poll( - async () => { - await page - .getByLabel("Lane", { exact: true }) - .selectOption("backfill"); - return lane.isVisible(); - }, - { timeout: 30_000 }, - ) - .toBe(true); + await expect(lane).toHaveAttribute("data-hydrated", "true", { + timeout: 30_000, + }); // THE PLAN IS DRAWN BEFORE ANYTHING IS ARMED. That is the whole point: the // commitment is readable before it is made. diff --git a/editor/e2e/navigation.spec.ts b/editor/e2e/navigation.spec.ts @@ -36,6 +36,10 @@ import { resetData } from "./helpers"; // the worst (4.5 s and 5.4 s) because each rendered a full corpus walk. const HEAVY_ROUTES = [ { link: "Channels", heading: "Channels", path: "/channels" }, + // The board: one row per operation, built from the registry catalog, with the + // arbiter above them. It reads every channel's snapshot to draw the rail, so + // it belongs with the heavy routes rather than beside the cheap ones. + { link: "Operations", heading: "Operations", path: "/operations" }, { link: "Actionable", heading: "Actionable items", path: "/actionable" }, { link: "Jobs", heading: "Jobs", path: "/jobs" }, ] as const; diff --git a/editor/next.config.ts b/editor/next.config.ts @@ -45,6 +45,20 @@ const nextConfig: NextConfig = { // views, and a stale queue is worse than a slow one. staleTimes: { dynamic: 15, static: 180 }, }, + // Every retired route redirects rather than 404s — the nav rule this repo + // shares with umtool (see editor/app/lib/nav.ts). /auto-queue was the console + // for two runner lanes and grew into a board over six operations; it is + // /operations now, one page per operation. + // + // TEMPORARY, not permanent: a 308 is cached by the browser forever, and this + // is a self-hosted admin surface where a wrong permanent redirect is a + // support call with no remedy but a profile wipe. The API paths under + // /api/auto-queue/* are NOT redirected — they never moved. + async redirects() { + return [ + { source: "/auto-queue", destination: "/operations", permanent: false }, + ]; + }, // Serve the built export artifacts (stats/summaries/transcripts) through a // route handler so the charts authoring tab can preview real data, mirroring // the static viewer's served paths. diff --git a/plans/FACTS.md b/plans/FACTS.md @@ -820,6 +820,9 @@ anything with a settings surface gets one live run and one assertion. | Actionable sections | `editor/app/actionable/page.tsx:30-46` | `SectionConfig` is module-private. Counters live in `actionable/lib/loadActionable.ts`. | | Widget sync payload | `editor/app/api/widget/sync/route.ts:15-26` | Comment at `:10-14` states it deliberately stays "a handful of scalars". `buildWidgetSyncPayload()` (`:30`) is exported for SSR seeding. | | Pipeline band | `editor/app/components/dashboard/PipelineBand.tsx:63-114` | Pure props, no fetching. `<Instrument dotClass=…>` encodes state color. | +| Operations board | `editor/app/operations/page.tsx` | `/operations`. Rail + arbiter + sync row, SSR-seeded, polls `/api/auto-queue/status` every 3 s. `data-board="operations"` carries `data-hydrated`. | +| One operation | `editor/app/operations/[id]/page.tsx` | `/operations/<id>`, **routed off `operationCatalog()`** — an unknown id is `notFound()`, a new registry entry needs no route work. Runner ids (`download`, `transcription`) render `RunnerOperationView`; everything else renders `SweepLane` for its lane inside `<section data-lane="digest"|"backfill">`. | +| Retired editor routes | `editor/next.config.ts` `redirects()` | `/auto-queue` → `/operations`, **temporary (307)**, not permanent — a 308 on a self-hosted admin surface is a support call with no remedy. The API paths `/api/auto-queue/{status,control}` did **not** move. | --- diff --git a/plans/README.md b/plans/README.md @@ -2,7 +2,7 @@ The local-AI derived-corpus work spans many phases and months, and **context is cleared between phases** to keep token cost down. That only works if a cold agent can resume without -re-deriving anything. Four artifacts with deliberately different lifetimes: +re-deriving anything. Five kinds of artifact, with deliberately different lifetimes: | File | Lifetime | Contents | | --- | --- | --- | @@ -10,6 +10,7 @@ re-deriving anything. Four artifacts with deliberately different lifetimes: | [`FACTS.md`](FACTS.md) | Append-only | Verified codebase facts with `file:line` anchors. | | [`STATE.md`](STATE.md) | Rewritten each session | Phase status, decisions log, open questions, commands known to pass. | | `phase-N-<slug>.md` | Written at phase start, archived at merge | File-level detail for the phase in flight. | +| `<topic>.md` design docs | Live until the design is fully shipped | A model that spans phases and outlives any one of them: [`unified-operations-model.md`](unified-operations-model.md) (how media-derived work is dispatched — the backend half) and [`editor-operations-ia.md`](editor-operations-ia.md) (what the editor's screens are about — the UI half, nine slices, slice 1 shipped). Read the matching one before touching a dispatcher or a page. | `FACTS.md` is the most valuable file here. It holds exactly the material that is expensive to establish and cheap to get wrong — the original draft of this plan contained several diff --git a/plans/STATE.md b/plans/STATE.md @@ -3,7 +3,8 @@ The working memory for the local-AI derived-corpus work. Rewritten at the end of every session, before context is cleared. See [`README.md`](README.md) for the protocol. -**Last updated:** 2026-08-24 — **`main` is green again, `feat/channel-groups` is merged, and +**Last updated:** 2026-08-25 — editor IA slice 1 (`/operations`) landed; see the dated +section below. Previously: 2026-08-24 — **`main` is green again, `feat/channel-groups` is merged, and this file caught up with two sessions it had no entry for.** The 2026-08-12 session below landed in `7d32438` on 08-18; everything from 08-19 to 08-24 — recency ordering, the arbiter, lane guards, the pipelines band, worker tags and slots, LLM fan-out, unit @@ -82,6 +83,42 @@ verified against the tree, **not started**, and its real prerequisite is operati `sdb1` (1.8 T platter) is not mounted. **Not covered here:** `umtool` (27 commits on 08-18, a separate workspace package) is documented in `umtool/docs` and AGENTS.md. +### 2026-08-25 — the editor gets its noun: `/operations`, and a nav with four groups + +**The backend found its organizing noun — the operation — and the UI had not.** Counted +rather than estimated: 28 pages behind a 19-link nav whose "Pool" group held ten unrelated +tools, four channel tables, four job lists, three renderings of lane state, and four +different answers to "what needs doing". Digest and attribution are first-class in the +REGISTRY and were second-class in the UI: no route, no stage, no per-video view, two settings +fieldsets for one concept. + +[`editor-operations-ia.md`](editor-operations-ia.md) is the UI half of +[`unified-operations-model.md`](unified-operations-model.md), written down whole — the four +nouns (Corpus, Operations, Sites, Machine), the seven places "operation as the noun" honestly +breaks and what to do about each, the nav end state, the reconciliations with unified-ops +steps 5/6, PLAN.md Phase 11a and relocate-channel-media §6, and **nine slices, each +shippable, each deleting something**. + +**Slice 1 shipped.** `git mv app/auto-queue app/operations`; `/operations` is the board (the +rail, the arbiter, a sync row) and `/operations/<id>` is one operation, resolved against +`operationCatalog()` so an unknown id 404s and a NEW registry entry gets a page with no route +work. `/auto-queue` redirects. The **lane `<select>` is deleted** — with one lane per page +there is no switch to lose policy-edit state across, which was the only reason the +non-selected lanes stayed mounted. The runner section's ~1,200-line structural e2e contract +survived the move verbatim (`RunnerOperationView.tsx` states every clause of it). Nav is four +groups; *Monitor* left it (the widget is a projection of the board payload, reached from the +dashboard's pipeline band). + +**Decisions worth keeping:** +- **The sweep scope is NOT pre-ticked to the operation being viewed.** The scope is written + at arm time, so a per-page default would arm a narrower sweep than the operator sees. +- **A speaker operation's page draws its OWN band** (the shared panel could only say "this + lane covers several operations"), and says in a sentence that the lane, sweep and pause are + shared. A per-operation page is not a per-operation switch. +- **The arbiter is the BOARD's control**, not a lane's — it decides *between* lanes. +- **Retired routes redirect, never 404**, and the nav cap is umtool's rule, quoted in + `nav.ts` so two consoles do not drift into two nav philosophies. + **Recommended next** (supersedes the list under "Pre-sweep decisions", which still describes the reasoning): @@ -92,6 +129,12 @@ describes the reasoning): 4. **Phase 6 Ollama `/ask`** — genuinely independent; a good parallel task. 5. Decide `attribution-text`'s fate on that one channel: ~194,000 chunk-level calls is a sweep-sized commitment that was never priced. +6. **Editor IA slice 2** — speakers become a real stage: `runOperationChannelJob()` extracted + from the arbiter and shared, `laneFor` on every kind, one `OperationStage.tsx`, the stage + list's middle derived from `OPERATION_GROUP_ORDER`. Slice 3 (per-operation settings) + follows it; **slice 4 needs unified-ops step 1** and slice 9 needs the sweep to have run + once, so those two wait on #1 and #2 above. See + [`editor-operations-ia.md`](editor-operations-ia.md). --- diff --git a/plans/editor-operations-ia.md b/plans/editor-operations-ia.md @@ -0,0 +1,192 @@ +# The editor: one noun, not a pile + +**Status: the model is decided; slice 1 has shipped.** This is the UI half of +[`unified-operations-model.md`](unified-operations-model.md), which states the backend half +("four schedulers for what is really one kind of work"; steps 1, 5, 6 open). Read that one +first if you are touching a dispatcher; read this one if you are touching a page. + +## The diagnosis, in one line + +**The backend has found its organizing noun — the operation — and the UI has not.** + +The editor grew page-by-mechanism. Each scheduler, sweep, runner and derived feature landed +with its own route, its own fieldset, its own pause switch and its own counter. What that +adds up to, counted rather than estimated: + +- **28 pages** behind a **19-link nav**, whose "Pool" group holds **10 unrelated tools**; +- **four channel tables** (`/channels`, the dashboard's, `/actionable`'s, `/cleanup`'s); +- **four job lists** (`/jobs`, `/jobs/active`, `/jobs/queue`, the per-channel one); +- **three renderings of lane state** (the rail, the dashboard cards, `/jobs/active`'s lanes); +- **eight settings surfaces**; and +- **four different answers to "what needs doing"**. + +Digest and attribution are first-class in the **registry** — `common/lib/backfillKinds.ts` +says in its own header that registering attribution "lit the channel stage card, +`/actionable`, the dashboard instrument and the widget strip with ZERO UI changes" — and +second-class in the **UI**: no route, no stage of their own, no per-video view, and two +settings fieldsets for one concept. + +The coherent parts of the UI are exactly the parts that already treat the operation as the +noun: `StateBand` drawn at three scales off one fold, and `operationCatalog()` driving both +the `/channels` columns and the auto-queue rail. The pile is everything that does not. + +## The vision statement + +> The editor is the operator console for turning a large, mostly-verbatim transcript archive +> into a derived corpus and publishing it. Its nouns are the **Corpus** (channels and videos, +> read from disk), the **Operations** that produce derived artifacts from it (download, +> transcribe, digest, diarize, attribute — one registry entry each, with a state per video, a +> cost per unit, a lane it runs on, and a rule in the policy tree that dispatches it), the +> **Sites** that publish slices of the corpus, and the **Machine** that supplies capacity and +> storage. Every screen answers one question about one of those nouns; a number shown in two +> places is the same fold over the same snapshot; a pause is a hold and never a stop; and +> reachable work is never summed with work that is blocked, deferred or gone. A new operation +> is a registry entry, and it appears on the board, the channel page, the video page and the +> policy tree without a new page. + +The last sentence is the test. Today registering an operation lights four *counters* and no +*surfaces*. When it lights the surfaces too, this work is done. + +## Where "operation as the noun" honestly breaks + +A model is worth writing down only with its exceptions attached, or the next person meets one +and concludes the model was wrong. + +| Thing | Verdict | +| --- | --- | +| **Sync** | An operation, but channel-scoped and cadence-triggered rather than per-video and backlog-driven. Add `scope: "video" \| "channel"` and `trigger: "backlog" \| "cadence"` to `OperationDescriptor`; it gets a board row and `/operations/sync` (today's `/scheduler`) and keeps its own runner. The keep-latest checks and the saved-video backup that ride the same heartbeat (`editor/app/scheduler/runTick.ts:25-27`) are **Storage chores**, not sync, and should be labelled as such. | +| **Build / deploy** | **Not** operations. Per-site, no per-video state, no lane. They are the Site's *publish* verb → `/sites/[siteId]`. | +| **Cleanup / saved-videos / relocate** | **Not** operations: they consume outputs rather than producing derived artifacts. Third noun, **Storage**, filed under Machine. The channel `cleanup` stage stays where it is. | +| **Transcode** | A per-video media operation that is simply **missing from the catalog** — it has a state per video, a lane and a dependency. Register it as an external descriptor, group `media`, `dependsOn: ["download"]`, with an `appliesTo?(channelConfig)` so a channel that never transcodes does not grow a dead row. Do it when the stage list becomes registry-derived (slice 2). | +| **Social channels** | A channel whose operation set is `{fetch-posts}`. An explicit **non-goal** here: leave the short-circuit at `editor/app/channels/[slug]/page.tsx:156` alone. | +| **The channel stage list** | Half-derived and that is correct. The middle (`download … backfill`) derives from `OPERATION_GROUP_ORDER`; the bookends (`configure`, `playlist` / `cleanup`, `diagnostics`, `danger`) are **channel chores**, not operations, and stay hand-listed. Say so in the code so the next reader does not "finish" the derivation. | +| **Digest's two lanes** | A **registry defect**, not a noun problem. `common/controller/arbiter.ts:118-124` special-cases `DIGEST_KIND_ID` because digest runs on its own queue. Give every kind a `laneFor(settings)` defaulting to `lane` — diarization already has one (`common/lib/backfillKinds.ts:599`) — and the special case dissolves. | +| **The widget** | A **projection** of the board payload, not a noun. Drop "Monitor" from the primary nav; reach it from the dashboard's pipeline band, next to the thing it mirrors. | + +## The nav, end state + +Four groups, eleven top-level entries, down from three groups and nineteen: + +- **Corpus** — Dashboard, Channels +- **Operations** — Operations *(+ Actionable until slice 4, Schedule until slice 8)* +- **Sites** — Sites *(+ Charts, Search aliases, Deploy, Build, Homepage until slice 5)* +- **Machine** — Jobs, Active, Workers, Cleanup, Saved videos, Settings, Changelog + +**The rule, and it is umtool's rule, not a new one.** `umtool/components/AppNav.tsx` states +it after the same disease: "SEVEN visible entries, and the cap is still NINE … The next tool +goes UNDER one of these, not beside them." umtool folded three judging piles into one +`song ▸` group **at their old URLs**, which is the second half of the rule: fold rather than +add, and **every retired route redirects rather than 404s**. Both apps now cite the same +comment, deliberately — two consoles drifting into two nav philosophies is how this started. + +## Reconciliations with existing plans + +Recorded here because each of these is a plan that says something slightly different, and +silence would read as a contradiction nobody noticed. + +- **unified-ops step 5** says the four pause fields become "node state on the tree". The + operator-facing gate is **per operation**, not per node, so the two meet at a definition: + *"pause operation X" = "pause the root of the tree that dispatches X"*. Ship one + `pauseOperationAction(id)` that maps onto the four legacy fields now; step 6 migrates the + storage underneath it. **No new settings block** — that would be a fifth pause. +- **unified-ops step 6** is slice 9 here. Untouched, and last. +- **PLAN.md Phase 11a** says "extend `/actionable`, don't build a parallel page". Slice 4 + *dissolves* `/actionable`, so 11a is rewritten rather than contradicted: digest review and + uncertain attribution become the **attention section of `/operations/<id>`**; duplicates, + viewer feedback and channel-context promotion become a small **`/review` under Corpus**. + The `SectionConfig` extension point moves with the sections; it does not disappear. +- **relocate-channel-media.md §6** asks for a Storage panel on the channel page. It goes + **under the existing `cleanup` stage**, not as a new stage id — media location is a storage + chore, and a new stage would be a sixth answer to "what does this channel need". The badge + on the channels table is unchanged. + +## The nine slices + +Each is shippable on its own, each **deletes something**, and the order is the order the +dependencies allow. Sizes are S/M/L. + +1. **The frame — SHIPPED.** `/operations` (the board) + `/operations/<id>` (one operation), + four-group nav, `/auto-queue` → redirect. Deletes the lane `<select>`. UI-only. **M.** +2. **Speakers become a real stage.** Extract `runOperationChannelJob()` from + `arbiter.ts:226-240` and share it with `channels/groupActions.ts:162-175` and the channel + page; `laneFor` on every kind; `DigestStage.tsx` + `BackfillStage.tsx` → one + `OperationStage.tsx`; `StageId "backfill"` → `"speakers"`; the middle of `stageOrder` + derived from `OPERATION_GROUP_ORDER`. **Keep `snapshot.backfill` on disk** — this is a UI + rename, not a data migration. e2e: `helpers.ts:336-346` `ChannelStage`, + `channelStage(SLUG, "backfill")` ×10 in `backfill.spec.ts` and ×3 in + `attribution.spec.ts`, heading "Speaker work" at `attribution.spec.ts:347`. **M.** +3. **Per-operation settings on the operation page.** Move the Digest + (`SettingsForm.tsx:526`), Diarization (`:708`), Speaker-work lane (`:889`) and Speaker + attribution (`:982`) fieldsets to `/operations/<id>`; split `saveSettingsAction` + (`settings/actions.ts:286-420`) along its existing per-block markers — the sanitizers are + already per-block (`common/lib/settings.ts:1096-1241`). Deletes ~560 lines of + `SettingsForm`. e2e: `settings.spec.ts:190`. **M.** +4. **`/actionable` dissolves.** Depends on unified-ops step 1. The per-operation sections + become the "channels with reachable work" table on `/operations/<id>` — which is what + `SweepPlan.tsx` already draws; `NeedsWorkPanel.tsx:61-76` drops `noDigest` in favour of + band `reachable` (`buildBands.ts:163`); the non-operation sections go to `/cleanup`, Sites + and the dashboard. **Keep `actionable/lib/loadActionable.ts`** — the widget payload reads + it. e2e: `actionable.spec.ts`, `cleanup-actionable.spec.ts`, `site-scope.spec.ts:101`, + `backfill.spec.ts:769`, `attribution.spec.ts:358`, `navigation.spec.ts:38`. **M–L, medium + risk** — dashboard numbers move, and the changelog must say so. +5. **Sites absorb charts / aliases / deploy / build / homepage.** `/sites/[siteId]/{charts, + aliases,publish}`; `?site=` stays for Dashboard and Channels. Deletes five pages. e2e: + `build.spec.ts`, `deploy-page.spec.ts`, `site-scope.spec.ts:63-87`, `aliases.spec.ts`, + `channel-build-toggle.spec.ts:17`. **M.** +6. **The video page: one panel per operation.** `DigestPanel.tsx` generalizes to + `OperationPanel {state, target, provenance, sections?}`, built per enabled kind from + `readVideoFiles()`; deletes `hasDigest()` (`digest-server.ts:89`) and `hasAttribution()` + (`attribution-server.ts:36`). Attribution and diarization get a per-video view for free. + The **"Open in umtool" link** (`VideoPanel.tsx`, `data-umtool-link`, landed in `818bfc7`) + stays above the panels, unchanged: it is a Corpus→umtool bridge, not an operation. **M.** +7. **One pause.** unified-ops step 5 as reconciled above: + `Pause{Downloads,Transcriptions,Backfill}Button` → `PauseOperationButton({operation})`; + the six actions in `jobs/actions.ts:172-380` collapse to one pair; the polarity + normalization at `laneState.ts:16-19` goes away. **Keep the aria labels** + (`backfill.spec.ts:803,814`, `widget.spec.ts:803,810`). **M, medium risk.** +8. **Machine.** `/jobs`, `/jobs/active` and `/jobs/queue` become one page with a mode and one + table; `WorkersField.tsx` (556 lines) moves to `/workers`; `/scheduler` becomes + `/operations/sync`. umtool's `/` already reads "one activity list over both job + registries" — this is the same shape. **M–L.** +9. **Sweeps retire; the tree dispatches everything.** unified-ops step 6 plus arbiter + persistence. Deletes `digestSweep.ts`, `backfillSweep.ts`, `SweepLane.tsx`, + `SweepScope.tsx`, the sweep checks at `arbiter.ts:205-213` and the resume hooks at + `instrumentation.ts:91-122`. **L, high risk — and not before the sweep has run once for a + day and been watched** (STATE.md "Recommended next" #1: GPU yield on a quiet box). **L.** + +**Vocabulary renames follow slice 2, mechanically and in one commit:** `BackfillKind` → +`Operation`, `BackfillLane` → `Lane`, `backfillKinds.ts` → `operations.ts`, and "unit" +disambiguated (a worker-pool unit is not a cost unit). Doing them earlier would collide with +every slice above; doing them later means writing `backfill` in surfaces that no longer say it. + +## Out of scope, stated so silence is not read as a decision + +- The **reader-facing export site** — that is PLAN.md Phase 10, a different audience and a + different set of nouns. +- **The sweep retirement itself** is in scope only as slice 9, and it is gated on the sweep + having run in production once. +- **Social channels** keep their short-circuit. +- **The widget's internals.** It is re-parented in the nav and otherwise untouched. + +## Slice 1, as shipped + +Routes: **`/operations`** (the board — the rail, every row a link, the arbiter bar, a sync +row) and **`/operations/<id>`** (one operation: the runner lanes verbatim for `download` and +`transcription`, the sweep lane for the registry kinds, with the band and the running jobs +for that operation above it). `/auto-queue` **redirects** to `/operations`; the API paths stay +`/api/auto-queue/*` (three specs hit them directly). + +Two things worth knowing before touching it again: + +- **The lane `<select>` is gone, and its structural contract is not.** `AutoQueueView.tsx` + carried a ~1,200-line e2e contract — a literal `<section data-lane data-hydrated>` per + runner kind with an `<h2>` named exactly "Auto-transcribe"/"Auto-download", nothing nested + inside it that is itself a `<section>`, native `<select>`/checkbox controls, and + `role="status"` reserved for "Saved." — and every clause of it survives in + `RunnerOperationView.tsx`. What did not survive is the *reason the non-selected lanes + stayed mounted*: with one lane per page there is no switch to lose policy-edit state + across, and the one page-wide option count the suite asserts + (`auto-subs-replace.spec.ts:409`) is satisfied on `/operations/transcription`. +- **The sweep scope is not pre-selected to the operation you are looking at.** It would be + the obvious convenience and it is a trap: the scope is written **at arm time**, so a page + default would arm a narrower sweep than the operator believes they are looking at.