commit df58ee9d3a6f511549b086ee0bd76378b0538b37
parent 0930b12baa86c072ebd26614d4b95ce8b4497585
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Sat, 22 Aug 2026 00:37:45 -0400
auto-queue: one console for four pipelines; a runner is a lane
Squashes the Stage 4 WIP commit's intent into a described change.
/auto-queue was two identical runner panels with no room for the other
two pipelines. It now opens on a comparison RAIL — one line per
pipeline, always present — over a focused lane chosen by a native
<select>.
The rail's band keeps five populations apart and never sums them
(reachable and needs-media are ~91x apart on this corpus), normalises
each band to ITS OWN eligible, and renders an unknown denominator as
an outline rather than 0% — presentBackfillWork already returns null
"because the honest answer there is unknown". Fill PATTERN leads and
hue follows: report-to-video established here by measurement that no
four-colour palette clears all-pairs CVD. One saturated colour on the
page, and it marks `reachable`.
Order and Reach move to the foot of every lane. Reach is disabled
under "Listed" rather than silently ignored. The sweep lanes keep BOTH
switches under their existing names — conflating sweep and pause is
how an operator loses a week of GPU time — and get no claim ladder,
because they have no policy tree and an empty one would read as "no
rule matches".
/jobs/active: the runner cards become a one-line lane strip in
deriveLaneState's vocabulary, with the idle reason in words. Every
lane keeps its <section aria-label> and <h2>, so it is addressable
exactly as before; "Other" stays at zero.
Three things this cost, all measured rather than assumed:
* `hidden` alone does not hide a Tailwind `flex` section — preflight's
[hidden] rule loses to the utilities layer. The class is switched
with the attribute.
* MOUNTED IS NOT ADDRESSABLE. The plan expected a hidden-but-mounted
lane to keep heading-scoped lookups working; it does not — a heading
in a display:none subtree is out of the a11y tree, so
locator("section", { has: heading }) resolves to ZERO, and
toBeHidden() then passes vacuously. That is correct behaviour, so
lanes gained a data-lane hook instead. What mounting does preserve
is the lane's unsaved policy edits.
* operationBands.ts had to split: it imports channelSnapshot, which
imports execa, so a client component importing it fails `next build`
with a module-not-found. e2e runs in dev and never prerenders, so
only the build caught it.
Also fixes a ~1-in-3 race in "strict priority": it polled the PICK log
(dispatch) and then asserted transcripts on DISK (completion) with
nothing ordering the two. The ordering assertion — the actual subject
— is untouched; only the disk check is polled.
Verification: 60/60 e2e across auto-queue, backfill, digest,
auto-subs-replace, disk-space and jobs-active; `pnpm build` clean;
8 new unit tests on the band model.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Diffstat:
19 files changed, 2075 insertions(+), 172 deletions(-)
diff --git a/.claude/settings.json b/.claude/settings.json
@@ -0,0 +1,5 @@
+{
+ "enabledPlugins": {
+ "hyperframes@claude-plugins-official": true
+ }
+}
diff --git a/editor/CHANGELOG.md b/editor/CHANGELOG.md
@@ -7,6 +7,9 @@
- **The editor can be started without resuming whatever it was in the middle of.** Booting arms the sync heartbeat, both auto-queue runners and the digest and backfill sweeps — right for a host install, where a restart interrupts work you own, and wrong the first time a container is pointed at a corpus somebody else configured: its stored policies may say *sweep*, and a corpus-wide digest sweep is GPU-**weeks** that would start seconds after `docker compose up`. `ARCHILYZER_IDLE_BOOT=1` starts the server with all of that stopped, and you can start any of it from the UI afterwards. The shutdown reaper and the persisted-pause restore stay armed either way, because both only ever *stop* work.
- **The digest and backfill lanes can do the newest uploads first too.** *Newest first* only ever existed on two of the four pipelines. The digest batch ordered a channel by **duration** — shortest first, which is the right default and turns a backlog into visible coverage fastest — and the backfill batch used whatever order a `readdir` happened to return, which is neither of the two orders anyone would choose. So on a channel with eleven thousand undigested videos, a video uploaded this morning sat behind every one of them, and there was no setting that changed that. Both lanes now take the same three-value **Order** the auto-queue runners already use: *Listed order* (exactly today's behaviour, and still the default), *Newest first*, *Oldest first*. It is opt-in, and with it left alone every list is byte-for-byte the list it was before — that is asserted, not assumed. Digest **composes** the two rules rather than choosing between them: videos are ordered by date first and by duration inside a date, so *newest first* reads as "newest day first, shortest video within that day" and the duration rule you already had is still doing its job. Measured on the live corpus, ordering a channel's 11,329 pending digests costs about a tenth of a second.
- **"Newest first" can now mean newest in the whole archive, not just newest in a channel.** Ordering videos inside a channel does nothing for the video uploaded this morning if its channel is fortieth in the queue — and a corpus-wide sweep visits channels heaviest-first, which on this archive means the largest channel gets eight days of attention before the second one is looked at. So Order is joined by **Reach**: *Within each channel* (today's behaviour, and the default) or *Across all channels*, which additionally orders the **channels** by the freshest — or oldest — piece of work each is holding. It is one control, not two mechanisms: an order without a reach is meaningless, so Reach is disabled while Order is left at *Listed*. The two levels share one rule for a video whose date is unknown — last under newest, first under oldest — which is what makes an archive with no dates at all come out in exactly the order it comes out in today, and heaviest-first survives as the tiebreak, so two channels holding work from the same day are still visited biggest-first. Measured on the live corpus: sixty-six channels re-ordered in about 1.5 seconds for digest, and the freshest channel under *Across all channels* is holding work from today.
+- **The auto-queue page is a console for all four pipelines, not two stacked copies of one.** It rendered auto-transcribe and auto-download as two identical full-height panels and had no room for the other two pipelines at all — so "the digest lane is idle because transcription has the GPU" was a fact you could only assemble by visiting three pages. The page now opens on a **rail**: one line per pipeline, always present, whichever lane you are looking at. Each line carries a band showing what that pipeline has done, what it can do **right now**, what is waiting on an operation upstream, what is held by a gate, and what has lost its media — five separate figures that are never added together, because on this archive they are ninety-one times apart and one "remaining" number would say the same thing about a finished lane and a lane that cannot start. Exactly one saturated colour appears on the page, and it marks work that can happen right now; everything else is texture, distinguished by **fill pattern before hue** — solid, hatched, dotted, hollow — because a four-colour scheme was measured here and none of them survives colour-blindness on every pair. Each band is drawn against **its own** population and says so, since a digest's eligible videos are genuinely a different set from a diarization's. Where an archive cannot yet say how many videos a pipeline is done with, the band draws as an empty outline and says *coverage unknown* rather than showing 0%. Below the rail, a **dropdown** puts one lane in full view: the rules ladder for the two runners, and for digest and backfill the state, both switches, and what is running. The rail is what makes the switch safe — you never have to change lanes to learn that this one is idle because another one is, which on this archive is the normal case rather than the exception.
+- **Order and Reach sit at the foot of the lane, and now cover every pipeline.** *Newest first* used to be a control on two of the four lanes, halfway up a form. It is now the same block on all four, with the trade-off spelled out beneath it, and it is joined by **Reach** — *Within each channel* or *Across all channels* — which is disabled while the order is left at *Listed*, because a reach with no order to apply is not a setting. The digest and backfill lanes keep **both** of their switches, under the names they have everywhere else: the **sweep** decides whether there is a corpus-wide pass at all, and the **pause** decides whether the lane consumes it. They are deliberately not merged into one button: one is cheap to undo and the other costs a week of GPU time.
+- **A runner is a lane, not a job, and the active-jobs screen finally says so.** Each always-on runner took a full bordered card — the same card a channel with real work in it gets — to say "running", which pushed the jobs that actually have progress bars below the fold. The runners are now a **strip** at the top: one line each, with the lane's state and, when it is not working, *why* — "nothing pending", "sweep armed, lane paused", "no enabled worker". Digest and backfill are on the strip too, so all four pipelines are visible at once from a screen that previously knew about two. Every control a runner had is still there, on its line. A screen with no jobs at all now shows the strip rather than only the words "No active jobs" — "every lane is idle and here is why" is the answer that screen was previously unable to give.
- **Newest-first could quietly hand a video another channel's upload date.** The date lookup's first and cheapest layer scans the transcript index, which is keyed by the id the *platform* reports; every video the auto-queue actually asks about is named by its **folder**, and on this archive those two disagree for about one video in seven — a Rumble folder is named for the URL, while its recorded id is the embed id. Matched across the whole corpus, one channel's folder name could collide with a different channel's recorded id and inherit its date, which is a *wrong* answer rather than a missing one, and wrong dates are exactly what an ordering setting cannot survive. The lookup is now scoped to the channel the video belongs to, so a folder name that is not an id in its own channel falls through to reading the date out of the folder it actually names. Separately, the first pass over a very large backlog can only date so many videos at once; it now says so once in the log, because the remainder sorting to the back and being picked up on the next pass is the design, not a fault.
- **The auto-queue can be told to do the newest uploads first, and it now genuinely does.** Both runners always took the first video off a rule's pile, and that pile's order came straight from the channel snapshot, which sorts most buckets **alphabetically by video id** — arbitrary for YouTube ids, and oldest-first for the date-prefixed folder names some sites use. So when a channel uploaded today, nothing made that video jump the nine-thousand-video backlog in front of it; the only reason auto-download roughly worked was that its one bucket happens to be left in playlist order. There is now an **Order** setting per runner — *Listed order* (what you have today, and still the default), *Newest first*, *Oldest first*. It sorts the videos **inside** each rule, across every channel and bucket that rule claims; the rule list still decides which rule goes first, because that is what the rule list is for. For a straight newest-first archive, use one catch-all rule. Worth knowing before you switch it on: under *Newest first* the retry and partial-download buckets lose their head start, so a half-finished download can end up waiting behind fresh work. The page says so next to the setting.
- **Working out how recent 79,000 videos are turned out to be nearly free, once we stopped guessing where the dates were.** The obvious source — reading each video's metadata file — is about six and a half minutes and 41 GB of reading, on every scheduling decision, which is a non-starter. The transcript index already holds a date per video in a form that can be scanned without decoding anything: **78,583 videos in well under a second**. That covers the corpus, but it turned out **not** to cover the videos auto-transcribe actually queues, because the index only holds videos that already *have* a transcript and auto-transcribe's whole job is the ones that don't — of the 870 videos genuinely pending here, it knew the date of **122**. The gap is closed by reading the last 8 KB of each remaining video's metadata file, where the upload date happens to sit: **868 of the 870, at a fifth of a millisecond each**, and remembered afterwards so it is paid once rather than every few seconds. Videos not downloaded yet have no date anywhere on disk at all, so auto-download estimates one from the video's position in the channel's newest-first listing; those show with a `≈`, and a brand-new upload with nothing dated above it goes to the front, which is the entire point. Anything still undatable sorts to the back rather than disappearing, and a missing or busy index degrades to the old ordering instead of stopping the runner.
diff --git a/editor/app/api/widget/sync/route.ts b/editor/app/api/widget/sync/route.ts
@@ -8,6 +8,7 @@ import {
laneBackfillKinds,
laneKindEntriesOf,
operationLabel,
+ presentBackfillWork,
reachableBackfillWork,
} from "yt-dlp-transcript-common/lib/backfillKinds";
import { buildScheduleView } from "yt-dlp-transcript-common/jobs/syncScheduler";
@@ -107,6 +108,14 @@ export type WidgetSyncPayload = {
// Sorted by `reachable` descending, tie-broken by id, so the order is stable
// across polls rather than following whatever order the snapshots were
// written in.
+ //
+ // `eligible` and `present` are the COVERAGE half, and they are `number |
+ // null` for the same reason `digest.eligible` above is: they are summed
+ // across channels, a third of the snapshots on disk predate `eligible`, and
+ // a partial sum is a denominator smaller than its own numerator. One
+ // channel that cannot report voids the kind's whole figure — see
+ // presentBackfillWork, which returns null rather than 0 "because the honest
+ // answer there is unknown".
kinds: {
id: string;
label: string; // operationLabel(id) — resolved here, not in the client
@@ -114,6 +123,12 @@ export type WidgetSyncPayload = {
needsMedia: number;
blocked: number;
deferred: number;
+ // How many videos this operation has an opinion about. null = unknown.
+ eligible: number | null;
+ // How many it is DONE with. null = unknown. Never derived by subtracting
+ // the work counts from `eligible` at a call site — that is exactly what
+ // presentBackfillWork does, once, with the right guard.
+ present: number | null;
}[];
};
};
@@ -175,7 +190,14 @@ export async function buildWidgetSyncPayload(): Promise<WidgetSyncPayload> {
// does not is simply added rather than dropped.
const backfillByKind = new Map<
string,
- { reachable: number; needsMedia: number; blocked: number; deferred: number }
+ {
+ reachable: number;
+ needsMedia: number;
+ blocked: number;
+ deferred: number;
+ eligible: number | null;
+ present: number | null;
+ }
>();
// Digest's two non-work counters, off the digestWorkOf() call already made
// below — no extra read.
@@ -207,6 +229,8 @@ export async function buildWidgetSyncPayload(): Promise<WidgetSyncPayload> {
needsMedia: 0,
blocked: 0,
deferred: 0,
+ eligible: 0 as number | null,
+ present: 0 as number | null,
};
acc.reachable += reachableBackfillWork(entry);
acc.needsMedia += entry.missingInput;
@@ -214,6 +238,16 @@ export async function buildWidgetSyncPayload(): Promise<WidgetSyncPayload> {
// existed lacks them, and undefined poisons the sum to NaN.
acc.blocked += entry.blocked ?? 0;
acc.deferred += entry.deferred ?? 0;
+ // The SAME null-voids-the-sum discipline digestEligibleKnown uses above,
+ // per kind rather than corpus-wide — a kind whose channels can all report
+ // must not be dragged to "unknown" by a kind whose channels cannot.
+ acc.eligible =
+ acc.eligible == null || entry.eligible == null
+ ? null
+ : acc.eligible + entry.eligible;
+ const done = presentBackfillWork(entry);
+ acc.present =
+ acc.present == null || done == null ? null : acc.present + done;
backfillByKind.set(id, acc);
}
}
diff --git a/editor/app/auto-queue/actions.ts b/editor/app/auto-queue/actions.ts
@@ -16,6 +16,7 @@ import {
type AutoQueueGroup,
type AutoQueueNode,
type AutoQueueOrder,
+ type AutoQueueReach,
} from "yt-dlp-transcript-common/jobs/autoQueuePolicy";
export type SaveResult = { ok: true } | { ok: false; error: string };
@@ -170,3 +171,52 @@ export async function prioritizeChannelDownloadAction(
revalidatePath("/auto-queue");
return { ok: true };
}
+
+// Order and reach for the two SWEEP-fed lanes.
+//
+// A separate action from saveAutoQueueAction because these lanes have no policy
+// tree to save alongside: their order lives in settings.digest / settings.
+// backfill, not in settings.autoQueue. Routing them through the policy form
+// would mean an operator could not change an order without also committing a
+// pending tree edit for a different lane.
+//
+// SPREAD-AND-OVERRIDE, never a rebuilt literal. settings.digest carries
+// `sweepEnabled` and its scope, and settings.backfill carries both plus
+// `allowRedownload`; a literal here that omitted one would disarm a multi-week
+// sweep, or write media to a 97%-full disk, on an unrelated save. That exact
+// bug is documented in the settings form's own digest block.
+//
+// It takes effect on the sweep's NEXT PASS — both sweeps re-read the setting per
+// pass rather than at launch — and on the next per-channel batch, which resolves
+// the order itself. Nothing needs restarting.
+export async function saveLaneOrderAction(
+ lane: "digest" | "backfill",
+ input: { order: AutoQueueOrder; reach: AutoQueueReach },
+): Promise<SaveResult> {
+ const current = getSettings();
+ const next: SiteSettings =
+ lane === "digest"
+ ? {
+ ...current,
+ digest: {
+ ...current.digest,
+ recencyOrder: input.order,
+ recencyReach: input.reach,
+ },
+ }
+ : {
+ ...current,
+ backfill: {
+ ...current.backfill,
+ order: input.order,
+ reach: input.reach,
+ },
+ };
+ try {
+ await writeSettings(next);
+ } catch (e) {
+ return { ok: false, error: (e as Error).message };
+ }
+ revalidatePath("/auto-queue");
+ return { ok: true };
+}
diff --git a/editor/app/auto-queue/components/AutoQueueView.tsx b/editor/app/auto-queue/components/AutoQueueView.tsx
@@ -1,33 +1,71 @@
"use client";
-import { useCallback, useEffect, useRef, useState } from "react";
+import { useCallback, useEffect, useId, useRef, useState } from "react";
import type { AutoQueueKind } from "yt-dlp-transcript-common/jobs/autoQueueState";
import type {
AutoQueueStatusPayload,
AutoQueueKindStatus,
PlatformCooldownView,
} from "../status";
-import { DispatchDeck } from "./DispatchDeck";
import { InFlightList } from "./InFlightList";
import { LaneHeader } from "./LaneHeader";
import { NextUp } from "./NextUp";
+import { OperationRail, type RailLaneState } from "./OperationRail";
import { PolicyTreeEditor } from "./PolicyTreeEditor";
import { SnoozeControl } from "./SnoozeControl";
-import { type Channel, formatClock, formatCooldown, leafOrder } from "./dispatch";
+import { SweepLane } from "./SweepLane";
+import {
+ type Channel,
+ SELECT_CLASS,
+ formatClock,
+ formatCooldown,
+ idleReasonText,
+ leafOrder,
+} from "./dispatch";
+import { deriveLaneState } from "../../components/lanes/laneState";
+import type { SweepLaneStatus } from "../lanes";
-// The dispatcher board. A deck showing both runners, then one lane per runner
-// answering, top to bottom: what is it about to do, what is it doing, by what
-// rules, and what did it just do.
+// THE CONSOLE. One grammar for four pipelines.
//
-// STRUCTURAL CONTRACT, load-bearing for ~1,200 lines of e2e:
-// * each kind is a literal <section> containing an <h2> named exactly
+// It reads top to bottom as: every lane at a glance (the rail), then ONE lane in
+// full (chosen by a dropdown). That split is the whole redesign. Before it, the
+// page stacked two identical runner panels and had no room for the other two
+// pipelines at all — so "the digest lane is idle because transcription is
+// hogging the GPU" was a fact you could only assemble by visiting three pages.
+//
+// THE RAIL IS THE SWITCHER'S CONTEXT, not a summary of it. You never have to
+// switch lanes to learn that THIS lane is idle because ANOTHER one is, which on
+// this corpus is the normal case: attribution-diarized is 99.9% blocked behind
+// diarization, diarization is 99.2% media-gone.
+//
+// STRUCTURAL CONTRACT, load-bearing for ~1,200 lines of e2e — every clause below
+// survives this redesign unchanged:
+// * each runner kind is a literal <section> containing an <h2> named exactly
// "Auto-transcribe" / "Auto-download", with the policy editor and the
// Start/Drain/Stop buttons inside it;
// * NOTHING inside a kind section is itself a <section> — the suite scopes with
// locator("section", { has: heading }), and a nested one would match two
// ancestors, failing every scoped lookup on strict mode;
-// * the deck names runners in plain text, never as headings, for the same
-// reason.
+// * the rail names pipelines in plain text, never as headings, for the same
+// reason;
+// * policy controls stay native <select> / <input type=checkbox>;
+// * role="status" stays reserved for "Saved.";
+// * data-hydrated stays on the section.
+//
+// THE NON-SELECTED LANES STAY MOUNTED, hidden rather than unmounted. Two reasons,
+// and the second is the load-bearing one: a runner lane holds live policy-edit
+// state that unmounting would silently discard, and the page-wide option counts
+// the suite asserts must keep resolving to the same number. Auto-transcribe is
+// the default selection, which is where every existing UI spec already scopes.
+
+type LaneId = AutoQueueKind | "digest" | "backfill";
+
+const LANE_OPTIONS: { id: LaneId; label: string }[] = [
+ { id: "transcription", label: "Auto-transcribe" },
+ { id: "download", label: "Auto-download" },
+ { id: "digest", label: "Digest" },
+ { id: "backfill", label: "Backfill" },
+];
export function AutoQueueView({
initial,
@@ -41,8 +79,10 @@ export function AutoQueueView({
bucketsByKind: Record<AutoQueueKind, string[]>;
}) {
const [data, setData] = useState<AutoQueueStatusPayload>(initial);
+ const [lane, setLane] = useState<LaneId>("transcription");
const mounted = useRef(true);
const now = useNow();
+ const switcherId = useId();
const refresh = useCallback(async () => {
try {
@@ -64,9 +104,45 @@ export function AutoQueueView({
};
}, [refresh]);
+ // The rail is keyed by OPERATION id; the switcher by LANE. They are not the
+ // same space — "backfill" is one lane covering several operations — so the
+ // highlight resolves through this rather than comparing the two directly.
+ const highlightedOperation =
+ lane === "transcription" ? "transcription" : lane === "download" ? "download" : lane;
+
return (
- <div className="flex flex-col gap-8">
- <DispatchDeck data={data} now={now} />
+ <div className="flex flex-col gap-6">
+ <OperationRail
+ bands={data.lanes.bands}
+ states={railStates(data)}
+ selectedId={highlightedOperation}
+ />
+
+ <div className="flex flex-wrap items-center gap-2">
+ <label
+ htmlFor={switcherId}
+ className="font-mono text-xs uppercase tracking-[0.14em] text-muted-foreground"
+ >
+ Lane
+ </label>
+ {/* A native <select>, NOT the kit's Radix one. Radix portals its options
+ and unmounts them while closed, which would take the page-wide option
+ counts the suite asserts to zero. Tabs are out for the same class of
+ reason — they unmount their panels. */}
+ <select
+ id={switcherId}
+ className={SELECT_CLASS}
+ value={lane}
+ onChange={(e) => setLane(e.target.value as LaneId)}
+ >
+ {LANE_OPTIONS.map((o) => (
+ <option key={o.id} value={o.id}>
+ {o.label}
+ </option>
+ ))}
+ </select>
+ </div>
+
<KindLane
kind="transcription"
title="Auto-transcribe"
@@ -75,6 +151,7 @@ export function AutoQueueView({
platforms={platforms}
buckets={bucketsByKind.transcription}
now={now}
+ hidden={lane !== "transcription"}
onRefresh={refresh}
/>
<KindLane
@@ -85,12 +162,102 @@ export function AutoQueueView({
platforms={platforms}
buckets={bucketsByKind.download}
now={now}
+ hidden={lane !== "download"}
onRefresh={refresh}
/>
+ {/* `hidden` ALONE IS NOT ENOUGH HERE. Tailwind's preflight sets
+ [hidden]{display:none} at low specificity, and a `flex` utility in the
+ utilities layer beats it — so a section carrying both would stay
+ visible while claiming to be hidden. The class is switched too, and
+ the attribute kept for semantics. */}
+ <section
+ data-lane="digest"
+ hidden={lane !== "digest"}
+ className={lane === "digest" ? "flex flex-col gap-4" : "hidden"}
+ >
+ <SweepLane
+ lane={data.lanes.digest}
+ band={data.lanes.bands.find((b) => b.id === "digest") ?? null}
+ onRefresh={refresh}
+ />
+ </section>
+ <section
+ data-lane="backfill"
+ hidden={lane !== "backfill"}
+ className={lane === "backfill" ? "flex flex-col gap-4" : "hidden"}
+ >
+ <SweepLane
+ lane={data.lanes.backfill}
+ // Backfill is ONE lane over SEVERAL operations, and their counts are
+ // in different units (attribution-text is ~1 model call per transcript
+ // chunk; diarization is one audio pass per video). There is no single
+ // band for it, and inventing one by summing is the exact mistake
+ // laneKindEntriesOf exists to make unspellable.
+ band={null}
+ onRefresh={refresh}
+ />
+ </section>
</div>
);
}
+// operation id -> the state of the lane that would dispatch it.
+//
+// SEVERAL OPERATIONS SHARE ONE LANE, and that is the fact worth seeing: every
+// backfill kind runs on one queue, so a diarization row reading "Holding" and
+// an attribution row reading "Holding" are one gate, not two. The rail draws
+// them separately because their WORK COUNTS are in different units, and maps
+// them onto the same lane state because their DISPATCH is not.
+//
+// Any operation the console has no lane for reads "Off" rather than "Idle" —
+// see deriveLaneState: an idle-looking lane claims "all caught up" where the
+// truth is "nothing would run this".
+function railStates(data: AutoQueueStatusPayload): Record<string, RailLaneState> {
+ const out: Record<string, RailLaneState> = {};
+
+ for (const status of [data.transcription, data.download]) {
+ const running = status.runner.running;
+ const inFlight = status.runner.inFlight.length;
+ out[status.kind] = {
+ // A runner has no sweep; its feed is the job queue, so `activeCount` is
+ // what tells running from idle. A STOPPED runner is "Off", not "Idle" —
+ // it will never pick anything up, which "Idle" does not say.
+ state: !running
+ ? "unavailable"
+ : deriveLaneState({ gateHeld: false, activeCount: inFlight }),
+ note: running ? idleReasonText(status.runner.idleReason, status.kind) : null,
+ };
+ }
+
+ const sweepState = (sweep: SweepLaneStatus): RailLaneState => ({
+ state: deriveLaneState({
+ available: sweep.available,
+ gateHeld: sweep.gateHeld,
+ feedRunning: sweep.sweeping,
+ activeCount: sweep.inFlight.length,
+ }),
+ note: !sweep.available
+ ? "no operation switched on"
+ : sweep.gateHeld
+ ? sweep.sweeping
+ ? "sweep armed, lane paused"
+ : "lane paused"
+ : sweep.sweeping || sweep.inFlight.length > 0
+ ? null
+ : "no sweep armed",
+ });
+
+ out.digest = sweepState(data.lanes.digest);
+ // Every OTHER catalog operation is dispatched by the one backfill lane, so it
+ // takes that lane's state — including the ones with no band of their own.
+ const backfill = sweepState(data.lanes.backfill);
+ for (const band of data.lanes.bands) {
+ if (out[band.id]) continue;
+ out[band.id] = backfill;
+ }
+ return out;
+}
+
function KindLane({
kind,
title,
@@ -99,6 +266,7 @@ function KindLane({
platforms,
buckets,
now,
+ hidden,
onRefresh,
}: {
kind: AutoQueueKind;
@@ -108,6 +276,7 @@ function KindLane({
platforms: string[];
buckets: string[];
now: number | null;
+ hidden: boolean;
onRefresh: () => Promise<void>;
}) {
const [busy, setBusy] = useState(false);
@@ -136,7 +305,25 @@ function KindLane({
// than once. `now` is null until the client mount effect runs, so it is an
// exact "React is live on this subtree" signal that already exists; exposing
// it lets a test wait for the page rather than race it.
- <section className="flex flex-col gap-4" data-hydrated={now !== null ? "true" : undefined}>
+ <section
+ // See the note at the digest section: the `hidden` attribute needs the
+ // class switched with it, or a `flex` utility overrides it.
+ className={hidden ? "hidden" : "flex flex-col gap-4"}
+ hidden={hidden}
+ // MEASURED, and it contradicts the plan this redesign came from: keeping
+ // a lane MOUNTED does NOT keep it addressable. A heading inside a
+ // display:none subtree is out of the accessibility tree, so
+ // locator("section", { has: getByRole("heading", …) }) resolves to ZERO
+ // elements for a hidden lane — and `toBeHidden()` passes vacuously on an
+ // element that does not resolve at all.
+ //
+ // That is correct behaviour (a screen reader must not see the hidden
+ // pane either), so the fix is an addressing hook rather than a rendering
+ // change. What mounting genuinely preserves is the lane's React state —
+ // an unsaved policy edit survives a switch away and back.
+ data-lane={kind}
+ data-hydrated={now !== null ? "true" : undefined}
+ >
<LaneHeader
title={title}
status={status}
diff --git a/editor/app/auto-queue/components/DispatchDeck.tsx b/editor/app/auto-queue/components/DispatchDeck.tsx
@@ -1,96 +0,0 @@
-"use client";
-
-import type { AutoQueueStatusPayload, AutoQueueKindStatus } from "../status";
-import { formatElapsed, idleReasonText } from "./dispatch";
-
-// Both runners at a glance, above the fold, so you never scroll to learn the
-// state of the other one. State ONLY — every control lives in the lane below.
-// That is not just tidiness: two Start buttons would give the page two elements
-// with the same accessible name, and the e2e suite addresses them by name.
-//
-// The runner names here are deliberately PLAIN TEXT, not headings. The whole
-// suite scopes itself with locator("section", { has: heading "Auto-transcribe" }),
-// and a second heading with that name would make every one of those lookups
-// ambiguous.
-
-export function DispatchDeck({
- data,
- now,
-}: {
- data: AutoQueueStatusPayload;
- now: number | null;
-}) {
- return (
- <div className="rounded-lg border border-border bg-card">
- <p className="border-b border-border px-4 py-2 font-mono text-xs uppercase tracking-[0.14em] text-muted-foreground">
- Dispatch
- </p>
- <div className="divide-y divide-border">
- <DeckRow label="Auto-transcribe" status={data.transcription} now={now} />
- <DeckRow label="Auto-download" status={data.download} now={now} />
- </div>
- </div>
- );
-}
-
-function DeckRow({
- label,
- status,
- now,
-}: {
- label: string;
- status: AutoQueueKindStatus;
- now: number | null;
-}) {
- const running = status.runner.running;
- const inFlight = status.runner.inFlight.length;
- const pending = Object.values(status.pendingByLeaf).reduce((a, b) => a + b, 0);
- const idle = running ? idleReasonText(status.runner.idleReason, status.kind) : null;
- // Working > held > stopped. A running runner with nothing in flight is not the
- // same as a stopped one, and the dot is the only thing that says so at a
- // glance — so it takes the "attention" tone rather than the neutral one.
- const tone = !running
- ? "bg-muted-foreground/40"
- : inFlight > 0
- ? "bg-info animate-pulse motion-reduce:animate-none"
- : "bg-warning";
-
- return (
- <div className="flex flex-wrap items-baseline gap-x-3 gap-y-1 px-4 py-2.5 text-sm">
- <span
- aria-hidden="true"
- className={`size-2 shrink-0 self-center rounded-full ${tone}`}
- />
- <span className="min-w-40 font-medium text-foreground">{label}</span>
- <span className="text-muted-foreground">
- {!running ? (
- "not running"
- ) : (
- <>
- up{" "}
- <span className="tabular-nums">
- {now !== null && status.runner.startedAt
- ? formatElapsed(now - status.runner.startedAt)
- : "—"}
- </span>
- {idle ? ` · idle: ${idle}` : ""}
- </>
- )}
- </span>
- <span className="ml-auto flex items-baseline gap-4 text-muted-foreground">
- <span>
- <span className="font-display tabular-nums text-foreground">
- {inFlight}
- </span>{" "}
- in flight
- </span>
- <span>
- <span className="font-display tabular-nums text-foreground">
- {pending.toLocaleString()}
- </span>{" "}
- pending
- </span>
- </span>
- </div>
- );
-}
diff --git a/editor/app/auto-queue/components/OperationRail.tsx b/editor/app/auto-queue/components/OperationRail.tsx
@@ -0,0 +1,273 @@
+"use client";
+
+import { bandCoverage, type OperationBand } from "../lib/operationBand";
+import type { LaneState } from "../../components/lanes/laneState";
+import { LANE_DOT, LANE_TEXT, LANE_WORD } from "../../components/lanes/laneState";
+
+// What a rail row says about the lane that would do this operation's work, in
+// the vocabulary deriveLaneState already established: Running / Holding / Idle /
+// Off. "Holding" is the state this codebase already named for "sweep armed,
+// gate shut", and it is precisely what a crowded card was failing to say.
+export type RailLaneState = {
+ state: LaneState;
+ // Why it is not working, in words — idleReasonText for a runner, the gate for
+ // a sweep. Null when it IS working, or when there is nothing to explain.
+ note: string | null;
+};
+
+// THE COMPARISON RAIL: every pipeline, one line each, always present.
+//
+// It is the lane switcher's context, not decoration. On this corpus the normal
+// case is that a lane is idle BECAUSE another one is — attribution-diarized is
+// 99.9% blocked behind diarization, diarization is 99.2% media-gone — and
+// without the rail you have to switch lanes to discover that, one lane at a
+// time, which is exactly the question a console should answer without being
+// asked.
+//
+// EXACTLY ONE SATURATED COLOUR ON THE PAGE: `reachable`. Everything else is
+// texture on neutral. Glancing down four bands shows you WHERE THE COLOUR IS,
+// which is "where work can happen right now" — and that is the whole boldness
+// budget, in agreement with LaneCard's own note that a third tinted background
+// would be the accessory to remove.
+//
+// FILL PATTERN FIRST, HUE SECOND. Not a stylistic whim: report-to-video's claim
+// rail established here BY MEASUREMENT that no four-colour palette clears
+// all-pairs colour-blindness. Pattern (solid / hatched / dotted / hollow)
+// survives CVD, greyscale, and a dim laptop at 2am, which is when this console
+// is actually read.
+//
+// EACH BAND IS NORMALISED TO ITS OWN `eligible`, stated on its own row. Digest's
+// 79,681 is genuinely a different population from diarization's 78,019, and
+// quietly sharing one denominator to make the bars comparable would be a lie
+// about what is being compared.
+
+// The five populations, in the order they are laid down. `present` first so a
+// band reads left-to-right as "done → can do → cannot do yet → cannot do at
+// all", which is the same direction the channel transit line runs.
+const SEGMENTS = [
+ {
+ key: "present",
+ label: "done",
+ // Solid, muted. The artifact exists; it is not where attention goes.
+ className: "bg-muted-foreground/45",
+ },
+ {
+ key: "reachable",
+ label: "reachable now",
+ // THE one saturated segment.
+ className: "bg-info",
+ },
+ {
+ key: "blocked",
+ label: "blocked upstream",
+ // Hatched — waiting on an operation this system produces, so it will clear
+ // itself without anyone doing anything.
+ className: "bg-warning/25 [background-image:repeating-linear-gradient(45deg,currentColor_0_1px,transparent_1px_4px)] text-warning/70",
+ },
+ {
+ key: "deferred",
+ label: "held by a gate",
+ // Dotted — a gate (stale cues, a duration window) is holding it.
+ className: "bg-transparent [background-image:radial-gradient(currentColor_0.5px,transparent_0.5px)] [background-size:3px_3px] text-muted-foreground",
+ },
+ {
+ key: "missingInput",
+ label: "media gone",
+ // Hollow outline — there is nothing to do. An opt-in re-download is the
+ // only thing that would ever change it, and it must never look like work.
+ className: "bg-transparent ring-1 ring-inset ring-border",
+ },
+] as const;
+
+type SegmentKey = (typeof SEGMENTS)[number]["key"];
+
+function valueOf(band: OperationBand, key: SegmentKey): number {
+ if (key === "present") return band.present ?? 0;
+ return band[key];
+}
+
+// The denominator a band's segments are drawn against.
+//
+// `eligible` when it is known. When it is NOT — a third of the snapshots on
+// disk predate the field — falling back to the sum of the work counts would
+// silently redraw the band as "100% accounted for", so instead the band renders
+// as an un-filled outline and says `coverage unknown`. Unknown is not zero, and
+// it is not full either.
+function denominatorOf(band: OperationBand): number | null {
+ if (band.eligible != null && band.eligible > 0) return band.eligible;
+ return null;
+}
+
+export function OperationRail({
+ bands,
+ states,
+ selectedId,
+}: {
+ bands: OperationBand[];
+ // operation id -> the state of the lane that dispatches it. Several
+ // operations can share one lane (every backfill kind runs on one queue), and
+ // that is exactly the fact the rail exists to make visible.
+ states: Record<string, RailLaneState>;
+ // The lane the console is focused on, highlighted so the rail and the
+ // dropdown are visibly the same control.
+ selectedId: string | null;
+}) {
+ return (
+ <div className="rounded-lg border border-border bg-card">
+ <p className="border-b border-border px-4 py-2 font-mono text-xs uppercase tracking-[0.14em] text-muted-foreground">
+ Pipelines
+ </p>
+ <ul className="divide-y divide-border">
+ {bands.map((band) => (
+ <RailRow
+ key={band.id}
+ band={band}
+ lane={states[band.id] ?? { state: "unavailable", note: null }}
+ selected={band.id === selectedId}
+ />
+ ))}
+ </ul>
+ <Legend />
+ </div>
+ );
+}
+
+function RailRow({
+ band,
+ lane,
+ selected,
+}: {
+ band: OperationBand;
+ lane: RailLaneState;
+ selected: boolean;
+}) {
+ const denominator = denominatorOf(band);
+ const coverage = bandCoverage(band);
+
+ return (
+ <li
+ data-operation={band.id}
+ className={`flex flex-wrap items-center gap-x-4 gap-y-1 px-4 py-2.5 text-sm ${
+ selected ? "bg-muted/40" : ""
+ }`}
+ >
+ <span
+ className={`flex min-w-36 shrink-0 items-center gap-2 ${
+ selected ? "font-medium text-foreground" : "text-muted-foreground"
+ }`}
+ >
+ <span
+ aria-hidden="true"
+ className={`size-2 shrink-0 rounded-full ${LANE_DOT[lane.state]}`}
+ />
+ {band.label}
+ </span>
+ <Band band={band} denominator={denominator} />
+ {/* PLAIN TEXT, never a heading — the suite scopes whole tests with
+ locator("section", { has: heading "Auto-transcribe" }), and a second
+ element with that accessible name would make every one ambiguous. */}
+ <span className={`shrink-0 text-xs ${LANE_TEXT[lane.state]}`}>
+ {LANE_WORD[lane.state]}
+ {lane.note ? (
+ <span className="text-muted-foreground">: {lane.note}</span>
+ ) : null}
+ </span>
+ <span className="ml-auto flex shrink-0 items-baseline gap-3 text-xs text-muted-foreground">
+ {/* THE FIGURES ARE STATED SEPARATELY AND NEVER SUMMED. On the measured
+ corpus reachable and needs-media are 91x apart; one "remaining"
+ number would say the same thing about a finished lane and a lane
+ that cannot start. */}
+ <Figure n={band.reachable} unit="reachable" tone="text-foreground" />
+ {band.blocked > 0 && <Figure n={band.blocked} unit="blocked" />}
+ {band.missingInput > 0 && (
+ <Figure n={band.missingInput} unit="no media" />
+ )}
+ {band.deferred > 0 && <Figure n={band.deferred} unit="held" />}
+ <span className="tabular-nums">
+ {coverage === null
+ ? "coverage unknown"
+ : `${(coverage * 100).toFixed(coverage < 0.1 ? 2 : 0)}% of ${denominator?.toLocaleString()}`}
+ </span>
+ </span>
+ </li>
+ );
+}
+
+function Figure({
+ n,
+ unit,
+ tone = "",
+}: {
+ n: number;
+ unit: string;
+ tone?: string;
+}) {
+ return (
+ <span>
+ <span className={`tabular-nums ${tone}`}>{n.toLocaleString()}</span>{" "}
+ {unit}
+ </span>
+ );
+}
+
+function Band({
+ band,
+ denominator,
+}: {
+ band: OperationBand;
+ denominator: number | null;
+}) {
+ if (denominator === null) {
+ // Unknown denominator: an outline and nothing else. Drawing the work counts
+ // against their own sum would claim the band is fully accounted for, which
+ // is a different (and worse) statement than "we cannot say".
+ return (
+ <span
+ aria-hidden="true"
+ className="h-2.5 min-w-32 flex-1 rounded-sm border border-dashed border-border"
+ />
+ );
+ }
+ return (
+ <span
+ aria-hidden="true"
+ className="flex h-2.5 min-w-32 flex-1 overflow-hidden rounded-sm bg-muted"
+ >
+ {SEGMENTS.map((seg) => {
+ const pct = (valueOf(band, seg.key) / denominator) * 100;
+ if (pct <= 0) return null;
+ return (
+ <span
+ key={seg.key}
+ // The ONE piece of motion on the page: a segment eases to its new
+ // width when the number actually changes. Never on poll (the width
+ // is the same, so no transition fires) and never at all under
+ // prefers-reduced-motion, which every animation in this codebase
+ // pairs with.
+ className={`h-full transition-[width] duration-500 motion-reduce:transition-none ${seg.className}`}
+ style={{ width: `${Math.min(100, pct)}%` }}
+ />
+ );
+ })}
+ </span>
+ );
+}
+
+// The legend is the only place the pattern vocabulary is named, and it is named
+// in WORDS as well as swatches — a pattern nobody can read the meaning of is
+// just noise, and this is the page's one non-obvious visual convention.
+function Legend() {
+ return (
+ <ul className="flex flex-wrap gap-x-4 gap-y-1 border-t border-border px-4 py-2 text-xs text-muted-foreground">
+ {SEGMENTS.map((seg) => (
+ <li key={seg.key} className="flex items-center gap-1.5">
+ <span
+ aria-hidden="true"
+ className={`h-2.5 w-4 shrink-0 rounded-sm ${seg.className}`}
+ />
+ {seg.label}
+ </li>
+ ))}
+ </ul>
+ );
+}
diff --git a/editor/app/auto-queue/components/OrderReach.tsx b/editor/app/auto-queue/components/OrderReach.tsx
@@ -0,0 +1,133 @@
+"use client";
+
+import { useId } from "react";
+import type {
+ AutoQueueOrder,
+ AutoQueueReach,
+} from "yt-dlp-transcript-common/jobs/autoQueuePolicy";
+import { ORDER_LABEL, SELECT_CLASS } from "./dispatch";
+
+// ORDER AND REACH, at the foot of the lane where the decision lives.
+//
+// Not in /settings. An operator asking "why is it working on a 2019 video" is
+// looking at this lane, and the answer is one control away or it is nowhere.
+//
+// TWO AXES, NOT ONE ENUM. Order says WHICH video; Reach says HOW WIDELY that
+// applies — within each channel (the sweep still visits channels heaviest-
+// first) or across all channels (the channel holding the freshest work goes
+// first). Reach is meaningless without an order, so it is DISABLED while
+// "Listed" is selected rather than being silently ignored.
+//
+// Native <select>, like every other policy control on this page. That is
+// load-bearing, not stylistic — see the note at the foot of dispatch.ts.
+
+export const REACH_LABEL: Record<AutoQueueReach, string> = {
+ channel: "Within each channel",
+ corpus: "Across all channels",
+};
+
+export function OrderReach({
+ order,
+ reach,
+ // The trade-off sentence for this particular lane, or null when there is
+ // none. Supplied rather than derived so the runner lanes keep using the
+ // existing orderTradeoff() copy verbatim.
+ tradeoff,
+ busy,
+ // Pinned by the e2e suite for the two runner lanes
+ // ("video order for auto-transcribe"). Optional because the sweep lanes have
+ // no such contract and the visible <label> already names the control.
+ orderAriaLabel,
+ onChange,
+}: {
+ order: AutoQueueOrder;
+ // NULL for the two RUNNER lanes, and that is a statement rather than a gap:
+ // a runner's order already applies across every channel and bucket a rule
+ // claims, so its reach is fixed at corpus-wide and there is no axis to offer.
+ // Rendering a disabled Reach dropdown there would imply a setting that does
+ // not exist; the fixed sentence below says what is actually true.
+ reach: AutoQueueReach | null;
+ tradeoff: string | null;
+ busy: boolean;
+ orderAriaLabel?: string;
+ onChange: (next: { order: AutoQueueOrder; reach: AutoQueueReach }) => void;
+}) {
+ const orderId = useId();
+ const reachId = useId();
+ const listed = order === "listed";
+
+ return (
+ <div className="flex flex-col gap-2 rounded-md border border-border bg-card px-3 py-2">
+ <p className="font-mono text-xs uppercase tracking-[0.14em] text-muted-foreground">
+ Order
+ </p>
+ <div className="flex flex-wrap items-center gap-x-6 gap-y-2 text-sm">
+ <span className="flex items-center gap-2">
+ <label htmlFor={orderId} className="text-muted-foreground">
+ Order
+ </label>
+ <select
+ id={orderId}
+ aria-label={orderAriaLabel}
+ className={SELECT_CLASS}
+ value={order}
+ disabled={busy}
+ onChange={(e) =>
+ onChange({
+ order: e.target.value as AutoQueueOrder,
+ reach: reach ?? "corpus",
+ })
+ }
+ >
+ {(Object.keys(ORDER_LABEL) as AutoQueueOrder[]).map((o) => (
+ <option key={o} value={o}>
+ {ORDER_LABEL[o]}
+ </option>
+ ))}
+ </select>
+ </span>
+ {reach !== null && (
+ <span className="flex items-center gap-2">
+ <label
+ htmlFor={reachId}
+ className={
+ listed ? "text-muted-foreground/50" : "text-muted-foreground"
+ }
+ >
+ Reach
+ </label>
+ <select
+ id={reachId}
+ className={SELECT_CLASS}
+ value={reach}
+ // Disabled, not hidden: the control staying in place is what
+ // tells you the axis exists and what turns it on.
+ disabled={busy || listed}
+ onChange={(e) =>
+ onChange({ order, reach: e.target.value as AutoQueueReach })
+ }
+ >
+ {(Object.keys(REACH_LABEL) as AutoQueueReach[]).map((r) => (
+ <option key={r} value={r}>
+ {REACH_LABEL[r]}
+ </option>
+ ))}
+ </select>
+ </span>
+ )}
+ </div>
+ <p className="text-xs text-muted-foreground">
+ {reach === null
+ ? listed
+ ? "Listed order is whatever order the work list already had — bucket order, then channel order."
+ : "A rule orders every video it claims, across every channel and bucket. Which RULE goes first is still the rule list's job; for a pure newest-first archive, use one catch-all rule."
+ : listed
+ ? "Listed order is whatever order the work list already had. Pick an order to enable Reach."
+ : reach === "corpus"
+ ? "Channels are visited by the freshest work each is holding, and each channel's own videos follow the same order. Heaviest-first is still the tiebreak."
+ : "Each channel's own videos are ordered. Which channel goes first is unchanged — heaviest first."}
+ </p>
+ {tradeoff && <p className="text-xs text-warning">{tradeoff}</p>}
+ </div>
+ );
+}
diff --git a/editor/app/auto-queue/components/PolicyTreeEditor.tsx b/editor/app/auto-queue/components/PolicyTreeEditor.tsx
@@ -6,7 +6,6 @@ import {
type AutoQueueLeaf,
type AutoQueueNode,
type AutoQueueOrder,
- AUTO_QUEUE_ORDERS,
isGroup,
} from "yt-dlp-transcript-common/jobs/autoQueuePolicy";
import type { AutoQueueKind } from "yt-dlp-transcript-common/jobs/autoQueueState";
@@ -14,12 +13,10 @@ import { saveAutoQueueAction, type SaveResult } from "../actions";
import type { AutoQueueKindStatus } from "../status";
import { ClaimLadder } from "./ClaimLadder";
import type { RungOps } from "./LadderRung";
+import { OrderReach } from "./OrderReach";
import { SaveBar } from "./SaveBar";
import {
type Channel,
- ORDER_HINT,
- ORDER_LABEL,
- SELECT_CLASS,
NUM_CLASS,
leafOrder,
orderTradeoff,
@@ -246,26 +243,6 @@ export function PolicyTreeEditor({
Enable auto-{kindWord}
</label>
<label className="flex items-center gap-2 text-sm">
- <span className="text-muted-foreground">Order</span>
- <select
- aria-label={`video order for auto-${kindWord}`}
- value={form.order}
- onChange={(e) =>
- setForm((f) => ({
- ...f,
- order: e.target.value as AutoQueueOrder,
- }))
- }
- className={SELECT_CLASS}
- >
- {AUTO_QUEUE_ORDERS.map((o) => (
- <option key={o} value={o}>
- {ORDER_LABEL[o]}
- </option>
- ))}
- </select>
- </label>
- <label className="flex items-center gap-2 text-sm">
<span className="text-muted-foreground">Runner max workers</span>
<input
type="number"
@@ -286,11 +263,6 @@ export function PolicyTreeEditor({
</label>
</div>
- <p className="text-xs text-muted-foreground">
- {ORDER_HINT}
- {tradeoff ? ` ${tradeoff}` : ""}
- </p>
-
<ClaimLadder
root={form.root}
channels={channels}
@@ -336,6 +308,25 @@ export function PolicyTreeEditor({
</span>
</label>
+ {/* ORDER AT THE FOOT OF THE LANE, in the same block the sweep lanes use,
+ so one control is learned once for all four pipelines. It stays part
+ of the POLICY FORM rather than saving on change: the tree above it has
+ an unsaved-changes discipline, and a control that saved itself while
+ sitting inside a dirty form would be the one thing on the page that
+ did not mean what the Save button says.
+
+ Reach is null here, and that is a statement rather than a gap — a
+ runner's order already applies across every channel and bucket a rule
+ claims, so there is no second axis to offer. */}
+ <OrderReach
+ order={form.order}
+ reach={null}
+ tradeoff={tradeoff}
+ busy={saving}
+ orderAriaLabel={`video order for auto-${kindWord}`}
+ onChange={(next) => setForm((f) => ({ ...f, order: next.order }))}
+ />
+
<SaveBar
dirty={dirty}
saving={saving}
diff --git a/editor/app/auto-queue/components/SweepLane.tsx b/editor/app/auto-queue/components/SweepLane.tsx
@@ -0,0 +1,301 @@
+"use client";
+
+import Link from "next/link";
+import { useState, useTransition } from "react";
+import { Button } from "yt-dlp-transcript-common/components/ui/button";
+import {
+ deriveLaneState,
+ LANE_DOT,
+ LANE_TEXT,
+ LANE_WORD,
+} from "../../components/lanes/laneState";
+import {
+ pauseBackfillAction,
+ pauseDigestsAction,
+ resumeBackfillAction,
+ resumeDigestsAction,
+ startBackfillSweepAction,
+ startDigestSweepAction,
+ stopBackfillSweepAction,
+ stopDigestSweepAction,
+} from "../../jobs/actions";
+import { saveLaneOrderAction } from "../actions";
+import type { SweepLaneStatus } from "../lanes";
+import type { OperationBand } from "../lib/operationBand";
+import { OrderReach } from "./OrderReach";
+import { formatElapsed } from "./dispatch";
+
+// A sweep-fed lane: digest or backfill.
+//
+// TWO CONTROLS, NOT ONE, and they keep the names they already have everywhere
+// else. The SWEEP is the feed — is there a corpus pass at all. The PAUSE is the
+// gate — is the lane consuming what the feed produces. LaneDeck says why they
+// must stay apart: conflating them is how an operator loses a week of GPU time,
+// because one is cheap to undo and the other is not. Both are here, and the
+// dashboard's cards keep both too.
+//
+// NO CLAIM LADDER. These lanes have no policy tree — settings.autoQueue has
+// exactly two keys — and an empty ladder would read as "no rule matches" where
+// the truth is "rules are not how this lane is dispatched". It gets one when
+// the arbiter does.
+//
+// A <div>, never a nested <section>: the e2e suite scopes with
+// locator("section", { has: heading }), and a section inside a section makes
+// every one of those lookups match two ancestors.
+
+export function SweepLane({
+ lane,
+ band,
+ onRefresh,
+}: {
+ lane: SweepLaneStatus;
+ // This lane's own band. For backfill that is one of several — the rail shows
+ // each kind separately, because summed they are a figure in no unit — so the
+ // panel names the kinds rather than pretending to a single total.
+ band: OperationBand | null;
+ onRefresh: () => void | Promise<void>;
+}) {
+ const [pending, startTransition] = useTransition();
+ const [busy, setBusy] = useState(false);
+ const digest = lane.id === "digest";
+ const state = deriveLaneState({
+ available: lane.available,
+ gateHeld: lane.gateHeld,
+ feedRunning: lane.sweeping,
+ activeCount: lane.inFlight.length,
+ });
+
+ const run = (action: () => Promise<unknown>) => {
+ setBusy(true);
+ startTransition(async () => {
+ try {
+ await action();
+ await onRefresh();
+ } finally {
+ setBusy(false);
+ }
+ });
+ };
+
+ const working = busy || pending;
+
+ return (
+ <div className="flex flex-col gap-4">
+ <div className="flex flex-wrap items-center gap-x-3 gap-y-2">
+ <h2 className="font-display text-lg font-semibold tracking-tight">
+ {lane.label}
+ </h2>
+ <span className="flex items-center gap-2 text-sm">
+ <span aria-hidden="true" className={`size-2 rounded-full ${LANE_DOT[state]}`} />
+ <span className={LANE_TEXT[state]}>{LANE_WORD[state]}</span>
+ </span>
+ <span className="ml-auto flex flex-wrap gap-2">
+ {/* THE FEED. Named "sweep" on both surfaces. */}
+ <Button
+ type="button"
+ size="sm"
+ variant={lane.sweeping ? "outline" : "default"}
+ disabled={working || !lane.available}
+ aria-label={`${lane.sweeping ? "Stop" : "Start"} ${lane.label} sweep`}
+ onClick={() =>
+ run(
+ lane.sweeping
+ ? digest
+ ? stopDigestSweepAction
+ : stopBackfillSweepAction
+ : digest
+ ? startDigestSweepAction
+ // Unscoped on purpose: every enabled lane kind, whole
+ // corpus. A scope belongs to the sweep controls that own
+ // one, not to a console button whose label says "sweep".
+ : () => startBackfillSweepAction(),
+ )
+ }
+ >
+ {lane.sweeping ? "Stop sweep" : "Start sweep"}
+ </Button>
+ {/* THE GATE. Backfill's is inverted on the wire (`enabled`); it is
+ normalised in lanes.ts so this button means the same thing on both
+ lanes. */}
+ <Button
+ type="button"
+ size="sm"
+ variant="outline"
+ disabled={working || !lane.available}
+ aria-label={`${lane.gateHeld ? "Resume" : "Pause"} ${lane.label}`}
+ onClick={() =>
+ run(
+ lane.gateHeld
+ ? digest
+ ? resumeDigestsAction
+ : resumeBackfillAction
+ : digest
+ ? pauseDigestsAction
+ : pauseBackfillAction,
+ )
+ }
+ className={
+ lane.gateHeld
+ ? ""
+ : "border-warning/30 text-warning hover:bg-warning-soft hover:text-warning"
+ }
+ >
+ {lane.gateHeld ? "Resume" : "Pause"}
+ </Button>
+ </span>
+ </div>
+
+ {!lane.available && (
+ <p className="text-sm text-muted-foreground">
+ No backfill operation is switched on, so there is no lane to run. That
+ is not the same as being finished — turn one on in Settings.
+ </p>
+ )}
+
+ {/* THE STATE SENTENCE. `holding` is the one an operator has no word for:
+ a sweep armed behind a shut gate looks exactly like a wedged runner
+ unless something says otherwise. */}
+ {state === "holding" && (
+ <p className="text-sm text-warning">
+ <span className="font-mono text-xs uppercase tracking-[0.14em]">
+ Holding
+ </span>{" "}
+ — {lane.sweeping ? "the sweep is armed but " : ""}the lane is paused, so
+ nothing is being consumed. Resume to let it move.
+ </p>
+ )}
+
+ <LaneFigures lane={lane} band={band} />
+ <InFlight lane={lane} />
+
+ <OrderReach
+ order={lane.order}
+ reach={lane.reach}
+ tradeoff={
+ lane.order === "listed"
+ ? null
+ : digest
+ ? "Digest still runs shortest-first inside a day, so a long VOD can wait behind shorter videos uploaded the same day."
+ : "Kinds are still walked in registry order, so the cheapest lane reaches a video first whatever the date order says."
+ }
+ busy={working}
+ onChange={(next) =>
+ run(() => saveLaneOrderAction(lane.id, next))
+ }
+ />
+ </div>
+ );
+}
+
+function LaneFigures({
+ lane,
+ band,
+}: {
+ lane: SweepLaneStatus;
+ band: OperationBand | null;
+}) {
+ if (!band) {
+ return (
+ <p className="text-sm text-muted-foreground">
+ This lane covers several operations; see the rail above for each one — a
+ single total across them would be a figure in no unit.
+ </p>
+ );
+ }
+ return (
+ <p className="flex flex-wrap items-baseline gap-x-2 gap-y-1 text-sm text-muted-foreground">
+ <Figure n={band.reachable} unit="reachable now" tone="text-foreground" />
+ {band.blocked > 0 && (
+ <>
+ <Sep />
+ <Figure n={band.blocked} unit="blocked upstream" />
+ </>
+ )}
+ {band.missingInput > 0 && (
+ <>
+ <Sep />
+ <Figure n={band.missingInput} unit="need media back" />
+ </>
+ )}
+ {band.deferred > 0 && (
+ <>
+ <Sep />
+ <Figure n={band.deferred} unit="held by a gate" />
+ </>
+ )}
+ <Sep />
+ <span>
+ {band.present == null || band.eligible == null
+ ? "coverage unknown until every channel has been re-reported"
+ : `${band.present.toLocaleString()} of ${band.eligible.toLocaleString()} done`}
+ </span>
+ {lane.sweepJobId && (
+ <>
+ <Sep />
+ <Link
+ href={`/jobs/${lane.sweepJobId}`}
+ className="underline underline-offset-2 hover:text-foreground"
+ >
+ sweep log
+ </Link>
+ </>
+ )}
+ </p>
+ );
+}
+
+function InFlight({ lane }: { lane: SweepLaneStatus }) {
+ const now = Date.now();
+ return (
+ <div className="flex flex-col gap-1">
+ <p className="font-mono text-xs uppercase tracking-[0.14em] text-muted-foreground">
+ In flight
+ </p>
+ {lane.inFlight.length === 0 ? (
+ <p className="text-sm text-muted-foreground">Nothing running.</p>
+ ) : (
+ <ul className="flex flex-col gap-0.5 text-sm">
+ {lane.inFlight.map((j) => (
+ <li key={j.id} className="flex flex-wrap gap-x-3">
+ <Link
+ href={`/jobs/${j.id}`}
+ className="font-mono text-foreground underline underline-offset-2"
+ >
+ {j.channelSlug ?? j.id}
+ </Link>
+ {j.startedAt !== null && (
+ <span className="tabular-nums text-muted-foreground">
+ {formatElapsed(now - j.startedAt)}
+ </span>
+ )}
+ </li>
+ ))}
+ </ul>
+ )}
+ </div>
+ );
+}
+
+function Figure({
+ n,
+ unit,
+ tone = "",
+}: {
+ n: number;
+ unit: string;
+ tone?: string;
+}) {
+ return (
+ <span>
+ <span className={`tabular-nums ${tone}`}>{n.toLocaleString()}</span> {unit}
+ </span>
+ );
+}
+
+function Sep() {
+ return (
+ <span aria-hidden="true" className="text-border">
+ ·
+ </span>
+ );
+}
diff --git a/editor/app/auto-queue/components/dispatch.ts b/editor/app/auto-queue/components/dispatch.ts
@@ -75,10 +75,6 @@ export const ORDER_LABEL: Record<AutoQueueOrder, string> = {
oldest: "Oldest first",
};
-export const ORDER_HINT =
- "Newest first orders the videos inside each rule. Rule order still wins; " +
- "for a pure newest-first archive, use one catch-all rule.";
-
export function orderTradeoff(
kind: AutoQueueKind,
order: AutoQueueOrder,
diff --git a/editor/app/auto-queue/lanes.ts b/editor/app/auto-queue/lanes.ts
@@ -0,0 +1,125 @@
+import { getPaths } from "yt-dlp-transcript-common/lib/paths";
+import { getSettings } from "yt-dlp-transcript-common/lib/settings";
+import type { AutoQueueOrder, AutoQueueReach } from "yt-dlp-transcript-common/jobs/autoQueuePolicy";
+import { allBackfillKinds } from "yt-dlp-transcript-common/lib/backfillKinds";
+import { getDigestSweepJobId } from "yt-dlp-transcript-common/controller/digestSweep";
+import { getBackfillSweepJobId } from "yt-dlp-transcript-common/controller/backfillSweep";
+import { getRegistry } from "yt-dlp-transcript-common/jobs/registry";
+import {
+ DIGEST_LOCAL_QUEUE,
+ DIGEST_REMOTE_QUEUE,
+ BACKFILL_QUEUE,
+} from "yt-dlp-transcript-common/lib/queueKeys";
+import { getChannelBriefs } from "../lib/requestCache";
+import {
+ buildOperationBands,
+ type OperationBand,
+} from "./lib/operationBands";
+
+// The two SWEEP-fed lanes, for a console that has to show four pipelines and
+// only has runners for two of them.
+//
+// Digest and backfill are dispatched by their own sweeps, not by the policy
+// tree, so they have no rules, no claim ladder and no next-up. What they do
+// have is the same four things every lane has — a state, a reason, work in
+// flight, and an order — and this is where those are read.
+//
+// NO CLAIM LADDER IS RENDERED FOR THEM, deliberately. A policy tree for these
+// lanes does not exist yet (settings.autoQueue has exactly two keys), and
+// drawing an empty ladder would say "no rules match" where the truth is "rules
+// are not how this lane is dispatched". It gets one when the arbiter does.
+
+export type SweepLaneId = "digest" | "backfill";
+
+export type SweepLaneStatus = {
+ id: SweepLaneId;
+ label: string;
+ // A corpus-wide sweep is ARMED — the feed. Persisted in settings, so it
+ // survives a restart.
+ sweeping: boolean;
+ // The gate is shut — the lane consumes nothing. SEPARATE from `sweeping` and
+ // deliberately so: conflating the two is how an operator loses a week of GPU
+ // time, because "stop the sweep" and "hold it at zero throughput" have very
+ // different costs to undo.
+ gateHeld: boolean;
+ // Whether the lane exists at all. False is not "idle": with no backfill
+ // feature registered there is nothing to hold, and an idle-looking lane would
+ // read as "all caught up" when the truth is "switched off".
+ available: boolean;
+ // The sweep's own job, for a link to its log. null when nothing is running.
+ sweepJobId: string | null;
+ // Per-channel jobs this lane has in flight right now, read from the registry
+ // rather than from a runner — there is no runner.
+ inFlight: { id: string; channelSlug: string | null; startedAt: number | null }[];
+ order: AutoQueueOrder;
+ reach: AutoQueueReach;
+};
+
+export type AutoQueueLanesPayload = {
+ digest: SweepLaneStatus;
+ backfill: SweepLaneStatus;
+ // One band per pipeline, in rail order. See lib/operationBands.ts.
+ bands: OperationBand[];
+};
+
+// Running jobs on a lane's queue keys. The digest lane has two (local and
+// metered) and they are genuinely separate concurrency, so both are read;
+// summing them into one "in flight" is correct here because the lane is one
+// lane to an operator even when it is two queues to the scheduler.
+function inFlightOn(queueKeys: ReadonlyArray<string>): SweepLaneStatus["inFlight"] {
+ const keys = new Set(queueKeys);
+ return getRegistry()
+ .list()
+ .filter((j) => j.status === "running" && keys.has(j.queueKey))
+ .map((j) => ({
+ id: j.id,
+ channelSlug: j.channelSlug ?? null,
+ startedAt: j.startedAt ?? null,
+ }));
+}
+
+export async function buildAutoQueueLanes(): Promise<AutoQueueLanesPayload> {
+ const paths = getPaths();
+ const settings = getSettings();
+ const briefs = await getChannelBriefs(paths);
+ // allBackfillKinds, not laneBackfillKinds: the rail is the CATALOG view —
+ // every operation that is switched on, whatever queue it runs on. Filtering
+ // to the shared backfill queue here would drop digest, which is the whole
+ // reason the rail exists.
+ const kinds = allBackfillKinds(settings);
+
+ const bands = buildOperationBands({
+ snapshots: briefs.map((c) => c.snapshot ?? null),
+ operationIds: kinds.map((k) => k.id),
+ });
+
+ return {
+ digest: {
+ id: "digest",
+ label: "Digest",
+ sweeping: settings.digest.sweepEnabled,
+ gateHeld: settings.digest.digestsPaused,
+ available: true,
+ sweepJobId: getDigestSweepJobId(),
+ inFlight: inFlightOn([DIGEST_LOCAL_QUEUE, DIGEST_REMOTE_QUEUE]),
+ order: settings.digest.recencyOrder,
+ reach: settings.digest.recencyReach,
+ },
+ backfill: {
+ id: "backfill",
+ label: "Backfill",
+ sweeping: settings.backfill.sweepEnabled,
+ // INVERTED: backfill's gate is `enabled`, where digest's is `paused`.
+ // Normalised here rather than at every reader, exactly as LaneDeck does.
+ gateHeld: !settings.backfill.enabled,
+ // With no backfill FEATURE switched on there is no lane — see `available`.
+ // Digest is always available because the operation is always registered.
+ available: kinds.some((k) => k.lane.queueKey === BACKFILL_QUEUE),
+ sweepJobId: getBackfillSweepJobId(),
+ inFlight: inFlightOn([BACKFILL_QUEUE]),
+ order: settings.backfill.order,
+ reach: settings.backfill.reach,
+ },
+ bands,
+ };
+}
diff --git a/editor/app/auto-queue/lib/operationBand.ts b/editor/app/auto-queue/lib/operationBand.ts
@@ -0,0 +1,69 @@
+// The band TYPE and the two pure functions over it.
+//
+// SPLIT FROM operationBands.ts DELIBERATELY, and the reason is a build error
+// rather than tidiness: the builder imports channelSnapshot, which imports
+// runYtdlp, which imports execa — so a client component importing the builder
+// drags a server-only process runner into the browser bundle, and `next build`
+// fails with a module-not-found on `node:child_process`. (Nothing catches that
+// in e2e, which runs in dev mode and never prerenders.)
+//
+// So the rail imports THIS, and the builder stays server-side. Nothing here may
+// import anything but types.
+
+// The five populations a pipeline can put a video in, plus the denominator.
+//
+// Rendered as fill PATTERNS first and hue second — solid / hatched / dotted /
+// hollow — because report-to-video's claim rail established here BY MEASUREMENT
+// that no four-colour palette clears all-pairs colour-blindness. Pattern
+// survives CVD, greyscale, and a dim laptop at 2am, which is when this console
+// is actually read.
+export type OperationBand = {
+ id: string;
+ label: string;
+ // How many videos this pipeline has an OPINION about. null = at least one
+ // channel cannot say.
+ eligible: number | null;
+ // Videos it is done with. null for the same reason.
+ present: number | null;
+ // What the lane can do RIGHT NOW — missing + stale + partial. The one
+ // saturated segment on the page: glancing down the rail shows where the
+ // colour is, i.e. where work can happen at all.
+ reachable: number;
+ // Waiting on an operation THIS SYSTEM produces (attribution on diarization,
+ // digest on transcription). Hatched: it will clear itself, upstream.
+ blocked: number;
+ // The media is gone. Hollow: there is nothing to do here, and an opt-in
+ // re-download is the only thing that would change it.
+ missingInput: number;
+ // Held by a gate — stale cues, a duration window. Dotted.
+ deferred: number;
+ // Whether the operation is dispatched by this system at all. False for the
+ // two external pipelines, which are here because the rail's whole point is
+ // that you never have to switch lanes to learn that this one is idle BECAUSE
+ // another one is.
+ dispatched: boolean;
+};
+
+// Sum that returns null if ANY term is null. Same rule digestEligibleKnown uses
+// in the widget payload: a partial sum is a denominator smaller than its own
+// numerator, which is a worse lie than admitting the number is not knowable.
+export function sumOrNull(
+ values: ReadonlyArray<number | null>,
+): number | null {
+ let total = 0;
+ for (const v of values) {
+ if (v == null) return null;
+ total += v;
+ }
+ return total;
+}
+
+// The fraction of a band that is `present`, or null when either half is unknown.
+// Null renders as an unfilled outline — never as 0, which on a fully-digested
+// channel would read as "nothing digested".
+export function bandCoverage(band: OperationBand): number | null {
+ if (band.present == null || band.eligible == null || band.eligible <= 0) {
+ return null;
+ }
+ return Math.min(1, band.present / band.eligible);
+}
diff --git a/editor/app/auto-queue/lib/operationBands.test.ts b/editor/app/auto-queue/lib/operationBands.test.ts
@@ -0,0 +1,200 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+import type { ChannelSnapshot } from "yt-dlp-transcript-common/controller/channelSnapshot";
+import type { BackfillSnapshotEntry } from "yt-dlp-transcript-common/lib/backfillKinds";
+import {
+ bandCoverage,
+ buildOperationBands,
+ sumOrNull,
+ type OperationBand,
+} from "./operationBands";
+
+// Run from this directory:
+// cd editor/app/auto-queue/lib && ../../../../node_modules/.bin/tsx --test operationBands.test.ts
+
+function snapshotOf(patch: Partial<ChannelSnapshot> = {}): ChannelSnapshot {
+ return {
+ generatedAt: "2026-08-21T00:00:00.000Z",
+ totals: { videos: 100, transcribed: 40, downloaded: 60 },
+ buckets: {} as ChannelSnapshot["buckets"],
+ ...patch,
+ } as ChannelSnapshot;
+}
+
+function entryOf(patch: Partial<BackfillSnapshotEntry>): BackfillSnapshotEntry {
+ return {
+ missing: 0,
+ stale: 0,
+ partial: 0,
+ missingInput: 0,
+ deferred: 0,
+ blocked: 0,
+ ids: [],
+ ...patch,
+ } as BackfillSnapshotEntry;
+}
+
+const bandOf = (bands: OperationBand[], id: string): OperationBand => {
+ const found = bands.find((b) => b.id === id);
+ assert.ok(found, `no band for ${id}`);
+ return found;
+};
+
+test("the four work states are kept apart and never summed", () => {
+ // The measured shape of this corpus in miniature: diarization is dominated by
+ // missing media and attribution-diarized by blocked work. Any code that added
+ // them would report both lanes as busy.
+ const bands = buildOperationBands({
+ snapshots: [
+ snapshotOf({
+ backfill: {
+ diarization: entryOf({
+ missing: 647,
+ missingInput: 77_276,
+ eligible: 78_019,
+ }),
+ "attribution-diarized": entryOf({
+ missing: 94,
+ blocked: 77_923,
+ eligible: 78_019,
+ }),
+ },
+ }),
+ ],
+ operationIds: ["diarization", "attribution-diarized"],
+ });
+ const dia = bandOf(bands, "diarization");
+ assert.equal(dia.reachable, 647);
+ assert.equal(dia.missingInput, 77_276);
+ assert.equal(dia.blocked, 0);
+ const attr = bandOf(bands, "attribution-diarized");
+ assert.equal(attr.reachable, 94);
+ assert.equal(attr.blocked, 77_923);
+ assert.equal(attr.missingInput, 0);
+});
+
+test("one channel that cannot report `eligible` voids the whole denominator", () => {
+ // The partial-sum trap. A snapshot predating `eligible` contributes videos to
+ // the corpus but nothing to the denominator, so summing what IS known gives a
+ // denominator smaller than its own numerator.
+ const bands = buildOperationBands({
+ snapshots: [
+ snapshotOf({
+ backfill: { diarization: entryOf({ missing: 1, eligible: 500 }) },
+ }),
+ snapshotOf({
+ // No `eligible` — an older snapshot.
+ backfill: { diarization: entryOf({ missing: 2 }) },
+ }),
+ ],
+ operationIds: ["diarization"],
+ });
+ const dia = bandOf(bands, "diarization");
+ assert.equal(dia.reachable, 3, "work counts still sum");
+ assert.equal(dia.eligible, null, "the denominator does not");
+ assert.equal(dia.present, null);
+ assert.equal(bandCoverage(dia), null, "and coverage renders as unknown");
+});
+
+test("coverage is null, never 0, when the denominator is unknown", () => {
+ // A 0 here would read as "nothing digested" on a fully digested channel.
+ assert.equal(
+ bandCoverage({ present: null, eligible: 10 } as OperationBand),
+ null,
+ );
+ assert.equal(
+ bandCoverage({ present: 5, eligible: null } as OperationBand),
+ null,
+ );
+ assert.equal(bandCoverage({ present: 5, eligible: 0 } as OperationBand), null);
+ assert.equal(bandCoverage({ present: 5, eligible: 10 } as OperationBand), 0.5);
+});
+
+test("digest falls back to the bucket when a snapshot predates its registry entry", () => {
+ // Do NOT stop reading buckets.noDigest until every snapshot has regenerated:
+ // the fallback is all that keeps a stale channel from reading "all digested".
+ const bands = buildOperationBands({
+ snapshots: [
+ snapshotOf({
+ buckets: { noDigest: ["a", "b", "c"] } as ChannelSnapshot["buckets"],
+ }),
+ ],
+ operationIds: ["digest"],
+ });
+ const digest = bandOf(bands, "digest");
+ assert.equal(digest.reachable, 3);
+ // The bucket cannot say how many are done, and it must not pretend to.
+ assert.equal(digest.eligible, null);
+ assert.equal(digest.present, null);
+});
+
+test("the external pipelines get bands from totals and buckets", () => {
+ const bands = buildOperationBands({
+ snapshots: [
+ snapshotOf({
+ totals: { videos: 100, transcribed: 40, downloaded: 60 },
+ undownloadedIds: ["u1", "u2", "u3"],
+ buckets: {
+ downloadedNoTranscript: ["d1", "d2"],
+ noTranscript: Array.from({ length: 60 }, (_, i) => `n${i}`),
+ untranscribable: ["x1", "x2"],
+ partialDownloads: ["p1"],
+ } as unknown as ChannelSnapshot["buckets"],
+ }),
+ ],
+ operationIds: [],
+ });
+ const download = bandOf(bands, "download");
+ // Every video the playlist knows about, not just the dirs that exist.
+ assert.equal(download.eligible, 103);
+ assert.equal(download.present, 60);
+ assert.equal(download.reachable, 4, "3 never fetched + 1 partial");
+ assert.equal(download.dispatched, false);
+
+ const transcription = bandOf(bands, "transcription");
+ assert.equal(transcription.eligible, 98, "100 videos less 2 untranscribable");
+ assert.equal(transcription.present, 40);
+ assert.equal(transcription.reachable, 2, "audio in hand");
+ // The rest of noTranscript is waiting on the DOWNLOAD lane — blocked on an
+ // operation this system produces, not reachable and not missing media.
+ assert.equal(transcription.blocked, 56);
+ assert.equal(transcription.missingInput, 0);
+});
+
+test("ids excluded from download are deferred, not reachable", () => {
+ // A channel deliberately not fetching members-only videos is not a lane with
+ // work to do, and the two must stay separable rather than one being netted
+ // off the other.
+ const bands = buildOperationBands({
+ snapshots: [
+ snapshotOf({
+ totals: { videos: 0, transcribed: 0, downloaded: 0 },
+ undownloadedIds: ["ok", "gone"],
+ excludedFromDownload: { deleted: ["gone"] },
+ } as Partial<ChannelSnapshot>),
+ ],
+ operationIds: [],
+ });
+ const download = bandOf(bands, "download");
+ assert.equal(download.reachable, 1);
+ assert.equal(download.deferred, 1);
+});
+
+test("a switched-off operation gets no band at all", () => {
+ // Absent, not zero: an empty work list because nobody enabled the feature is
+ // not the same as being finished, and a full green bar would claim it was.
+ const bands = buildOperationBands({
+ snapshots: [snapshotOf({ backfill: { diarization: entryOf({ missing: 5 }) } })],
+ operationIds: [],
+ });
+ assert.equal(
+ bands.find((b) => b.id === "diarization"),
+ undefined,
+ );
+});
+
+test("sumOrNull latches null and never returns a partial total", () => {
+ assert.equal(sumOrNull([1, 2, 3]), 6);
+ assert.equal(sumOrNull([1, null, 3]), null);
+ assert.equal(sumOrNull([]), 0);
+});
diff --git a/editor/app/auto-queue/lib/operationBands.ts b/editor/app/auto-queue/lib/operationBands.ts
@@ -0,0 +1,196 @@
+import type { ChannelSnapshot } from "yt-dlp-transcript-common/controller/channelSnapshot";
+import {
+ digestWorkOf,
+ excludedDownloadIdSet,
+} from "yt-dlp-transcript-common/controller/channelSnapshot";
+import {
+ DIGEST_KIND_ID,
+ operationLabel,
+ presentBackfillWork,
+ reachableBackfillWork,
+} from "yt-dlp-transcript-common/lib/backfillKinds";
+import { sumOrNull, type OperationBand } from "./operationBand";
+
+export type { OperationBand } from "./operationBand";
+export { bandCoverage, sumOrNull } from "./operationBand";
+
+// THE COMPARISON RAIL'S MODEL: one band per pipeline, summed across the corpus.
+//
+// This is the corpus-wide twin of channelFlow's transit line, and it holds the
+// same two invariants for the same reasons — they are the two ways every earlier
+// version of this number was wrong:
+//
+// 1. WORK THE LANE CAN DO IS NEVER SUMMED WITH WORK IT CANNOT. `reachable`,
+// `blocked`, `missingInput` and `deferred` are four separate fields on four
+// different axes, and nothing here adds them. On the live corpus that is not
+// pedantry: attribution-diarized is 94 reachable against 77,923 blocked, and
+// diarization is 647 against 77,276 with no media. A single "remaining"
+// figure would say the same thing about a lane that is finished and a lane
+// that cannot start.
+// 2. UNKNOWN IS NOT ZERO. `eligible` and `present` are `number | null`, and one
+// null poisons the whole sum deliberately — a third of the snapshots on disk
+// predate `eligible`, and "three channels are done and the fourth is
+// unknown" is not a number. The band renders a null denominator as an
+// unfilled outline, never as 0% progress.
+//
+// WHY THE RATIO IS THE STORY, AND WHY EACH BAND KEEPS ITS OWN DENOMINATOR.
+// Three of the four pipelines are dominated by a non-actionable state, so a
+// count renders them as "94" and "647" and tells you nothing. And digest's
+// eligible population is genuinely a different set from diarization's — sharing
+// one denominator across the rail to make the bars comparable would be a lie
+// about what is being compared. Each band states its own, in its own header.
+//
+// Pure and snapshot-only: common/controller/noCorpusWalkInRenderPaths.test.ts
+// bans a corpus walk from a render path, and this feeds a 3-second poll.
+
+function emptyBand(id: string, dispatched: boolean): OperationBand {
+ return {
+ id,
+ label: operationLabel(id),
+ eligible: 0,
+ present: 0,
+ reachable: 0,
+ blocked: 0,
+ missingInput: 0,
+ deferred: 0,
+ dispatched,
+ };
+}
+
+// Fold one snapshot's entry for a registry operation into a band. `null` for
+// either coverage half latches for the whole corpus.
+function addRegistryEntry(band: OperationBand, snapshot: ChannelSnapshot): void {
+ const entry = snapshot.backfill?.[band.id];
+ if (!entry) return;
+ band.reachable += reachableBackfillWork(entry);
+ band.missingInput += entry.missingInput;
+ // `?? 0` at every read: snapshots written before these fields existed lack
+ // them, and undefined poisons the sum to NaN.
+ band.blocked += entry.blocked ?? 0;
+ band.deferred += entry.deferred ?? 0;
+ band.eligible = sumOrNull([band.eligible, entry.eligible ?? null]);
+ band.present = sumOrNull([band.present, presentBackfillWork(entry)]);
+}
+
+export type BuildOperationBandsInput = {
+ snapshots: ReadonlyArray<ChannelSnapshot | null>;
+ // Registry operations to build a band for, in rail order. Comes from
+ // allBackfillKinds(), so a switched-off feature is simply absent — which is
+ // the honest rendering: an empty work list because nobody enabled it is not
+ // the same as being finished.
+ operationIds: ReadonlyArray<string>;
+};
+
+// The two pipelines this system counts but does not dispatch through the
+// operation registry. They are on the rail anyway, and deliberately:
+//
+// The rail exists so you never have to switch lanes to learn that THIS lane is
+// idle because ANOTHER one is — and on this corpus that is the normal case, not
+// the exception (diarization is 99.2% media-gone; attribution-diarized is 99.9%
+// blocked behind diarization). Leaving transcription and download off it would
+// remove exactly the two lanes whose state explains the other four.
+//
+// Their numbers do NOT come from BackfillKind.state() — they have no entry,
+// because EXTERNAL_OPERATIONS registers them for the dependency graph and not
+// for dispatch. They come from `totals` and the buckets, using the SAME
+// definitions the channel transit line already uses for its Download and
+// Transcribe stations, so a corpus figure and a channel figure cannot disagree
+// about what "downloaded" means.
+function addExternalBands(
+ bands: Map<string, OperationBand>,
+ snapshot: ChannelSnapshot,
+): void {
+ const totals = snapshot.totals ?? { videos: 0, transcribed: 0, downloaded: 0 };
+ const buckets = snapshot.buckets;
+ const undownloaded = snapshot.undownloadedIds ?? [];
+ const excluded = excludedDownloadIdSet(snapshot);
+
+ const download = bands.get("download");
+ if (download) {
+ // Eligible is every video the playlist knows about: the dirs that exist
+ // plus the ids that have never been fetched. `totals.videos` alone would be
+ // a denominator that grows only as work completes.
+ download.eligible = sumOrNull([
+ download.eligible,
+ totals.videos + undownloaded.length,
+ ]);
+ download.present = sumOrNull([download.present, totals.downloaded]);
+ // Partial downloads are reachable work like any other — the same rule
+ // backfillBatch applies to `partial`.
+ download.reachable +=
+ undownloaded.filter((id) => !excluded.has(id)).length +
+ (buckets?.partialDownloads?.length ?? 0);
+ // Excluded ids have LEFT the line: a channel deliberately not fetching
+ // them is not a lane with work to do. They are deferred, not reachable —
+ // and never subtracted from anything, so the two stay separable.
+ download.deferred += undownloaded.filter((id) => excluded.has(id)).length;
+ }
+
+ const transcription = bands.get("transcription");
+ if (transcription) {
+ // A video marked untranscribable is not eligible — it is not work anyone is
+ // waiting on, and counting it would put a permanent ceiling under 100%.
+ const untranscribable = buckets?.untranscribable?.length ?? 0;
+ transcription.eligible = sumOrNull([
+ transcription.eligible,
+ Math.max(0, totals.videos - untranscribable),
+ ]);
+ transcription.present = sumOrNull([
+ transcription.present,
+ totals.transcribed,
+ ]);
+ // Reachable = the audio is in hand. Everything else without a transcript is
+ // waiting on the DOWNLOAD lane, which is exactly what `blocked` means here
+ // — an operation this system produces, one station upstream.
+ const downloadedNoTranscript = (
+ buckets?.downloadedNoTranscript ?? []
+ ).filter((id) => !excluded.has(id)).length;
+ const noTranscript = buckets?.noTranscript?.length ?? 0;
+ transcription.reachable += downloadedNoTranscript;
+ transcription.blocked += Math.max(
+ 0,
+ noTranscript - untranscribable - downloadedNoTranscript,
+ );
+ }
+}
+
+// The rail, left to right. Download and transcription lead because everything
+// else depends on them; digest and the backfill kinds follow in registry order.
+export const EXTERNAL_BAND_IDS = ["download", "transcription"] as const;
+
+export function buildOperationBands({
+ snapshots,
+ operationIds,
+}: BuildOperationBandsInput): OperationBand[] {
+ const bands = new Map<string, OperationBand>();
+ for (const id of EXTERNAL_BAND_IDS) bands.set(id, emptyBand(id, false));
+ for (const id of operationIds) {
+ if (!bands.has(id)) bands.set(id, emptyBand(id, true));
+ }
+
+ for (const snapshot of snapshots) {
+ if (!snapshot) continue;
+ addExternalBands(bands, snapshot);
+ for (const id of operationIds) {
+ const band = bands.get(id);
+ if (!band) continue;
+ if (id === DIGEST_KIND_ID && !snapshot.backfill?.[id]) {
+ // A snapshot written before digest joined the registry has no entry, and
+ // digestWorkOf is the fallback that reads the same population off the
+ // buckets with the transcript and cues-staleness gates applied. Without
+ // it a stale channel reads "all digested", which is the one failure mode
+ // the source:"bucket" fallback exists to prevent.
+ const work = digestWorkOf(snapshot);
+ band.reachable += work.reachable;
+ band.blocked += work.blocked;
+ band.deferred += work.deferred;
+ band.eligible = sumOrNull([band.eligible, work.eligible]);
+ band.present = sumOrNull([band.present, work.present]);
+ continue;
+ }
+ addRegistryEntry(band, snapshot);
+ }
+ }
+
+ return [...bands.values()];
+}
diff --git a/editor/app/auto-queue/status.ts b/editor/app/auto-queue/status.ts
@@ -13,6 +13,7 @@ import {
readAutoQueueState,
} from "yt-dlp-transcript-common/jobs/autoQueueState";
import type { AutoQueuePolicy } from "yt-dlp-transcript-common/jobs/autoQueuePolicy";
+import { buildAutoQueueLanes, type AutoQueueLanesPayload } from "./lanes";
// Read-only payload for the Auto-Queue panel: per-kind runner status (running?,
// what's in flight, in-flight counts per tree node), the effective policy, the
@@ -54,6 +55,11 @@ export type AutoQueueKindStatus = {
export type AutoQueueStatusPayload = {
transcription: AutoQueueKindStatus;
download: AutoQueueKindStatus;
+ // The other two pipelines, plus the comparison rail's bands. On the SAME
+ // payload as the runners rather than a second endpoint, because the rail's
+ // whole purpose is that four lanes are read together — two polls would let
+ // the rail and the focused lane disagree about the same moment.
+ lanes: AutoQueueLanesPayload;
};
async function buildKind(kind: AutoQueueKind): Promise<AutoQueueKindStatus> {
@@ -87,9 +93,10 @@ async function buildKind(kind: AutoQueueKind): Promise<AutoQueueKindStatus> {
}
export async function buildAutoQueueStatusPayload(): Promise<AutoQueueStatusPayload> {
- const [transcription, download] = await Promise.all([
+ const [transcription, download, lanes] = await Promise.all([
buildKind("transcription"),
buildKind("download"),
+ buildAutoQueueLanes(),
]);
- return { transcription, download };
+ return { transcription, download, lanes };
}
diff --git a/editor/app/jobs/active/buildActiveJobs.ts b/editor/app/jobs/active/buildActiveJobs.ts
@@ -15,6 +15,19 @@ import {
type DiskGateReason,
} from "yt-dlp-transcript-common/lib/diskSpace";
import type { RunningJobsListItem } from "../components/RunningJobsList";
+import {
+ AUTO_DOWNLOAD_KIND,
+ AUTO_TRANSCRIBE_KIND,
+ getAutoRunnerStatus,
+} from "yt-dlp-transcript-common/controller/autoRunner";
+import { getDigestSweepJobId } from "yt-dlp-transcript-common/controller/digestSweep";
+import { getBackfillSweepJobId } from "yt-dlp-transcript-common/controller/backfillSweep";
+import { laneBackfillKinds } from "yt-dlp-transcript-common/lib/backfillKinds";
+import {
+ BACKFILL_QUEUE,
+ DIGEST_LOCAL_QUEUE,
+ DIGEST_REMOTE_QUEUE,
+} from "yt-dlp-transcript-common/lib/queueKeys";
export type DiskStatusView = {
// Whether the low-disk gate is configured (minFreeDiskGB > 0). When false the
@@ -35,10 +48,37 @@ export type DiskStatusView = {
message: string;
};
+// A RUNNER IS A LANE, NOT A JOB.
+//
+// The auto-queue runners are channel-less jobs, so this screen grouped them by
+// kind and gave each group the same bordered card a real channel gets — a whole
+// card, per always-on daemon, to say "running". With four pipelines converging
+// on this model that is four cards stacked above the work that actually has
+// progress bars.
+//
+// So a runner ships as one of these instead: a line on a strip, in the lane
+// vocabulary the dashboard already uses (Running / Holding / Idle / Off), with
+// its reason in words. The JOB is still in `jobs` and still carries its Drain
+// and Cancel controls — this is the same fact, said in one line instead of a
+// card.
+export type ActiveLaneView = {
+ // The job kind, so the client can find the runner's own job row.
+ kind: string;
+ label: string;
+ state: "running" | "holding" | "idle" | "unavailable";
+ // Why it is not working, in words. Null when it IS working.
+ note: string | null;
+ inFlight: number;
+};
+
export type ActiveJobsPayload = {
jobs: RunningJobsListItem[];
channels: { slug: string; displayName: string }[];
disk: DiskStatusView;
+ // Every lane, whether or not it has a job right now. A stopped runner still
+ // gets a line: "not running" is exactly the state a screen full of channel
+ // cards used to hide.
+ lanes: ActiveLaneView[];
};
// Estimate seconds remaining as: remaining tasks × average measured task
@@ -231,5 +271,138 @@ export async function buildActiveJobsPayload(): Promise<ActiveJobsPayload> {
message: diskStatus.message,
};
- return { jobs, channels, disk };
+ return { jobs, channels, disk, lanes: buildLanes(jobs) };
+}
+
+// The four lanes, from state this process already holds — getAutoRunnerStatus
+// is an in-memory read and the sweep job ids are registry lookups, so the strip
+// costs nothing on a 1-second poll.
+//
+// The wording comes from the SAME two helpers the console uses. A runner's
+// reason is its own idleReason, mapped to words here rather than on the client
+// so /jobs/active, the widget and the dashboard cannot describe one state three
+// ways.
+function buildLanes(jobs: RunningJobsListItem[]): ActiveLaneView[] {
+ const settings = getSettings();
+ const runningOn = (keys: ReadonlyArray<string>): number => {
+ const set = new Set(keys);
+ return jobs.filter((j) => j.status === "running" && set.has(j.queueKey))
+ .length;
+ };
+
+ const lanes: ActiveLaneView[] = [];
+
+ for (const [kind, label, jobKind] of [
+ ["transcription", "Auto-transcribe", AUTO_TRANSCRIBE_KIND],
+ ["download", "Auto-download", AUTO_DOWNLOAD_KIND],
+ ] as const) {
+ const status = getAutoRunnerStatus(kind);
+ const inFlight = status.inFlight.length;
+ lanes.push({
+ kind: jobKind,
+ label,
+ // A STOPPED runner is "unavailable", not "idle". It will never pick
+ // anything up, and an idle-looking lane reads as "all caught up".
+ state: !status.running
+ ? "unavailable"
+ : inFlight > 0
+ ? "running"
+ : "idle",
+ note: status.running ? autoIdleNote(status.idleReason, kind) : "not running",
+ inFlight,
+ });
+ }
+
+ const digestInFlight = runningOn([DIGEST_LOCAL_QUEUE, DIGEST_REMOTE_QUEUE]);
+ lanes.push(
+ sweepLane({
+ kind: "digest-sweep",
+ label: "Digest",
+ sweeping: settings.digest.sweepEnabled || getDigestSweepJobId() !== null,
+ gateHeld: settings.digest.digestsPaused,
+ available: true,
+ inFlight: digestInFlight,
+ }),
+ );
+
+ const backfillInFlight = runningOn([BACKFILL_QUEUE]);
+ lanes.push(
+ sweepLane({
+ kind: "backfill-sweep",
+ label: "Backfill",
+ sweeping:
+ settings.backfill.sweepEnabled || getBackfillSweepJobId() !== null,
+ // INVERTED on the wire: backfill's gate is `enabled` where digest's is
+ // `paused`. Normalised here so both lanes mean the same thing downstream.
+ gateHeld: !settings.backfill.enabled,
+ available: laneBackfillKinds(settings).length > 0,
+ inFlight: backfillInFlight,
+ }),
+ );
+
+ return lanes;
+}
+
+function sweepLane(l: {
+ kind: string;
+ label: string;
+ sweeping: boolean;
+ gateHeld: boolean;
+ available: boolean;
+ inFlight: number;
+}): ActiveLaneView {
+ // Same precedence as deriveLaneState: unavailable, then the gate, then work.
+ // HOLDING is the state that has no other name — a sweep armed behind a shut
+ // gate is not stopped and is not working, and it looked identical to wedged.
+ const state: ActiveLaneView["state"] = !l.available
+ ? "unavailable"
+ : l.gateHeld
+ ? "holding"
+ : l.sweeping || l.inFlight > 0
+ ? "running"
+ : "idle";
+ return {
+ kind: l.kind,
+ label: l.label,
+ state,
+ note: !l.available
+ ? "no operation switched on"
+ : l.gateHeld
+ ? l.sweeping
+ ? "sweep armed, lane paused"
+ : "lane paused"
+ : state === "running"
+ ? null
+ : "no sweep armed",
+ inFlight: l.inFlight,
+ };
+}
+
+// The runner idle reasons, in words. A copy of the console's idleReasonText,
+// kept server-side because this payload is consumed by three clients and the
+// sentence must be the same in all three.
+function autoIdleNote(
+ reason: string | null,
+ kind: "transcription" | "download",
+): string | null {
+ switch (reason) {
+ case "no-pending":
+ return "nothing pending";
+ case "capped":
+ return "every route to the work is at a worker cap";
+ case "cooldown":
+ return "every pending platform is in a rate-limit cooldown";
+ case "no-workers":
+ return "no enabled worker";
+ case "disk-gate":
+ return "disk gate closed";
+ case "downloads-paused":
+ return "downloads paused globally";
+ case "snoozed":
+ return "snoozed";
+ case "disabled":
+ return `auto-${kind === "transcription" ? "transcribe" : "download"} is switched off`;
+ default:
+ return null;
+ }
}
diff --git a/editor/app/jobs/components/ActiveJobsLive.tsx b/editor/app/jobs/components/ActiveJobsLive.tsx
@@ -2,9 +2,15 @@
import Link from "next/link";
import { useEffect, useState } from "react";
-import type { ActiveJobsPayload } from "../active/buildActiveJobs";
+import type {
+ ActiveJobsPayload,
+ ActiveLaneView,
+} from "../active/buildActiveJobs";
import { jobKindLabel } from "../jobKindLabels";
import { RunningJobsList, type RunningJobsListItem } from "./RunningJobsList";
+import { LANE_DOT, LANE_TEXT, LANE_WORD } from "../../components/lanes/laneState";
+import { DrainJobButton } from "./DrainJobButton";
+import { CancelJobButton } from "./CancelJobButton";
// Polls /api/jobs/active so per-task progress bars advance live (the server
// component only provides the initial paint). Replaces the coarser 2.5s
@@ -36,7 +42,7 @@ export function ActiveJobsLive({ initial }: { initial: ActiveJobsPayload }) {
};
}, []);
- const { jobs, channels } = payload;
+ const { jobs, channels, lanes } = payload;
const jobsBySlug = new Map<string, RunningJobsListItem[]>();
// Channel-less jobs (e.g. the cross-channel auto-queue runners) are grouped by
@@ -54,16 +60,17 @@ export function ActiveJobsLive({ initial }: { initial: ActiveJobsPayload }) {
}
}
- if (jobs.length === 0) {
- return (
- <p className="text-sm text-muted-foreground border border-dashed border-border rounded p-4">
- No active jobs.
- </p>
- );
- }
-
+ // A LANE IS NOT A JOB, so an empty work list is not an empty page. The strip
+ // still renders: "every lane is idle and here is why" is the answer this
+ // screen was previously unable to give at all.
return (
<div className="flex flex-col gap-4">
+ <LaneStrip lanes={lanes} jobs={jobsByKind} />
+ {jobs.length === 0 && (
+ <p className="text-sm text-muted-foreground border border-dashed border-border rounded p-4">
+ No active jobs.
+ </p>
+ )}
{channels.map(({ slug, displayName }) => {
const channelJobs = jobsBySlug.get(slug) ?? [];
if (channelJobs.length === 0) return null;
@@ -86,19 +93,106 @@ export function ActiveJobsLive({ initial }: { initial: ActiveJobsPayload }) {
</section>
);
})}
- {[...jobsByKind.entries()].map(([kind, kindJobs]) => {
- const label = jobKindLabel(kind);
- return (
- <section
- key={kind}
- aria-label={`System jobs: ${label}`}
- className="flex flex-col gap-3 border border-border rounded-md p-3 bg-card"
- >
- <h2 className="text-base font-medium">{label}</h2>
- <RunningJobsList jobs={kindJobs} />
- </section>
- );
- })}
+ {/* Channel-less jobs that are NOT lanes keep their own labelled card —
+ they are real units of work, not always-on daemons, and the "Other"
+ bucket they used to fall into is what this section exists to prevent.
+ Lane kinds are filtered out because the strip above already carries
+ them, one line each. */}
+ {[...jobsByKind.entries()]
+ .filter(([kind]) => !lanes.some((l) => l.kind === kind))
+ .map(([kind, kindJobs]) => {
+ const label = jobKindLabel(kind);
+ return (
+ <section
+ key={kind}
+ aria-label={`System jobs: ${label}`}
+ className="flex flex-col gap-3 border border-border rounded-md p-3 bg-card"
+ >
+ <h2 className="text-base font-medium">{label}</h2>
+ <RunningJobsList jobs={kindJobs} />
+ </section>
+ );
+ })}
</div>
);
}
+
+// THE LANE STRIP. One line per pipeline, in place of one card per runner.
+//
+// Each lane keeps its <section aria-label="System jobs: …"> and its <h2>, so a
+// runner is still addressable exactly as it was — what changed is that it costs
+// a line instead of a card, and that it now says WHY it is not working. Four
+// bordered cards saying "running" above the work with the progress bars is the
+// layout this replaces.
+function LaneStrip({
+ lanes,
+ jobs,
+}: {
+ lanes: ActiveLaneView[];
+ // kind -> the channel-less jobs of that kind. A lane has at most one (its own
+ // long-lived runner or sweep job); its controls ride on the lane's line.
+ jobs: Map<string, RunningJobsListItem[]>;
+}) {
+ if (lanes.length === 0) return null;
+ return (
+ <div className="rounded-md border border-border bg-card">
+ <p className="border-b border-border px-3 py-1.5 font-mono text-xs uppercase tracking-[0.14em] text-muted-foreground">
+ Lanes
+ </p>
+ <div className="divide-y divide-border">
+ {lanes.map((lane) => (
+ <LaneRow key={lane.kind} lane={lane} job={jobs.get(lane.kind)?.[0]} />
+ ))}
+ </div>
+ </div>
+ );
+}
+
+function LaneRow({
+ lane,
+ job,
+}: {
+ lane: ActiveLaneView;
+ job: RunningJobsListItem | undefined;
+}) {
+ return (
+ <section
+ aria-label={`System jobs: ${lane.label}`}
+ className="flex flex-wrap items-center gap-x-3 gap-y-1 px-3 py-2 text-sm"
+ >
+ <span
+ aria-hidden="true"
+ className={`size-2 shrink-0 rounded-full ${LANE_DOT[lane.state]}`}
+ />
+ {/* Kept an <h2>: the suite finds this runner by heading, and a lane is
+ still the thing a screen reader should be able to jump to. */}
+ <h2 className="min-w-40 text-sm font-medium">{lane.label}</h2>
+ <span className={`text-xs ${LANE_TEXT[lane.state]}`}>
+ {LANE_WORD[lane.state]}
+ {lane.note ? (
+ <span className="text-muted-foreground">: {lane.note}</span>
+ ) : null}
+ </span>
+ {lane.inFlight > 0 && (
+ <span className="text-xs text-muted-foreground">
+ <span className="tabular-nums text-foreground">{lane.inFlight}</span>{" "}
+ in flight
+ </span>
+ )}
+ {job && (
+ <span className="ml-auto flex items-center gap-2">
+ <Link
+ href={`/jobs/${job.id}`}
+ className="text-xs underline underline-offset-2 text-muted-foreground hover:text-foreground"
+ >
+ log
+ </Link>
+ {job.drainable && (
+ <DrainJobButton jobId={job.id} draining={job.draining} />
+ )}
+ <CancelJobButton jobId={job.id} />
+ </span>
+ )}
+ </section>
+ );
+}
diff --git a/editor/e2e/auto-queue.spec.ts b/editor/e2e/auto-queue.spec.ts
@@ -247,10 +247,24 @@ test("strict priority: drains the high-priority channel first, then falls back",
})
.toBe(4);
- // Every alpha video is picked before any beta video, and in-channel order holds.
+ // Every alpha video is picked before any beta video, and in-channel order
+ // holds. THIS is what the test is about, and it stays an exact assertion.
expect(pickOrder(await getStatus(request))).toEqual(["a1", "a2", "b1", "b2"]);
- expect(await allTranscribed("alpha", ["a1", "a2"])).toBe(true);
- expect(await allTranscribed("beta", ["b1", "b2"])).toBe(true);
+
+ // The transcripts are POLLED, not asserted outright. A pick is recorded when
+ // the runner hands work out; a transcript appears when that work finishes, so
+ // the fourth pick is observable before the fourth transcript is written — and
+ // asserting instantly is a ~1-in-3 race under load, which is how this landed
+ // red. Polling does not weaken the claim: all four transcripts must still
+ // exist, and the ordering assertion above is untouched.
+ await expect
+ .poll(
+ async () =>
+ (await allTranscribed("alpha", ["a1", "a2"])) &&
+ (await allTranscribed("beta", ["b1", "b2"])),
+ { timeout: 30_000 },
+ )
+ .toBe(true);
});
test("respects the saved worker default: a disabled worker is never used", async ({
@@ -1027,3 +1041,151 @@ test("snooze idles the runner without stopping it, and Wake now resumes", async
.toBe(true);
expect((await getStatus(request)).transcription.runner.running).toBe(true);
});
+
+// --- The console: the rail, the switcher, and the two sweep lanes -----------
+
+test("UI: the rail carries every pipeline, and the switcher focuses one", async ({
+ page,
+}) => {
+ await resetData(null);
+ await makeChannel("alpha", ["a1", "a2"]);
+ await writeSettings({
+ adminTitle: "Test Admin",
+ maxTranscriptPageBytes: 8388608,
+ sleepBetweenDownloadsSeconds: 0,
+ minFreeDiskGB: 0,
+ workers: ONE_WORKER,
+ autoQueue: transcriptionAutoQueue(ALPHA_ROOT),
+ });
+
+ await page.goto("/auto-queue");
+ const transcribe = page.locator("section", {
+ has: page.getByRole("heading", { name: "Auto-transcribe" }),
+ });
+ await awaitHydration(transcribe);
+
+ // The rail is always present and carries a row per pipeline, whichever lane
+ // is focused — that is the whole reason it exists.
+ const rail = page.locator("li[data-operation]");
+ await expect(rail.filter({ hasText: "Download" }).first()).toBeVisible();
+ await expect(rail.filter({ hasText: "Transcription" }).first()).toBeVisible();
+ await expect(rail.filter({ hasText: "Digest" }).first()).toBeVisible();
+
+ // Auto-transcribe is the default focus. Auto-download stays MOUNTED — which
+ // is what preserves its unsaved policy edits across a switch — but a hidden
+ // subtree is out of the accessibility tree, so it is addressed by data-lane
+ // rather than by its heading. A heading-scoped locator resolves to ZERO here,
+ // which is correct: a screen reader must not see the hidden pane either.
+ const download = page.locator('section[data-lane="download"]');
+ await expect(transcribe).toBeVisible();
+ await expect(download).toHaveCount(1);
+ await expect(download).toBeHidden();
+ await expect(
+ page.locator("section", {
+ has: page.getByRole("heading", { name: "Auto-download" }),
+ }),
+ ).toHaveCount(0);
+
+ await page.getByLabel("Lane", { exact: true }).selectOption("download");
+ await expect(download).toBeVisible();
+ await expect(transcribe).toBeHidden();
+
+ // The digest lane is a real lane on this page now, with both of its controls.
+ await page.getByLabel("Lane", { exact: true }).selectOption("digest");
+ const digest = page.locator('section[data-lane="digest"]');
+ await expect(digest.getByRole("heading", { name: "Digest" })).toBeVisible();
+ await expect(
+ digest.getByRole("button", { name: "Start Digest sweep" }),
+ ).toBeVisible();
+ // SWEEP AND PAUSE ARE TWO CONTROLS, and both must be here — conflating them
+ // is how an operator loses a week of GPU time.
+ await expect(
+ digest.getByRole("button", { name: "Pause Digest" }),
+ ).toBeVisible();
+});
+
+test("UI: Reach is disabled until an order is chosen, and it persists", async ({
+ page,
+}) => {
+ await resetData(null);
+ await makeChannel("alpha", ["a1"]);
+ await writeSettings({
+ adminTitle: "Test Admin",
+ maxTranscriptPageBytes: 8388608,
+ sleepBetweenDownloadsSeconds: 0,
+ minFreeDiskGB: 0,
+ workers: ONE_WORKER,
+ autoQueue: transcriptionAutoQueue(ALPHA_ROOT),
+ });
+
+ await page.goto("/auto-queue");
+ await awaitHydration(
+ page.locator("section", {
+ has: page.getByRole("heading", { name: "Auto-transcribe" }),
+ }),
+ );
+ await page.getByLabel("Lane", { exact: true }).selectOption("digest");
+
+ // Scoped to the lane: every lane on the page carries its own Order/Reach
+ // pair, and getByLabel does not filter by visibility.
+ const digest = page.locator('section[data-lane="digest"]');
+ const order = digest.getByLabel("Order", { exact: true });
+ const reach = digest.getByLabel("Reach", { exact: true });
+ // Reach is meaningless without an order, so it is disabled rather than
+ // silently ignored.
+ await expect(reach).toBeDisabled();
+
+ await order.selectOption("newest");
+ await expect(reach).toBeEnabled();
+ await reach.selectOption("corpus");
+
+ // Both land in settings.digest — NOT in autoQueue, which has no digest key.
+ await expect
+ .poll(
+ async () => {
+ const s = await readJson<{
+ digest?: { recencyOrder?: string; recencyReach?: string };
+ }>("test-settings.json").catch(() => null);
+ return `${s?.digest?.recencyOrder}/${s?.digest?.recencyReach}`;
+ },
+ { timeout: 15_000 },
+ )
+ .toBe("newest/corpus");
+
+ // The armed sweep and its scope are UNTOUCHED by an order save — a rebuilt
+ // settings block here would disarm a multi-week run.
+ const after = await readJson<{ digest?: { sweepEnabled?: boolean } }>(
+ "test-settings.json",
+ );
+ expect(after.digest?.sweepEnabled ?? false).toBe(false);
+});
+
+test("Active jobs: a runner is a lane on a strip, not a card", async ({
+ page,
+ request,
+}) => {
+ await resetData(null);
+ await makeChannel("alpha", []); // enabled, nothing pending -> running but idle
+ await writeSettings(IDLE_SETTINGS(ALPHA_ROOT));
+ await startRunner(request);
+
+ await page.goto("/jobs/active");
+ const lane = page.locator(
+ "section[aria-label='System jobs: Auto-transcribe']",
+ );
+ // Still addressable exactly as before — heading, aria-label, controls.
+ await expect(lane.getByRole("heading", { name: "Auto-transcribe" })).toBeVisible();
+ await expect(lane.getByRole("button", { name: /^Cancel$/ })).toBeVisible();
+ // What is NEW: it says why it is not working, instead of a card that said
+ // nothing at all.
+ await expect(lane.getByText(/nothing pending/)).toBeVisible({
+ timeout: 15_000,
+ });
+
+ // The sweep-fed lanes are on the strip too, with no job of their own.
+ await expect(
+ page.locator("section[aria-label='System jobs: Digest']"),
+ ).toBeVisible();
+ // And the generic bucket stays gone.
+ await expect(page.getByRole("heading", { name: "Other" })).toHaveCount(0);
+});
