commit 6bfa76f2f05ca01440363deda0edc12bb72d14dc
parent b0d77057b5c08feea65e2ec1e1e00975d5e85051
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Mon, 7 Sep 2026 23:25:19 -0400
plans: slice 1.3 is shipped, and the record says what it cost
The `1.3, as shipped` section: the seven commits, 2,974 lines of deleted
dispatcher in a table, the migration's ten-field table with a reason beside each
of the four fields that are RETIRED rather than carried, the numbers diff quoted
in full (0 lines removed, 18 added), what the rewritten specs pin where they
pinned a sweep, and every divergence from the plan's bullets.
Two things review found before the e2e run are in it, because both are the shape
of mistake this slice invites: a block written for two lanes drawn on all four
(the transcription page claiming auto-transcribe was switched off in settings
that do not exist), and `armLaneAction(lane, {})` writing a catch-all root over
an authored tree from a dashboard click.
e2e: 511 passed, 1 failed. The failure is `video-page.spec.ts` asserting a video
DIRECTORY still exists after a refused delete — a file this slice does not touch,
doing something nothing in this slice can do, and it passes 20/20 alone on the
same commit. Reported, not fixed.
`plans/FACTS.md` also gains the two ways the `editor/content` symlink follows you
into a worktree, both hit today: `git add -A editor` STAGES it (untracked is not
ignored), and a worktree three directories deeper makes the panic read
`join("../../../../../recipe-content")`, which a grep for the recorded string
does not find.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Diffstat:
2 files changed, 343 insertions(+), 2 deletions(-)
diff --git a/plans/FACTS.md b/plans/FACTS.md
@@ -3287,8 +3287,16 @@ regression. `next build` is unaffected. A `git worktree` of the same commit has
symlink and runs the suite normally — **511 passed / 0 failed in 23.8 min** on `1691c4f`,
one worker. A worktree needs one extra step first: its `export/public` holds only the tracked
static assets, so the export webServer 500s and Playwright dies with "Timed out waiting
-120000ms from config.webServer". Copy a composed fixture site into it (16 files) and it
-serves 200.
+120000ms from config.webServer". Copy a composed fixture site into it (20 files, ~4.8 MB —
+`.claude/worktrees/duplicates-page/export/public` is one) and it serves 200.
+
+**Two ways it follows you into the worktree, both hit on 2026-09-07 (slice 1.3).**
+`git add -A editor` from the primary checkout STAGES THE SYMLINK — it is untracked, not
+ignored — so a commit made that way carries `editor/content` into every worktree of it and
+the panic is back. Add by path, or `git rm --cached editor/content` and amend. And a
+worktree under `.claude/worktrees/<name>/` is three levels deeper, so the panic reads
+`join("../../../../../recipe-content")` rather than `join("../../recipe-content")` — same
+fault, different arithmetic, and a grep for the old string will not find it.
### The bake-off harnesses do not take `--help`
diff --git a/plans/one-core-phase-1.md b/plans/one-core-phase-1.md
@@ -687,3 +687,336 @@ fails, 1.3 does not land.
- **`countOperationWork` opens a run with `useClusters: false`.** Counting must
not read the corpus-wide duplicates report, and a mirror is `present` or
`missing` on its own merits either way — which is what the snapshot counts.
+
+## 1.3, as shipped
+
+`253826b` (the migration, before anything is deleted) → `b172d5a` (the backend
+retirement) → `ca83051` (the editor half) → `65c0a49` (the e2e rewrites) →
+`ca28fb5` (the numbers script and the prose) → `43b2f9f` and `cdaccc4` (two
+fixes, below) → the commit after them, which carries this note and so cannot name
+its own sha. On `one-core/phase-1` off `0438a72`.
+
+**Gate A is run from `one-core/phase-1-gate-a` (which points at `0438a72`) and it
+gates the MERGE of this slice, not its authorship.** Everything below stands on
+"the digest lane's runner has driven a production pass and resumed after a
+restart"; the branch exists so the operator can do that against the code as it
+was when 1.2 finished, while 1.3 was written.
+
+Verification: `tsc --noEmit` clean in common, editor, export, mcp, homepage and
+umtool; `yt-dlp-transcript-common` **888** tests (897 before: **+8**
+`laneMigration`, **-17** — seven `sweepPreview`, seven `arbiter`, three
+`backfillLimit`/`sanitizeBackfill` weight cases), `yt-dlp-transcript-mcp` **205**;
+`next build` clean; full editor e2e from a `p13-e2e` worktree of the final commit
+— **511 passed, 1 failed, 31.2 min**, one worker behind the queue lock. 515 at
+1.2, minus the three arbiter tests, is **512**, which is the total run: the three
+rewritten `backfill.spec.ts` tests and the rewritten Reach test are one-for-one
+replacements, so no other count moved.
+
+**The one failure is not this slice's**, and it is worth saying how that was
+established rather than asserted: `video-page.spec.ts:216 "Delete directory
+wrong-id confirmation shows an error"` asserts a video DIRECTORY still exists
+after a refused delete. Nothing in 1.3 deletes a video directory, the file is
+untouched by the slice, and re-running that spec alone on the same commit passes
+**20/20**. It is an ordering or reset flake, reported and not fixed.
+
+### Deleted
+
+| File | Lines |
+|---|---|
+| `common/controller/backfillSweep.ts` | 519 |
+| `common/controller/arbiter.ts` | 418 |
+| `common/controller/digestSweep.ts` | 368 |
+| `common/controller/sweepPreview.test.ts` | 321 |
+| `common/controller/sweepPreview.ts` | 167 |
+| `common/lib/sweepPlan.ts` | 163 |
+| `common/controller/arbiter.test.ts` | 177 |
+| `common/controller/sweepRecency.ts` | 141 (git says `Bin` — the NUL byte) |
+| `common/controller/arbiterWork.ts` | 67 |
+| `editor/app/operations/components/SweepLane.tsx` | 422 |
+| `editor/app/operations/components/SweepPlan.tsx` | 264 |
+| `editor/app/operations/components/SweepScope.tsx` | 154 |
+| `editor/app/operations/components/ArbiterBar.tsx` | 113 |
+| `editor/app/api/test/resume-backfill-sweep/route.ts` | 35 |
+
+Plus, in place: the `backfill-sweep` and `operations-arbiter` job kinds, both
+`instrumentation.ts` resume hooks, `laneBlockedReason` (the last reader of a
+sweep flag) with the `lane-blocked` idle reason and its two copies of the
+wording, `saveLaneOrderAction`, `startArbiterAction` / `stopArbiterAction`, the
+four sweep server actions, `sweepLaneIdFor`, `SweepLaneStatus`, `ArbiterStatus`,
+`laneState.ts`'s `feedRunning` axis and `sweepLaneNote`, the `backfill-sweep`
+entry in `stageStatus`, and the `LaneSettingsForm` "Resource share" field.
+`operations/lanes.ts` goes 370 → 131. **Net across the slice: 1,448 insertions,
+4,939 deletions over 71 files.**
+
+`controller/digestPlan.buildDigestSweepPlan` STAYS — `bin/digest-plan` prices the
+backlog with it — and needed nothing kept beside it: it imports `planOrder`,
+which `digestPlan` already owned.
+
+**The architecture allow-list shrank by ONE**, not three. `lib/sweepPlan.ts ->
+controller/planOrder` died with the file. The plan expected `jobs/* -> controller`
+to go too, but those two entries are `jobs/snapshotScheduler.ts ->
+controller/channelSnapshot` and `jobs/workerPool.ts -> controller/remoteCapacity`
+— a snapshot builder and a capacity probe, unrelated to the sweeps and still real
+back-edges. The guard fails on a stale entry, which is how that was checked
+rather than assumed. **No entry was added.**
+
+### The migration
+
+`getSettings` runs `migrateSweepsToLanes(parsed)` before `sanitizeAutoQueue`, on
+the PARSED FILE rather than the merged object — "absent from the file" is the
+trigger, and a merged object has already had the defaults folded in and can no
+longer tell absent from default.
+
+| Retired field | Becomes |
+|---|---|
+| `digest.sweepEnabled` | `autoQueue.digest.enabled` |
+| `digest.sweepChannels` | `autoQueue.digest.root` — a strict group of channel leaves; empty ⇒ one `{type:"all"}` leaf |
+| `digest.recencyOrder` | **nothing** — see below |
+| `digest.recencyReach` | **nothing** — see below |
+| `backfill.sweepEnabled` | `autoQueue.backfill.enabled` |
+| `backfill.sweepChannels` | `autoQueue.backfill.root` channel leaves |
+| `backfill.sweepKinds` | `match.operation` on those leaves; empty ⇒ leaves naming none, which draw the lane's whole union |
+| `backfill.order` | `autoQueue.backfill.order` |
+| `backfill.reach` | **nothing** — see below |
+| `backfill.weight` | **nothing** — the yield is `contendsFor`, the share is `concurrency` + the lane's `maxWorkers` |
+
+**`digest.recencyOrder` is not migrated onto `order`, and this is the decision the
+1.2 note asked 1.3 to take.** The digest order is a COMPOSITION — shortest inside
+a day, newest day first — which the lane already spells `cheapest`;
+`recencyOrder` was only its date half. Migrating it would have replaced the
+composition with one of its two terms. The date half is fixed at newest-first (the
+live value) in `recencyOrdering`, and a digest lane wanting pure recency sets
+`order: "newest"` and gets no duration term at all.
+
+**Neither `reach` is migrated, and `AutoQueuePolicy` gains no `reach` field.** The
+plan's bullet says `recencyReach → reach`, and that sentence predates what 1.1
+actually shipped: `OrderReach.tsx` renders `reach={null}` for a runner lane and
+says why in a comment — a rule already orders every video it claims across every
+channel and bucket, so there is no second axis, and which RULE goes first is the
+tree. Adding a stored `reach` nothing reads would be config with no reader,
+contradicting a decision documented in three places. Behaviour-neutral either
+way: nothing reads it once the sweeps are gone.
+
+Written as a pure function, `common/lib/laneMigration.ts`, with
+`common/jobs/laneMigration.test.ts` (in `jobs/` because `architecture.test.ts`
+forbids `lib/` importing `jobs/`, and the migration has to be asserted THROUGH
+`sanitizeAutoQueue` — it deliberately emits raw nodes for the sanitizer to
+normalize rather than being a second implementation of it). Eight cases: the live
+shape, an armed digest sweep with three channels, an armed backfill sweep with two
+kinds and one channel, an operation scope with no channel scope, a file that
+already carries the blocks (untouched, and idempotent over its own output), an
+empty file, blank/duplicate scope entries, and the arm-equals-migration check.
+
+**The arm-equals-migration check is a unit test, and it holds by construction.**
+`armLaneAction` and `migrateSweepsToLanes` both call `laneRootFromScope`, so
+"arming the digest lane on channels X" and "migrating `sweepChannels: X`" are the
+same function; the test asserts it so that a second leaf builder added later fails
+there rather than in production. It is also visible in the numbers diff: the
+migrated leaf ids are `digest-all` and `backfill-all`, which only that builder
+produces. `backfill.spec.ts`'s rewritten scope test asserts the other half from
+the browser — a rule authored on the console persists as `match.operation` on the
+lane's root.
+
+### Numbers
+
+Before/after `plans/tools/phase1-numbers.ts` over the live corpus: **0 lines
+removed, 18 added.** Every pre-existing line is byte-identical and in place,
+which is why the file's own lanes are still printed first and the migrated ones
+appended.
+
+```
+> download leaves = [{"type":"channel","value":"quartering-live"}, … ,{"type":"all"}]
+> transcription leaves = [{"type":"channel","value":"quartering-live"}, … ,{"type":"all"}]
+> digest enabled = false
+> digest order = cheapest
+> digest maxWorkers = null
+> digest replaceAutoSubs = false
+> digest leaves = [{"type":"all"}]
+> backfill enabled = false
+> backfill order = newest
+> backfill maxWorkers = null
+> backfill replaceAutoSubs = false
+> backfill leaves = [{"type":"all"}]
+> digest leaf digest-all = 56223
+> digest TOTAL = 56223
+> digest nextUp = digest-all/lv474Pl-H_k
+> backfill leaf backfill-all = 80193
+> backfill TOTAL = 80193
+> backfill nextUp = backfill-all/qXow_UvcHkI
+```
+
+`enabled: false` on both, one catch-all leaf apiece — exactly what the live
+file's disarmed, unscoped sweeps meant. The two totals match 1.2's poll-timing
+measurement (56,223 / 80,193) to the video. The two `leaves` lines on the runner
+lanes are the one addition outside the migrated sections: leaf shapes are printed
+for every lane rather than only the new two, because a section only some lanes get
+is the kind of asymmetry that rots.
+
+One new guard in the script: diagnostics from `common/` are dropped. Pricing the
+digest lane makes the recency index log a `[recency] dating N of M undated
+candidates` line whose numbers move between runs on the same corpus, and one
+nondeterministic line in a file whose whole purpose is byte-for-byte comparison
+would make every later slice's evidence unreadable.
+
+### Divergences from the 1.3 bullets, and why
+
+- **`reach` is retired, not migrated.** Above.
+- **`digest.recencyOrder` is retired, not migrated.** Above — and this one the
+ plan already asked for.
+- **The allow-list shrank by one, not three.** Above.
+- **`backfill.weight` retiring is a real behaviour change, and it is named.** One
+ 0..1 scalar answered two questions: "does this run contend with transcription"
+ and "how much of the machine may it have". Its default of 0 meant idle-only for
+ the WHOLE lane, so a `contendsFor: "network"` attribution run parked itself
+ behind a GPU transcription it was not competing with. `backfillLimit` takes
+ `idleOnly` now, keyed on the operation's declared lane exactly as 1.2's
+ carve-out already was. A diarization run is unaffected on a GPU backend; an
+ attribution or a CPU diarization run no longer stands aside. The lane is
+ disarmed on this corpus, so the live path this changes is the per-channel
+ **manual** Run.
+- **`laneState.ts` keeps four states; `sweepLaneNote` is what actually died.** The
+ plan says the `feedRunning` axis and the note both go. The axis did go — every
+ lane is a runner lane, so "a corpus sweep is armed" is not a state anyone has —
+ but `deriveLaneState` still has four values, because `holding` (gate shut,
+ runner up) is the state with no other name and deleting it was never the point.
+- **`JOB_KINDS_BY_LANE` is rekeyed, not deleted.** It is the MANUAL per-channel
+ jobs — `digest-channel-local`, `backfill-channel`, `diarize-channel` — and
+ without them a hand-started digest is invisible on the digest page. The two
+ sweep orchestrator kinds are gone from it and it is keyed by `AutoQueueKind`
+ with `[]` for the two bucket lanes.
+- **`armLaneAction` takes an OPTIONAL scope.** Given one it writes the root;
+ absent it keeps the stored tree. The dashboard and widget cards call it with
+ `{}` — the whole corpus, which is what "Sweep every channel" meant — and the
+ scope is authored on the lane's console. That keeps the invariant "the runner
+ console is the only surface an arm can be SCOPED from" while not deleting the
+ one-click corpus pass the two cards have always had.
+- **`disarmLaneAction` drains rather than stops.** The sweep's stop drained — the
+ video in flight finishes — and that is worth more than symmetry with the
+ console's Stop button, which is a different control with a different promise.
+- **The dashboard's arm control is renamed, not kept.** `start digest sweep` /
+ `stop digest sweep` / `start backfill sweep` / `stop backfill sweep` become
+ `arm|disarm digest|backfill lane`, and the labels become "Run every channel" /
+ "Stop the lane". The retired-routes rule is honoured by asserting the old names
+ gone in the specs that pinned them (`backfill.spec.ts`, `auto-queue.spec.ts`).
+- **The widget sync payload's `sweeping` becomes `armed`** (`autoQueue[lane].
+ enabled`) rather than being deleted: the deck's control needs to know which of
+ its two states to draw, and the question is the same one asked of the thing that
+ now answers it. A pinned widget tab that predates this reads `undefined ??
+ false` — not armed — until it is reloaded, exactly as the `held` rename did.
+- **`startAutoRunnerBlockedReason` survives with one reason.** It had two (a
+ disabled policy, an armed sweep); it now answers "this lane's policy is
+ switched off". Deleting it would have made the console's Start a silent no-op
+ on a disabled lane, which is the failure the function was written for.
+- **`buildActiveJobs.buildLanes` takes no argument now.** It counted in-flight
+ work on the sweep lanes by scanning the job rows for their queue keys; four
+ runner rows read `getAutoRunnerStatus(lane).inFlight`, which is in-memory.
+ **A held transcription or download lane reads `Holding` on the /jobs strip
+ now**, where the two runner rows could only say running / idle / unavailable —
+ the strip and the operations rail (which has derived it that way since slice 7)
+ finally agree. One fold, one answer; that is what folding the two halves
+ together buys.
+- **The sweep panel's surviving prose moved rather than died.** The band, the
+ "this operation is switched off" warning and the "shares one lane, one runner
+ and one pause with …" sentence are a `LaneOperationContext` **div** above the
+ console — a nested `<section>` would break every `locator("section", { has })`
+ in the suite. The "also runs on …" line is in `LaneHeader`, per LANE: the slots
+ belong to the machine, so two operations served by one endpoint are one row.
+- **`e2e/backfill.spec.ts` had three sweep tests, not seven.** The other four
+ sweep-shaped assertions the bullet counted are `data-sweep-lane` selectors
+ inside tests about other things (the diarization page's populations, the two
+ pause tests, `attribution.spec.ts`'s per-operation bands); those were rehomed
+ onto the runner section and the rail's row rather than rewritten.
+- **`operation-settings.spec.ts` needed no change.** Its two lane-form tests use
+ "Run the backfill lane" and "Save lane settings", both of which survive; only
+ the "Resource share" field went.
+
+### What the specs pin now, where they pinned a sweep
+
+- **`the lane arms from the dashboard and disarms without losing its rules`** —
+ the sweep pair pinned "the flag is persisted without the scope" and "stopping
+ leaves the scope behind". Both are unreachable now (the scope IS the tree, in
+ the same write as the switch), so the test pins the other half of that bargain:
+ disarming must NOT clear the tree.
+- **`a lane can be scoped to one operation from its console, and the scope is a
+ tree`** — drives the ladder's `rule operation` select and asserts
+ `match.operation` on the persisted root.
+- **`an armed lane comes back after a restart`** — `/api/test/resume-lane?lane=
+ backfill`, which 1.2 added for exactly this.
+- **`the digest lane offers Shortest first, and has no Reach axis`** — replaces
+ the Reach test. Reach asserted GONE, `cheapest` asserted present on the one lane
+ that can price its work, and the order asserted to land on
+ `autoQueue.digest.order` with `digest.recencyOrder`/`recencyReach` gone from the
+ file.
+- **`section[data-sweep-lane]` is asserted to have COUNT 0** on both an operation
+ page and the digest page — not merely unused.
+- **The three arbiter tests are deleted.** Its leaf-order case already lives in
+ `autoQueuePolicy.test.ts`.
+- **`backfill.spec.ts`'s concurrency test survives**, as the plan requires: it
+ covers the per-channel jobs, which still exist and still need distinct queue
+ keys. `lane-runner.spec.ts` test 4 covers the property at the runner.
+
+### What review found before the e2e run
+
+**`LaneOperationContext` was drawn on every page with a runner** (`43b2f9f`).
+It is about THIS OPERATION — its band, its "switched off" warning, the sentence
+naming the operations it shares a queue with — and on
+`/operations/transcription` the lane's operation list is `[]` by construction, so
+`switchedOn` was false and the page claimed auto-transcribe was switched off in
+settings that do not exist. It is gated on `bucketsByKind[lane].length === 0`
+now: exactly the two lanes whose work list is an operation's ids rather than a
+snapshot bucket. Worth naming because it is the shape of mistake this slice
+invites — a block that was written for two lanes, moved onto a component that
+serves four.
+
+**The dashboard's arm button was discarding the lane's rules** (`cdaccc4`).
+`armLaneAction(lane, {})` is not the same call as `armLaneAction(lane)` — an
+empty scope object is still a scope, so the deck wrote a catch-all root over
+whatever tree an operator had authored on the console. Three channel rules, one
+click, gone, and disarming would not bring them back. The button it replaces did
+the opposite: `startDigestSweepAction()` with no argument INHERITED
+`sweepChannels` off disk. It passes no scope now, and the label follows the truth
+— "Run the lane", not "Run every channel", because on a narrowed lane every
+channel is not what it would do.
+
+**`git add -A editor` staged the operator's `editor/content` symlink** into the
+e2e commit, which put it in the worktree, which panicked Turbopack before the
+first spec — the exact failure `plans/FACTS.md` documents for the primary
+checkout, arriving by a route that entry did not cover. The two commits were
+rewritten to drop it and FACTS.md now says so. Add by path in this repo.
+
+### Traps for the 1.4 implementer
+
+- **Every GATE read of the four legacy fields is inside `lib/pauseGates.ts`** —
+ `transcriptionsPaused`, `downloadsPaused`, `digest.digestsPaused`,
+ `!backfill.enabled`, in `isGateHeld` (`:85-92`) and `withGateHeld` (`:114+`).
+ Nothing in `common/controller/`, `common/jobs/` or `editor/app/` asks the
+ question any other way. The files that touch the FIELDS for another reason,
+ and which 1.4's deletion step has to visit:
+ `editor/app/settings/actions.ts:222-223` (spreads `transcriptionsPaused` and
+ `downloadsPaused` through an unrelated save — this is the "settingsActions
+ spread" the 1.4 bullet names), `editor/app/operations/settingsActions.ts:102`
+ (same, for `digestsPaused`), `LaneSettingsForm` + `settingsActions` for
+ `backfill.enabled` behind "Run the backfill lane", `editor/instrumentation.ts`
+ re-applying `transcriptionsPaused` to the worker pool at boot (the intent-vs-pool
+ asymmetry that must survive), and `editor/app/workers/buildWorkers.ts:54`, which
+ already goes through `isGateHeld`. The e2e reads on disk are
+ `auto-queue.spec.ts` (`downloadsPaused`), `dashboard.spec.ts`,
+ `backfill.spec.ts` (`backfill.enabled`, three tests, via its `laneEnabled`
+ helper) and `widget.spec.ts`.
+- **`held` should land on `AutoQueuePolicy`, beside `enabled` and `snoozeUntil`,
+ and `sanitizePolicy` is the one place to default it.** `armLaneAction` and
+ `saveAutoQueueAction` both spell every policy field EXPLICITLY when they write
+ — the note on `saveAutoQueueAction` says why — so both must gain the key or a
+ save will silently unhold a lane. That is two call sites, and they are the whole
+ list.
+- **The numbers script already prints `held <lane>` for all four lanes** through
+ `isGateHeld`, so 1.4's before/after diff over the live corpus is a direct test
+ of the fallback: it must stay `true/false/false/false` while the file carries
+ only the legacy fields.
+- **`withGateHeld` must keep spreading.** `pauseGates.test.ts`'s "holding the
+ backfill lane leaves the rest of its block alone" now also asserts
+ `held.autoQueue` deep-equals the input's — the tree is the thing a rebuilt
+ literal would drop next.
+- **`backfill.enabled` is a lane gate AND a form checkbox.** 1.4's "delete the
+ four legacy fields" step has to decide what "Run the backfill lane" writes;
+ today it writes the same field the pause does, on purpose.
