commit 12ec3b2cd66d8a66bf2b061e7563fb4005af8134
parent c8a24bddc8c7b018a9fad00c3d118eeba60bcecb
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Thu, 1 Oct 2026 12:39:06 -0400
plans: slice RM — the review (SHIP AFTER FIXES), its fixes, the guard as it now is, the merge of main, the gates after them
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Diffstat:
| M | plans/FACTS.md | | | 54 | ++++++++++++++++++++++++++++++++++-------------------- |
| M | plans/release-16.md | | | 70 | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------ |
2 files changed, 92 insertions(+), 32 deletions(-)
diff --git a/plans/FACTS.md b/plans/FACTS.md
@@ -8115,8 +8115,8 @@ phase deletes from the destination.
the new files and refused on the counts, 1755 against 1750 — the five scratch files on the
destination, which a copy that never deletes could not settle.
- **The three closes.** (1) The move refuses over a writer at its preview, its job's FIRST STEP and a
- third time right after the copy phase's marker is written (`relocateChannelMedia.ts:162`
- `assertNoWriters`, `:488`, `:546`, `:667` `assertNoWritersUnderMarker` — the third removes a marker
+ third time right after the copy phase's marker is written (`relocateChannelMedia.ts:167`
+ `assertNoWriters`, `:493`, `:551`, `:672` `assertNoWritersUnderMarker` — the third removes a marker
this run created). (2) A `needsMedia` job's guard is asked again when the queue STARTS it
(`jobs/streamCommand.ts:429`, `refuseForUnreachableMedia` `:327`), so a job queued before a marker
and started after it fails before `fn` runs. (3) The per-video writers that were not in
@@ -8126,12 +8126,12 @@ phase deletes from the destination.
(`/jobs` unchanged). Not in the table and still not refused: `worker-transcribe`/`worker-unit` (this
box as a worker: a scratch dir or another machine's mount), `refresh-report` (asserts reachability
itself), and the corpus-wide `normalize-live-chat`/`archive-*` (no slug).
-- **Who is a writer: `controller/channelWriters.ts`.** `channelWriters(slug, opts)` (`:86`): registry
+- **Who is a writer: `controller/channelWriters.ts`.** `channelWriters(slug, opts)` (`:89`): registry
jobs whose `channelSlug` is the slug, running (queued too with `includeQueued`), minus
`ignoreKinds`, then every lane's in-flight units for the slug (`getAutoRunnerStatus(lane).inFlight`).
A lane download unit is also an `auto-download-unit` job; it is named once, as the lane's.
- `describeChannelWriter` (`:151`) names the first task's video ("a transcription of v50t5yt is running
- (Transcribe all, job …)"); `channelWritersRefusal` (`:177`) is the move's sentence. The move's job
+ `describeChannelWriter` (`:164`) names the first task's video ("a transcription of v50t5yt is running
+ (Transcribe all, job …)"); `channelWritersRefusal` (`:190`) is the move's sentence. The move's job
passes `ignoreKinds: ["relocate-channel-media"]` (it is itself running on the slug; the relocation
queue runs one at a time); the preview passes nothing. **The editor's `channelMediaBusyReason`
(`editor/app/channels/lib/mediaBusy.ts`) reads the same list with `includeQueued`**, keeps its
@@ -8149,33 +8149,47 @@ phase deletes from the destination.
(imported by the hold module) is never pulled into a client file by this. The runners skip on
`isMediaHeld` (`autoRunner.ts:641`); their skip log is now `[auto] skipping <slug>, held: <reason>:
media <status> — <detail>` (the old `media <status> — <detail>` is its tail).
-- **The mirror: `controller/relocateDir.ts`.** `MIRROR_ARGS = ["-a", "--delete"]` (`:344`); the
- verify's dry run is `VERIFY_ARGS` (`:348`, `--delete` included, so an extra on the destination is a
- difference, `*deleting`). **`--delete` only ever targets the copy under construction**:
- `mirrorTree` (`:487`) calls `assertMirrorDirection` (`:461`) before rsync is spawned — the
- destination must equal the caller's `underConstruction` (the directory the move made to copy into:
- `<root>/<slug>/data` out, `data.incoming` back, `<root>/saved-videos` for the store) and neither
- tree may contain the other (realpaths). `copyMirrorVerify` (`:550`) is the copy phase for both
+- **The mirror: `controller/relocateDir.ts`.** `MIRROR_ARGS = ["-a", "--delete", "--info=del"]` —
+ every removal is a `deleting <path>` line in the job log; the verify's dry run is `VERIFY_ARGS`
+ (`--delete` included, so an extra on the destination is a difference, `*deleting`). **`--delete`
+ never reaches the live media**: `assertMirrorDirection` (`:468`, asked by `copyMirrorVerify` before its
+ FIRST rsync, and by `mirrorTree`) takes `live` from the caller — `channels/<slug>/data` for a
+ channel (both directions; resolved through its link, so on the way back it is the relocated
+ target) and `paths.savedVideosDir` for the store — and refuses when the destination's realpath is
+ the live media's, contains it or is inside it; and when the source and destination contain one
+ another. `live` is never derived from `src`/`dest`, so a swapped call is refused with nothing
+ spawned (review M1: the first cut compared `dest` with an `underConstruction` every caller set to
+ `dest`, which could not fail). `copyMirrorVerify` (`:565`) is the copy phase for both
movers and both directions: the progress copy (`COPY_ARGS`), the mirror pass, `verifyCopy` in mirror
- mode (`verifyMirrored`, `:690`): an empty itemized dry run plus equal counts; a difference gets one
- more mirror pass (`retried`), a second refuses with `CopyVerificationError` (`:425`) carrying the
- differences by kind (`classifyDrift`, `:386`: `*deleting` → extra on the destination, all-`+`
+ mode (`verifyMirrored`, `:709`): an empty itemized dry run plus equal counts; a difference gets one
+ more mirror pass (`retried`), a second refuses with `CopyVerificationError` (`:430`) carrying the
+ differences by kind (`classifyDrift`, `:391`: `*deleting` → extra on the destination, all-`+`
attributes → missing on the destination, anything else → changed). **A swap-phase re-verify
- mirrors only from the live media** (`data/` still a real directory, `relocateChannelMedia.ts:863`);
+ mirrors only from the live media** (`data/` still a real directory, `relocateChannelMedia.ts:872`);
against a parked `data.relocated-*` it is the strict legacy verify — the target is never mirrored
from a copy that is no longer live. Resume takes the copy phase again, so stale files on a
destination are removed.
- **Reconcile and resume** = `relocateChannelMedia({ reconcile: true })`, refused without a marker
- (`:551`): the copy phase skips the progress copy, logs `diffTrees` (`relocateDir.ts:510`) by kind
+ (`relocateChannelMedia.ts:556`): the copy phase skips the progress copy, logs `diffTrees` by kind
("Reconciling: the destination copy differs from the source — …"), and the mirror pass carries the
- progress sink. Editor: `reconcileRelocationAction` (`storageActions.ts:136`, Resume's guards), the
+ progress sink. `RelocateChannelMediaResult.reconciled` is true only when the copy phase ran as a
+ reconcile; a reconcile of a marker past the copy phase is a plain resume and says "resumed". Editor: `reconcileRelocationAction` (`storageActions.ts:136`, Resume's guards), the
Storage panel's button beside Resume move (log `"Reconcile and resume output"`), the job's done line
"(reconciled and resumed an interrupted move)".
- **The saved-video store shares the pipeline** (`relocateSavedVideos.ts` copy phases →
`copyMirrorVerify`; the swap re-verify mirrors while the store is still a real directory) and takes
`busy` (`:246`, asked first, `:259`); the editor's job passes the store check with `runningOnly`
(`editor/app/storage/lib/storeBusy.ts`). It has no reconcile button.
-- **Not covered:** a lane tick that inspected the channel before the marker landed and dispatches
- after the third ask (milliseconds; the pick→run backstop and the mirror cover it); a corpus-wide
+- **A cancel is a request, not an exit.** `registry.cancel` on a RUNNING job sets `cancelled` at once
+ (no `endedAt`) and the function winds down; `markTerminal` then skips it, so `registry.finalize`
+ stamps `endedAt` itself for a cancelled record that has none, and `forceRelease` does for one never
+ finalized. `channelWriters` counts `cancelled` with no `endedAt` as a writer, status "stopping"
+ ("— wait for it to stop"). Before this, a job cancelled while running never got an `endedAt` at all
+ (the 2026-09-30 Transcribe all's meta has none).
+- **The race the third ask leaves is closed by ordering** (the review's trace): `registry.enqueue`
+ marks a job running before `start()`, whose guard then reads the marker; a lane unit is in
+ `inFlight` before the pick→run backstop reads the marker. Either the third ask sees the writer or
+ the writer sees the marker.
+- **Not covered:** a corpus-wide
writer mid-channel when a move starts; a resumed or reconciled move-out is charged the whole tree
against the destination's free space, not the remainder (move-back charges the remainder).
diff --git a/plans/release-16.md b/plans/release-16.md
@@ -823,7 +823,10 @@ so they were never refused for a moving channel.
has been touched.` (a lane unit: "wait for it, or hold the transcription lane"). Asked by
`previewRelocation`, by the job's first step, and a third time the moment the copy phase's marker is
written — after which the lanes skip the channel and a media job refuses to start, so that answer
- cannot go stale; a refusal there removes a marker this run created. The editor's
+ cannot go stale; a refusal there removes a marker this run created. A job that has been cancelled
+ but whose function has not returned yet still counts ("is stopping — wait for it to stop"): the
+ registry stamps `endedAt` on a cancelled-while-running job only when it has actually stopped
+ (`registry.finalize`, and `forceRelease` for one never finalized). The editor's
`channelMediaBusyReason` (the panel's blocked message, rename, delete, the bulk move's skips) reads
the same list and appends `Now: <the first writer>.` to its counts.
- **The writers are held while the marker stands.** A media job's guard (`needsMedia`) is asked again
@@ -842,10 +845,15 @@ so they were never refused for a moving channel.
copy under construction, then a verify whose dry run carries `--delete` plus equal counts. One
change seen between the mirror and the check gets one more mirror pass; a second refuses with the
paths by kind — extra on the destination, missing on the destination, changed — and "Something is
- still writing into <src>: stop it, then Reconcile and resume." `mirrorTree` asserts the direction
- before rsync is spawned: the destination must be the directory the move created to copy into, and
- neither tree may contain the other. A resume takes the same path, so the realcandaceo leftovers are
- mirrored away. A swap-phase re-verify mirrors only while `data/` is still the live directory; against
+ still writing into <src>: stop it, then Reconcile and resume." The mirror runs with `--info=del`,
+ so every file it removes is a `deleting <path>` line in the job log. **What the direction guard
+ proves** (`assertMirrorDirection`, asked by `copyMirrorVerify` before its first rsync and by
+ `mirrorTree`): the destination's real path is not the live media's, does not contain it and is not
+ inside it — `live` being what the caller knows is live, `channels/<slug>/data` resolved through its
+ link (so the relocated target on the way back) or the saved-video store, never derived from `src`
+ or `dest`; and the source and destination do not contain one another. A call with the two swapped
+ is refused with nothing spawned, in either direction. A resume takes the same path, so the
+ realcandaceo leftovers are mirrored away. A swap-phase re-verify mirrors only while `data/` is still the live directory; against
a parked `data.relocated-*` it stays the strict verify, so the target is never mirrored from a copy
that is no longer live.
- **Reconcile and resume** (the remediation bullet): a button beside **Resume move**, with its own log
@@ -854,7 +862,8 @@ so they were never refused for a moving channel.
by kind ("Reconciling: the destination copy differs from the source — 2 extra on the destination
(…), 0 missing on the destination, 2 changed (…)"), mirrors (the progress bar rides the mirror
pass), verifies, swaps and reclaims; the done line says "(reconciled and resumed an interrupted
- move)". Without a marker it refuses.
+ move)" — only when the copy phase ran as a reconcile (`result.reconciled`): a marker already past the
+ copy phase has nothing to reconcile, and that run says "(resumed …)". Without a marker it refuses.
- **The saved-video store's move shares the pipeline** — `relocateSavedVideos` uses
`copyMirrorVerify` both ways and mirrors its swap re-verify while the store is still a real
directory — and takes a `busy` first step, which the editor's job fills with the store check
@@ -868,8 +877,12 @@ so they were never refused for a moving channel.
|---|---|
| `50fc580c` | `common:` `channelWriters.ts` + test; the mirror, the direction assertion, `copyMirrorVerify`, the verify by kind (`relocateDir.ts` + test); both movers (`reconcile`, `writers`, `busy`) + tests; the start-time media guard + test; seven `needsMedia` kinds; the hold's words, the lanes' skip + test; `ChannelRowView.mediaHold` |
| `1c5b9739` | `editor:` the Storage panel's hold line and Reconcile and resume; the rack's chip; `mediaBusy` names the writer; the store job's first step |
-| `0ee291f9` | `editor(e2e):` `channel-storage.spec.ts` — four cases (below); `/api/test/stuck-job` takes `slug` and `task` |
-| this commit | `plans:` this section, the slices table's RM row; FACTS; the editor changelog |
+| `0ee291f9` | `editor(e2e):` `channel-storage.spec.ts` — three new cases and Resume extended (below); `/api/test/stuck-job` takes `slug` and `task` |
+| `05383fe6` | `plans:` this section, the slices table's RM row; FACTS; the editor changelog |
+| `f9dcf2d4` | `common, editor:` the review's fixes — M1 the guard checks the live media the caller names; L1 `--info=del` and the preview's partial-copy line; L2 a cancelled job counts until it has stopped (`registry.finalize`/`forceRelease` stamp `endedAt`); L3 `reconciled` on the result; tests and one e2e case |
+| `01de2613` | `editor:` the changelog bullet — the fixes, and a made-up video id in its example (N3) |
+| `3cc7a5da` | merge of `main` `a2229d68` (plans only: slice XL's ruling) — the slices table keeps both rows |
+| this commit | `plans:` the review, its fixes and the gates after them |
#### Gates (logs `$T/rm-*.log`)
@@ -899,7 +912,7 @@ so they were never refused for a moving channel.
| 3 | `0ee291f9` | the full editor suite | **683 passed**, 3 failed, 12 skipped (the rack-audit shots), 51.8 min — `jobs-channel` "auto-refreshes the jobs list": every button on the Playlist stage still disabled after 30 s (the page never hydrated); `perf-budget` "the dashboard renders within budget": `resetData` met `EEXIST` making `test-transcripts/channels` (the fixture race helpers.ts describes); `widget` "+N more expands": `ERR_CONNECTION_REFUSED` — the log's one `[WebServer] ⚠ Server is approaching the used memory threshold, restarting...` landed on it |
| 4 | `0ee291f9` | `jobs-channel`, `perf-budget`, `widget` | **34 passed**, 0 failed, 1.4 min (after ~4 min waiting for the queue). None of the three touches what the slice changed beyond the shared job start |
- The four new `channel-storage` cases: **a move that starts while a job writes into the channel
+ Three new `channel-storage` cases and one extended: **a move that starts while a job writes into the channel
refuses, naming the job** (the 2026-09-30 shape: a stuck job holds the relocation queue for 4 s, the
move goes in through `/api/ops/relocate`, a fake running Transcribe all on the channel's video
appears, the move starts and fails with the exact sentence; no marker, no copy; the panel's blocked
@@ -928,9 +941,10 @@ so they were never refused for a moving channel.
- **Kinds still outside `JOB_KINDS` with a slug:** `worker-transcribe` / `worker-unit` (this box as a
worker, writing into a scratch dir or another machine's mount), `refresh-report` (it asserts
reachability itself), and the slug-less `normalize-live-chat` / `archive-*`.
-- **A lane tick that inspected the channel just before the marker landed** can dispatch a unit after
- the third ask (milliseconds); the pick→run backstop reads the marker, and the mirror covers the
- rest.
+- ~~A lane tick that inspected the channel just before the marker landed can dispatch a unit after
+ the third ask.~~ **Closed, by ordering** (the review's trace): a unit is in `inFlight` before the
+ pick→run backstop reads the marker, so a unit the third ask missed reads the marker itself; and a
+ job is marked running before its start-time guard reads the marker.
- **A resumed or reconciled move-out is charged the whole tree against the destination's free space**,
not what is still missing (the move back charges the remainder). Unchanged; a near-full destination
can refuse a resume it has room for.
@@ -948,6 +962,38 @@ so they were never refused for a moving channel.
| The rack's hold is a chip in the Tier cell, shown for every held status (unreachable, stalled and inconsistent too: the lanes skip all of them) | Only for a moving channel, as the ruling's sentence names |
| Reconcile and resume skips the progress copy and lets the mirror pass carry the bar | Run the copy pass first, as Resume does — the same bytes either way |
+#### Review
+
+**Verdict: SHIP AFTER FIXES** (`rm-review.md` in the job's scratch): one Medium, three Lows, three
+notes. Every `--delete` was traced and found pointed the right way; the fixes are what the guard and
+the wording promise.
+
+| Finding | Where |
+|---|---|
+| M1: half the `--delete` direction check could not fail — every caller set `underConstruction` to `dest`, so "dest equals the copy under construction" held by construction, and a swap across two volumes would have passed the nesting check | `f9dcf2d4`: `assertMirrorDirection` takes `live` from the caller (`channels/<slug>/data` through its link, or the store) and refuses a destination whose realpath is, contains or sits inside it; `copyMirrorVerify` asks before its first rsync. Tests: swapped across two roots, out and back, nothing spawned; a destination inside and one containing the live media; the containment check kept |
+| L1: the mirror's removals were not logged, and the preview did not say a pre-existing copy loses files | `f9dcf2d4`: `--info=del` on `MIRROR_ARGS` (a test reads the `deleting` line); the partial-copy line adds "files there that the channel no longer has are removed from that copy (never from the channel)" |
+| L2: a job cancelled but still winding down was not a writer | `f9dcf2d4`: `registry.finalize` stamps `endedAt` on a cancelled-while-running job when it actually stops (`forceRelease` on one never finalized); `channelWriters` counts "cancelled, no `endedAt`" as "stopping — wait for it to stop". Unit (injected and through the live registry) and e2e ("a job cancelled but still stopping holds the move until it has stopped", the stuck-job route cancelled from `/jobs`) |
+| L3: Reconcile on a marker past the copy phase said "reconciled" | `f9dcf2d4`: `RelocateChannelMediaResult.reconciled`; the done line says "resumed" otherwise; a unit test on a swap-phase marker |
+| N1: "four new" cases — three are new, Resume was extended | This commit |
+| N2: any running job on the slug refuses a move, `refresh-report` included | No action (errs safe) |
+| N3: the changelog's example quoted a real video id | `01de2613`: a made-up id |
+
+The review also traced the race "Found and left" listed (a lane tick dispatching after the third
+ask) and found it closed by ordering; that item is struck above.
+
+#### Gates after the review and the merge (at `3cc7a5da`; logs `$T/rm-*2.log`, `rm-tsc3.log`, `rm-e2e5.log`)
+
+- **tsc** (all workspaces) clean, 52 s. **common:** **2,438/2,438**, 94 s (+6: three direction
+ tests in place of one, the mirror's logged removal, two stopping-writer tests, the reconcile past
+ the copy phase). **Editor unit:** 109/109. **test:scripts:** 302 passed, 2 skipped.
+- **Build:** the capped editor build with the corpus linked: exit 0, 63 s, 1.67 GB, the link removed.
+- **e2e** (after ~1.5 min waiting for the queue):
+
+ | Run | At | Specs | Result |
+ |---|---|---|---|
+ | 5 | `3cc7a5da` | `channel-storage` (13, the stopping case included), `storage-locations`, `channels-storage-columns` | **24 passed**, 1 failed, 5.4 min — `storage-locations` "a volume that came up somewhere else is re-pointed": the first step's `clickUntil` on **Refresh** never saw the volume's identity written to settings in 30 s (the page showed the probe's identity; the action's write did not land). Nothing in that step is the slice's |
+ | 6 | `3cc7a5da` | `storage-locations` × 2 | **18 passed**, 0 failed, 2.0 min |
+
## Rollout
Both slices are export- and homepage-side; the editor and umtool are not rebuilt for this release.
