commit 67946b369d61965f999875a7b85d3678453c382d
parent 6d65595ff74f1c219f9162f844a008d757728533
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Fri, 2 Oct 2026 01:46:40 -0400
plans: slice T2's review — H1, L2-L4, N5-N9 to their commits, the merge of main 9ee7f1ee and the gates after it
The record's Review subsection, FACTS "Release 17 slice T2" (the re-point of a
legacy channel, the refused rename, tierMigrationRefusal, the 503 rule), and
the changelog bullets.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Diffstat:
3 files changed, 57 insertions(+), 15 deletions(-)
diff --git a/editor/CHANGELOG.md b/editor/CHANGELOG.md
@@ -33,8 +33,8 @@
- **The operations pages share one count of the lanes' pending work.** Every open operations page asks for the lanes' status every 3 seconds, and each request used to count every lane's pending videos afresh from every channel's report. That count is now made once and handed to every request in the next 3 seconds. Changing a lane's rules, a focus or a channel's priority counts again at once; otherwise a pending count can be up to 3 seconds behind a report that was just rewritten or a video a lane just picked. A lane's hold, its runner and its picks are still read fresh on every request.
- **Jobs a stopped editor left "running" are closed when it starts again.** A job that was still running when the editor's process ended (killed, crashed, or shut down before the job had finished unwinding) kept "running" in its record for good, and `/jobs` listed it as archived. On start the editor now marks each one **cancelled**, with "interrupted: the process running it stopped before it finished" as the reason on the job's page, and its end time is the last time its log was written. Nothing is run again; **Retry** works as for any cancelled job. A job that another live process is running, such as `archilyzer run`, is left alone, and the same check now keeps the start-up pass from closing that process's queued jobs. Such leftover jobs never blocked a media move.
- **A channel's text stays on the fast disk when its media moves, so a slow or unplugged media drive no longer holds its transcripts.** A channel's big files — the audio and the raw live-chat replay — can now live in the channel's own `media` folder, on this disk or another, while its transcripts, cues, metadata and every other small file stay in `data/` where they always were; each big file that moves leaves a small link behind, so everything that opens it by name still finds it. New downloads, transcodes and live-chat normalizes put their big files there as they finish, and every cleanup that deletes audio removes the file the link points to, not just the link. What that changes when a media drive is stalled, unplugged or mid-move: **the index and stats builds never wait on it or are held by it** (a live chat whose transcript cues are out of date keeps the cues the last build read until the drive answers), the channel's report still refreshes (its media size reads as unknown until the drive answers), **digests keep running — even during a move of that channel's media** — and so do normalize, the availability checks, the metadata scan and clip eviction. Transcription, downloads, the backfill lane and anything else that opens the audio are held as before. **A channel moved the old way — its whole `data/` on the other drive — is now shown as "Media layout retired" and held by everything, the builds and digests included, until `archilyzer storage migrate-tier <channel>` brings its text home;** every refusal says so. Deleting a video from its page is refused while its channel's media drive is not reachable, so its audio is never left behind on the drive. Needs a rebuild and restart of the editor.
-- **Moving a channel's media now moves only its big files.** The Storage panel's **Move media** copies the channel's audio and raw live-chat replays to `<root>/<channel>/media` on the destination and leaves its transcripts, metadata and every other small file in `data/` on this disk; a channel whose audio is still in `data/` has it put into the channel's `media` folder on this disk first, and the preview says how many files ("2 file(s) tiered first"). **Move back in place** brings the `media` folder home; the links in `data/` are not touched either way. The panel shows the media path and the text path, each with its size. While a channel's media is held — moving, or on a drive that is unplugged or not answering — its text stays readable: the video page and the videos list open as usual and list the audio with its size unknown, the transcript opens, the audio answers "not reachable, try again" (503) instead of "not found", and a digest may run, during a move too; only the jobs and lanes that open the audio wait. A move, its preview and **Resume move** refuse a channel still on the retired whole-directory layout and name `archilyzer storage migrate-tier`; `/storage` counts such channels as unreachable "(n to migrate)". On `/storage` a location's figure is now the media on it, and the corpus volume's row adds every channel's text and clip windows ("text … + clips … on the corpus volume, plus … media of in-place channels"). Re-pointing a location, renaming a channel and deleting one follow the `media` folder. Needs a rebuild and restart of the editor.
-- **An export build no longer reads a raw live-chat replay from a media drive that is unplugged, not answering or mid-move.** It publishes the live-chat cues already on disk for that channel, and its log says how many are older than their replay.
+- **Moving a channel's media now moves only its big files.** The Storage panel's **Move media** copies the channel's audio and raw live-chat replays to `<root>/<channel>/media` on the destination and leaves its transcripts, metadata and every other small file in `data/` on this disk; a channel whose audio is still in `data/` has it put into the channel's `media` folder on this disk first, and the preview says how many files ("2 file(s) tiered first"). **Move back in place** brings the `media` folder home; the links in `data/` are not touched either way. The panel shows the media path and the text path, each with its size. While a channel's media is held — moving, or on a drive that is unplugged or not answering — its text stays readable: the video page and the videos list open as usual and list the audio with its size unknown, the transcript opens, the audio answers "not reachable, try again" (503) instead of "not found", and a digest may run, during a move too; only the jobs and lanes that open the audio wait. A move, its preview and **Resume move** refuse a channel still on the retired whole-directory layout and name `archilyzer storage migrate-tier`; `/storage` counts such channels as unreachable "(n to migrate)". On `/storage` a location's figure is now the media on it, and the corpus volume's row adds every channel's text and clip windows ("text … + clips … on the corpus volume, plus … media of in-place channels"). Re-pointing a location, renaming a channel and deleting one follow the `media` folder; a re-point also carries a channel not yet migrated (its whole `data/` link), so a drive holding both kinds still follows its disk, while renaming such a channel is refused until it is migrated, and **Clear marker** never removes an interrupted migration's marker. Needs a rebuild and restart of the editor.
+- **An export build no longer reads a raw live-chat replay from a media drive that is unplugged, not answering or mid-move.** It publishes the live-chat cues already on disk for that channel, and its log says how many are older than their replay and how many videos with a replay and no cues yet were left out of that build.
- **A video whose YouTube subtitles answer "Too Many Requests" (HTTP 429) is downloaded anyway, and YouTube is not put in a cooldown for it.** YouTube refuses a subtitle file per video while the video itself downloads fine; every one of the day's 429s on 2026-10-01 was a subtitle fetch, and each failed its download, put all of YouTube in a cooldown that reached 30 minutes, and deferred the video for 6 hours to fail the same way again. Now that refusal is noted and the download goes on to the audio, as for a video with no captions, so the transcription lane transcribes it; nothing platform-wide is backed off. The video's subtitles are deferred: **Download missing subs** skips them for 6 hours, and from the third time they are refused, for 7 days; a download from the video's own page still fetches them. The download lane's page lists them under **Deferred subtitles**, and the video page says how many times and when. **Download missing subs** also goes on to the next video when one video's subtitles are refused, instead of stopping. Any other subtitle failure (a 403, a missing file, a chat replay that fails) still fails the download, as before. Needs a rebuild and restart of the editor.
- **The download pace adapts to rate limits, a rate limit that outlasts the cooldown holds the platform, and the auto-download lane waits between downloads.** Every yt-dlp run against a platform now waits its platform's current pace between requests: 1 second for YouTube and Rumble, doubled by each real rate limit (up to 16 seconds) and eased back one step after every 5 clean downloads, and one step for every hour with no rate limit; a subtitle-only refusal never raises it. When a platform has failed three times in a row at the 30-minute cooldown, it is **held**: auto-download tries it once an hour instead of every 30 minutes, and until that try is due a manual **Sync**, download or metadata scan on it is refused with a sentence giving its time. A clean try lifts the hold — the lane's, or a manual Sync, download or scan once the try is due, which is how a hold ends while auto-download is off or has nothing to fetch on that platform — and so does a try whose video came down although its subtitles were refused. The lane page keeps a held platform listed until then (saying when the lane is off), with a **Clear hold** button that drops the hold, the cooldown and the raised pace at once and says so in a job log. The auto-download lane now waits **Sleep between downloads** between two downloads on one platform, as a channel's batch downloads always did, plus whatever the pace was raised by; batch downloads add that too. The lane page's **Rate-limit cooldown** box shows held platforms, the raised paces and the deferred subtitles, the lane says when it is idle because a platform is held or it is pausing between downloads, and `archilyzer doctor` warns about a platform in a cooldown or held, and about a raised pace. **Download missing subs** also waits that gap between videos. The four numbers are the new `pacing` block in `settings.json` (SETTINGS.md). **After the restart, YouTube may be held at its first real failure:** its cooldown count from before the update (the subtitle refusals) still stands, so one failure puts it straight past the cap — **Clear hold** on the download lane's page resets it, and a clean download does too. Needs a rebuild and restart of the editor.
diff --git a/plans/FACTS.md b/plans/FACTS.md
@@ -3951,12 +3951,18 @@ Supersedes, where they differ, the mover and storage-location facts above (the w
`channelsOnLocation` (with `textBytes`, `unknownTextBytes` and a `legacy` count inside
`unreachable`), `channelVolumeOf(config, locations)` (now takes the config), the storage watch (one
auto-pause per channel, as before), `deleteChannel` (removes both targets). The re-point rewrites the
- `media` link and `mediaDir`, and refuses a legacy channel on the location naming `migrate-tier`. The
- rename moves a convention-shaped `<root>/<slug>/media` and re-points the `media` link.
+ `media` link and `mediaDir`, and a legacy channel on the same location the RETIRED way — its `data`
+ link and `dataDir` to `relocatedDataDir(newRoot, slug)`, in the same ledger and rollback
+ (`RepointPreflight.legacy`), so `dataDir` is always `<root>/<slug>/data` for the tier migration. The
+ rename moves a convention-shaped `<root>/<slug>/media` and re-points the `media` link; it REFUSES a
+ legacy channel (`migrate-tier`), for the same invariant. `tierMigrationRefusal` (exported from the
+ mover) is the one sentence the job, the preview, Resume/Reconcile and Clear marker refuse a
+ `scope: "tier-migration"` marker with.
- **`/storage` bytes.** A location row's figure is its channels' `totalMediaBytes` (the media tier),
with no clips (`clipsBytes` 0 there); the internal row is in-place media + every channel's
`totalTextBytes` + `totalClipsBytes` (`internal.corpus`, folded in `editor/app/storage/buildStorage.ts`;
- a report with no text figure counts unmeasured, once per channel) with the breakdown in `tiersText`
+ a report with no text figure counts unmeasured, once per channel; a legacy channel is left out of the
+ sums and named "legacy (n to migrate)") with the breakdown in `tiersText`
(`aria-label="location tier bytes"`). `channelsText` adds "(n to migrate)".
- **The two video pages read the text on the corpus disk.** They gate on `channelTextStall` (a legacy
channel only), not `channelMediaStall`, and list a directory through ONE loader,
@@ -3964,14 +3970,16 @@ Supersedes, where they differ, the mover and storage-location facts above (the w
are listed (a dirent `isFile()` hid them), a link's time is its `lstat` (the hook copies the file's
times onto it), and the media links' sizes are asked of the media drive in ONE `onDrive(mediaDir)`
call; unreachable → `VideoFile.size: null`, rendered "— (media drive not reachable)".
-- **The file route picks the drive by the file's kind** (`classifyEntry(name) === "media"` →
- `mediaDir`, else none; a legacy channel's retired `dataDir` for both) and answers **503 with
- `retry-after: 15`** when the stat fails on a path whose `lstat` is a symlink — a tiered file whose
- drive is away — never 404. `deleteVideoFileAction` refuses such a file rather than unlink the link
- alone.
+- **The file route picks the drive by where the file is**: a link (`lstat`) with an `isTierable` name
+ → `mediaDir`; anything else, `source-media.*` included, the corpus disk (a legacy channel's retired
+ `dataDir` for both). It answers **503 with `retry-after: 15`** when the stat of such a link fails AND
+ the channel has `mediaDir` — a tiered file whose drive is away; a dangling link on an in-place
+ `media/` is a missing file, 404. `deleteVideoFileAction` refuses the first (unlinking alone would
+ orphan the bytes) and removes the second.
- **The live-chat archive (export build) reads no raw replay while the channel's media is not
`ok`/`in-place`**: it stages the cues on disk as they are (`isLiveChatCuesFresh` `lstat`s the link)
- and logs how many are older than their replay.
+ and logs how many are older than their replay and how many videos with a raw replay and no cues it
+ left out; it never drops the channel.
- **`noCorpusWalkInRenderPaths.test.ts` greps render-path files for the walkers' NAMES**, comments
included: naming `measureTree` in a comment of a file a page imports fails it.
diff --git a/plans/release-17.md b/plans/release-17.md
@@ -1742,10 +1742,11 @@ in §2, the `onDrive`-by-file-kind editor changes in §4, and the T2 row; T1's "
delete still ask every writer.
- **Locations key on `mediaDir`** (a legacy channel on its retired `dataDir`): `channelsOnLocation`
(new `textBytes`, `unknownTextBytes`, `legacy`), `channelVolumeOf(config, …)`, the storage watch (one
- auto-pause per channel, as before), the re-point (rewrites the `media` link and `mediaDir`; a legacy
- channel on the location is refused naming `migrate-tier`), the rename (a convention-shaped
- `<root>/<slug>/media` moves and the `media` link is re-pointed; the per-file links are relative and
- move with the channel dir), `deleteChannel` (removes `mediaDir`, and a legacy channel's `dataDir`).
+ auto-pause per channel, as before), the re-point (rewrites the `media` link and `mediaDir`; since the
+ review a legacy channel on the same location is re-pointed the retired way — see "Review"), the
+ rename (a convention-shaped `<root>/<slug>/media` moves and the `media` link is re-pointed; the
+ per-file links are relative and move with the channel dir; since the review a legacy channel's rename
+ is refused), `deleteChannel` (removes `mediaDir`, and a legacy channel's `dataDir`).
- **`/storage`**: a location row's figure is its channels' media tier, with no clips; the internal row is
in-place media + every channel's text + every channel's clip windows (`internal.corpus`, folded in
`buildStorage.ts`), with the breakdown "text N + clips N on the corpus volume, plus N media of in-place
@@ -1868,4 +1869,37 @@ No helper was added to T1's `mediaTier-server.ts` or `channelMedia.ts`.
- For T3: the migration writes `mediaDir` and unsets `dataDir`; everything in this slice reads a channel
with `mediaDir` and no `dataDir` as relocated on the new layout, and a channel with both as legacy.
+#### Review (SHIP AFTER FIXES) and the fixes
+
+The review (`$T/T2-review.md`) found one HIGH, three LOWs and five NITs. Rulings: the live-chat archive
+publishes the cues on disk while a channel's media is away and never drops the channel; legacy channels
+stay placed by their retired `dataDir`; the digest job stands for the digest lane in e2e.
+
+| Finding | Fix |
+|---|---|
+| H1 a legacy channel refused the re-point of its whole location | `22bdf3d7`: the preflight lists a legacy channel (its `data/` a link) with the others and `RepointPreflight.legacy` names it; the job re-points it the retired way in the same ledger and rollback — its `data` link and `dataDir` to `relocatedDataDir(newRoot, slug)` after checking that tree exists — so `dataDir` stays `<root>/<slug>/data` for T3. A recorded `dataDir` over a real `data/` is refused by name. Tests: one legacy and one `mediaDir` channel on one location, both re-pointed; a legacy tree missing under the new root is named and nothing moves. |
+| L2 a legacy channel's rename left `<root>/<oldSlug>/data` | `b9de2318`: refused before anything moves, with the `migrate-tier` sentence. Test. |
+| L3 Clear marker removed a tier-migration marker | `0e53239a`: `tierMigrationRefusal` is exported and Clear marker, Resume/Reconcile, the preview and the job give its one sentence. |
+| L4 videos with a raw replay and no cues dropped silently | `ec3a7e59`: counted in the build log ("N video(s) with a raw replay and no cues left out of this build"); the channel is never dropped. |
+| N5 the job tiered before checking the destination's identity | `0e53239a`: `assertRelocationRootPresent` runs before the preflight tier (and again before the mkdir). |
+| N6 legacy channels counted "unmeasured" on the corpus row | `250c02fa`: left out of the corpus volume's sums and named "legacy (n to migrate)" in its breakdown. Test. |
+| N7 the drive was chosen by `classifyEntry === "media"` | `7069e5b8`: a file is on the media tier only when its `lstat` is a link and its name `isTierable` (`source-media.*` never is), in the file route, the file delete and the directory loader. |
+| N8 a dangling link on an in-place `media/` answered 503 forever | `7069e5b8`: 503 only when the channel's media is relocated (`mediaDir`); otherwise 404, and the file delete removes the dangling link. |
+| N9 the preview did not refuse a tier-migration marker | `0e53239a`: it does, and tiers nothing. Test. |
+
+**Gates after the fixes** (logs `$T/T2-regate.log`, `$T/T2-e2e-4.log`, `$T/T2-regate2.log`,
+`$T/T2-e2e-5.log`):
+- Before the merge, at `250c02fa`: tsc clean; the four mover/storage test files (`relocateChannelMedia`,
+ `storageLocations`, `renameChannel`, `storageWatch`) 88/88; common 2,613 passed, 0 failed, 0 skipped;
+ editor unit 109/109; e2e (`channel-storage`, `storage-locations`, `channel-rename`, `video-page`,
+ `$T/T2-specs-review.txt`) **46 passed, 0 failed, 3.5 min**.
+- **Merge of `main` `ce9ec612`** (slice RL and a plans commit) at `a001f17e`: two conflicts, both
+ appends — `editor/CHANGELOG.md` keeps every `[Unreleased]` bullet, this file keeps U1, U2, XP, D0, T1,
+ RL, then T2 under `## Record`. On the merged tree: tsc clean; **common 2,653 passed, 0 failed, 0
+ skipped** (2,653); editor unit 109/109; e2e (the same four specs) **46 passed, 0 failed, 3.3 min**.
+- Not re-run: the capped build and test:scripts (no package, route or script surface changed by the
+ fixes; tsc covers the editor's types). Privacy: 0 added lines carry the user or host name.
+- From this point a commit's `Co-Authored-By` names the model that wrote it (Opus); the earlier
+ commits are not rewritten.
+
## Rollout
