commit 52bdc26c0e9bc4c326dbe4feaa6984b198a1e40e
parent 5163aefc675e52ff1bf2d7994414305726771d17
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Fri, 25 Sep 2026 20:46:23 -0400
plans: release 9 slice C2 — review fixes (5163aefc) and re-gate (e2e:hub 17/17); follow-ups named; CHANGELOG
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Diffstat:
2 files changed, 33 insertions(+), 19 deletions(-)
diff --git a/editor/CHANGELOG.md b/editor/CHANGELOG.md
@@ -1,7 +1,7 @@
# Changelog
## [Unreleased]
-- **The hub's search shows each archive's state, lets you choose which archives to search, and no longer waits for the slowest.** Under the line "Searching N archives …" is a row of chips, one per archive on the hub (official and added). Each says whether that archive is loading, how many videos it has in the search once it is in, or that it failed, with a Retry beside it. Pressing a chip takes that archive out of the search: nothing more is fetched from it and its results disappear. The choice is kept in this browser, and an archive it has never seen is searched. The search now runs as soon as one archive has loaded and runs again as each further one arrives, where it used to wait for all of them. When an archive does not answer, a line above the results says so ("4 of 5 archives answered. Hasanalyzer did not, so its videos are not in these results.") with a Retry, and the other archives' results show as usual; it used to contribute nothing, silently. Each result names its archive in text before the channel ("Jeralyzer · TheQuartering · 2026-09-25"). The hub now loads at most six summaries pages at a time from each archive, so a large archive does not hold up the small ones. `/ask` on the hub is unchanged: it searches every archive and waits for all of them. Published sites are unchanged. Needs a rebuild and deploy of the hub.
+- **The hub's search shows each archive's state, lets you choose which archives to search, and no longer waits for the slowest.** Under the line "Searching N archives …" is a row of chips, one per archive on the hub (official and added). Each says whether that archive is loading, how many videos it has in the search once it is in, or that it failed, with a Retry beside it. Pressing a chip takes that archive out of the search: nothing more is fetched from it and its results disappear. The choice is kept in this browser, and an archive it has never seen is searched. The search now runs as soon as one archive has loaded and runs again as each further one arrives, where it used to wait for all of them. When an archive does not answer, a line above the results says so once the others have loaded ("4 of 5 archives answered. Hasanalyzer did not, so its videos are not in these results.") with a Retry, and the other archives' results show as usual. None of that archive's videos, posts or live chat are searched until a Retry succeeds; before, it dropped out silently or in part. Each result names its archive in text before the channel ("Jeralyzer · TheQuartering · 2026-09-25"). The hub now loads at most six summaries pages at a time from each archive, so a large archive does not hold up the small ones. The line under the archives now counts "videos", as the chips do. `/ask` on the hub is unchanged: it searches every archive and waits for all of them. Published sites are unchanged. Needs a rebuild and deploy of the hub.
- **The hub and the homepage drop their subtitles, and both carry a Ko-fi link.** The headings are in Title Case on both pages: **Official Instances**, **Archives You Added** (hub) and **What It Does** (homepage). The line "The archives I run. Anyone can run their own." is gone from both, and the official-instance cards no longer show the site's description under its name; the four figures stay. The footer of the homepage and of the hub has a plain link reading "Ko-fi" to `https://ko-fi.com/archilyzer`. A published site's footer does not: an archive someone else hosts never carries it. Needs a rebuild and deploy of the hub and the homepage.
- **A job that waited in a queue no longer ends `failed` after doing its work.** A sync, download or other job queued behind another on the same platform ran its final page refresh outside any request, where Next refuses it, so the job read `failed` and `pnpm ops … --wait` exited 1 even though the work was done (the teamrcn sync on 2026-09-25). The refresh is now skipped there with one warning in the server log; the pages re-read disk on their next load anyway.
- **Downloads no longer sleep after a video the download filter declined.** The "sleep between downloads" (30 s by default) ran after every video, including each one the channel's download filter declined before fetching anything. A filtered channel's download-missing slept 193 times for 14 downloads on 2026-09-25. It still sleeps after every real fetch and after every failure, per-video ones included.
diff --git a/plans/release-9.md b/plans/release-9.md
@@ -475,19 +475,22 @@ and pages are fetched per archive behind its manifest. Scoping: `$T/r9-hub-scope
gains `enabled` (default true): false means none of that archive's feeds are fetched (`enabled: false`
on its manifest, subs, posts and alias queries; no page descriptors) and nothing already cached from it
is merged — TanStack returns cached `data` for a disabled query, so every merge checks scope itself.
-Per archive, its state is derived from its manifest query and its page queries: `ready` (manifest in
-and every page in), `failed` (manifest or a page errored and nothing of it is fetching), `loading`
+Per archive, its state is derived from its manifest, page, subs and posts queries: `ready` (manifest
+in, every page in, and its subs + posts manifests settled — a posts 404 is an empty manifest, a subs
+error counts as settled), `failed` (manifest or a page errored and nothing of it is fetching), `loading`
(otherwise, including while a Retry is in flight), `off`. `SearchDataValue` gains two OPTIONAL fields,
set only by the multi-site provider: `federation: {sites: FederatedSiteState[], retry(origin)}`
(`{origin, siteTitle, accent?, status, count?, error?}`, `count` = the manifest's `totalCount`) and
-`siteTitleOf(origin)`. `retry` is `refetchQueries` over every query whose key's second element is that
-origin and whose status is `error`. `summariesState.error` is no longer always null: it is set when
+`siteTitleOf(origin)`. `retry` is `refetchQueries` over this provider's five feeds (`manifest`, `summaries-page`,
+`subs-manifest`, `posts-manifest`, `search-aliases`) of that origin whose status is `error`. `summariesState.error` is no longer always null: it is set when
EVERY in-scope archive failed (read by `/ask`'s corpus-error line). `SingleSiteDataProvider` is not
touched.
- **Merge.** An archive's records join the merged list only when it is `ready`, so the list grows one
whole archive at a time; the merged list's identity is keyed on the ready set alone, so the search
session (which re-runs its pipeline on every new `summaries` identity) re-runs once per archive, not
- per page or per manifest. A failed archive contributes nothing (before: whatever pages had loaded).
+ per page or per manifest. The subs, posts and alias merges are keyed on the same ready set, so every contribution of an
+archive lands in one re-run and a failed archive contributes nothing — no videos, posts or chat
+(before: whatever pages had loaded, and its posts regardless).
The synthetic merged manifest (HubStats' transcript figure) covers the ready archives. Channel groups
and channels cover the in-scope archives whose manifest has arrived.
- **Progressive.** A new `progressive` prop: `summariesReady` turns true at the first ready in-scope
@@ -498,7 +501,8 @@ touched.
- **Speed.** Pages are requested per archive only once its manifest has settled and only while it is in
scope (as before for the manifest gate; new for scope), and each page fetch goes through a per-origin
gate of **6 in flight** (`PAGE_FETCHES_PER_SITE`), so a big member cannot take the browser's
- connections from the small ones. No new build-time data.
+ connections from the small ones. The page `queryFn` consumes TanStack's `signal`, so switching an
+ archive off mid-load cancels its queries and a page still waiting at the gate bails unfetched. No new build-time data.
**The page** (`export/app/components/hub/`). `useHubScope.ts` keeps the set of origins switched OFF in
`localStorage["ytdlp-tb:hub-scope"]` (`{off: [...]}`, every access in try/catch), so an archive the
@@ -506,21 +510,26 @@ browser has not seen — a new official instance, an added one — is searched.
of chips under the live line, above the query builder: one per archive on the page (official and
added), `data-testid="hub-scope-chip-<origin>"` + `data-status`, a toggle button (`aria-pressed`, the
archive's title and `loading…` / its record count / `failed` / `off`) and, when failed, a `Retry
-<title>` button. `HubStats` counts the archives in scope. `common/components/SearchResults.tsx`: one line
-above the results when an in-scope archive failed, `data-testid="hub-scope-status"`: "N of M archives
-answered. X did not, so its videos are not in these results. Retry" — nothing while every archive is
-fine or still loading (the chips show loading). A result card's meta line reads
+<title>` button. `HubStats` counts the archives in scope and says "videos" (the chips' word for the same figure; was
+"transcripts"). Every chip button and the line's Retry carry a literal `focus-visible:outline-2
+focus-visible:-outline-offset-2 focus-visible:outline-ring` (the chip's `overflow-hidden` clipped the
+default outline). `common/components/SearchResults.tsx`: one line
+above the results when an in-scope archive failed, `role="status"`, `data-testid="hub-scope-status"`:
+"N of M archives answered. X did not, so its videos are not in these results. Retry" ("archive" when
+M = 1) — only once NOTHING in scope is loading, so M = ready + failed and every archive not counted is
+named; nothing while every archive is fine (the chips show loading). A result card's meta line reads
`<archive> · <channel> · <date>` (`data-testid="result-source"`, from `ResultGroup.source`, set via
`siteTitleOf` in `SearchSessionContext`); single-site groups carry no `source` key. The accent stripe
and the `data-result-slug` contract are unchanged.
-**Specs.** New `export/e2e-hub/federated-search.spec.ts` (+4), two official members route-mocked with
+**Specs.** New `export/e2e-hub/federated-search.spec.ts` (+5), two official members route-mocked with
CORS: (1) both searched by default, both chips `ready`, each card names its archive, no status line;
(2) toggling Origin B off removes its card and after a reload it stays off with **zero requests** to its
origin, and back on it is fetched and returns; (3) Origin B's pages aborted → chip `failed` with `Retry
Origin B`, "1 of 2 archives answered." / "Origin B did not", Origin A's card renders; fixed + the line's
Retry → chip `ready`, line gone, both cards; (4) Origin B's page held → Origin A's search result renders
-while B is `loading`, and the same search covers B once it is released. `federation.spec.ts`: "Archives
+while B is `loading`, and the same search covers B once it is released. (5, review) three members, B failed and C
+held → no line; C released → "2 of 3 archives answered." naming B. `federation.spec.ts`: "Archives
You Added" exact (C1b's Title Case). Every existing hub test id / accessible name is unchanged.
| sha | what |
@@ -533,6 +542,7 @@ You Added" exact (C1b's Title Case). Every existing hub test id / accessible nam
| `6a69c034` | `federation.spec.ts`: "Archives You Added", exact |
| `a0bd7b02` | a chip's title says what its number counts and what pressing it does |
| `40818bbb` | the merged list's identity keys on the ready set alone (no re-run when another manifest lands) |
+| `d54e6814` | review fixes: ready needs subs + posts settled and their merges key on the ready set; the answered line waits for every archive (+ `role="status"`, singular); focus rings; page `signal`; "videos"; Retry scoped to the five feeds; spec +1 |
**Gates.** tsc clean (per commit; full workspace on `9d20d3fc`+ and on `40818bbb`). Common tests
**1,845/1,845** (no new unit tests: the provider is a hook, covered by the e2e). test:scripts **162 passed
@@ -541,7 +551,9 @@ corpus: "0 built-in pool site(s) …; hub-summary.json skipped") + `INSTANCE_MOD
32 s. e2e, queued, all on the merged tree: **`e2e:hub` 15 passed** (pre-merge, 48.9 s), **16 passed**
(merged, 53.2 s), **16 passed** (`40818bbb`, 47.0 s); **export full suite 195 passed**, 8.5 min (single
site unchanged); **`e2e:2origin`** (`TWO_ORIGIN_REBUILD=1`) **3 passed**, 46.9 s on `a0bd7b02`, and **3 passed**,
-1.0 min on `40818bbb`. The seven compose-hub outputs + `sw.js` in the worktree's `export/public` were
+1.0 min on `40818bbb`.
+
+**Review (SHIP AFTER FIXES) re-gate** on `d54e6814`: tsc clean; **`e2e:hub` 17 passed** (16 + 1), 52.9 s. The seven compose-hub outputs + `sw.js` in the worktree's `export/public` were
swapped for copies before every build/e2e and relinked after; the primary's copies were not written by
this slice (one change seen, `hub-summary.json` at 20:18:20, is the parent's build-hub, which wrote the
other six at 20:17:53). Editor build/unit and mcp not run: nothing they build changed. **Numbers tools:
@@ -557,15 +569,17 @@ with Retry, 72,360 records from the other four listed. Checked at 1280 dark and
- The chip's number is the member's summaries `totalCount` — its record (video) count, the number the
results header counts ("All videos (N)") — not the card's "transcripts" figure, which is the
homepage's build-time `transcribed.total`. The chip's title says "N videos from X are in the search";
- the live line still says "transcripts" (C1's wording, left).
+ the live line now says "videos" too.
- A member whose manifest loaded but whose pages failed keeps its channel group in the filter panel
(the channels are known); its records are not searched. Hiding the group on failure would churn the
groups on every Retry.
-- The `/ask` hub has no scope chips and searches every archive on the page, as before; the scope is
- the front page's. It waits for every archive to settle (not progressive), so a chat never grounds in
+- **Follow-up:** the `/ask` hub has no scope chips and searches every archive on the page, as before;
+ the scope is the front page's. Honouring it is `useHubScope().isOn` in `AskHub`. It waits for every archive to settle (not progressive), so a chat never grounds in
a half-loaded federation silently.
-- Official cards have no `accent` (C1: they wear `seriesColor` on the page), so result stripes and chip
- dots appear only for an archive that sets one — the text attribution is what names the source.
+- **Follow-up:** official cards have no `accent` (C1: they wear `seriesColor` on the page), so result
+ stripes and chip dots appear only for an archive that sets one — the text attribution is what names
+ the source. The fix is one shared fallback (a `shelfAccent(site, summary, i)` out of `ArchiveShelf`)
+ used by HubHome and AskHub too.
- `route()` does not see requests a service worker makes; a manual check of the built hub needs
`serviceWorkers: "block"`. The e2e-hub suite runs `next dev`, where no SW is registered.
