commit 531f72173b1e4958845f2ccd45cdd9f28d8707d4
parent 872c984c70320e396fa1e61c5e607ebc09cfb4f6
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Sat, 12 Sep 2026 11:43:27 -0400
plans: S2a's record — one reader per origin, and the offline gap closed
Sha range, every gate's number, and seven divergences with the reason each
one was made. The two worth reading before the next slice:
The caches did NOT adopt ArchiveReader's tolerant methods; the reader gained
raw document reads underneath them. Its interface has no method at all for
four documents the viewer reads, and where it does have one its answer to
absence is the opposite of what a browser needs — react-query retries a
rejected query and will never retry a resolved null. The tolerant methods are
rebased on the raw reads, so the MCP's behaviour and the bench's read counts
cannot move.
And the service workers needed more than a wider SHARD_RE. Downloading
/duplicates.json into the cache closes nothing while the fetch handler never
matches it, so both workers gained FLAT_RE and ROOT_RE answered network-first.
Writing those lists down surfaced three drifts: a dead `summaries` in the
per-channel matcher and in both eviction lists, `/posts/` and `/digests/`
missing from those lists, and no `digests` anywhere in sw-hub.
Also recorded: the one red editor spec is a subset-run artifact, verified by
re-running the same two spec files at the base commit and getting the same
failure; and what S2c still owes the widened hub worker (the _headers CORS
set).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Diffstat:
1 file changed, 155 insertions(+), 0 deletions(-)
diff --git a/plans/one-core-phase-2.md b/plans/one-core-phase-2.md
@@ -382,3 +382,158 @@ client chunk. The prerender is the only step that fails, and it fails at base.
`plans/tools/jeralyzer-corpus-2026-09-12.json` was not re-fetched; the live diff
is S3's gate, after the search pipeline lands.
+
+### S2a — shipped 2026-09-12
+
+Branch `one-core/phase-2-s2a`, off `7f86aef` (the
+`integrate/2026-09-storage-priority` tip, S1 merged). Four commits,
+three code commits `dc3aef1` → `2eb9c3b` plus this note, unmerged. **No URL shape moved, no `corpus.json` byte
+moved, no CONTRACT version moved, no architecture allow-list entry added or
+burned, and no `common/package.json` exports line added.**
+
+| commit | what |
+|---|---|
+| `dc3aef1` | the eight caches — plus `SearchDataContext`, `siteRegistry` and `DuplicatesClient` — onto one `RemoteSource` per origin; `lib/archive/readers.ts`; the reader's raw document reads |
+| `8ca7022` | `offlineCache` enumerates `ARCHIVE_TREES` + `ROOT_FILES`; both service workers widened; the SW and search-worker guard cases in `contract.test.ts` |
+| `2eb9c3b` | `archive/readers.test.ts` + `components/archiveCaches.test.ts` |
+| (this commit) | this note |
+
+#### Seven divergences from the plan above, each because the code said so
+
+**1. The caches do NOT adopt `ArchiveReader`'s tolerant methods. The reader
+gained a block of RAW DOCUMENT READS underneath them instead.** The slice was
+specified as `memo(originId, () => reader.X(...))`, and for transcripts that is
+exactly what it is. For everything else `ArchiveReader` is the wrong surface,
+twice over.
+
+It has no method at all for four of the documents the viewer reads:
+`/subs/manifest.json` and `/posts/manifest.json` (the SITE-level index of which
+channels ship a tree — a different document from a channel's own manifest), and
+the raw summaries/stats manifest-plus-pages. What it exposes for those is
+`videoIndex()` / `statsIndex()`, folded slug-keyed maps, which are the wrong
+shape for a UI that renders arrays and loads page by page under react-query.
+
+And where a method does exist, its answer to absence is the opposite of the
+viewer's. `subsManifest`/`postsManifest`/`digestsManifest` resolve `null` for
+anything that is not a 200 — right for a tool probing thirty channels, wrong
+for a browser, because **react-query retries a rejected query and will never
+retry a resolved `null`**. Folding both policies into one method is exactly how
+a transient blip becomes a panel that is empty for the rest of the session.
+
+So `RemoteSource` now has twelve one-to-three-line reads that do URL + fetch +
+parse and THROW, and every tolerance policy sits on top and picks its own
+answer. **The MCP's behaviour is unchanged by construction**: each tolerant
+method is rebased on the raw read it duplicated, and `buildVideoIndex` /
+`buildStatsIndex` already treat a throw from their readers as absence, so the
+degradation paths are the same code they were. This is not the `record(layer,
+slug, id)` the plan forbids — these are the DOCUMENTS of the walk, not records
+inside a shard, and no read count moves.
+
+`ArchiveHttpError` came with them, for the one caller that has to tell a 404
+from a dropped connection (the alias dictionary, whose query has `staleTime:
+Infinity`). Its `message` is byte-identical to the `Error` it replaced.
+
+**2. Two memos stay in `components/`, and neither is a leftover.** The
+subs/posts CHANNEL-manifest memos keep the throwing policy of (1). The posts
+and digest PAGE memos exist because the reader caches subs and transcript pages
+but not those two — and `fetchThread` walks every page of a channel, so without
+a memo opening two posts in one thread re-downloads the whole channel. Adding
+those caches to `RemoteSource` instead would have moved the bench's read counts,
+which is not a thing to do in a slice that claims the walk is all that changed.
+
+**3. `io-stats` blind spots are preserved deliberately.** The raw reads take an
+OPTIONAL `kind`, and the ones that pass none are exactly the reads that
+recorded nothing before (the summaries/stats/duplicates/alias folds used a bare
+`fetch`). Closing them would move the bench's structural counters for a reason
+unrelated to the walk. That is also why the bench was not re-run for this slice
+— nothing in the reader's semantics or read counts moved.
+
+**4. `contract.ts` needed `treeManifestUrl`, and `ARCHIVE_TREES` /
+`PER_CHANNEL_TREES` beside it.** `manifestUrl("subs")` throws with no slug, and
+correctly so — but `/subs/manifest.json` is a real published document. A
+per-channel tree has BOTH a root manifest and a per-channel one; `manifestUrl`
+builds the second, `treeManifestUrl` the first, and for a flat tree they are the
+same file. The two lists are what the offline cache and the SW guard test
+enumerate; `CONTRACT.layers` stays the frozen published list.
+
+**5. The service workers needed more than a wider `SHARD_RE`, and the plan's
+stated goal is why.** `SHARD_RE` matches three path segments, so
+`/duplicates.json` and `/summaries/*` never matched it and the fetch handler let
+them through to the network — downloading a root file into the cache and never
+serving it from there closes nothing. Both workers gain `FLAT_RE` and `ROOT_RE`
+answered NETWORK-FIRST (those documents have no per-channel `generatedAt` to
+evict against, so cache-first would be stale until the SW version moved).
+
+Three drifts surfaced while writing the lists down. `summaries` was in
+site-sw's `SHARD_RE` and in both eviction prefix lists: dead in both places,
+because it is a flat tree and `/summaries/<slug>/` does not exist. `/posts/` and
+`/digests/` were absent from both eviction lists, so a rebuilt channel kept
+serving its old posts and digests from cache. And sw-hub had no `digests` at
+all — the hub federated a layer it could never cache.
+
+**6. `offlineCache` keeps its own `cache: "no-store"` fetch** rather than taking
+the shared reader. The reader is a plain `fetch`, so it would be answered from
+the very service-worker cache that function exists to refill, and would compute
+its download list from the stale copy. What is shared is the thing that actually
+drifted: the URL shape.
+
+**7. One new `components/*.ts`, against the invariant, and why it is not the
+thing the invariant protects.** `components/archiveCaches.test.ts` is a test: it
+is imported by nothing, the `"./components/*"` catch-all maps to `.tsx`, and it
+needs no `package.json` exports line — which is the cost the invariant names. It
+cannot live under `lib/` because S3 adds `"components"` to `FORBIDDEN.lib` in
+the architecture test, and a test for the caches has to import them.
+
+Three walk sites beyond the eight caches moved too, all of them covered by the
+brief's "only the fetch walk moves": `SearchDataContext.tsx` (four URLs, state
+untouched), `siteRegistry.ts` (two), and `export/app/duplicates/
+DuplicatesClient.tsx`'s bare `fetch("/duplicates.json")`, which became a new
+`fetchDuplicateReport()` export on `duplicatesCache`. A repo-wide sweep for
+hand-written archive paths now returns only `searchIndex.worker.ts` — the
+deliberate copy, and it is pinned.
+
+#### Gates
+
+- `pnpm -r exec tsc --noEmit` — clean in all six packages, after every commit.
+- `pnpm --filter yt-dlp-transcript-common test` — **1095 passed / 0 failed**
+ (baseline 1077 at `7f86aef`, + 3 `contract.test.ts`, + 6 `readers.test.ts`,
+ + 9 `archiveCaches.test.ts`; none lost). Architecture test green, allow-list
+ untouched.
+- `pnpm --filter yt-dlp-transcript-mcp test` — **205 passed / 0 failed**.
+- `pnpm test:scripts` — **71 passed / 1 skipped**.
+- `pnpm --filter export exec next build` — **compiled successfully**, 11 static
+ pages. The only proof `reader-fs.ts` is unreachable from a client module, and
+ it now has eight more `components/` modules reaching `lib/archive/` to prove
+ it about. (The one warning is S1's pre-existing NFT trace.)
+- **compose-site byte-identity** over the committed fixture: `IDENTICAL modulo
+ the build clock` for both trees — 16 files under `public/`, 7 under `index/`.
+- **The mcp bench was NOT run**, per the brief — the reader's semantics and read
+ counts do not move (see divergence 3).
+- **e2e**, behind the queue lock from worktree #10 (ports 4001/4011/4010/4020):
+ export `e2e` **172 passed / 0 failed** (S1's baseline exactly), `e2e:hub`
+ **5 passed / 0 failed**. `e2e:2origin` was skipped: it is known-red on the
+ base for the hub `/ask` prerender, which S1 verified at `c7f7b90` and which
+ nothing here is in the path of.
+- The editor specs `export-search` + `export-player-platform-cache`: **20 passed
+ / 1 failed**, and the one failure is a SUBSET-RUN ARTIFACT, not a regression.
+ `export-search.spec.ts:612` opens `editor/test-settings.json` with `readJson`;
+ that file is gitignored and is only ever written by `writeSettings()`, i.e. by
+ an earlier spec in the full suite. A fresh worktree running just these two
+ files has never had one written, so the test dies on `ENOENT` before it
+ touches any code. **Confirmed by re-running the same two spec files at
+ `7f86aef` in the same worktree: the same spec fails the same way.** Worth
+ fixing independently — the spec should seed the file rather than assume a
+ predecessor left one — but it is not this slice's.
+
+#### What this slice deliberately leaves for S2c
+
+A hub caching a member's `/duplicates.json` or `/digests/*` still depends on
+those paths being in the composed `_headers` CORS set, which they are not.
+S2c's `_headers` commit is what makes the widened `sw-hub.js` reach them
+cross-origin; same-origin (the site SW) works today.
+
+The offline DOWNLOAD path has no e2e coverage and gains none here: the service
+worker registers in production builds only, so `pwa.spec` can assert the control
+renders but never that a download lands. The guard that the SW lists and the
+download list agree with the contract is `contract.test.ts`, verified by
+mutation (dropping `digests` from sw-hub's `SHARD_RE` turns it red).
