commit 52858224b72bd9eede5b47a6f89ba2a0624cb062
parent 65f6e7d3d424374479cacb33a045be9c5fb8c027
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Mon, 28 Sep 2026 14:37:47 -0400
plans: slice R's review record — M1 and the Lows fixed, the five questions as ruled, the gates re-run; source-mirror.md's "As shipped" for R
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Diffstat:
2 files changed, 193 insertions(+), 0 deletions(-)
diff --git a/plans/release-12.md b/plans/release-12.md
@@ -550,4 +550,158 @@ doctor's home-directory paths.
3. The worktree's `homepage/public/source`, `homepage/out` and `homepage/.source-publish.json` were
made with the scratch rule. They are disposable, and nothing here deployed them.
+**Review** (verdict **SHIP AFTER FIXES**; a read-only Opus review of `e6c5d2e3..6ddfd498`,
+`$T/r-review.md`). The parent then added the missing scrub rule to the operator file:
+`source publish --check` with the real files exits 0. The coordinator ruled that M1 and the
+listed Lows be fixed on the branch. `main` had moved by one `plans:` commit (the release 11
+rollout record), merged at `439eb106` with no conflict.
+- **M1 — fixed** (`7d64054c`, and `78b32636` below). **A refusal left the previous publish in
+ place**, in `homepage/public` and in the last build's `homepage/out`, for a deploy-only or a
+ raw `next build` to ship under rules it was never audited against. Now it is withdrawn at three
+ points:
+ 1. **`publishSource`:** once the rules are loaded, any outcome but success — an audit hit, a
+ limit, a missing tool, a cancel, a crash — runs `removePublishedSource`. That removes the
+ manifest first, then the mirror, the tree, the tarball, `snapshot.json` and the skip key.
+ `--check` still writes nothing, this included: it is a dry run, and the deploy check below
+ covers what it cannot.
+ 2. **`buildHomepage`:** a non-zero source result also removes `out/source` and the two download
+ files from `homepage/out`.
+ 3. **`deployHomepage` asks `publishedSourceProblem` before every deploy**, preview included.
+ - An `out/` with source artefacts ships only when the skip key says the publish was made
+ under TODAY's rules and step (`rulesHash`), of TODAY's `main`, and is the publish in `out/`:
+ the same mirror head, and a tarball whose sha256 matches.
+ - It keys on the artefacts and the `/source` page, not on `out/source` existing (`78b32636`).
+ A `--no-source` build still renders the page into `out/source/index.html`, and the first
+ version refused exactly that build; I found it by running one.
+ - The page with no artefacts beside it is a `--no-source` build, and deploys as before.
+ - No page at all is a refused build (or one from before the page), and it refuses.
+ - **Tests:** the round trip denies a literal the mirror holds, and the publish then exits 1 with
+ `public/source`, the tarball, `snapshot.json` and the key gone. The skip test covers a changed
+ rule. `build.test` covers the `out/` removal and the deploy refusals. `publishedSourceProblem`
+ is covered on a real publish: ok, tampered tarball, newer `main`, other rules, no record.
+- **Q3/L5 — fixed** (`6936f6e2`). A label says where the literal was written: `denylist line 3
+ (len 5)`, `scrub line 2 lhs (len 11)`, `built-in home rule (len 11)`. No character of the
+ literal appears.
+- **L4 — fixed** (`6936f6e2`).
+ - A hit is listed by kind, object id (with the blob's path in history), byte offset, and for a
+ commit or tag the header field (`author`, `committer`, `tagger`, …) or `message`; a tree hit
+ by entry number. `redactHit` is deleted: no byte of an object is printed.
+ - The test asserts the report carries none of the planted text's neighbours (`/srv/`,
+ `example.invalid`, `Planted <`, the message).
+ - PUBLISH.md and the editor changelog show the new format.
+- **L1 — fixed** (`6936f6e2`): a tracked `404.html` anywhere in the tree is refused, with a test.
+- **L2 — fixed** (`7d64054c`).
+ - A UTF-8 BOM and CRLF are dropped from both operator files (`operatorLines`), and a trailing
+ `/` from the home dir.
+ - An empty left side (`==>x`, `literal:==>x`, `regex:==>x`, `glob:`) is a refusal naming its
+ line.
+ - The reviewer's reproduction is a test: a BOM + CRLF scrub file still scrubs, and still
+ denies its left side. The round trip now runs with a BOM + CRLF scrub file and denylist.
+- **L3 — fixed** (`7d64054c`): the published manifest (and `SourceManifest`) no longer carries
+ `audit.literals`. The log still gives the count.
+- **L6 — left, as ruled:** compressed content is opaque to the byte search. The reviewer
+ decompressed every compressed or binary blob in the history (a zip, PNGs, icons) and found 0
+ hits. PUBLISH.md records it as a known limit, and says a binary is audited, never scrubbed.
+- **L7 — fixed** (`d3c680ea`). `.dockerignore` excludes `homepage/public/source/` and
+ `homepage/.source-publish.json`, and no longer names `create-archives.sh`.
+- **L8 — fixed** (`7d64054c`), for a checkout with no git repository.
+ - `buildHomepage` passes `noRepository: "empty"`. The step logs `[source] no git repository
+ here; nothing to mirror — the /source page will show its empty state`, removes an old publish,
+ and returns 0.
+ - `archilyzer source publish` exits 1 with the same sentence, and no raw git error.
+ - A test runs both over a temp dir with no operator files.
+- **L9 — fixed** (`7d64054c`, `91728a21`).
+ - The round trip reads EVERY object in the published packs from a plain file copy (`cat-file
+ --batch-all-objects`), and asserts `objects/pack` holds only `pack-*.{pack,idx}`.
+ - `homepage/app/lib/headers.test.ts` pins `_headers` with a replica of wrangler's parse and
+ attach:
+ - each `/source/tree` override says `! Content-Type` first;
+ - each path gets ONE value;
+ - the file stays inside wrangler's limits.
+ - `E2E_EXPECT_SOURCE=1` makes the empty state FAIL `source.spec.ts`' three data tests. It is
+ declared in the homepage playwright config and the env registry. It bites: in the empty state
+ with the flag set, **3 failed, 2 passed**.
+- **L10 — fixed** (`7d64054c`, `91728a21`). `parseSourceManifest` checks every number the page
+ reads, and `loadSourceManifest` takes the public dir as a seam. Unit tests cover common (2) and
+ homepage (2): a malformed, wrong-version or orphaned manifest is null, never a throw.
+- **L11 — fixed** (`7d64054c`).
+ - `SOURCE_STEP_VERSION` (now 2, with a comment to bump it whenever the scrub or the audit
+ changes) is in the rules hash.
+ - The skip key also holds the filter-repo label and version, and the mirror head.
+ - A test shows another filter-repo version does not skip.
+- **L12 — fixed** (`7d64054c`).
+ - A scratch root that lands (through symlinks) inside the checkout or the public dir is refused.
+ - `--keep-scratch` deletes `replace.txt` (written mode 600) and says so.
+ - Tests cover both.
+- **L13 — fixed** (`d3c680ea`).
+ - The `project.ts` comment names the mirror.
+ - PUBLISH.md's tsconfig line says `public` and `out`. The "new spelling" line now says the
+ implied denial is the exact bytes, and a new spelling is caught only by the denylist.
+ - The editor's /sites Homepage section gains one `<p>`: Build also publishes the source and can
+ refuse; a refusal removes it from `homepage/out` too; Deploy refuses an unaudited source.
+ I did not run the editor e2e for it:
+ - `sites-homepage.spec.ts`' assertions on that group are `toContainText` substrings of the
+ existing paragraph, role-and-name lookups for buttons, and `getByText(/^Queued/)` and
+ `"Cancelled"` (exact). A new sibling paragraph cannot break any of them.
+ - Its build job is held on the queue and cancelled, so the source step never runs there.
+- **The runbook point (PUBLISH.md, `d3c680ea`):**
+ - a Pages preview is public, and every deployment stays reachable at its hash URL until it is
+ deleted;
+ - the private literals go in the denylist before ANY deploy;
+ - if something private ships, delete that deployment, because a newer deploy does not remove it;
+ - the editor's process needs `~/.local/bin` on its PATH.
+- **The five questions, as ruled:**
+ 1. The rules hash in the gitignored `.source-publish.json` is **acceptable**. The step version
+ has been added (L11).
+ 2. `--no-source` removing the previous publish is **right**.
+ 3. The first character is **dropped** (above).
+ 4. The loose `refs/heads/main` is **acceptable**.
+ 5. `.dockerignore` is **fixed here** (L7).
+
+| sha | what |
+|---|---|
+| `439eb106` | merge `main` (the release 11 rollout record) |
+| `6936f6e2` | `common:` labels by provenance, no object bytes in the report, `404.html` refused (Q3, L4, L1) |
+| `7d64054c` | `common:` a refusal withdraws; the build's `out/` copy; the deploy check; L2, L3, L8, L10, L11, L12 |
+| `91728a21` | `homepage:` `E2E_EXPECT_SOURCE`; the loader and `_headers` unit tests (L9, L10) |
+| `d3c680ea` | `docs:` PUBLISH.md (withdrawal, previews, no-repo, report format, L6); `.dockerignore`; `project.ts`; the /sites copy; the changelog bullet |
+| `78b32636` | `common:` the deploy check keys on the artefacts and the `/source` page (found by running `--no-source` against it) |
+| _this_ | `plans:` this review record and `source-mirror.md`'s "As shipped" note for R |
+
+**Gates after the fixes** (logs `$T/r-m-*.log`):
+- **tsc:** clean before every commit (36–55 s).
+- **Unit:**
+ - common **2,146/2,146** (+8: sourceAudit +2, source +3, build +1, sourceManifest +2). The
+ filter-repo tests ran, 0 skipped.
+ - homepage unit **7/7** (+5); editor unit **85/85**; `test:scripts` **185 + 1 skip**.
+ - `docs env --check` exits **0**.
+- **`source publish --check` with the REAL files** exits **0** in 19 s:
+ - `audit clean: 21,447 objects (1,703 commits), 2,459 staged files against 6 denied literals;
+ gitleaks clean`;
+ - it would publish main `e56101fdee5d` as `20c367613f75`;
+ - a count-only grep of the log for `$(id -un)` gives **0**.
+- **A real `build homepage` in the worktree, with the real files:**
+ - **ok in 38 s** (the source step 19 s); `homepage/out` holds **2,640 files, 77.9 MB**;
+ - the mirror is 37.9 MB in 2 packs, the largest 20,966,235 bytes (19.99 MiB, against the 24 MiB
+ limit); the tree is 2,035 files and 412 dirs; the tarball 7,249,703 bytes.
+- **Dumb-HTTP clone** from `python3 -m http.server 8765 --bind 127.0.0.1` in `homepage/out`:
+ - HEAD `20c367613f75…` = `manifest.mirrorHead`;
+ - the user-name grep over its 1,703 revisions gives **0**;
+ - `:8765` was free before, the server was killed, and it was free after.
+- **The refusal, exercised through `build homepage`** (`SOURCE_DENYLIST_FILE` = a scratch file of
+ `Co-Authored-By`):
+ - exit **1** in 13 s: `1477 hits … denylist line 1 (len 14): 8 in blobs, 1469 in commits`, with
+ each listed as `commit <id> (message, byte N)`;
+ - the log holds the literal **0** times, and the source step's lines hold 0 of the 6 real
+ literals;
+ - afterwards `public/source`, the public tarball, the skip key, `out/source` and the out
+ tarball are **all gone**, and `out/downloads/index.html` stays;
+ - a good rebuild restored them all, and the deploy check then says "ok".
+ - Before the refusal, the deploy check over the real `out/` said ok under the real rules, and
+ "audited under other rules" with the scratch denylist.
+- **`build homepage --no-source`:** `out/source` holds only the page. The tarball is gone, and the
+ deploy check says ok.
+- **homepage e2e with `E2E_EXPECT_SOURCE=1`:** **36 passed**, 0 skipped, 51 s, with the manifest
+ present. The bite run is above: 3 failed and 2 passed in the empty state.
+
## Rollout
diff --git a/plans/source-mirror.md b/plans/source-mirror.md
@@ -419,6 +419,45 @@ empty state; `PATH` without pipx → the install-line refusal; `archilyzer docto
block; homepage e2e 31 → 36 with the manifest present, `downloads.spec.ts` unchanged. Record the
resolved filter-repo command and version.
+**As shipped (2026-09-28, `r12/source-mirror`; record: `release-12.md`, "Slice R, as shipped" and
+its "Review"):**
+- **R1–R6 as written, with these corrections found by running them:**
+ - **`_headers`:** Pages APPENDS a header a later matching rule sets again. Each override now
+ detaches with `! Content-Type` first, and `homepage/app/lib/headers.test.ts` pins it.
+ - **The homepage tsconfig excludes `out` as well as `public`,** and its eslint config ignores
+ `public/source/**`.
+ - **The mirror carries a loose `refs/heads/main`** beside `packed-refs`, because git needs a
+ `refs/` directory before a `file://` clone or `source audit` will read it.
+ - **filter-repo reads `#` lines as literals,** so `replace.txt` is written without them.
+ - **The clone takes `--no-tags`,** and filter-repo `--replace-refs delete-no-add --quiet`.
+ - **`--no-source` REMOVES the previous publish.**
+ - **The header nav moves to `md`.**
+- **No `rulesHash` and no literal count in the published manifest.**
+ - The skip key is `homepage/.source-publish.json` (gitignored and dockerignored):
+ `{sourceCommit, mirrorHead, rulesHash, filterRepo}`.
+ - `rulesHash` covers the rules, the literals and `SOURCE_STEP_VERSION`. Bump it whenever the
+ scrub or the audit changes.
+- **After the review (M1): a refusal withdraws the source everywhere it could ship from.**
+ - The step removes the last publish from `homepage/public` (not under `--check`).
+ - `buildHomepage` removes it from `homepage/out`.
+ - `deployHomepage` refuses an `out/` whose source was not audited under today's rules, of
+ today's `main`, or that has no `/source` page at all.
+- **The audit report names a literal by where it was written** (`denylist line 3 (len 5)`) and a
+ hit by object, field and byte offset. It prints no byte of any object.
+- **The review's other fixes:**
+ - a tracked `404.html` is refused;
+ - a BOM, CRLF and a trailing `/` on the home dir are handled, and an empty left side refuses;
+ - a checkout with no git repository builds with the `/source` empty state (and the CLI refuses
+ with the same sentence);
+ - a scratch root inside the checkout or the public dir is refused, and `--keep-scratch` deletes
+ `replace.txt`;
+ - `E2E_EXPECT_SOURCE=1` makes the empty state fail the spec;
+ - a malformed manifest is the empty state, never a crash.
+- **Left:** compressed content is opaque to the byte search (review L6; a known limit in
+ PUBLISH.md).
+- **The baselines moved:** common 2,114 → **2,146**; homepage unit 2 → **7**; homepage e2e 31 →
+ **36**.
+
## Rollout (parent; nothing here edits the primary's tracked files)
Release 11 is merged and NOT rolled out. A production homepage deploy of release 12 also ships
