commit 2a6906654fee3611f697e56428a849c1916beb81
parent debce0f3c493e33d751ee89ecf21549a4cb38f8a
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Mon, 28 Sep 2026 13:42:53 -0400
plans: slice R's record — `archilyzer source publish` + /source, as shipped
What shipped (R1–R6), the corrections running it found (`_headers` appends,
the tsconfig must exclude `out`, no rules hash in the published manifest, a
loose refs/heads/main so the mirror is a git dir, filter-repo reads `#` lines
as literals), the operator files' refusal with its redacted report and the
one scrub rule that clears it, every R7 gate with its numbers (common 2,138,
homepage e2e 36/36 published and 16/16 empty, 2,640 files / 78.1 MB out,
largest 19.93 MiB, both clones = mirrorHead, the user-name grep 0), the commit
table, and what was found and left.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Diffstat:
| M | plans/release-12.md | | | 269 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
1 file changed, 269 insertions(+), 0 deletions(-)
diff --git a/plans/release-12.md b/plans/release-12.md
@@ -281,4 +281,273 @@ No High or Medium findings. The coordinator asked for two of the Lows to be fixe
- **After the fixes:** tsc is clean (41 s, all seven packages), and the grep gate is empty at the
new tip. Per the coordinator, e2e was not re-run for a type comment and a changelog line.
+### Slice R, as shipped — `archilyzer source publish` + `/source` (2026-09-28)
+
+Branch `r12/source-mirror` off `main` `e6c5d2e3` (slice Q merged), worktree
+`~/Projects/r12-source-mirror` (worktree #10: editor 4001, homepage e2e 4040), one Opus
+implementer. Plan: [`source-mirror.md`](source-mirror.md), "Slice R", R1–R7. Why: the operator
+asked for the repo on the project site, read-only, clonable from static files, with a gate that
+refuses to publish a denied literal. `main` did not move during the slice (`git merge main`:
+already up to date). Scratch files `r-m-*` in the job's `tmp`.
+
+**What shipped.**
+- **R1, `common/publish/source.ts`: `publishSource(opts)`** — 0 published / skipped / checked,
+ 1 refused or cancelled; a `SourceRefusal` is logged as `[source] REFUSED: …`. The steps are the
+ plan's 1–17:
+ - `git rev-parse` of the git COMMON dir's `refs/heads/main`;
+ - the two operator files, parsed; a missing one is a refusal naming it (`~/`-relative);
+ - the skip;
+ - `resolveFilterRepo()`: `git filter-repo`, else `pipx run --spec git-filter-repo==2.47.0`,
+ else the install line;
+ - `git clone --no-local --bare --single-branch --no-tags --branch main` into
+ `mkdtemp(<ARCHILYZER_SOURCE_SCRATCH>/archilyzer-source-)`, origin removed;
+ - filter-repo `--force --quiet --replace-refs delete-no-add --replace-text R --replace-message R`,
+ then `filter-repo/` deleted;
+ - `repack -a -d -q --max-pack-size=20m`, `prune-packed`, `pack-refs --all`, `update-server-info`.
+ It refuses on loose objects, no `P` line, any ref but `refs/heads/main`, or a HEAD that is not
+ main.
+ - the object gate; the tree (`git archive` → `tar -x` → pages); the tarball (`--format=tar.gz
+ -9 --prefix=archilyzer/`) and `snapshot.json` in the `Snapshot` shape;
+ - the mirror staged from an allowlist; the manifest staged; the file gate; the limits;
+ `--check` stops; the link-safe install (manifest removed first, written last); one summary line.
+
+ Every child goes through `runChildIntoLog` with `AbortSignal.any([signal, timeout])`, in an
+ environment with the `GIT_DIR`-family variables cleared. The one exception is the audit's
+ binary `cat-file` stream (a `spawn`, with the signal). `clearPublishedSource` is what
+ `--no-source` runs. `common/lib/sourceManifest.ts` is the leaf contract: `MIRROR_DIR`,
+ `CLONE_URL`, `TREE_HREF`, `TARBALL_HREF`, `SourceManifest` and `parseSourceManifest`.
+- **R2, `sourceAudit.ts`.**
+ - `parseDenylist` (`i:`, `#` comments, trimmed, deduped).
+ - `scanBuffer`, and `redactHit`: ±24 bytes, every byte any hit covers masked, a half-shown
+ occurrence included.
+ - `maskLiterals` for every path or line quoted from the repo.
+ - `auditObjects`: ONE `cat-file --batch-all-objects --unordered --batch` stream with a framing
+ parser — blobs and commits/tags whole, trees by entry NAME (not their binary ids).
+ - `auditFiles`: contents, paths, a `.gz` decompressed; a symlink is a refusal.
+ - `runGitleaks`: cwd = scratch, so no checkout's ignore file applies. 0 is clean, 3 is parsed
+ findings, anything else refuses; not on PATH is "skipped" with a WARNING.
+ - `auditBare`: blob hits get their path in history, and gitleaks runs only when the literal
+ audit is clean.
+ - `formatAuditReport`: `#n (x…, len L)`, counts per literal and kind, the first 20 contexts,
+ "… and N more", then the plan's closing line.
+- **R3, `sourceTree.ts`** (`hrefFor`, `escapeHtml`, `renderTreeIndex`, `writeTreeIndexes`, which
+ refuses a tracked `index.html` or a symlink), and the `_headers` block — with the correction
+ below.
+- **R4, the homepage.**
+ - `app/lib/source.ts` `loadSourceManifest()` (also needs `info/refs` and the tarball).
+ - Nav: Source after Docs.
+ - `app/source/page.tsx`: the plan's copy verbatim; `source-clone`, `source-mirror-head`,
+ `source-tree-link`, `source-tarball-link`, and `source-tarball-sha` for the spec; the empty
+ state `source-empty`.
+ - A shared `components/Fact.tsx`.
+ - Downloads: one sentence linking "read-only git mirror", and the empty state names
+ `archilyzer source publish`; every phrase `downloads.spec.ts` asserts is kept.
+ - `marketing.spec.ts`' nav list gains Source.
+- **R5, the docs.** `create-archives.sh` is deleted. README and SETUP lead with the clone and keep
+ the tarball as the no-git path. PUBLISH.md gains "The source mirror (homepage)". The homepage's
+ *Install*, FAQ, *What is Archilyzer* and `content/README.md` stop saying "there is no public
+ repository". `grep -rn 'create-archives' README.md SETUP.md homepage/` is empty.
+- **R6 and the rest.**
+ - `getPaths()` gains `configDir`, `sourceScrubFile`, `sourceDenylistFile` and
+ `sourceScratchDir`, from `ARCHILYZER_CONFIG_DIR`, `SOURCE_SCRUB_FILE`, `SOURCE_DENYLIST_FILE`
+ and `ARCHILYZER_SOURCE_SCRATCH`, declared as `paths` in `envVars.ts`.
+ - `HOMEPAGE_PUBLIC_DIR`'s `readBy` names `source.ts`, and `TRANSCRIPTS_DIR`'s doc says umtool
+ reads `<it>/channels` too (review Q(c)). `ENVIRONMENT.md` is regenerated.
+ - The CLI rows `build homepage [--no-source]`, `source publish [--force] [--check]
+ [--keep-scratch]` and `source audit [<git dir>]`.
+ - `buildHomepage`: compose, then the source step (a lazy import; `publishSource` and
+ `clearSource` are seams), then `next build`.
+ - doctor's "source publish" block. It is never a failure; it WARNs when the operator files exist
+ but no filter-repo is installed, or a file is readable by others. It prints rule counts and
+ modes, never contents.
+
+**Corrections to the plan** (each found by running it):
+- **`_headers`: later rules APPEND, they do not win** (`f218ed86`).
+ - wrangler 4.88's `attachHeaders` (the Pages asset server's code, read in its `cli.js`) `set`s
+ a header on the first matching rule and `append`s it on every later one. The plan's exact
+ text served a directory page as `text/plain; charset=utf-8, text/html; charset=utf-8` and a
+ font as `text/plain; charset=utf-8, font/ttf`.
+ - Each override now starts with `! Content-Type`. A replica of wrangler's parse and attach
+ (`$T/r-m-headers-sim.mjs`) gives single values for `/source/tree/`, `…/common/`, a `.ts`, the
+ bracketed `.ttf` and `page.tsx`, the `.onnx` and `README.md`.
+ - The preview deploy is still the live proof.
+- **The tsconfig must exclude `out` too** (`3fa5ff18`).
+ - `next build` copies `public/` into `out/`. The SECOND build with a mirror failed its type check
+ on `out/source/tree/common/jobs/registry.ts`, a second `declare global var __yttJobRegistry__`.
+ - With `out` excluded, the homepage program is 871 files, none from the mirror; homepage tsc
+ takes 11 s with a mirror in both dirs.
+ - The homepage's `eslint.config.mjs` ignores `public/source/**` (it already ignored `out/**`).
+ Lint itself was not run.
+- **The published manifest carries no `rulesHash`.**
+ - `plans/` ships in the mirror, and so does the plan's description of the two files. What is
+ left unknown is the hostname and the address, so a published hash of the rules would confirm
+ a guess at them.
+ - The skip key (`{sourceCommit, rulesHash}`) is `homepage/.source-publish.json`, beside
+ `public/` and gitignored. A missing key rebuilds; the round trip asserts it is absent from the
+ manifest.
+- **The mirror has a loose `refs/heads/main` beside `packed-refs`.** git treats a directory as a
+ repository only with a `refs/` dir, so without it the `file://` clone and `source audit` of the
+ published dir fail. An empty dir would not survive a deploy.
+- **filter-repo's `--replace-text` does not skip `#` lines** (it would replace a comment as a
+ literal; `get_replace_text` in 2.47). The step writes `replace.txt` without comments or blank
+ lines.
+- **The step adds three safety flags:** `--no-tags`, `--replace-refs delete-no-add` (2.47's
+ default is `update-no-add`, stated for determinism), and a check that the mirror holds exactly
+ `refs/heads/main`.
+- **`--no-source` removes the published source** (manifest first; a linked `downloads/` goes as a
+ link, its target untouched). It does not leave the source there: a copy audited against older
+ rules would ship ungated. That removal gives the empty state R7 expects.
+- **The header nav moves from `sm` to `md`:** five labels do not fit beside the wordmark at 640 px.
+- **The tree has 4 `.ttf`**, not 3 (IBM Plex Mono Bold, O5); `*.ttf` covers them.
+- **No `branch` option:** `SOURCE_BRANCH = "main"`, an operator decision. `now` is `() => Date`.
+- **The packs are 37.5 MB in two** (20,897,277 + 16,633,904 bytes), not the plan's 21.5 MB in one.
+ The history grew by about 150 commits since the plan, and the 20 MB split costs deltas across
+ the two packs. That is well inside the limits.
+
+**The operator's files, as created, refuse the publish** (an action for the rollout, not the
+slice):
+- **The run:** `pnpm archilyzer source publish --check` with `~/.config/archilyzer/*` as the
+ parent created them, at `main` `e6c5d2e3`. It ended `AUDIT REFUSED: 8 hits in 21,441 objects
+ (1,702 commits) against 6 denied literals`, with every hit `#1 (r…, len 5)` in a blob:
+ - `plans/source-mirror.md`, two versions, 3 hits each: the plan's own grep gates (`git grep -c
+ -i '<it>' …`, `git -C <clone> grep -c -i <it> …`) and the "a future transcript quote "<it>
+ that"" risk line;
+ - `plans/release-12.md`, two versions, 1 hit each: slice Q's grep-gate line.
+- **Why the scrub rules miss them.** They cover the backticked spelling and the `/run/media/` path
+ (review I2: no hit came from there), not the bare, single-quoted or double-quoted name.
+- **The operator's step:** add a rule for the bare name (`<name>==>user` covers every spelling,
+ the backticked one included) or one per spelling. Then `archilyzer source publish --check`
+ (about 25 s) must end `check passed`.
+- **How R7 ran anyway,** with the operator files untouched and the gate not weakened:
+ - `SOURCE_SCRUB_FILE` = a scratch file of ONE rule, `<name>==>user`, written with `$(id -un)`,
+ with the REAL denylist. The check was clean: 4 literals, gitleaks clean.
+ - `source audit` of both clones and of the published dir with the REAL files (6 literals) is
+ clean, below.
+
+**Gates** (from the worktree root; logs `$T/r-m-*.log`). Every log of a run over the real files was
+first scanned by `$T/r-m-leakcheck.mts` (counts per literal label, any ASCII case) and read through
+`$T/r-m-maskview.mts`. The step's own lines carried 0 literals; the only hits were pnpm's and
+doctor's home-directory paths.
+- **tsc** was clean before every commit (`r-m-tsc-{0..3}.log`):
+ - 215 s at the first run, 168 s, 59 s, and 73 s with a mirror in `public/` AND `out/`. Another
+ session's tsc was running during the first and the last.
+ - The homepage alone: 11 s, 871 files, none from the mirror.
+- **Unit:**
+ - common **2,138/2,138** (2,114 + 24: sourceAudit 7, sourceTree 5, source 7, build +1, `_cli`
+ +3, doctor +1). The two filter-repo tests RAN (`ok 1832` round trip, `ok 1833` planted-literal
+ refusal); 0 skipped.
+ - `test:scripts` **185 + 1 skipped**; editor unit **85/85**; mcp **269/269**; homepage unit
+ **2/2**.
+- `pnpm archilyzer docs env --check` exits **0**. The editor's `next build` is **ok** (43 s); it
+ bundles `buildHomepage`'s lazy import.
+- **`archilyzer build homepage`** (from `common/`, `SOURCE_SCRUB_FILE` as above):
+ - **ok in 38 s**, against 19 s for `--no-source`. The source step took **19 s**: filter-repo
+ 5.3 s, gitleaks 7.3 s, `[source] published main e6c5d2e322b3 as 78dc0126382b: 2459 files,
+ 69.5 MB (mirror 2 packs, tree 412 dirs), tarball 6.9 MB sha256 c4ceb6d110ee`.
+ - A rebuild with nothing changed logged `[source] up to date at e6c5d2e322b3; skipping`.
+ - **Deterministic:** two runs gave the same `mirrorHead` and tarball sha; the real files' two
+ runs gave `9b880270c49d` both times.
+- **What it published** (`homepage/out`):
+ - **2,640 files, 78.1 MB** in all; `source/` 2,465 files, 65.8 MB;
+ - the mirror: 9 files, 38.1 MB, 2 packs;
+ - the tree: 2,035 files + 412 pages, 27.5 MB;
+ - the tarball: 7,246,992 bytes.
+ - **Against the limits:** 2,459 staged of the step's 15,000 (2,640 of Pages' 20,000); the
+ largest file is 19.93 MiB, against 24 MiB (25 MiB).
+- **Two clones, both HEAD = `manifest.mirrorHead` `78dc0126382b…`:**
+ - `git clone file://$WT/homepage/out/source/archilyzer.git`: 2 s.
+ - `git clone http://127.0.0.1:8765/source/archilyzer.git` from `python3 -m http.server 8765
+ --bind 127.0.0.1` in `homepage/out`: 1 s. The protocol was dumb: `info/refs?service=…` 200,
+ then `HEAD`, `objects/info/packs`, both `.idx` and both `.pack`; the loose-object probes 404.
+ - `:8765` was free before, the server was killed, and it was free after.
+- **The user name:** `git grep -c -i -F "$(id -un)" $(git rev-list --all) | wc -l` is **0** in both
+ clones (1,702 revisions). It is 0 in the identities and messages and 0 in the paths too; the
+ hostname is 0.
+- **`archilyzer source audit`** with the real files (6 literals) is **clean** on the file clone
+ (13 s), the http clone (11 s) and the published dir (10 s): 21,441 objects, 1,702 commits,
+ gitleaks "1530 commits scanned … no leaks found".
+- **The gate, exercised:** `SOURCE_DENYLIST_FILE` = a scratch file of `Co-Authored-By`, then
+ `source publish --check`:
+ - exit **1** in 14 s, `AUDIT REFUSED: 1476 hits … #1 (C…, len 14): 8 in blobs, 1468 in
+ commits`, 20 context lines and "… and 1456 more";
+ - the log holds the literal **0** times;
+ - `public/source/manifest.json`'s mtime and size are unchanged (`1790616152 1057`).
+- **`build homepage --no-source`:** `out/source/index.html` holds `data-testid="source-empty"`
+ and "No source published in this build."; `/downloads/` says "no source snapshot attached".
+- **No filter-repo:** with PATH = a scratch dir of two symlinks (`git`, `node`; nothing
+ uninstalled), `git filter-repo` is "not a git command", and `source publish --check` exits 1 with
+ `REFUSED: git-filter-repo is not installed and pipx is not on PATH — install it once: \`pipx
+ install git-filter-repo\` …`.
+- **The resolved filter-repo** is `git filter-repo` (`~/.local/bin/git-filter-repo`, pipx-installed
+ 2.47.0), whose `--version` prints `a40bce548d2c`; git is 2.55.0. The `pipx run` fallback was not
+ run (it needs the network).
+- **`archilyzer doctor`** prints the block:
+ ```
+ source publish
+ ok filter-repo git filter-repo a40bce548d2c
+ ok gitleaks gitleaks version is set by build process
+ ok scrub rules ~/.config/archilyzer/source-scrub.txt (2 rules, mode 600)
+ ok denylist ~/.config/archilyzer/source-denylist.txt (3 literals, mode 600)
+ -- published main e6c5d2e322b3 as 78dc0126382b, 2026-09-28T17:22:29.301Z (2458 files)
+ ```
+ The real output prints the absolute paths. `2458` is the manifest's `files`, which does not
+ count the manifest itself.
+- **homepage e2e** (`node scripts/worktree.mjs run -- pnpm --filter homepage run e2e`; the queue
+ was free):
+ - **36 passed, 0 skipped, 0 failed, 1.1 min**, WITH the manifest present:
+ `loadSourceManifest()` from `homepage/` gave mirror head `78dc0126382b`, so the published
+ branch of every `source.spec` test ran.
+ - The empty state too: with `public/source` and `public/downloads` moved aside and restored
+ after, `source.spec` + `downloads.spec` + `marketing.spec` gave **16 passed** (30 s).
+ - `downloads.spec.ts` is unchanged.
+- **Numbers tool:** none.
+
+**They bite:**
+- The planted-literal test fails if the object walk misses the blob.
+- The redaction test pins a half-shown second occurrence.
+- `build.test.ts`' fake refusal proves `next build` never runs.
+- The restricted-PATH run proves the install line.
+- The `Co-Authored-By` run proves the gate refuses a literal the scrub does not touch, in commits
+ AND blobs.
+
+| sha | what |
+|---|---|
+| `25f5e241` | `homepage:` `/homepage/public/source` gitignored (the Tailwind note), tsconfig excludes `public` — first, before any mirror |
+| `7fdbe2dc` | `common:` `sourceAudit.ts`, `sourceTree.ts`, `sourceManifest.ts` + tests |
+| `fc31aab5` | `common:` `publishSource` (`source.ts`) + tests; `getPaths()` / `envVars.ts` / `ENVIRONMENT.md` |
+| `2f08cb47` | `common:` `buildHomepage` runs the step; the CLI rows; doctor's block + tests |
+| `1a11a2bb` | `homepage:` `/source/`, nav, `Fact`, Downloads, `_headers`, docs content, `source.spec.ts`, marketing nav |
+| `2cd189f3` | `docs:` README, SETUP, PUBLISH "The source mirror (homepage)"; `create-archives.sh` deleted |
+| `c448b392` | `changelog:` the editor and homepage `[Unreleased]` bullets |
+| `f218ed86` | `homepage:` `_headers` overrides detach `Content-Type` first (correction) |
+| `3fa5ff18` | `homepage:` tsconfig excludes `out`; eslint ignores `public/source/**` (correction) |
+| _this_ | `plans:` this record |
+
+**Found and left** (not this slice's files):
+- **`.dockerignore:20-22`** still names `create-archives.sh`, and it does not exclude
+ `homepage/public/source/`. A `docker build` from a checkout that has published would send about
+ 66 MB of mirror and tree in the build context.
+- **`common/lib/project.ts:37-38`** (`PROJECT_DOWNLOADS_URL`'s comment) says "there is no public git repository".
+- **The editor's /sites Homepage section** (`HomepageBuildButtons.tsx:170`) does not say that the
+ build publishes the source, or that it can refuse.
+- **`export/CHANGELOG.md`'s released entry** names `create-archives.sh`. It is history, and is left.
+- **The label `#n (x…, len L)`** shows a literal's first character and length in the logs. This is
+ the plan's format; nothing of it is published.
+- **gitleaks** scans 1,530 of 1,702 commits: its `git log -p` skips merges. The literal audit reads
+ every object.
+
+**Changelog.**
+- **`editor/CHANGELOG.md`:** one `[Unreleased]` bullet. The build step and the gate; the operator
+ files, and that the build refuses without them; `pipx install git-filter-repo`; the skip, the
+ CLI rows and doctor; `create-archives.sh` gone.
+- **`homepage/CHANGELOG.md`:** one `[Unreleased]` bullet. `/source/` and its nav entry, the empty
+ state, the tarball regenerated per build, the docs, the nav at `md`, and `_headers`.
+
+**For the parent and the operator:**
+1. Add the scrub rule above, then run `archilyzer source publish --check`.
+2. The editor's /sites homepage job finds `git filter-repo` only if the editor's PATH includes
+ `~/.local/bin`. Otherwise it falls back to `pipx run`, which needs the network on first use.
+3. The worktree's `homepage/public/source`, `homepage/out` and `homepage/.source-publish.json` were
+ made with the scratch rule. They are disposable, and nothing here deployed them.
+
## Rollout
