Archilyzer · Source

archilyzer

Archilyzer
git clone https://archilyzer.pages.dev/source/archilyzer.git
Log | Files | Refs | README | LICENSE

commit 8cf542e1f31bf51986ff517c69ede358d06fa370
parent f985d12aa555e32054e059d01cce64a971c070df
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date:   Wed, 30 Sep 2026 09:22:26 -0400

plans: slice SG, as shipped — the source's history and diffs at /source/git/, rendered by stagit (519cae2f): the step, the post-pass, the stylesheet, the gate, the key, the cache, the no-stagit path, the History block; the gates (common 2,318, homepage e2e 11 + the full 103, the proof build: 1,872 commits in 1,878 files, 141.3 MB, the largest 5.0 MB, the user name and hostname 0), what was found and left (the file limit about three months away), the decisions; the SG row; FACTS "The source's history (stagit)"

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Diffstat:
Mplans/FACTS.md | 73+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Mplans/release-15.md | 176+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 249 insertions(+), 0 deletions(-)

diff --git a/plans/FACTS.md b/plans/FACTS.md @@ -7754,3 +7754,76 @@ The record is [`release-15.md`](release-15.md), "Slice DS, as shipped", with the reconcile pass, are not capped); a hand-typed root is capped but never marked; a drive already stalled at boot before the second counter sample, unless a page reaches it; per-click server actions; the file route's stream; the corpus disk itself. + +## The source's history (stagit) (verified 2026-09-30, branch `r15/stagit`) + +Release 15 slice SG ([`release-15.md`](release-15.md)); the operator-facing doc is `PUBLISH.md`, "The +source mirror (homepage)". Anchors are at the branch. + +- **stagit** (codemadness.org, C over libgit2; built from `519cae2f`, "bump version to 1.3", + 2026-09-14, against the system libgit2 1.9.7) renders the scrubbed mirror into `/source/git/`. + It is operator-installed like git-filter-repo: never vendored. `resolveStagit` + (`publish/sourceHistory.ts:104`): `paths.stagitBin` (`STAGIT_BIN`, else `"stagit"`, `lib/paths.ts:304`) + — a name with a slash is that file or nothing; a bare name is PATH, then `~/.local/bin/<name>`. + It has no version flag: `stagitIdentity` (`:118`) is the first 12 hex of its binary's sha256, + never the path (the manifest is published, and a home-dir path is a denied literal). +- **What stagit does** (read in `stagit.c`): it writes into its CWD; the repository's NAME is its + directory's basename minus `.git` — so the step's scratch clone is `<scratch>/archilyzer.git` + (`source.ts:792`), no longer `bare`; `description` (read with `fgets`, the newline KEPT — write it + without one) and `url` (newline stripped) come from the repository directory; the header links + `<relpath>file/README.md.html` and `LICENSE` when HEAD has them, and the logo `<a href="../<relpath>">` + lands on `/source/` from every depth; `file/` pages are written for HEAD's whole tree on EVERY run; + every text from the repository is xmlencoded (`"` → `&quot;`), so no page text can fake an + `href="…"`; a diff's paths are percent-encoded (`/` and `,-.` kept); a commit over 1,000 files or + 100,000 added or deleted lines prints "Diff is too large, output suppressed"; `-c <file>` walks from + HEAD to the commit the cache file names, KEEPS every `commit/<sha>.html` already in the CWD, and + appends the cached log lines — so the cache is the whole output directory, and a cache whose commit + is not an ancestor duplicates the log. Its temp cache file (`cache.XXXXXXXXXXXX`) is made in the CWD. +- **The step** (`source.ts`, step 12b, `:920`) runs after the object audit and the mirror's staging, + before the manifest and the file audit: `stageHistoryPages` (`:1070`) → `renderHistory` + (`sourceHistory.ts:454`). **An allowlist is staged** (`TOP_FILES`, `COMMIT_PAGE`, `:76`): `log.html`, + `files.html`, `refs.html`, `atom.xml`, `tags.xml`, `commit/<40 hex>.html`, plus `style.css`. Never + `file/`, never a leftover. Pages are rewritten byte for byte (latin1 in and out; every injection is + ASCII: the back link says `&middot;`). +- **The post-pass** `rewriteHistoryPage` (`:285`), pages only: `<script>` + the homepage's pre-paint + script (`homepageThemeScript`, `:70` = `buildThemeScript({ defaultBase: HOMEPAGE_DEFAULT_BASE })`) + before `</head>`; `HISTORY_BACK_LINK` (`:67`) after `<body>`; `href="(../)*file/<p>.html"` → + `href="$1../tree/<p>"`; `logo.png`/`favicon.png` → `/icons/icon-32.png` (a route the homepage renders + at build). The theme config moved to `lib/themeConfig.ts` for this (publish may not import + components/; `components/themeConfig.ts` re-exports it); `HOMEPAGE_DEFAULT_BASE` (`:62`) is what the + homepage layout passes. The attribute the script sets is `data-base`, not `data-theme`. +- **`style.css`** is `historyStylesheet` (`:197`) over `common/styles/tokens.css`: the light block + (`:root, html[data-base="light"]`), the dark block for `html[data-base="dark"]` AND for + `:root:not([data-base])` under `prefers-color-scheme: dark` (no JS); `HISTORY_TOKENS` and whatever + they reach through `var()`. Diff `+` lines are `--success`, `−` lines `--destructive`; there is no + chart-safe red in the tokens (the chart palette is blue, green, violet, amber, magenta, rust). +- **Never a failed build over the history.** No stagit: one line with the install (`source.ts:931`). + A render that fails (`HistoryProblem`), a stagit timeout (600 s), unreadable tokens, more files + than the step's 15,000 in all, or a page over 24 MiB: a `[source] WARNING: …` and no history. A + cancel still cancels; any other error is a crash as before. +- **The manifest's `history`** (`lib/sourceManifest.ts:88`, parsed at `:137`, all-or-nothing): + `href`, `commits` (`git rev-list --count`), `head`, `files`, `bytes`, `sha256` (`historyDigest`, + `source.ts:500`: `sourceDigest`'s walk over `source/git/**` alone), `tool`. `sourceDigest` (`:484`) + now walks `source/git` too. The state (`.source-publish.json`) gains `stagit` and `history: + {files, digest} | null`; the skip needs both to match, and a stagit present with no history never + skips. `SOURCE_STEP_VERSION` is 4 (`:109`). +- **The deploy check** (`publishedSourceProblem`, `:1204`) names the history on its own (`:1286`): + `out/source/git`'s digest must be the state's (null when none) and the manifest's. +- **The render cache** is `<scratch root>/archilyzer-source-history/` (`historyCacheDir`, `:556`): + `key.json`, `stagit.cache`, `out/`, and `lock` while held. `holdHistoryCache` (`:408`): the lock + names the holder's pid; a live holder → render without the cache; a dead one → emptied and taken. + Usable only when the key (`historyCacheKey`, `:565`: rules hash, filter-repo, stagit, description, + clone URL, base URL) matches, `out/log.html` exists, and the cache's commit is an ancestor of the head + (`git merge-base --is-ancestor`). The key is removed before a render and written after it. A cached + render whose commit pages ≠ commits is rendered again from nothing, once. `--force` renders + afresh; `--check` renders in its own scratch; a refusal removes the cache (`source.ts:676`). +- **Measured (2026-09-30, `main` `6b8aa450` → mirror `8c924ca31701`):** 1,872 commits; 1,878 files, + 141,299,532 bytes; the largest `commit/7ddfc955….html`, 5,017,646 bytes; stagit 8 s of a 32 s + publish (every page); the whole publish 4,396 files, `homepage/out` 4,576. On the published mirror + at 1,834 commits: 8.0 s every page, 2.0 s from the cache with nothing new. The cache is 138 MB — on + this machine's tmpfs `/tmp`, memory. One file per commit: at about 100 commits a day (September + 2026) the step's 15,000 is some three months away, and the history is what is dropped then. +- **The homepage** believes `history` only beside `source/git/log.html` (`homepage/app/lib/source.ts:43`; + without it the manifest comes back without the block). Its e2e reads a fixture publish from + `E2E_SOURCE_PUBLIC_DIR` while it holds a manifest (`sourcePublicDir`, `:69`; never in a production + build). diff --git a/plans/release-15.md b/plans/release-15.md @@ -20,6 +20,7 @@ prompt carries its ruling, and this record carries what was built. Rules: | DS | `r15/drive-stall` | A stalled drive does not stop the editor answering | new `common/lib/storageHealth.ts`; `lib/{storageVolumes,channelMedia,channelMediaHold}.ts`, `controller/storageWatch.ts` and the gated callers; `/storage`, `/channels`, the videos pages; `UV_THREADPOOL_SIZE` (`editor/package.json`, `docker/entrypoint.sh`, `envVars.ts`) | | UT | `r15/umtool-trace` | umtool's build stops tracing the whole `umtool/` folder | per its prompt | | SS | `r15/site-scope` | The editor's site picker paints the stored site at once: the selection is a cookie | `editor/app/lib/activeSite{,Server,Actions}.ts` + `activeSite.test.ts`, `editor/app/components/SiteScope{Provider,Select}.tsx`, `editor/app/layout.tsx`, the scope lines of `editor/app/page.tsx` and `editor/app/channels/page.tsx`, a comment in `editor/next.config.ts`, `editor/e2e/site-scope.spec.ts`; records: `plans/FACTS.md` | +| SG | `r15/stagit` | The source's history and diffs on the homepage, rendered by stagit at `/source/git/` | new `common/publish/sourceHistory.ts` + test; `common/publish/source.ts` (step 12b, the key, the digest, the deploy check) + test; `common/lib/{sourceManifest,paths,envVars}.ts`, `common/lib/themeConfig.ts` (moved from `components/`, which re-exports it); `common/bin/doctor.ts`; `homepage/app/{source/page.tsx,lib/source.ts,layout.tsx}`, `homepage/e2e/{source,source-history}.spec.ts`, `homepage/e2e/fixture-source.ts`, `homepage/playwright.config.ts`; `ENVIRONMENT.md`, `PUBLISH.md`; records: `plans/FACTS.md` | **Order:** IG → DS. DS adds a health gate inside `inspectChannelMedia`, which IG's hold calls through its public signature. UT is independent. The shared files are `editor/CHANGELOG.md`'s @@ -1088,6 +1089,181 @@ It reproduced M1 with a two-tab probe under the queue lock. editor's built bundle, so all of it takes effect only after the editor is rebuilt and restarted. After that, each browser's first visit migrates its localStorage selection once. +### Slice SG, as shipped — the source's history and diffs, rendered by stagit (2026-09-30) + +Branch `r15/stagit` off `main` `6b8aa450`, worktree `~/Projects/r12-source-mirror` (block #13: +editor 4301, test 4311, export 4310, homepage e2e 4340), one Opus implementer. Scratch files +`sg-*` in the job's `tmp`. `main` did not move during the slice. The ruling (2026-09-30): visitors +read the commit history and each commit's diff on the homepage; the tool is **stagit**; its +per-file pages are dropped, and browsing stays the raw tree at `/source/tree/`. No vendor file: +stagit is operator-installed, like git-filter-repo. + +**What shipped.** +- **Where it runs** — `source.ts` step 12b, after the object audit and the mirror's staging, before + the manifest and the file audit. `stagit -c <cache> -u https://archilyzer.pages.dev/source/git/ + <the scrubbed clone>`, in the render's directory. The scratch clone is now named + `archilyzer.git` (stagit names the repository after its directory; it was `bare`). Its + `description` is "Archilyzer" (no newline: stagit keeps it) and its `url` is the clone URL. Neither + file is published: the mirror is staged from its allowlist. +- **What is published** — an allowlist of stagit's output: `log.html`, `files.html`, `refs.html`, + `atom.xml`, `tags.xml` and `commit/<40 hex>.html`, plus `style.css`. stagit writes `file/` + (HEAD's whole tree) on every run; it is deleted where it was rendered and never staged. +- **The post-pass** (`rewriteHistoryPage`, pure, unit-tested; pages only, never the two feeds): + - the homepage's pre-paint theme script before `</head>`. It is the string `ThemeScript` emits: + `buildThemeScript({ defaultBase: HOMEPAGE_DEFAULT_BASE })`; + - one line after `<body>`: `Archilyzer · Source`, linking to `/source/`; + - `href="(../)*file/<path>.html"` → `href="$1../tree/<path>"`. This covers the Files index, the + header's README and LICENSE, and both sides of every diff header; + - stagit's `logo.png` and `favicon.png` → the site's `/icons/icon-32.png`. Without it, every page + has a broken image in its header. + + The pages are rewritten byte for byte: latin1 in and out, and every injection is ASCII. +- **The theme config moved to `lib/`.** The publish layer may not import `components/` + (`architecture.test.ts`), and the script lived in `components/themeConfig.ts`. That file now + re-exports `lib/themeConfig.ts`, so its importers are unchanged. `HOMEPAGE_DEFAULT_BASE` ("dark") + is new; the homepage layout passes it to `ThemeScript` and `ThemeProvider`. +- **`style.css`** — `historyStylesheet(tokens.css)`, stagit's own rules recoloured: + - the light block, and the dark one for `html[data-base="dark"]`; + - the dark one also for `:root:not([data-base])` under `prefers-color-scheme: dark`, the no-JS + case; + - links in `--brand` (Signal), diff insertions in `--success`, deletions in `--destructive`; + - the clone line wraps at phone width. + + Only the tokens the rules read are copied, with what they reach through `var()`. +- **The gate covers it.** The pages are in the stage the file sweep reads. A denied literal planted in + a fake stagit's output refuses the publish, named as `file + source/git/commit/<sha>.html (contents, byte N)` and never by its bytes. The refusal withdraws the + publish and removes the render cache. +- **The manifest and the key.** + - The manifest gains `history: {href, commits, head, files, bytes, sha256, tool}`. + `sha256` is `historyDigest`, the content digest's walk over `source/git/**`; `tool` is `stagit + (sha256 <12>)`, since stagit has no version flag. + - `sourceDigest` walks `source/git` too. + - The state gains `stagit` and `history: {files, digest} | null`. The skip needs both, and a stagit + present with no history never skips. + - `publishedSourceProblem` names history pages that are not the audited ones: "homepage/out's + history pages (/source/git/) are not the ones that were audited". + - `SOURCE_STEP_VERSION` is 4. +- **The render cache** — `<ARCHILYZER_SOURCE_SCRATCH>/archilyzer-source-history/`: the `-c` file, + stagit's output, a key. + - It is used only when the key matches (the rules hash, filter-repo, stagit, the header text), + the cached commit is an ancestor of the head, and the last run finished (the key is removed + before a render). + - One holder at a time, by a pid lock. A live holder means render without the cache; a dead one + means empty it and take it. + - A cached render whose page count is not the commit count renders every page again, once. + - `--force` renders afresh, `--check` renders in its own scratch, and a refusal removes the cache. +- **Without stagit** there is one line: `[source] stagit not found (not on PATH, not in ~/.local/bin) + — publishing without the history pages (/source/git/); install it once: git clone + git://git.codemadness.org/stagit && make -C stagit && cp stagit/stagit ~/.local/bin/`. The manifest + then has no `history`, and `/source/` shows no History links. + - A render that fails, a timeout (600 s), unreadable tokens, or pages over the step's limits (15,000 + files in all, 24 MiB a file) are a `[source] WARNING` and no history. The build never fails over + it. + - `STAGIT_BIN` (a `paths` variable, `getPaths().stagitBin`) overrides the lookup: stagit on PATH, + then `~/.local/bin/stagit`. + - `archilyzer doctor` prints `stagit <path>` right after filter-repo; a `STAGIT_BIN` that names + nothing is a warn. +- **`/source/`'s History block**: the commit count, the newest commit (short, linking to its page), + and Log · Refs · Atom feed. It shows only when the manifest has a history and `source/git/log.html` + is beside it: the loader drops an orphaned block, and a malformed one untrusts the manifest. +- **Docs.** PUBLISH.md's source-mirror section gains the history. There is no `operate.md` mention + of `/source/`; Install and the FAQ each gain half a sentence. + +**stagit, installed.** Cloned `git://git.codemadness.org/stagit` into `$T/sg-stagit/src` at +**`519cae2fcab6f2427dc8f2805d6a5d1eee4bdb05`** ("bump version to 1.3", 2026-09-14). `make` built it +against libgit2 1.9.7. The binary was copied to `~/.local/bin/stagit` (sha256 `898752011b07…`). + +| Commit | What | +|---|---| +| `b482f5fc` | `common:` the theme config moves to `lib/`; `components/themeConfig.ts` re-exports it; `HOMEPAGE_DEFAULT_BASE`, used by the homepage layout. | +| `3cdf4773` | `common:` `sourceHistory.ts` + 11 tests; `source.ts` step 12b, the key, the digest, the deploy check, the withdrawal + 4 tests (and the old ones: `stagit: null`, the `archilyzer.git` scratch name, step ≥ 4); the manifest's `history` + 2 tests; `STAGIT_BIN`; `ENVIRONMENT.md`. | +| `8394f4ca` | `common:` doctor's stagit line (its test extended). | +| `17965d58` | `homepage:` the History block; the loader; `E2E_SOURCE_PUBLIC_DIR` (declared, `ENVIRONMENT.md`); `fixture-source.ts`, `source-history.spec.ts` (2), `source.spec.ts` (+1); +2 unit tests. | +| `cbe0ac3a` | `homepage:` the head as a short link to its page; the clone line wraps. | +| `3dd5238d` | `docs:` PUBLISH.md, Install, FAQ; the editor and homepage `[Unreleased]` bullets. | +| this commit | `plans:` this record, the SG row, FACTS "The source's history (stagit)". | + +#### Gates (at `3dd5238d`; logs `$T/sg-*.log`) + +- **tsc** clean before every commit (`sg-tsc-{1..4}.log`, each `exit=0`). +- **common 2,318/2,318** (2,301 at `main`, +17: `sourceHistory` 11, `source` +4, `sourceManifest` + +2; doctor's test extended), 0 skipped, 77 s. The filter-repo round trips and the real-stagit + test RAN. +- **homepage unit 22/22** (+2). **editor unit 95/95.** **test:scripts 195 + 1 skipped.** +- `docs env --check` and `docs files --check` both exit 0. +- **Builds:** + - editor `next build` ok, 34 s. It bundles `buildHomepage`'s lazy import; no `.nft.json` names + `homepage/public`, `tokens.css` or the cache. + - export `next build` ok, 19 s (`export/public` linked from the primary, no dangling link). + - homepage `next build` under `systemd-run … MemoryMax=5G` ok, 12 s. +- **homepage e2e**, `E2E_EXPECT_SOURCE=1`, over this worktree's publish (the queue was free): + - `source.spec` + `source-history.spec` + `downloads.spec`: **11 passed, 0 failed, 56 s**. The + history test RAN its published branch (10.1 s): the no-JS light and dark, the stored light + choice, a commit, the Files link into the tree, the feed. + - The full suite: **103 passed, 0 failed, 2.3 min**. +- **The proof** — `pnpm -s archilyzer build homepage` in the worktree, with the operator's files + (read by the step; never printed), `~/.local/bin` on PATH: + - exit 0 in 50 s: `[source] history: stagit (sha256 898752011b07) — 1872 commits, 1872 pages + rendered; 1878 files, 134.8 MB, the largest git/commit/7ddfc955….html 4.8 MB (8 s)`; + - `[source] audit clean: 23,028 objects (1,872 commits), 4,396 staged files against 10 denied + literals; gitleaks clean`; + - `[source] published main 6b8aa450ad78 as 8c924ca31701: 4396 files, 211.5 MB (mirror 3 packs, + tree 414 dirs, history 1872 commits in 1878 files), tarball 7.3 MB sha256 a072630c6c3a (32 s)`. + - **`homepage/out/source/git/`:** 1,878 files, 141,299,532 bytes; the largest + `commit/7ddfc955….html` at 5,017,646 bytes. `homepage/out` holds 4,576 files, against Pages' + 20,000 and 25 MiB. + - `grep -rci "$(id -un)"` gives 0 files and 0 hits; the hostname gives 0. No `file/` directory, no + `.git` path segment in `out/`. + - `publishedSourceProblem` over the worktree's `out/` is null. With one byte added to + `out/source/git/log.html`, it gives the history sentence; restored, it is null again. + - A second build: `[source] up to date at 6b8aa450ad78; skipping` (17 s). + - `source publish --force` gave the same mirror head and tarball sha. + - `STAGIT_BIN=/nonexistent/stagit source publish --check`: the one line ("STAGIT_BIN names no + executable"), then `check passed … 2518 files, 76.8 MB (…, no history)`. + - doctor: `ok stagit ~/.local/bin/stagit`, right after filter-repo. +- **The cache, measured** on a copy of the published mirror (1,834 commits): 8.0 s for every page, + 2.0 s with nothing new. The real cache is 138 MB in `/tmp`, which is tmpfs here. +- **Numbers tool:** none. + +#### Found and left + +- **The file limit is months away, not years.** There is one file per commit, and main gained about + 100 commits a day in September 2026. The publish is 4,396 files, and the step refuses at 15,000: + about 10,600 commits of headroom. When the history would cross it, the history is dropped with a + WARNING and the rest is published. The choices then (raise `MAX_FILES` toward Pages' 20,000, or + publish only recent commits' pages) are a later ruling. +- **`/source/git/` has no index page**, so the directory itself is a 404; `/source/` links + `log.html`. A copy of `log.html` as `index.html` would double 0.6 MB. +- **No `_headers` rule for `/source/git/*`.** The pages are HTML by extension, and stagit encodes + every repository string. No CSP was added, because the inline theme script would need its hash in + `_headers` on every change. +- **The skip does not see a code change to the step** (the post-pass, the stylesheet). The next + `main` does, and every merge moves `main`. `--force` does it at once. +- **The mirror has 1,872 commits against 1,874 on the private main.** filter-repo prunes commits + that become empty. This predates the slice. + +#### Decisions the operator could overturn + +| What I assumed | The alternative | +|---|---| +| Diff lines use `--success` and `--destructive`, with no new tokens: the chart palette has no red (it is categorical: blue, green, violet, amber, magenta, rust); both are text-grade on both grounds; every changed line keeps its `+`/`−`, so hue is never the only signal. | Two new tokens (`--diff-add`, `--diff-del`), validated as a pair. A green/red pair cannot be told apart by hue for every colour-blind reader, so the sign column carries it either way. | +| A failed render, a timeout, unreadable tokens or pages over the limits are a WARNING, with no history. | Refuse the publish: louder, but it takes the mirror down with the history. | +| stagit's logo and favicon point at `/icons/icon-32.png`. | Leave them broken, or drop the `<img>`. The ruling said "nothing else is injected"; this is a link rewrite, like `file/`. | +| The History block's head is 12 characters, linking to its commit page. The full sha is the Mirror head fact just above. | The full 40-character sha, as text. | +| The cache lives under the scratch root, a tmpfs `/tmp` by default here (138 MB of memory). | A cache dir of its own, on disk. | +| `--force` empties the cache. | Keep it, and force only the publish. | +| `--check` renders without the cache (8 s rather than 2 s): it writes nothing outside its own scratch. | Use the cache. | +| The e2e shows the page with and without a history from a fixture publish, through a non-production-only `E2E_SOURCE_PUBLIC_DIR`. The real pages are walked by `source.spec` when the checkout has a history. | Only the checkout's real state, as the other source specs do. | + +**For the rollout** (the parent's): +- **Install stagit on the publishing machine**, the same way (`~/.local/bin/stagit`). +- **Rebuild and restart the editor** before any `/sites` Homepage job; its built bundle runs the + old step. +- The first build publishes the source again (step 4), with the history. +- A review should read one live page after the deploy; the preview's `_headers` do not touch + `/source/git/`. + ## Rollout Release 15 is slices IG (`r15/index-hold`, merged `ccf90892`), UT (`r15/umtool-trace`, `07c991be`),