commit d85aafb4b2f2f415f6e30c0cb77a25cbf0049273
parent 8d7b0ab03b11abf16a93f0c472cfa99c4fa9eb48
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Wed, 30 Sep 2026 10:13:17 -0400
plans: slice SG — the review (SHIP AFTER FIXES) and its fixes: M1 ruled noindex, L1–L6 and L8 to their commits, the corrections (the cache path, `total`, "every other link resolves"); the re-gates (common 2,347, the three source specs 12/12, publish and --check, every tree and commit link resolving, the capped editor and homepage builds with the corpus linked); FACTS for the unlinking, the stale lock, the unusable cache and noindex
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Diffstat:
2 files changed, 86 insertions(+), 27 deletions(-)
diff --git a/plans/FACTS.md b/plans/FACTS.md
@@ -7782,13 +7782,13 @@ source mirror (homepage)". Anchors are at the branch.
- **stagit** (codemadness.org, C over libgit2; built from `519cae2f`, "bump version to 1.3",
2026-09-14, against the system libgit2 1.9.7) renders the scrubbed mirror into `/source/git/`.
It is operator-installed like git-filter-repo: never vendored. `resolveStagit`
- (`publish/sourceHistory.ts:115`): `paths.stagitBin` (`STAGIT_BIN`, else `"stagit"`, `lib/paths.ts:314`)
+ (`publish/sourceHistory.ts:122`): `paths.stagitBin` (`STAGIT_BIN`, else `"stagit"`, `lib/paths.ts:314`)
— a name with a slash is that file or nothing; a bare name is PATH, then `~/.local/bin/<name>`.
- It has no version flag: `stagitIdentity` (`:129`) is the first 12 hex of its binary's sha256,
+ It has no version flag: `stagitIdentity` (`:136`) is the first 12 hex of its binary's sha256,
never the path (the manifest is published, and a home-dir path is a denied literal).
- **What stagit does** (read in `stagit.c`): it writes into its CWD; the repository's NAME is its
directory's basename minus `.git` — so the step's scratch clone is `<scratch>/archilyzer.git`
- (`source.ts:831`), no longer `bare`; `description` (read with `fgets`, the newline KEPT — write it
+ (`source.ts:834`), no longer `bare`; `description` (read with `fgets`, the newline KEPT — write it
without one) and `url` (newline stripped) come from the repository directory; the header links
`<relpath>file/README.md.html` and `LICENSE` when HEAD has them, and the logo `<a href="../<relpath>">`
lands on `/source/` from every depth; `file/` pages are written for HEAD's whole tree on EVERY run;
@@ -7797,9 +7797,11 @@ source mirror (homepage)". Anchors are at the branch.
commits remaining, fetch the repository", but still writes a page for EVERY commit (it only skips
the diffstat of a page that exists); `-c` with `-l` is a usage error; a diff's paths are percent-encoded (`/` and `,-.` kept); a commit over 1,000 files or
100,000 added or deleted lines prints "Diff is too large, output suppressed"; `-c <file>` walks from
- HEAD to the commit the cache file names, KEEPS every `commit/<sha>.html` already in the CWD, and
- appends the cached log lines — so the cache is the whole output directory, and a cache whose commit
- is not an ancestor duplicates the log. Its temp cache file (`cache.XXXXXXXXXXXX`) is made in the CWD.
+ HEAD to the commit the cache file names — in COMMIT-DATE order, so a `--no-ff` merge of commits
+ older than that commit leaves them out (the step's count check then renders every page again,
+ about 8 s: common with parallel slices, and correct) — KEEPS every `commit/<sha>.html` already in
+ the CWD, and appends the cached log lines — so the cache is the whole output directory, and a cache
+ whose commit is not an ancestor duplicates the log. Its temp cache file (`cache.XXXXXXXXXXXX`) is made in the CWD.
- **The cap** (`SOURCE_HISTORY_MAX_COMMITS`, 10,000; ruled 2026-09-30): up to it, `-c`; past it,
`-l 10000`. Either way the pages published are exactly the commits the log links
(`loggedCommits`: every `href="commit/<sha>.html"` in log.html), each of which must exist, and the
@@ -7808,53 +7810,64 @@ source mirror (homepage)". Anchors are at the branch.
diffstats (no `-c`): 8.75 s at 1,834 commits with every page present, against 0.58 s with `-c`.
The oldest published page's parent link is then a 404.
- **The step** (`source.ts`, step 12b) runs after the object audit and the mirror's staging,
- before the manifest and the file audit: `stageHistoryPages` (`:1109`) → `renderHistory`
- (`sourceHistory.ts:497`). **An allowlist is staged** (`TOP_FILES`, `COMMIT_PAGE`, `:87`): `log.html`,
+ before the manifest and the file audit: `stageHistoryPages` (`:1114`) → `renderHistory`
+ (`sourceHistory.ts:573`). **An allowlist is staged** (`TOP_FILES`, `COMMIT_PAGE`, `:94`): `log.html`,
`files.html`, `refs.html`, `atom.xml`, `tags.xml`, `commit/<40 hex>.html`, plus `style.css`. Never
`file/`, never a leftover. Pages are rewritten byte for byte (latin1 in and out; every injection is
ASCII: the back link says `·`).
-- **The post-pass** `rewriteHistoryPage` (`:296`), pages only: `<script>` + the homepage's pre-paint
- script (`homepageThemeScript`, `:81` = `buildThemeScript({ defaultBase: HOMEPAGE_DEFAULT_BASE })`)
- before `</head>`; `HISTORY_BACK_LINK` (`:78`) after `<body>`; `href="(../)*file/<p>.html"` →
+- **The post-pass** `rewriteHistoryPage` (`:327`), pages only: `<script>` + the homepage's pre-paint
+ script (`homepageThemeScript`, `:88` = `buildThemeScript({ defaultBase: HOMEPAGE_DEFAULT_BASE })`)
+ before `</head>`; `HISTORY_BACK_LINK` (`:85`) after `<body>`; `href="(../)*file/<p>.html"` →
`href="$1../tree/<p>"`; `logo.png`/`favicon.png` → `/icons/icon-32.png` (a route the homepage renders
- at build). The theme config moved to `lib/themeConfig.ts` for this (publish may not import
+ at build). With the published set (`PublishedSet`: the staged raw tree's files, decoded; the log's
+ commits), a link to a file main no longer has (a diff of a deleted or renamed file: 3,342 anchors at
+ 1,882 commits) or to a commit with no page loses its ` href="…"` and keeps its text and its `id`
+ (a diff header is the diffstat's `#h<n>` target); `a:not([href])` is styled as text. At `4dfe21e5`
+ every one of the 32,609 remaining tree links and every commit link resolves. The theme config moved to `lib/themeConfig.ts` for this (publish may not import
components/; `components/themeConfig.ts` re-exports it); `HOMEPAGE_DEFAULT_BASE` (`:62`) is what the
homepage layout passes. The attribute the script sets is `data-base`, not `data-theme`.
-- **`style.css`** is `historyStylesheet` (`:208`) over `common/styles/tokens.css`: the light block
+- **`style.css`** is `historyStylesheet` (`:215`) over `common/styles/tokens.css`: the light block
(`:root, html[data-base="light"]`), the dark block for `html[data-base="dark"]` AND for
`:root:not([data-base])` under `prefers-color-scheme: dark` (no JS); `HISTORY_TOKENS` and whatever
they reach through `var()`. Diff `+` lines are `--success`, `−` lines `--destructive`; there is no
chart-safe red in the tokens (the chart palette is blue, green, violet, amber, magenta, rust).
-- **Never a failed build over the history.** No stagit: one line with the install (`source.ts:970`).
+- **Never a failed build over the history.** No stagit: one line with the install (`source.ts:973`).
A render that fails (`HistoryProblem`), a stagit timeout (600 s), unreadable tokens, more files
than the step's 15,000 in all, or a page over 24 MiB: a `[source] WARNING: …` and no history. A
cancel still cancels; any other error is a crash as before.
- **The manifest's `history`** (`lib/sourceManifest.ts:88`, parsed at `:140`, all-or-nothing):
`href`, `commits` (`git rev-list --count`), `head`, `files`, `bytes`, `sha256` (`historyDigest`,
- `source.ts:505`: `sourceDigest`'s walk over `source/git/**` alone), `tool`. `sourceDigest` (`:489`)
+ `source.ts:508`: `sourceDigest`'s walk over `source/git/**` alone), `tool`. `sourceDigest` (`:492`)
now walks `source/git` too. The state (`.source-publish.json`) gains `stagit` and `history:
{files, digest} | null`; the skip needs both to match, and a stagit present with no history never
skips. `SOURCE_STEP_VERSION` is 4 (`:109`).
-- **The deploy check** (`publishedSourceProblem`, `:1244`) names the history on its own (`:1326`):
+- **The deploy check** (`publishedSourceProblem`, `:1254`) names the history on its own (`:1336`):
`out/source/git`'s digest must be the state's (null when none) and the manifest's.
- **The render cache** is `${XDG_CACHE_HOME:-~/.cache}/archilyzer/source-history/`
(`getPaths().sourceHistoryCacheDir`; an empty `XDG_CACHE_HOME` is unset; ruled 2026-09-30):
`key.json`, `stagit.cache`, `out/`, and `lock` while held. `historyCacheFor` (`source.ts`) gives
- none for `--check` and none, with a line, inside the checkout or the public dir. `holdHistoryCache` (`sourceHistory.ts:443`): the lock
- names the holder's pid; a live holder → render without the cache; a dead one → emptied and taken.
+ none for `--check` and none, with a line, inside the checkout or the public dir. `holdHistoryCache`
+ (`sourceHistory.ts:508`): the lock names the holder's pid; a live holder with a lock under an hour
+ old (`HISTORY_LOCK_STALE_MS`, `:498`) → render without the cache; a holder not running, or a lock
+ over an hour old → emptied and taken, with one line. An I/O error on the dir, the lock or the key
+ (`isIoError`, `:653`: an errno code — EACCES, EROFS, ENOSPC) → one line and a render without the
+ cache; a key that cannot be written is no key. Every line `renderHistory` logs goes through
+ `maskLiterals` (`source.ts`' `onLog` for it).
Its pages are kept when the key (`historyCacheKey`: rules hash, filter-repo, stagit, description,
clone URL, base URL) matches and `out/log.html` exists; its `-c` file only when the commit it names
is an ancestor of the head (`git merge-base --is-ancestor`), else that file alone goes. A page of a
commit no longer in history is never published (only the log's commits are). The key is removed
before a render and written after it. A cached render whose log names a page that is not there
is rendered again from nothing, once. `--force` renders
- afresh; `--check` renders in its own scratch; a refusal removes the cache (`source.ts:713`).
+ afresh; `--check` renders in its own scratch; a refusal removes the cache (`source.ts:716`).
- **Measured (2026-09-30, `main` `6b8aa450` → mirror `8c924ca31701`):** 1,872 commits; 1,878 files,
141,299,532 bytes; the largest `commit/7ddfc955….html`, 5,017,646 bytes; stagit 8 s of a 32 s
publish (every page); the whole publish 4,396 files, `homepage/out` 4,576. On the published mirror
at 1,834 commits: 8.0 s every page, 2.0 s from the cache with nothing new. The cache is 138 MB.
At `main` `4dfe21e5`: 1,882 commits, 1,888 files, 4,409 staged in all. One file per commit: the cap
keeps the history at 10,006 files at most; the 15,000-file drop is the last resort.
+- **`_headers`**: `/source/git/*` is `X-Robots-Tag: noindex` (ruled at review, M1), like the raw
+ tree; no other rule matches it, so the pages keep their own types (`headers.test.ts` pins both).
- **Doctor** (`bin/doctor.ts`): the stagit line ends `; cache: <path>, <size>`. Since the merge of
slice DT it reads the settings before the corpus and applies `settings.storage.health`
(`applyHealthTimings`) before it inspects any drive, and prints the timings in force on a
diff --git a/plans/release-15.md b/plans/release-15.md
@@ -1345,7 +1345,8 @@ stagit is operator-installed, like git-filter-repo.
source/git/commit/<sha>.html (contents, byte N)` and never by its bytes. The refusal withdraws the
publish and removes the render cache.
- **The manifest and the key.**
- - The manifest gains `history: {href, commits, head, files, bytes, sha256, tool}`.
+ - The manifest gains `history: {href, commits, total, head, files, bytes, sha256, tool}`
+ (`total` since ruling 1, below: `commits` is how many have a page).
`sha256` is `historyDigest`, the content digest's walk over `source/git/**`; `tool` is `stagit
(sha256 <12>)`, since stagit has no version flag.
- `sourceDigest` walks `source/git` too.
@@ -1354,8 +1355,9 @@ stagit is operator-installed, like git-filter-repo.
- `publishedSourceProblem` names history pages that are not the audited ones: "homepage/out's
history pages (/source/git/) are not the ones that were audited".
- `SOURCE_STEP_VERSION` is 4.
-- **The render cache** — `<ARCHILYZER_SOURCE_SCRATCH>/archilyzer-source-history/`: the `-c` file,
- stagit's output, a key.
+- **The render cache** — ~~`<ARCHILYZER_SOURCE_SCRATCH>/archilyzer-source-history/`~~, since ruling 2
+ `${XDG_CACHE_HOME:-~/.cache}/archilyzer/source-history/` (below): the `-c` file, stagit's output, a
+ key.
- It is used only when the key matches (the rules hash, filter-repo, stagit, the header text),
the cached commit is an ancestor of the head, and the last run finished (the key is removed
before a render).
@@ -1443,9 +1445,10 @@ against libgit2 1.9.7. The binary was copied to `~/.local/bin/stagit` (sha256 `8
last resort.
- **`/source/git/` has no index page**, so the directory itself is a 404; `/source/` links
`log.html`. A copy of `log.html` as `index.html` would double 0.6 MB.
-- **No `_headers` rule for `/source/git/*`.** The pages are HTML by extension, and stagit encodes
- every repository string. No CSP was added, because the inline theme script would need its hash in
- `_headers` on every change.
+- ~~**No `_headers` rule for `/source/git/*`.**~~ Since the review (M1, ruled): `X-Robots-Tag:
+ noindex`, like the raw tree. The pages are HTML by extension, and stagit encodes every repository
+ string. No CSP was added, because the inline theme script would need its hash in `_headers` on
+ every change.
- **The skip does not see a code change to the step** (the post-pass, the stylesheet). The next
`main` does, and every merge moves `main`. `--force` does it at once.
- **The mirror has 1,872 commits against 1,874 on the private main.** filter-repo prunes commits
@@ -1518,12 +1521,55 @@ against libgit2 1.9.7. The binary was copied to `~/.local/bin/stagit` (sha256 `8
`source.ts`, `sourceHistory.ts`, doctor and one page component; the source specs cover the page.
**Found and left, from the rulings:**
-- **Past the cap, the oldest published commit page links its parent's page, which is not
- published** (a 404). Every other link resolves.
+- ~~**Past the cap, the oldest published commit page links its parent's page, which is not
+ published** (a 404). Every other link resolves.~~ Corrected at review (L4): 3,327 links to files
+ main no longer has were 404s too. Since the fix, a link to what is not published is text (below).
- **Past the cap the render is not incremental**: stagit cannot combine `-l` with `-c`, so each
publish computes 10,000 diffstats (about 48 s by the per-commit rate measured here). The pages
themselves are still kept.
+#### Review: SHIP AFTER FIXES, and the fixes (2026-09-30)
+
+The review (`$T/sg-review.md`, at `10ae765b`) found no High, one Medium and eight Lows; the gate needs
+no change. The parent ruled on each; L7 (the live check greps the live history pages) is the
+parent's, for the runbook.
+
+| Finding | Ruling | Where |
+|---|---|---|
+| M1: the history pages are indexable; the raw tree is `noindex` | **Ruled: `noindex`** (the alternative, indexed, was not taken) | `dd5d03ee`: `_headers` `/source/git/*` `X-Robots-Tag: noindex`; `headers.test.ts` pins it and the pages' own types; PUBLISH.md `2f02cc7e` |
+| L1: the retry line quoted stagit's words unmasked (the home dir in its argv) | Mask | `8a960a0b`: `renderHistory`'s `onLog` masks every line; the cache-location line too; a test plants a literal in the cache's path |
+| L2: a cache I/O error failed the build | One line, render without the cache | `8a960a0b`: EACCES/EROFS/ENOSPC on the dir, the lock or the key → `the render cache <dir> is unusable (<code>); rendering without it`; a key that cannot be written is no key; tests with a read-only cache dir and an unmakeable one |
+| L3: stagit's `-c` misses a `--no-ff` merge of older commits | Say so | `8a960a0b`: the retry line says it ("stagit's -c stops at the last head it rendered, and a merge of older commits falls behind it"); a test merges two commits dated 2001; PUBLISH.md and FACTS |
+| L4: 3,327 tree links were 404s; the record said every link resolves | Unlink | `8a960a0b`: a link to a file not in the staged raw tree, or to a commit with no page, keeps its text and loses its `href` (a diff header keeps its `id`, the diffstat's target); the real render has 3,342 such anchors and 32,609 tree links, **every one resolving**, and every commit link; tests with a deleted file and past the cap (the parent) |
+| L5: stale records | Fix | this commit (the cache path, `total`, the link sentence); PUBLISH.md's timings say which is stagit's and which the step's (`2f02cc7e`); `source publish`'s usage names the history (`8a960a0b`) |
+| L6: the file-count drop was untested | Test | `8a960a0b`: a `historyFileLimit` seam; the test drops nine history files with the WARNING, and publishes the rest |
+| L8: a reused pid could hold the lock forever | Stale by pid or age | `8a960a0b`: a lock whose pid is not running, or over an hour old (`HISTORY_LOCK_STALE_MS`), is replaced with one line |
+
+| Commit | What |
+|---|---|
+| `dd5d03ee` | `homepage:` `/source/git/*` noindex; `headers.test.ts` +1. |
+| `8a960a0b` | `common:` L1, L2, L3, L4, L5 (usage), L6, L8; `sourceHistory` +4 tests, `source` +2. |
+| `2f02cc7e` | `docs:` PUBLISH.md. |
+| this commit | `plans:` this subsection, the corrections, FACTS. |
+
+**Re-gates** (at `2f02cc7e`; logs `$T/sg-*-3.log`, `sg-tsc-8.log`, `sg-pub3.log`, `sg-capped.log`):
+- **tsc** clean before each commit.
+- **common 2,347/2,347**, 0 skipped (+6: `sourceHistory` 17, `source` 21).
+- **homepage unit 23/23** (+1); **test:scripts 195 + 1 skipped**; `docs env --check` 0, `docs files
+ --check` 0.
+- **The three source specs** (`E2E_EXPECT_SOURCE=1`, over a fresh `--force` publish): **12 passed,
+ 0 failed, 17.2 s**.
+- **`source publish --force`**: `history: … 1882 commits, 1882 pages rendered; 1888 files, 135.1 MB`;
+ `audit clean: 23,128 objects (1,882 commits), 4,411 staged files …; gitleaks clean`; published as
+ `f81edb46f7e3`. Over the rendered pages: 32,609 tree links, 0 that do not resolve; 0 commit links to
+ an unpublished page; `grep -rli` of the user name and of the hostname: 0 files each.
+- **`source publish --check`**: `check passed — would publish main 4dfe21e5714a as f81edb46f7e3: 4411
+ files, 212.5 MB (…, history 1882 commits in 1888 files) …; nothing written (32 s)`.
+- **Capped builds with the corpus linked** (`transcripts/` moved aside, `ln -sT` to the primary's,
+ restored after; `systemd-run … MemoryMax=5G`): editor `next build` **ok, 34 s**; homepage `next
+ build` **ok, 13 s**. No editor `.nft.json` names `transcripts/channels`, `homepage/public/source` or
+ the cache.
+
## Rollout
