commit aa79dfcc411326b2615fe8e9aabb2c417b8ddfcd
parent b8b2cab4b370f637ac99c9af0a9820a1712fb4f4
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Tue, 6 Oct 2026 09:50:18 -0400
plans: release 18 — slice S1 review fixes (the table, the re-gates); the changelog says deploy all's skips and the no-branch refusal
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Diffstat:
2 files changed, 47 insertions(+), 16 deletions(-)
diff --git a/editor/CHANGELOG.md b/editor/CHANGELOG.md
@@ -1,7 +1,7 @@
# Changelog
## [Unreleased]
-- **Publishing is stages, from the command line: `archilyzer publish`.** `publish index` updates the index — the LMDB index, the stats datasets and the chart templates, in one child process with an 8 GB heap — and writes an index stamp (`export/.export-index/stamp.json`) naming, for each site, a signature of everything that site's build reads. `publish build <id|all>` builds a site from that index (no data phase of its own) into its own bundle, `export/.export-builds/<id>/out`, and stamps it (`built.json`); a site whose bundle already matches the index is a no-op unless `--force`. `publish deploy <id|all> [--preview <branch>] [--to local]` ships that bundle — to Cloudflare Pages, or with `--to local` into the directory the docker `site` service serves — and records the deploy (`deployed.json`); deploying the same build again is a no-op unless `--force`. `publish hub [--deploy]` and `publish homepage [--deploy]` do the same for the hub (`_hub/out`) and the homepage. A stage whose input is not there says so and exits 3: "update the index first", "no build of jeralyzer — archilyzer publish build jeralyzer". Production refuses a bundle built on a branch other than `main` (a preview of it is fine). Exit codes: 0 done or nothing to do, 1 failed, 2 usage, 3 precondition not met, 130 cancelled.
+- **Publishing is stages, from the command line: `archilyzer publish`.** `publish index` updates the index — the LMDB index, the stats datasets and the chart templates, in one child process with an 8 GB heap — and writes an index stamp (`export/.export-index/stamp.json`) naming, for each site, a signature of everything that site's build reads. `publish build <id|all>` builds a site from that index (no data phase of its own) into its own bundle, `export/.export-builds/<id>/out`, and stamps it (`built.json`); a site whose bundle already matches the index is a no-op unless `--force`. `publish deploy <id|all> [--preview <branch>] [--to local]` ships that bundle — to Cloudflare Pages, or with `--to local` into the directory the docker `site` service serves — and records the deploy (`deployed.json`); deploying the same build again is a no-op unless `--force`. `all` passes over private sites and, to Pages, sites with no Pages project; any other site it cannot deploy is a failure, said after the rest are tried. `publish hub [--deploy]` and `publish homepage [--deploy]` do the same for the hub (`_hub/out`) and the homepage. A stage whose input is not there says so and exits 3: "update the index first", "no build of jeralyzer — archilyzer publish build jeralyzer". Production refuses a bundle built on a branch other than `main`, or with no branch recorded (a detached checkout; an image sets `ARCHILYZER_BRANCH`) — a preview of it is fine. Exit codes: 0 done or nothing to do, 1 failed, 2 usage, 3 precondition not met, 130 cancelled.
- **One publish at a time on a machine.** Every stage takes `export/.export-builds/.publish.lock`; a second one — an `archilyzer publish` beside the editor, say — waits for it, saying once whom it waits for, and Ctrl-C ends the wait. A lock left by a process that is gone is taken over. A cancelled stage takes the whole process tree it started with it (`next build`'s workers, wrangler, docker).
- **`export/out` is now a link to the bundle built last.** Each site, and the hub, keeps its own bundle, so building one site no longer replaces another's; `export/out` points at whichever was built most recently, so `serve out` and anything else that read it keeps working.
- **`build site`, `build all` and `deploy site` are aliases of the publish commands** and print what they run: `build site <id>` is `publish index` (skipped with `--nodata`) then `publish build <id> --force`; `build all` is `publish index` then `publish build all --runner auto` (containers when an engine answers, else one site at a time on the host); `deploy site <id>` is `publish deploy <id>`, which now ships the site's own bundle and refuses a site never built that way. `publish build all --runner docker` builds every stale site in containers on a Linux host and refuses with "the docker runner needs an engine on this host" where there is none.
diff --git a/plans/release-18.md b/plans/release-18.md
@@ -371,12 +371,13 @@ rewire, the status view, the lane and `publish status|now` S3's.
- **`needs()`, row by row.** update-index: stale with no stamp, an ingest ended `done` after `stamp.scannedAt`, or a
config newer than it. build-site: BLOCKED "update the index first" with no stamp (forced too), "waiting for the
index update this run started" under `indexAfter`, "the index has not seen site X" when the stamp has no entry;
- then stale by `changedChannels` ("N channels changed (a, b, …)"), a config change after `built.builtAt`, an
+ then stale by `changedChannels` ("N channels changed (a, b, …)"), a config change after `builtCheckedAt(built)`, an
`inputSig` mismatch ("data changed"), a bundle problem, never built; `--force` stale; a `built.commit` that differs
is NOT stale. `_all`: per site. deploy-*: blocked with no build ("no build of X — archilyzer publish build X"), a
- private target (every kind), no Pages project (pages kinds only), a bundle problem, `builtAfter`, and a PRODUCTION
- deploy of a bundle built on a branch other than `main` (a null branch — an image with no `ARCHILYZER_BRANCH` —
- passes); fresh exactly when `deployed[kind/branch].builtStampId === built.stampId`. build-hub: `built.inputSig ===
+ private target (every kind), no Pages project (pages kinds only), a bundle problem, `builtAfter` (unless the
+ bundle matches the index this run updated — see the review fixes), and a PRODUCTION deploy of a bundle whose
+ `branch` is not `main` — a null branch (a detached HEAD, an image built without `ARCHILYZER_BRANCH`) is refused the
+ same way; fresh exactly when `deployed[kind/branch].builtStampId === built.stampId`. build-hub: `built.inputSig ===
stamp.hubSig` (+ `changedChannels`). build-homepage: `indexStampId` current and `sourceCommit === mainHead` when a
repository answers.
- **`common/publish/stamps.ts`** — `IndexStamp`, `BuiltStamp`, `DeployRecord`, `DeployedFile`, `LiveCheck`, `Probe`
@@ -384,19 +385,26 @@ rewire, the status view, the lane and `publish status|now` S3's.
writes (`writeJsonAtomic`); tolerant reads (missing, unparseable or wrongly shaped = null); `recordDeploy` keeps
every other record; `newStampId` sorts by time; `imageBuildFacts` (the image's `ARCHILYZER_COMMIT` /
`ARCHILYZER_BRANCH`, empty = null — S5's names, read by name here until S5's helper of the same name replaces it).
+ `BuiltStamp.checkedAt?` (review fix): when a later no-op build last found the bundle still matching its inputs.
+- **Commit and branch in a stamp (the S4 seam):** `ARCHILYZER_COMMIT` / `ARCHILYZER_BRANCH`, when set, WIN over git,
+ each on its own — the runtime image bakes them (no `.git`), and S4's e2e webServer sets `ARCHILYZER_BRANCH=main`,
+ because a worktree's branch is never `main` and production refuses any other. Else git's HEAD and branch; a
+ detached HEAD records `branch: null`.
- **`common/publish/stageLock.ts`** — `<exportBuildsDir>/.publish.lock` `{pid, host, kind, target, since, pidStart}`,
- `open(…, "wx")`; stale when same host and the pid is dead (`processIsAlive`) or answers with another
- `/proc/<pid>/stat` start time (a container's small pids come back after a restart); another host's is never
- stolen; a torn file is taken over after 60 s; a live holder is waited for (5 s poll, ONE log line, the signal
- cancels the wait); release removes only its own.
+ `open(…, "wx")`; the host is `ARCHILYZER_HOST_ID` when set (S5 fixes one in compose), else `os.hostname()`; stale
+ when same host and the pid is dead (`processIsAlive`), answers with another `/proc/<pid>/stat` start time, or names
+ a process that STARTED AFTER the lock's `since` (starttime/100 + `/proc/stat` btime, 2 s slack; a recreated
+ container's pid 1) — with no `/proc`, pid-alive alone; another host's is never stolen, and its wait line names
+ both hosts and how to clear it; a torn file is taken over after 60 s; a live holder is waited for (5 s poll, ONE log
+ line, the signal cancels the wait); release removes only its own.
- **`common/publish/stageRun.ts`** — `runStage(req)` (in-process, under the lock, never throws: `{code, outcome,
message}`), exit codes 0 / 1 / 2 / 3 / 130, `stageMain` (the child: SIGTERM/SIGINT abort the stage, a second one
- exits, tree-kill on), and **`stageCommand(paths, req)`** — `<common>/node_modules/.bin/tsx bin/archilyzer.ts stage
+ SIGKILLs the child process groups and exits, tree-kill on), and **`stageCommand(paths, req)`** — `<common>/node_modules/.bin/tsx bin/archilyzer.ts stage
<kind> <target> [flags]`, cwd `common/`, the caller's env + `NODE_OPTIONS=… --max-old-space-size=8192` for
update-index only — what S3's `enqueueStage` hands `runManagedCommand`.
- **`common/publish/stageBodies.ts`** — every body but update-index first asks its `needs()` over the state ON DISK
(`readNeedsInput`: the stamps, the bundles, `siteDeployProblem`, the Pages project, `main`'s head; no job metas):
- blocked → exit 3, fresh and not forced → a no-op. **update-index**: `buildIndex` → `buildStats` → the chart
+ blocked → exit 3, fresh and not forced → a no-op (a build's no-op writes `checkedAt`). **update-index**: `buildIndex` → `buildStats` → the chart
templates in ONE process, then the stamp — `generation`, `scannedAt` and each site's `siteFp`/`statsFp` (sha1 of
the LMDB keys, read-only), each site's `inputSig` and the `hubSig` (`common/publish/inputSig.ts`). An index that
rebuilt nothing and whose every signature is unchanged KEEPS its stamp id (status `noop`), so the builds made from
@@ -413,13 +421,18 @@ rewire, the status view, the lane and `publish status|now` S3's.
`.export-index/sites/<id>/` tree (chart-templates.json by its BYTES — `build templates` rewrites it every run), each
PUBLISHED member's shared transcripts/subs/posts/digests tree (`manifest.json` ignored, a manifest-only tree as
compose's constant), `site.json`'s bytes, the `sites/<id>/` dir, the global aliases, curated tags and duplicates
- files (size + mtime), and `archiveStorage` + `social.x.visibility`. A superset of compose's skip inputs:
+ files (size + mtime), `archiveStorage` + `social.x.visibility` + `buildArchives`, and — what the export BUILD
+ renders beyond compose (review fix) — the resolved social links, the resolved hub url and the footer's sibling
+ sites (`resolveRelatedSites(site, listSites())`: each sibling's url, title and listing). The rule: a site is fresh
+ exactly when compose AND the export build would produce the same bundle. A superset of their inputs:
conservative. `hubSig` = sha1(stampId, homepage.json, each listed site's id + siteUrl + title).
- **`common/lib/dirSignature.ts`** — compose's `dirSignature`, moved unchanged; `compose-site.ts` imports it (that
line, and its now-unused `createHash` import removed, are the only compose-site edits).
- **`common/publish/build.ts`** — per-target bundles: `bundleDir(paths, target)` (= `dockerSiteOutDir` for a site),
`installBundle(src, <target>/out)` (rename into `out.next`, `out → out.prev`, `out.next → out`, `out.prev` removed, a
- leftover `out.next` deleted first; EXDEV → `fs.cp` + remove, injectable `BundleFs`), `unlinkExportOut` (before a
+ leftover `out.next` deleted first; EXDEV → `fs.cp` + remove, injectable `BundleFs`), `recoverInterruptedInstall` (an
+ `out.prev` with no `out` is renamed back before a build or an install touches the target; the old `built.json` is
+ removed before the swap and the new one written last), `unlinkExportOut` (before a
build: a link at export/out is removed so a failed build cannot leave an older bundle there), `pointExportOutAt`
(export/out → a RELATIVE symlink, replaced atomically), `stageSiteArchives` (the host compose's `.r2-staging/<id>`
moved to `dockerSiteStagingDir`, a no-op where they are one place), `bundleCounts`, `corpusGeneratedAtIn`,
@@ -429,8 +442,8 @@ rewire, the status view, the lane and `publish status|now` S3's.
unchanged; the editor's actions still build into `export/out` (S4 rewires them).
- **`common/bin/archilyzer.ts` + `common/bin/publish.ts`** — rows `publish index` (the SAME child the editor spawns,
for its heap), `publish build <id|all> [--runner local|docker|auto] [--force] [--skip-archives]`, `publish deploy
- <id|all> [--preview b] [--to local] [--force]` (`all` skips a site refused with 3 — private, no project, never
- built — and fails on anything else), `publish hub [--deploy] [--preview b] [--force]`, `publish homepage [--deploy]
+ <id|all> [--preview b] [--to local] [--force]` (`all` passes over, one line each, only a private site and — to
+ Pages — a site with no project; every other refusal is a failure, the rest are still tried and the run exits 1), `publish hub [--deploy] [--preview b] [--force]`, `publish homepage [--deploy]
[--preview b] [--to local] [--force]`, and the internal `stage <kind> <target> --run-id …`. A comment marks where
S3's `publish status` / `publish now` rows go. **Aliases, printed first**: `build site <id>` = `publish index` (not
with `--nodata`) + `publish build <id> --force`; `build all` = `publish index` + `publish build all --runner auto`;
@@ -496,6 +509,23 @@ envVars names above.
| `3033d9f2` | stamps fall back to the image's commit/branch; `deploy-homepage --to local` → `ARCHILYZER_HOMEPAGE_OUT` |
| `89b16716` | `.gitignore`: `/export/out` |
+**Review fixes** (review `s1-review.md`: SHIP AFTER FIXES; the coordinator's list, plus S5's host-id note):
+
+| sev | fix | commit |
+|---|---|---|
+| HIGH | `inputSig` also signs the resolved social links, the hub url, `buildArchives` and the footer's sibling sites; one test each | `b88cbe8d` |
+| MEDIUM | the lock's host is `ARCHILYZER_HOST_ID` (else the hostname); a pid whose process started after the lock's `since` is not its holder (`/proc` start time, injectable; no `/proc` = pid-alive alone); a foreign host's wait line names both hosts and how to clear it | `99a8a939` |
+| MEDIUM | a run's no-op build no longer holds its deploy: under `builtAfter` the bundle counts as current when it matches the current index (`inputSig` / `hubSig` / `indexStampId`) and that index ran at or after `builtAfter`; a no-op build writes `built.checkedAt`, and `changedChannels` / config changes are measured against `builtCheckedAt` = max(builtAt, checkedAt) — S3's chip uses the same | `d1d19add` |
+| LOW | a detached HEAD records `branch: null`, and production refuses null like any branch but `main` | `d1d19add` |
+| SEAM | `ARCHILYZER_BRANCH` / `ARCHILYZER_COMMIT` win over git in the stamps (S4's e2e sets `ARCHILYZER_BRANCH=main`) | `d1d19add` |
+| MEDIUM | `publish deploy all` skips only private and (to Pages) project-less sites; every other refusal fails the run (exit 1) after the rest | `32f8a37d` |
+| LOW | an interrupted install (`out.prev`, no `out`) is restored before anything else; `built.json` is removed before the swap, written last | `32f8a37d` |
+| LOW | a second SIGTERM / Ctrl-C (CLI and stage child) SIGKILLs the detached process groups before exiting (`killChildTreesNow`) | `32f8a37d` |
+| LOW | the `stage` usage names `--allow-missing-media` | `32f8a37d` |
+
+Not taken (the review's other lows, left for S6's list): the read-then-`rm` race in `removeIfUnchanged`; the lock's
+place beside the checkout's builds dir while a worktree's `export/public` links into the primary's.
+
**Gates** (all from the worktree root): tsc clean at every commit; common **3219 passed** (58 new: stamps 6,
stageLock 8, stages 21, bundle 7, stageRun 6, inputSig 4, runChild 2, `_cli` +4); editor unit **142**;
`test:scripts` **596 + 3 skipped**; mcp **289**; export unit **116**; homepage unit **23**; `pnpm --filter editor exec
@@ -505,7 +535,8 @@ suite, `s1-specs.txt`: build, deploy-page, site-publish-preview, sites-homepage,
ops-api): **52 passed, 0 failed, 2.5 min** — after a first launch died on "Timed out waiting 120000ms from
config.webServer" (the linked primary `export/public` held a cited site's compose, so the export dev server 500ed on
`summaries/manifest.json`); re-run with a scratch FULL site composed into the worktree's own `public/` (FACTS :3485's
-"Copy a composed fixture site into it"), cleaned after. Numbers tool: none. Live smoke over a scratch corpus (`s1-smoke-build.sh`): `publish index`
+"Copy a composed fixture site into it"), cleaned after. **After the review fixes:** tsc clean; common
+**3229 passed** (10 new: inputSig +1, stageLock +3, stages +2, stageRun +2, bundle +1, runChild +1); e2e (same seven, same seeding) **52 passed, 0 failed, 2.1 min**. Numbers tool: none. Live smoke over a scratch corpus (`s1-smoke-build.sh`): `publish index`
(9 s) → `publish build smoke` (85 s, bundle installed by rename, export/out a relative link) → again: no-op →
`stage deploy-site … --to local` without the env: refused → `publish deploy smoke --to local`: copied + recorded →
`publish hub`: refused by `builtHubProblem` (above) → `build site smoke --nodata`: alias printed, forced rebuild.
