# Release 13 — the lows bundle, the build image refreshed, one-core Phase 5 slices 1–3 `main` at `bf6904e8`: release 11 merged in full and **cut as 0.10.0** on 2026-09-28 (editor `24c8352e`, export `bf6904e8`), its rollout in progress (`release-11.md`, "Rollout, as done"). Release 13 is the second half of the operator's 2026-09-28 choice of all four bundles: the lows left by release 11, the `Dockerfile.build` image refreshed and proved, and one-core Phase 5 ([`one-core.md`](one-core.md), "Phase 5") slices 1–3, after a corrected plan doc. Rules: `plans/tools/implementer-rules.md`, with the commit trailer this release's prompts give. **Why 13, not 12:** a parallel session owns **release 12** — the source mirror (`plans/release-12.md` on `r12/paths-fix`; slices Q then R; plan [`source-mirror.md`](source-mirror.md)). The two releases share `editor/CHANGELOG.md`'s `[Unreleased]` and a few files (`common/bin/doctor.ts`, `common/lib/envVars.ts`, `common/lib/paths.ts`, umtool's path defaults, the docs); whichever merges second resolves the join, and a code conflict goes back to its slice. **The operator's standing choices** (2026-09-28): - **The LM chat-only tier is closed as moot:** Legal Mindset's filter has `includeLivestreams: true`, so the tier never changes what is kept. - **Merging:** each slice merges to `main` after its review, `--no-ff`. No cut, no deploy and no restart inside a slice; the parent does the one build-image rebuild, the one build-only Build all and (at the end of Phase 5) the one umtool rebuild + restart. - **Worktrees:** the seven `r11-*` worktrees stay until release 12's worktrees are gone. Ports are index-based, and removing them would move `r12-paths-fix`'s block while that session runs. Release 13's worktrees are named `r13-*`, so they sort after it. ## The slices | Slice | Branch | What | Owns | |---|---|---|---| | W1 | `r13/lows-editor` | Diagnostics cards keep their retry log when a retry empties the bucket; the `transcript-source.spec.ts:76` flake; `/sites` "built " refreshes when a homepage build ends; `/jobs` labels for the four hub/homepage kinds; a queued-cancel writes its final sidecar status; doctor's worker engine binary from `paths` (L7); an `editor/package.json` `test` script | `editor/app/channels/[slug]/components/stages/**`, `editor/app/sites/components/{JobLane,HomepageBuildButtons}.tsx`, `common/jobs/{jobKinds,registry}.ts`, `common/lib/streamCommand.ts`, `common/bin/doctor.ts` (L7 only), the two specs | | W2 | `r13/lows-export` | `ChartView` reaches `--chart-6` through `seriesColor(i)`; an export unit `test` script (named in the rules and CONTRIBUTING); O5's wording lows; the 2origin stage guard; `no-data.spec` skips on `workers > 1`; the mcp docs move to `archilyzer mcp` | `common/components/charts/ChartView.tsx`, `export/package.json`, `umtool/report-to-video/{svg-faces.mjs,README.md}` (comments), `export/playwright.2origin.config.ts`, `homepage/e2e/no-data.spec.ts`, `mcp/README.md`, `AGENTS.md`, `README.md`, `CONTRIBUTING.md`, `plans/tools/implementer-rules.md` | | W3 | `r13/build-image` | `Dockerfile.build` on Node 22 and pnpm 11; the stale `--network=none` comments; `archilyzer doctor` checks the build image's presence and age | `Dockerfile.build`, `common/publish/build.ts` (comments), `common/bin/doctor.ts` (the image check); from W3b `docker/build-site.sh`, `Dockerfile.build.dockerignore` | | P0–P3 | `r13/phase-5` | one-core Phase 5: the corrected plan doc (P0, to the operator first), then slices 1–3 stacked | per `plans/one-core-phase-5.md` | **Order:** W1 ‖ W2 ‖ W3, merged W1 → W2 → W3; then the parent rebuilds the build image and runs one build-only Build all. Then Phase 5, which merges to `main` only when all three slices pass. Then one integration gate on `main` (`r13/integration`) and the runbook (`~/reports/release-13/MORNING.html`). ## Record ### Slice W3, as shipped — the build image refreshed (2026-09-28) Branch `r13/build-image` off `main` `bf6904e8`, worktree `/home/user/Projects/r13-build-image`, one Opus implementer. `git merge main` first fast-forwarded to `441bdbb2` (this file). Three items: `Dockerfile.build` on the workspace's Node and pnpm, its stale `--network=none` comments, and an `archilyzer doctor` check of the image. The proof is a scratch-tag image and a fixture Build all, not e2e. **It found that a container Build all fails every site on `main` as well** — see "Found and left" 1, before the parent's build-only Build all. Fixed in this slice by "Follow-up W3b" below. **`Dockerfile.build`.** - `ARG NODE_IMAGE=node:22.23.2-bookworm-slim` and `ARG PNPM_VERSION=11.26.0`. - Node is the host's `node --version`; nothing else pins it (no `.nvmrc`, no `engines`, no `packageManager`). It is bookworm like the root Dockerfile's build stage. - pnpm is the host's `pnpm --version`, the one the worktree installed with. pnpm 11 needs Node >= 22.13. - `ensureBuildImage` passes no build args, so these defaults are what Build all builds. - **pnpm 11 needed one more line, and the proof found it:** `ENV pnpm_config_verify_deps_before_run=false pnpm_config_update_notifier=false`. - Before every `pnpm exec` / `pnpm run`, pnpm 11 checks that the whole workspace is installed, and runs `pnpm install` when it is not. - The image installs root, common and export. After `COPY . .` the workspace has 8 projects, so the first `pnpm --filter … exec` in `build-site.sh` ran `pnpm install`. As the host uid that died with `EACCES: permission denied, open '/repo/_tmp_…'` (`w3-image-versions.log`). - pnpm 11 reads `pnpm_config_*` from the environment, not `npm_config_*` (both were tried). The update notice is off because every site's log would otherwise print it. - **Comments:** - The header no longer calls the containers "hermetic". It says the network is on (`runDockerBuildOne`, for `next/font/google`) and names the entry command as `archilyzer build site --nodata`. - The corepack reason is now "no `packageManager` pin, so corepack would download a pnpm of its own choosing in every container". It used to say "offline". - The chmod comment rests on `--rm` and "writes nothing back but /site". It used to rest on `--network=none`. - The contract with `build.ts` is unchanged: `WORKDIR /repo`, the entrypoint, the mounts, `-u`, and the chmod of `/repo/export`. **`common/publish/build.ts`** (no behaviour change). - `dockerBin(env = process.env)` is now exported. It was private and read only `process.env`. - `buildImageArgs(pipeline)` is new, and `ensureBuildImage` now runs it. It is the one spelling of the image's `docker build` argv, which the doctor prints. - The comments at `:437-456`: `ensureBuildImage`'s comment now says it runs before every Phase B, what a Dockerfile or lockfile change costs there, and that the doctor warns about it ahead of time. `runDockerBuildOne`'s network comment was already right and is unchanged. **`archilyzer doctor`: a "build image" section** between umtool's and the ports. It is a block of its own: W1 changes the worker-engine line in "tools", and release 12 adds a source block; neither is adjacent. - **The probe.** ` version` must exit 0, which is what `dockerAvailable` asks. Then ` image inspect --format '{{.Created}}|{{.Size}}' `. - **The tag and the Dockerfile** come from the effective settings (`settingsFromFile`, or the defaults when the file is absent). That is the same `getSettings().buildPipeline` that `build.ts` reads, so `yt-dlp-transcript-browser-build` is not spelled a second time. - **The Dockerfile's last change.** - In a checkout it is the file's last commit (`git log -1 --format=%ct`), unless `git status --porcelain` shows uncommitted edits; then it is the file's mtime. - With no checkout, or no git, it is the mtime. - `GIT_OPTIONAL_LOCKS=0` stops `git status` from refreshing `.git/index`, so the doctor stays read-only. Without it, the test's tree comparison fails because `.git/index`'s mtime moves. - **Grading:** - With no engine there is one `--` line, "skipped: `docker version` did not answer …". - With an engine, an absent image is a WARN, and so is an image created before the Dockerfile's last change. Both print the command. Anything else is `ok`, with the date, age and size. - A Dockerfile the settings name that is not there gets its own `dockerfile` WARN. - **Never a FAIL.** The WARNs apply only beside a corpus: with no channels the same lines are notes, as for a tool nothing needs yet (Question 1). - **The command** is `rebuild it now: cd && docker build -f Dockerfile.build -t .`, built from `buildImageArgs` and shell-quoted. `DOCKER_BIN` changes both the engine asked and the command printed. - `parseEngineTime` reads docker's RFC 3339 with nanoseconds and podman's Go default format. - On this machine, run from the worktree (no corpus there, so a note): ``` build image -- build-image "yt-dlp-transcript-browser-build" built 2026-07-07 16:04 (83 days ago), 7.72 GB — before Dockerfile.build's last change (2026-09-28 16:47, its last commit), so the next Build all rebuilds it from the changed step on rebuild it now: cd /home/user/Projects/r13-build-image && docker build -f Dockerfile.build -t yt-dlp-transcript-browser-build . ``` With the fixture corpus and the scratch tag, the same line is a `WARN`. | sha | what | |---|---| | `bb3d0f6a` | `docker:` `Dockerfile.build` on node:22.23.2-bookworm-slim + pnpm 11.26.0 (build args), the pnpm 11 `verify_deps_before_run` / `update_notifier` env, the network and corepack comments | | `40dfc008` | `common:` the doctor's build-image block + 7 tests; `build.ts` exports `dockerBin(env)` and `buildImageArgs`, and `ensureBuildImage`'s comment | | _this_ | `plans:` this record; two `[Unreleased]` bullets in `editor/CHANGELOG.md` | **Gates**, all from the worktree root: - **tsc** (`pnpm -r --no-bail --workspace-concurrency=1 exec tsc --noEmit`) was clean before both commits: `w3-tsc-1.log` (168 s), then `w3-tsc-2.log` (50 s) on the final tree. The only change between the two runs was one comment. - **common: 2,121/2,121** (`w3-common-test.log`, 132 s). The 7 new tests are all in `bin/doctor.test.ts`, so `main` has 2,114; the prompt said 2,112. `test:scripts` does not cover `common/bin`, so it was not run. - No generated doc, no editor or export build, and no e2e: nothing under `editor/app` or `export/` changed, and the proof below replaces e2e. - **The new tests bite.** `w3-bite.log` ran the new `doctor.test.ts` against `main`'s `doctor.ts`, with a one-line `parseEngineTime` shim so the file loads: **8 passed, 7 failed**, and the 7 failures are exactly the new tests. Without the shim the whole file fails to import. Separately, with `GIT_OPTIONAL_LOCKS: "0"` removed, the git test fails on `.git/index`'s mtime. **The proof, instead of e2e.** Every log is under `$T`. The scratch tags were `r13-build-image-test`, `-old` and `-probe`, all removed afterwards. `yt-dlp-transcript-browser-build` was never tagged, built or replaced. 1. **The image.** `docker build -f Dockerfile.build -t r13-build-image-test .` from the worktree root took **53 s** on the first build (base pulled, no cached layers). The final `--no-cache --pull` build took **52 s**. Size **1.67 GB** (1,673,627,497 B), from a 26.7 MB context (`w3-docker-build*.log`). 2. **Inside it** (`w3-image-versions*.log`): - node **v22.23.2**, pnpm **11.26.0**. - As the host uid with `HOME=/tmp`, which is how `runDockerBuildOne` runs it: `pnpm --filter export exec next --version` gives Next.js v16.2.3, and `pnpm --filter yt-dlp-transcript-common exec tsx --version` gives tsx v4.21.0. `lmdb`'s native module opens, writes and reads. - `pnpm ls --depth 0` over export and common lists 53 packages in 2 projects. This was run as root: as another uid, pnpm 11's `ls` opens the store index under `/root` and fails EACCES. `ls` is not on the build path. 3. **Build all's own code over a fixture** (`w3-fixture-build.log`, 40 s). - The run: `pnpm archilyzer build all` from the worktree, with `TRANSCRIPTS_DIR`, `SETTINGS_FILE` and `EXPORT_{PUBLIC,INDEX,BUILDS}_DIR` pointed at `$T/w3-fixture`. - The fixture: editor e2e's `one-youtube-channel-with-data` channel and one site, `w3site`. Settings: `buildPipeline.dockerImage: "r13-build-image-test"`, `maxParallelBuilds: 1`. - Phase A on the host passed: index, stats, templates and the archive warm. - `ensureBuildImage` rebuilt the scratch tag from cached layers. - In the Phase B container, `build-site.sh` composed into `/site/public` (with the read-only archive cache materialized), and `next build` compiled and type-checked. - It then **failed prerendering `/favicon.ico`**: `Failed to load external module next/dist/compiled/@vercel/og/index.node.js: … Cannot find package 'next' imported from /site/.next/server/chunks/[turbopack]_runtime.js`. 4. **The failure predates W3.** `main`'s `Dockerfile.build` (node:20, pnpm 9.15.4) was built as `r13-build-image-old` through the same Build all and failed identically (`w3-fixture-build-old.log`, 136 s). 5. **Two probes of the proposed fix** replaced the entrypoint with a copy from `$T` over the `docker run` argv `runDockerBuildOne` builds. Nothing was committed under `docker/`. - **Probe 1** kept `.next` in the container instead of linking it to `/site/.next`. The run used a derived image with `export/public`'s dangling links deleted, which is what a clean clone has. - It exits 0 with 149 files, but `/site/out` has no `corpus.json`, `_headers`, `llms.txt`, `robots.txt`, `site.json`, `summaries/`, `transcripts/`, `archives/` or `stats/`. All of these are only in `/site/public` (`w3-probe1-run.log`). - **Probe 2** made the same `.next` change and, in addition, ran `cp -an export/public/. /site/public/ && rm -rf export/public && ln -s /site/public export/public`. - It exits 0 with **167 files**, in 51 s. `out/` holds `w3site`'s `corpus.json`, `transcripts/test-youtube/{manifest,page-0000}.json`, `summaries/` and `archives/`, all host-owned (`w3-probe-run.log`). 6. **What only the parent's post-merge Build all can prove:** - the image built from the primary's context: its size, and the time to transfer that context; - every real site at `maxParallelBuilds`, with the real fonts; - the deploy phase. As `main` stands, that Build all fails every site at `/favicon.ico` on the container path (bug A below), with or without W3. **Found and left.** 1. **`docker/build-site.sh` has two bugs, and a container Build all cannot ship a site until they are fixed.** `docker/**` is not W3's. - **(A) `export/.next` is a symlink to `/site/.next`.** - Turbopack's runtime imports next's externals from its own real path, and under `/site` there is no `node_modules`. `next build` dies at `/favicon.ico`. - **(B) `next build` copies `export/public` into `out/`.** - `export/public` is the copy BAKED into the image, not the composed `EXPORT_PUBLIC_DIR=/site/public`. - Built from a clean clone, `out/` would have none of the site's data (probe 1). - Built from the primary, `out/` would carry the primary's stale export/public. That is whatever site the host composed last, without `summaries/` and `transcripts/`, which `.dockerignore` excludes. - **Why no one saw it.** The live image dates from 2026-07-07, and `ensureBuildImage` runs before every fan-out, so no container Build all has run since then. - **The host path is unaffected:** single-site builds, `archilyzer build site`, and Build all when no engine answers. - **The proposed fix** is proven by probe 2 and is uncommitted: `$T/w3-build-site-probe.sh`. Its cost is that `.next` stops persisting between runs, so incremental `next build` is lost. Persisting only `.next/cache` could win that back, but it is untested. 2. **The build context bakes the checkout's generated data, twice.** - From the primary it includes: - the gitignored entries of `export/public` that `.dockerignore` does not name (`subs/` 1.8 GB, `posts/`, `stats/`, `digests/`, `corpus.json`, …); - `.diarize/` (1.3 GB); - umtool's data (1.1 GB). - `/repo/export` is in the image twice, once in the `COPY` layer and once in the `chmod` layer. The live image is 3.23 GB + 3.18 GB of its 7.72 GB. - From a worktree, `export/public`'s entries are absolute links into the primary and are baked dangling. The first probe died on `stat '/repo/export/public/_headers'`. - Fix B keeps this data out of `out/`. It still sizes the image, and it invalidates the `COPY` layer after every host build. - `.dockerignore` is shared with the root `Dockerfile` and `Dockerfile.test`, so W3 left it. The follow-up is a BuildKit per-Dockerfile ignore (`Dockerfile.build.dockerignore`) or an allow-list. 3. **The root `Dockerfile` and `Dockerfile.test` still pin node 20 and pnpm 9.15.4.** They are not W3's, and they have the same pnpm 9 gap with `allowBuilds`. The root Dockerfile's `deps` stage installs all seven packages, so pnpm 11's check would pass there. Its runtime stages' `npm install -g pnpm@9.15.4` needs the same review. 4. **Built from uncommitted edits, then committed, an image reads as stale** under the doctor's commit-time rule until it is rebuilt. That happened here: the scratch image was built at 16:39Z and the commit is 16:47Z. **Questions for the reviewer.** 1. The prompt says "WARN when it is absent". Here the WARN applies only beside a corpus, and with no channels it is a note. The doctor's own rule is that a clone with no corpus must not be told it is broken, and the tools block grades a binary nothing needs yet the same way. Keep it, or WARN everywhere? 2. Bugs A and B above belong to `docker/build-site.sh`, which is outside W3. Should a slice fix them before the parent's build-only Build all? As things stand, that Build all fails every site in containers. #### Follow-up W3b — the container Build all works (2026-09-28) **The parent's rulings.** - **Q1: keep it.** The doctor warns about an absent or stale image only beside a corpus. - **Q2: fix bugs A and B in this slice**, before the parent's build-only Build all. - **Owns, added:** `docker/build-site.sh` and a new `Dockerfile.build.dockerignore`. - **Also touched:** `common/lib/builtExport.ts` + its test. It holds the one reader of a bundle's identity, and the container check imports only `node:fs` from it. `PUBLISH.md`'s container section, three sentences that this change made false. - **Not touched:** the root `Dockerfile`, `Dockerfile.test`, the shared `.dockerignore` and the rest of `docker/**`. - **The runtime image does not use `build-site.sh`.** It is `Dockerfile.build`'s entrypoint and what `runDockerBuildOne` mounts. The runtime image's `publish-site.sh` only names it in a comment, and inside that container there is no engine, so Build all falls back to the host. **`git merge main` (`e6c5d2e3`, release 12 slice Q):** `644bd2d5`. One conflict, in `editor/CHANGELOG.md`'s `[Unreleased]`: both sides are kept, main's umtool bullet first, then W3's block. There was no code conflict, and tsc on the merge was clean (`w3b-tsc-1.log`, 60 s). **(A) `.next` stays inside the container** (`docker/build-site.sh`). - The symlink to `/site/.next` is gone. That symlink put turbopack's chunks at a real path under `/site`, from which `next` cannot be resolved. - **Incremental cache: tried, measured, dropped.** - The try: copy `.next/cache` into the container before the build and back out after it. It holds the TypeScript check's `.tsbuildinfo` and the fetch cache, 492 KB. - Two consecutive two-site runs: - TypeScript: **30.3 s → 32.6 s**; - compile: 18.3 s → 21.4 s; - the whole run: **90 s → 88 s** (`w3b-fixture-build-{1,2}.log`). - `export/next.config.ts` does not set `experimental.turbopackFileSystemCacheForBuild`, so a Turbopack production build keeps no filesystem cache. The old symlinked `.next` never made a container build incremental either. - **The cost, plainly:** none measured. Each container's `next build` starts from an empty `.next`. **(B) `export/public` is the composed `/site/public`.** - `build-site.sh` copies the image's tracked assets (`export/public/*.svg` only) into `/site/public`, never over a file that is already there. It then replaces `export/public` with a link to `/site/public` before `archilyzer build site --nodata`. - Only `.svg` is copied, so an image that did bake data (see the proof) cannot leak it into a site through the copy. **A wrong-site bundle cannot ship: two checks, both over `builtBundleProblem`.** - **The rule** (`common/lib/builtExport.ts`, new): `site.json`'s `siteId` and `corpus.json`'s `site.id` must both be there and both name the site. The one sentence it returns names the directory and the file that disagreed. - **In the container.** `build-site.sh` runs the check through `tsx -e` after the build and before publishing. A refusal prints `[build-site] REFUSED: …`, exits 1 and leaves `/site/out` as it was. A pass prints `out/ is the bundle of (site.json, corpus.json)`. - **On the host.** `runDockerDeployAllPhase` now checks every per-site `out/` FIRST, before the "no Cloudflare project" skip, the R2 upload and the Pages deploy. - Before this, it shipped whatever `export/.export-builds//out` held. The host deploy checks `builtSiteProblem`; this path checked nothing. - A refusal is `failed` with the sentence as its reason, and the log line is `[] deploy REFUSED — …`. **`Dockerfile.build.dockerignore`: an allow-list.** BuildKit reads `.dockerignore` in place of the shared file. - **It admits:** - `package.json`, `pnpm-lock.yaml`, `pnpm-workspace.yaml` and `tsconfig.base.json`; - `common/` and `export/`; - `docker/build-site.sh`. - **Inside those it excludes:** - dot-entries, `node_modules`, `.next` and `out`; - `*.tsbuildinfo`, `next-env.d.ts`, `.env*` and `*.pem`; - `export/test-*`, and the playwright report dirs; - `export/public/*` except `*.svg`. No tracked file matches an exclusion except the five svgs, which are let back in. - **The export build reads nothing outside these.** The reads relative to `monorepoRoot` on the build path are `export/service-worker/*` and `export/CHANGELOG.md`, and the rest is mounted. - **Measured:** - The context is **807 files, 7.4 MB** from the primary checkout, and the same from the worktree. `w3b-context.py` emulates the rules as a read-only walk, and its worktree list matched the built image's `/repo` file-for-file (807 = 807, `w3b-image-files.txt`). No docker build was run from the primary. - The image's `/repo` is `common docker export node_modules` and the four manifests. `/repo/export/public` holds only `file.svg globe.svg next.svg vercel.svg window.svg`. - Under the shared `.dockerignore`, the primary sent `export/public`'s generated data (`subs/` alone 1.8 GB), `.diarize/` (1.3 GB), umtool's data and the rest. The live image's `COPY . .` layer is 3.23 GB, and its chmod layer 3.18 GB more. - Now `COPY . .` is **7.14 MB** and the chmod layer **1.04 MB**. `COPY --chmod` was not worth a portability question to podman, so the chmod stays. - The image is **1.65 GB** (1,654,559,800 B; the live one is 7.72 GB). The final `--no-cache --pull` build took **48 s**. Of the 1.65 GB, 1.39 GB is the pnpm install (`w3b-image-final.log`). - **pnpm 11's check now passes on its own.** The image holds exactly the installed projects, so `pnpm exec` works as the host uid even with `verify_deps_before_run=install` or `=error`. The ENV stays because a builder that reads only the shared `.dockerignore` bakes all seven packages, and then the first exec dies (W3's proof). `Dockerfile.build`'s comment now says so. | sha | what | |---|---| | `644bd2d5` | merge `main` `e6c5d2e3` (release 12 slice Q); `[Unreleased]` joined | | `698ecc16` | `common:` `builtBundleProblem` + 3 tests | | `d22faf21` | `docker:` `build-site.sh` (A) `.next` in the container, (B) `export/public` → `/site/public` (the `.svg` assets only), the bundle check before publishing | | `84988249` | `common:` `runDockerDeployAllPhase` refuses a bundle that is not the site's, first; 1 test | | `2c7b57be` | `docker:` `Dockerfile.build.dockerignore` (the allow-list); `Dockerfile.build`'s comments | | `ab75a5b8` | `docs:` `PUBLISH.md`'s container section (the per-site mount, the image's context, the refusals) | | `8b7c3864` | `common:` the doctor's git test turns off git's background maintenance (a W3 flake, below) | | _this_ | `plans:` this follow-up and the W3 table row's Owns; one `[Unreleased]` bullet | **Gates.** - **tsc:** clean before the code commits (`w3b-tsc-2.log`, 121 s) and before `8b7c3864` (`w3b-gates-1.log`, 121 s). - **common: 2,125/2,125** (`w3b-gates-1.log`). That is 2,121 + 4: `builtExport` 3, `build.test` 1. - The first full run (`w3b-common-test.log`) was 2,124/2,125. The failure was W3's own git test: git 2.55's detached `maintenance run --auto` after the test's commit took `.git/objects/maintenance.lock` while the doctor ran. - The fix turns maintenance off in the temp repo. After it the test passed 8/8 alone, and it still fails without `GIT_OPTIONAL_LOCKS=0` (`.git/index` moves). - **`test:scripts`:** none of `scripts/*.test.mjs` covers `build-site.sh` or `Dockerfile.build`, so it was not run. - **They bite** (`w3b-bite.log`), against `46d9b0cd`, before W3b: - `builtExport.test.ts`, with a permissive `builtBundleProblem` shim so it loads: **9 passed, 2 failed**. The two are the refusal tests; the third new test asserts a pass. - `build.test.ts`: **10 passed, 1 failed**, the new deploy test. The old phase `skipped` both sites for having no project. It could not deploy, because the test's sites have no Cloudflare project on purpose: a real `wrangler` is never reachable from this test, even with the check removed. **The proof: Build all's own code over a two-site fixture.** `$T/w3-fixture`: `w3site` over `test-youtube` (editor e2e's `one-youtube-channel-with-data`), and `w3other` over `tagchan` (`curated-tags-channel`). Settings name the scratch tag, with `maxParallelBuilds: 2`. 1. **`pnpm archilyzer build all` passed** (`w3b-fixture-build-3.log`, 110 s, the final script): - Phase A ran on the host, then `ensureBuildImage` on the scratch tag, then Phase B with both sites in parallel. **`2/2 built`, exit 0.** - Each `out/` has 167 files, and 0 files in either per-site dir are not host-owned. - Each has `corpus.json`, `site.json`, `_headers`, `llms.txt`, `robots.txt`, `summaries/manifest.json`, `archives/manifest.json`, `favicon.ico`, `index.html` and the svgs. - `w3site`'s `site.json` and `corpus.json` say `w3site`, and its `transcripts/` is `test-youtube`. `w3other`'s say `w3other`, over `tagchan`. - `builtBundleProblem` on the host is null for both (`w3b-fixture-outputs.log`). 2. **The dangerous case, reproduced and refused** (`w3b-poisoned-runs.log`). - The setup: a scratch image `FROM` the test tag, with `w3site`'s composed `public/` copied into `/repo/export/public`. That is what a checkout that last composed `w3site` baked under the shared ignore file. Then `w3other` was built, through `runDockerBuildOne`'s own `docker run` argv. - **The old `public/` handling** (the committed script minus the link block, bundle check kept) produced `w3site`'s bundle for `w3other`: - `[build-site] REFUSED: /repo/export/out holds a build of "w3site", not "w3other" (site.json) — /site/out is left as it was`; - exit 1; - `/site/out` still held only its earlier `MARKER`. - **The committed script** over the same image produced exit 0, with `site.json` and `corpus.json` both `w3other`, `transcripts/` `tagchan` only, and 0 paths or files anywhere in `out/` or `public/` mentioning `test-youtube`. 3. **Every scratch tag has been removed:** `r13-build-image-test` and `r13-build-image-poisoned`. `yt-dlp-transcript-browser-build` is still `ddb3fbce9f60`, from 2026-07-07. **What only the parent's post-merge Build all can prove:** the real sites at real parallelism, the real fonts and corpus sizes, and memory under `maxParallelBuilds`. The image itself is the same 807 files from the primary. **Found and left.** - **Legacy per-site dirs.** A host that ran the old container path has `export/.export-builds//.next` from those runs, now unused. This machine has no `.export-builds/` at all. - **The in-container bundle check adds a few seconds per site**: `pnpm exec tsx -e` is about 5 s on the host, with `builtExport.ts` importing only `node:fs`. - **W3's "Found and left" 1 and 2 are fixed here.** Item 3 (the root `Dockerfile` and `Dockerfile.test` on node 20 and pnpm 9.15.4) stands, and so does item 4. #### Follow-up W3c — every site deploy checks its bundle; the assets stay in sync (2026-09-28) **The review** (`w3-review.md`) is **SHIP**, with five findings: - **S1:** a should-fix, release-level and pre-existing. It is now this slice's. - **L1** and **L2:** fixed here. - **L3** and **N1:** recorded below as found and left. - **N2:** a nit, fixed in the doc. `main` was not merged again, as instructed. There was no docker build, e2e run or `next build`: the primary was running a live six-site build and deploy. **S1: every SITE deploy checks its bundle right before wrangler** (`common/publish/build.ts`, `runDeployIntoLog`). - **The gap.** Two host paths shipped `export/out` with no identity check between their build and wrangler: - `buildAndDeployAction`: the Publish tab's Build & deploy, and `pnpm ops build-deploy`; - `basicBuildAndDeployAll`: Build & deploy all with no engine. The deploy queue runs beside the build queue. A build of another site, or of the hub, that rewrote `export/out` during the R2 upload would therefore have shipped to this site's Pages project. - **The fix.** - `runDeployIntoLog` now calls `builtBundleProblem(outDir, site.siteId)` first. Nothing runs between the check and the spawn. - A refusal logs `[deploy] REFUSED — . Nothing was sent to Cloudflare Pages; build again, then deploy.` and returns 1. - Every caller already turns 1 into a failed deploy: - Phase C: `failed`; - `deploySite` and `buildAndDeployAction`: they throw `Deploy failed (exit 1).`; - `basicBuildAndDeployAll`: `deploy FAILED — exit 1`. - The hub and the homepage deploy through `runPagesDeployIntoLog`, so they are unaffected. - **Every caller still passes a real site bundle.** - Phase C passes the per-site `out/`, already checked first by W3b. - `deploySite`, `buildAndDeployAction` and `basicBuildAndDeployAll` pass `export/out`, which after a site build always holds both files: compose writes `corpus.json` whenever it writes `site.json`, from the same descriptor. - **The two checks now agree.** `builtSiteProblem` is the fast answer that `deploySite` and the editor's `deployAction.ts` give before any job. It now also refuses a `site.json` naming the site whose `corpus.json` does not, as `export/out holds an incomplete build of "" (its corpus.json does not name it) — build first`. - So it refuses exactly what `builtBundleProblem` refuses. A new test walks seven bundle shapes through both: good, nothing, another site, no corpus, torn, unnamed corpus, hub. - The existing e2e regex (`ops-api.spec.ts:1022`) still holds: the fixture site is never in `export/out`, so the new sentence cannot arise there. - **The R2 upload still runs before the check** in the two editor paths (`editor/app` is not mine). The upload sends this site's own staged archives, from the per-site `.r2-staging/` and not from `export/out`, so it cannot carry another site's data. A refusal after it leaves what a wrangler failure leaves today: R2 has the newer archives, and Pages is unchanged. - **The test** (`build.test.ts`) cannot reach a real wrangler. - Its `PATH` holds only a fake `pnpm` that records its argv, and the test proves the fake answers the same `runChildIntoLog` spawn before anything deploys. - Four refusals are checked: another site, the hub's shape, torn, no corpus. Each returns 1 with one log line, and the fake is never spawned. - A good bundle returns 0 with the argv unchanged (`dlx wrangler pages deploy --project-name …`) and the `[deployed]` line. **L1: the tracked assets are synced every run** (`docker/build-site.sh`, `sync_public_assets`). - Each tracked `.svg` is copied over whatever copy is there, so a changed asset ships. - A name the previous run copied that the repo no longer has is removed, so a dropped icon stops shipping. The Ko-fi mark was such an icon. - The copied names are listed in `/site/.tracked-public-assets`, outside `public/`, so the list never ships. Only a listed name, and only as a regular file, is ever removed. So nothing compose wrote, and no svg the image did not put there, is touched. - The sync runs before compose, so compose would win over any shared name anyway. - **N3 (the re-read):** the removal loop now skips any listed name that is not an `.svg`, so a hand-edited or damaged list naming `corpus.json` cannot delete what compose wrote. The sync test lists `corpus.json`, and without the guard it fails with ENOENT on `site-public/corpus.json` (`w3d-bite.log`). - **Review question 1:** no case exists where a file already in `/site/public` must win over the image's svg. compose writes no top-level `.svg`: its files are `_headers`, `site.json`, `corpus.json`, `llms.txt`, `robots.txt`, `sitemap.xml`, `sw.js`, the JSON data files and the data dirs. **L2:** a test that every file `git ls-files export/public` lists is a top-level `.svg` (`common/publish/buildImage.test.ts`, the image contract's own file). - Its failure names both `Dockerfile.build.dockerignore` and `build-site.sh` (`sync_public_assets`). - It skips when the tree is not a git checkout (`Dockerfile.test`'s context has no `.git`). - L1's test lives beside it: the function is read out of `build-site.sh` and run with bash over temp dirs. **N2** (`PUBLISH.md`): - The doctor sentence gets its own paragraph break. - BuildKit reads the per-Dockerfile ignore file. A builder that reads only the shared one sends several GB and stays safe: `out/` is built from the composed data, and only the svgs are copied. - **Review question 2:** podman was not checked. There is no podman or buildah on this machine, and the doc now says it is unverified. | sha | what | |---|---| | `526e9af0` | `docker:` `sync_public_assets` in `build-site.sh`; `common/publish/buildImage.test.ts` (L1 sync + L2 svg-only, 2 tests) | | `85929901` | `common:` `runDeployIntoLog` refuses first; `builtSiteProblem` agrees with `builtBundleProblem`; 1 + 1 tests, and the good-bundle test gains its `corpus.json` | | `c52e116c` | `docs:` `PUBLISH.md` (N2) | | _this_ | `plans:` this follow-up; one `[Unreleased]` bullet (a refused deploy names why) | **Gates.** - **tsc:** clean before the code commits (`w3c-tsc-2.log`, 39 s, the final tree). - **common:** **2,129/2,129** (`w3c-common.log`, 66 s): 2,125 + 4 (`buildImage` 2, `build.test` 1, `builtExport` 1). - **`test:scripts`:** not run. No script was touched. - **They bite** (`w3c-bite.log`), against `31a74988`, before W3c: - `build.test.ts` + `builtExport.test.ts`: **22 passed, 2 failed**. - The two are `runDeployIntoLog refuses…`: on "another site's bundle" the old code returned 0, having spawned the fake `pnpm`. - And `builtSiteProblem refuses exactly what builtBundleProblem refuses`. - `buildImage.test.ts`'s sync test: - against the old script it fails at "defines sync_public_assets()"; - with the old no-clobber copy wrapped as the function, it fails at "a changed asset is shipped" (`v1` where `v2` was expected). - The svg-only test: with an intent-to-add `export/public/brand.png`, it fails with its message naming both files. The file was then un-staged and deleted. **Found and left.** - **L3: the doctor's stale rule can stick.** A fully cached `docker build` keeps the image's old `Created`, so after a comment-only commit to `Dockerfile.build`, with nothing in common/export changed, the WARN persists. Its "rebuild it now" command cannot clear it; only `--no-cache` or a context change can. This is rare, since nearly every commit touches common/export. If it ever matters, bake a `--label` with the Dockerfile's hash in `buildImageArgs` and compare that. - **N1: the doctor's image block loads the AWS SDK.** It does `await import("../publish/build")` for `dockerBin` and `buildImageArgs`. A small `publish/buildImage.ts`, re-exported by `build.ts`, would keep the doctor light. - **The two editor host paths upload to R2 before the check** (above). Moving a check ahead of the upload there is an `editor/app` change. #### Brought to main (2026-09-30) Release 13's W slices were reviewed SHIP on 2026-09-28 and never merged. `main` then moved through releases 14, 15 and 16 and the 0.11.0 cut. This brings the branch to today's `main` and proves it again. It is a merge, not a rebase, so the reviewed commits keep their shas. **The merge:** `abefa892` merges `main` `7a77536b` (238 commits) into the branch tip `83ee15ef`. It was first committed as `f7a047ee`, then reworded to this round's uniform commit trailer; the tree is the same (`git diff f7a047ee abefa892` is empty), so the gates below, run at `f7a047ee`, hold for it. Git marked four content conflicts and no modify/delete conflict. Each was resolved by keeping both sides. | File | Conflict | Resolution | |---|---|---| | `common/bin/doctor.ts` | Three hunks against SG's source-publish block: the `DoctorDeps` fields, the section, and the helpers | Both kept. `DoctorDeps` has W3's `buildImage`, `dockerfileChanged` and `now`, then SG's `sourceTools`. The sections run umtool, **build image**, **source publish**, then ports: W3's block stays right after umtool's and SG's stays right before the ports. The helpers are W3's (`probeBuildImage`, `parseEngineTime`, `dockerfileChangedAt`, `shellQuote`, `stamp`, `ago`, `gigabytes`) and then SG's (`probeSourceTools`, `dirSizeText`); no name is defined twice. DT's drive-health line, in the corpus block, merged with no conflict. | | `common/bin/doctor.test.ts` | W3's seven tests against SG's source-publish test, the social-icons test and DT's drive-health test | All kept, in that order: 8 + 7 + 3 = **18**. main's two tests that call `collectDoctorReport` directly (source publish, social icons) now also pass `buildImage: noEngine`. Without it they would ask the machine's real container engine, and W3's rule is that a unit test never does. | | `common/publish/build.test.ts` | The import lines: W3 added `readFileSync`, `tmpdir` and `runChildIntoLog`; main used `os` | One import block. W3's two `tmpdir()` calls now read `os.tmpdir()`, as main's do. 10 + 2 + 2 = **14** tests. | | `editor/CHANGELOG.md` | The cut renamed the old `[Unreleased]` to `[0.11.0]`, and W3's four bullets sat in it | W3's four bullets moved to a fresh `## [Unreleased]` above `## [0.11.0]`, which is now exactly main's (`git diff main` on the file adds only those six lines). | **Also touched by both sides, merged by git:** - `common/publish/build.ts`: W3's import, `runDeployIntoLog`'s check, `dockerBin(env)`, `buildImageArgs`, `ensureBuildImage` and Phase C's check. `git diff main` on the file shows only W3's hunks. On the merged tree, the call sites of `runDeployIntoLog` (four), `builtSiteProblem` (two) and `builtBundleProblem` (two) are the same as on the branch, so main added no path that deploys a site bundle. - `PUBLISH.md`: W3b's container paragraphs, beside SG's source-mirror history. `git diff main` shows only W3's two hunks. **What moved under W3, and why nothing yields:** - main's `.dockerignore` gained the homepage's published source (`homepage/public/source/`, `homepage/.source-publish.json`). `Dockerfile.build.dockerignore` admits nothing under `homepage/`, so it needs no change. - No file tracked under `common/` or `export/` matches one of the allow-list's exclusions, by `git ls-files`. `export/public` still tracks only the five svgs. - The export build reads nothing new from outside `common/` and `export/`. Since the fork, no package manifest changed except `editor/package.json` (DS's `UV_THREADPOOL_SIZE`), and neither did the lockfile or `export/next.config.ts`. DX's compose changes are URL strings. The image's `pnpm install` layer is therefore the same. - Nothing of main's was removed or rewritten. The only line of W3's that changed is the `os.tmpdir()` spelling. **Gates** (logs `$T/w3-*.log`, all from the worktree root): - **tsc** (all workspaces): clean on the merged tree before the merge commit, 137 s (`w3-tsc-1.log`). - **Unit:** | Suite | Result | |---|---| | `buildImage`, `builtExport`, `build`, `doctor` tests | **46/46**, 12 s (2 + 12 + 14 + 18) | | common | **2,396/2,396**, 146 s. That is main's 2,381 (release 16's last count) + W3's 15. | | editor unit (`app/**/*.test.ts`) | **101/101**, 9 s | | `test:scripts` | **194 passed, 2 skipped** of 196, 12 s. The skips are umtool's trace check (no umtool build in this worktree) and the `LIVE=1` archive check. | | mcp | **271/271**, 45 s | - **The build image, built for real** (`w3-docker-build.log`, `w3-docker-build-cold.log`). `docker build -f Dockerfile.build -t archilyzer-build:w3-check .` from the worktree at `f7a047ee`. - **With the build cache: 15 s.** The base, apt, pnpm and `pnpm install` layers were cached from W3b, since the lockfile has not moved; `COPY . .` onward ran. - **`--no-cache --pull`: 69 s.** apt 6.8 s, pnpm 6.0 s, `pnpm install --frozen-lockfile` 23.6 s, and the layer export 28.4 s. - **Size: 1,655,428,299 B (1.66 GB)** from `docker image inspect --format '{{.Size}}'`. The cached build was 1,655,410,190 B. W3b's was 1,654,559,800 B, and the live image is 7.72 GB. - `COPY . .` is **7.91 MB** (W3b: 7.14 MB) and the chmod layer 1.12 MB. `/repo` holds 849 files outside `node_modules`: releases 14–16 added code to `common/` and `export/`. - Inside it: node **v22.23.2** and pnpm **11.26.0**. `/repo` is `common docker export node_modules` and the four root manifests, and `/repo/export/public` is the five svgs. - **No site build ran through it.** The tag was removed with `docker rmi`, along with the first build's image, which the cold build's re-tag had left untagged. `yt-dlp-transcript-browser-build` is untouched: still `ddb3fbce9f60`, from 2026-07-07. - **The doctor on the merged tree** (`w3-doctor.log`, run from the worktree, so no corpus). It exits 0, and its sections run workspace, corpus, settings, social icons, tools, report pipeline (umtool), build image, source publish, then ports. The build-image line is a note: the live image predates `Dockerfile.build`'s last commit (W3b's). - **The editor's `next build`**, because the editor bundles `publish/build.ts` and `lib/builtExport.ts` (it does not import `doctor.ts`). It ran with the primary's `transcripts/` linked in (`ln -sT`) and was capped at 5 GB with no swap: **119 s, max RSS 1,648,884 KB**, exit 0. None of its 81 traces names the corpus. The link was removed after the build, and nothing ran through it. - **e2e:** W3's record names no spec, since its proof was the fixture Build all. This run covers the editor's paths into W3's changed code: the deploy refusals through `builtSiteProblem` (`ops-api`), the Build all controls (`deploy-page`) and the publish tab's cannot-deploy line (`site-scope`). It ran detached and queued. | Run | At | Specs | Result | |---|---|---|---| | 1 | `f7a047ee` | `ops-api`, `deploy-page`, `site-scope` | **40 passed**, 0 failed, 3.2 min; 7.6 min wall, of which 4.3 min waiting in the queue behind W1 | ### Slice W1, as shipped — the editor lows (2026-09-28) Branch `r13/lows-editor` off `main` `bf6904e8` (`441bdbb2` merged first, fast-forward), worktree `/home/user/Projects/r13-lows-editor`, one Opus implementer beside W2 and W3. Seven lows left by release 11. No settings, site or channel key; nothing on disk moves. Scratch files `w1-*` in the job's `tmp/overnight`. **1 — a Diagnostics retry keeps its card and its log** (`DiagnosticsStage.tsx`). - The O3 hole on the other stage: both Diagnostics grids dropped an emptied bucket's card (`populated` for channel health, `listed` for availability), and a retry empties its bucket while it streams (the per-video snapshot regen, then the end-of-run refresh). The card took `RetryBucketControl`, its `StreamActionLog` and the log with it. - `useRanBuckets()`: a `ran` set of the buckets whose Retry ran on this page, filled by `RetryBucketControl`'s existing `onRun`. Keyed by the card's aria label (channel health) or its status (availability). A ran card stays, its button disabled at **Retry (0)** (the control's own `ranHere` already did that). `AvailabilitySummary`'s hook sits above its `total === 0` early return, and that return yields to a ran card. A reload drops an empty card. Labels and test ids unchanged. - **An availability bucket is never emptied by its own retry:** a download records availability *history* only (`updateTopLevel: false`), so only a probe rewrites `availability.json`. The spec lands one before the click (as a check running beside the retry would), and the regen the retry's download triggers is the first to read it. - `diagnostics-retry-log.spec.ts` (new, 2), modelled on `cookies-mode.spec.ts:241`: a **Missing metadata.info.json** retry (an empty `data/vidnometa1/`; the fake writes the metadata) and a **Needs auth** retry. Each waits for the refreshed EMPTY list, which renders only inside a card that survived, then asserts the log, the disabled **Retry (0)**, and that a reload drops it. **2 — the `transcript-source.spec.ts:76` flake** (spec only). - The cause is the prompt's: `resetData`'s invalidate-cache clears the snapshot scheduler's TIMER, but a regeneration already in flight runs on and writes `snapshot.json` from the tree it read, before the spec's swap. `generateReport` returns as soon as any `snapshot.json` exists. The race is the harness reset's, not product code's: in production a change made through the app arms its own regen after the fact. - The test now deletes `snapshot.json` after the swap and clicks **Refresh report** until the report shows `nonStandardVtt` with one video (`chat-only.spec`'s `refreshReport` shape). **3 — `/sites` "built " refreshes when a homepage build lane ends** (`JobLane.tsx`, `HomepageBuildButtons.tsx`). - `JobLane` gets `onSettled(outcome)`, with `LaneOutcome = "done" | "failed" | "cancelled" | "error"` ("error" = the trigger refused or threw, no job). It is called once, from one `settle()` that also sets the chip. The prop is read through a ref, because the one-shot launch effect would otherwise call the first render's. A `settledRef` makes "once" the prop's contract, not the effect's shape (O4's Strict Mode fix is untouched). It is not called for a lane that a newer launch unmounted. - `HomepageBuildButtons` calls `router.refresh()` when a lane that BUILDS (**Build homepage**, **Build & deploy homepage**) ends with any job outcome. **Wider than the prompt's "ends done", on purpose:** a failed or cancelled build may already have rewritten or emptied `homepage/out`, and the line is what **Deploy homepage** would ship. `StreamActionLog` refreshes after any run that started, for the same reason. No refresh on "error" or for a deploy lane. - **AutoRefresh already covered most of it:** the pulse token carries each job's status and `endedAt`, so with passive refresh on (5 s by default) `/sites` re-rendered once the job ended anyway. The explicit refresh is immediate, and it is the only one when `autoRefreshIntervalSeconds` is 0. - **e2e cannot build the homepage** (O4's rule: `homepage/out` is the checkout's own directory), so the new `sites-homepage.spec` test proves the RE-RENDER on the one terminal state it may reach, a cancel: - passive refresh is off, and both queues are held, as before; - Build homepage queues, then a site is written to disk; - the queued job is cancelled from OUTSIDE the page, through the harness reset (newest first, so nothing is promoted). The lane's own Cancel is a server action that revalidates, and its response re-renders the page by itself (`server-action-reducer.js`: a revalidating action navigates to the current URL); - the new site's link appears without a reload. **4 — `/jobs` labels** (`jobKinds.ts`). - Six entries: `build-hub` **Build hub**, `deploy-hub` **Deploy hub**, `build-deploy-hub` **Build & deploy hub**, `build-homepage` **Build homepage**, `deploy-homepage` **Deploy homepage**, `build-deploy-homepage` **Build & deploy homepage** (the `/sites` lanes' titles). - Shape: `queueKeyStrategy: "custom"` (`BUILD_QUEUE` / `DEPLOY_QUEUE`), not drainable, not replayable (no JobSpec), no `needsMedia` (no channelSlug, and none of them opens a channel's `data/`). - **Correction to the prompt:** - There are no "existing build/deploy entries" to copy. No publish kind has an entry (O4's record says so), so the shape is the table's own. - The prompt named four kinds. `deploy-hub` and `build-deploy-hub` are in too, so the hub's trio is not half-labelled. - **Consumers checked:** - `jobKindLabel` renders on `/jobs` (`JobRow`, `JobsTable`) and on the job page. - `jobKinds.test.ts` enumerates the table: `ADDED_KINDS` +6. - No spec matches these kinds by row text. `ops-api` uses them as API verbs only, and `sites-homepage` reads `kind` from the metas. - The publish kinds (`build-export`, `build-deploy`, `deploy-export`, `build-index`, …) are still unlabelled. Left: not asked. **5 — a job cancelled while queued writes its cancelled sidecar** (`registry.ts`, `streamCommand.ts`, `shutdownCancel.ts`). - **The bug.** `onCancel` only closed the stream and settled `done`, and `registry.cancel()` marked the record terminal only after `scheduler.cancel()` had fired it. The sidecar kept its enqueue's `queued`. So the release-9 boot pass could **re-queue a job the operator had cancelled** (the newest of its spec, under 24 h), and `/api/media/fetch-window/` (MCP `fetch_clip`) could report a cancelled, evicted fetch as still `queued`. (`/jobs` never showed it as queued: such a job opened no log, so once evicted it is not listed, and a non-terminal meta reads "archived".) - **The fix.** `registry.cancel()` marks a queued record terminal (`markTerminal`, now shared with `finalize`) BEFORE `scheduler.cancel()`. `finalize()` cannot go first: its `scheduler.complete` would drop the entry without firing `onCancel`, and `done` would never settle. Both `onCancel`s persist the sidecar when the record is terminal. - **Found: the naive fix breaks the boot pass.** `shutdownCancel.ts` cancels every queued job on SIGTERM/SIGINT (so the exit cannot promote one into a child), and `bootQueuedJobs.ts` exists to settle exactly those `queued` metas on the next start. A graceful restart would have written `cancelled` over each (or torn it mid-exit). So `registry.beginShutdown()`, called first by the reaper, makes `cancel()` skip the early mark. A queued job then reaches `onCancel` still queued, and nothing is written. **`shutdownCancel.ts` is outside the prompt's Owns list** (no other slice owns it); the change is three lines. The call is optional (`?.()`), because under `next dev` the registry on `globalThis` can predate the method. - **Meta writes are chained** (`metaWriter`, over `serialWriter` since the fix round). This is **defensive, not a fix for an observed race**: - `writeJobMeta` snapshots the record before its first `await`, so two writes are issued in order, and only the fs threadpool could complete them out of order; - the review's probe cancelled 300 jobs in the same tick as their enqueue, and all 300 ended `cancelled` with or without the chain. - **Tests.** `registry.test.ts` +1: `onCancel` sees the record already `cancelled`, with `endedAt`. `streamCommand.test.ts` (new, 3): a function job and a command job queued behind a holder, cancelled, end `cancelled` on disk with no `startedAt`; after `beginShutdown()` the sidecar stays `queued` (that test replaces the process registry afterwards). `sites-homepage.spec`'s Build homepage test polls the job's sidecar to `cancelled` (the UI path). **6 — L7: doctor's worker engine from `paths`** (`doctor.ts`, the engine line only). The default engine was `app.defaultBin()`, which is this process's `getPaths()`. Now it is `paths.whisperBin` for whisper-cpp and `paths.parakeetBin` for parakeet, as the model line beside it uses `paths`. chough has no Paths field, so it keeps its app default. `doctor.test.ts` +1, and the test Paths gain `whisperBin` and `parakeetBin`. W3 adds an image check to the same file. **7 — `editor/package.json` `test`:** `tsx --test "app/**/*.test.ts"`. It works as `pnpm test` in `editor/` and as `pnpm --filter editor test` from the root: 85/85. `plans/tools/implementer-rules.md` is W2's, so the gate-list line is in the report for the parent to join. | sha | what | |---|---| | `2fa120cd` | 1: `useRanBuckets`, both grids keep a ran card; `diagnostics-retry-log.spec.ts` (new, 2) | | `aad7b3bf` | 2: `transcript-source.spec` drops `snapshot.json` after the swap and refreshes until the bucket reads 1 | | `cd0781f5` | 5: `markTerminal` before `onCancel`, `onCancel` persists a terminal record, `beginShutdown`, `metaWriter`; `registry.test.ts` +1, `streamCommand.test.ts` (new, 3) | | `5c6118df` | 3: `JobLane` `onSettled`; `HomepageBuildButtons` refreshes after a build lane; `sites-homepage.spec` +1, and the Build test polls the sidecar | | `79997a8d` | 4: six `JOB_KINDS` entries; `jobKinds.test.ts` `ADDED_KINDS` +6 | | `6a805323` | 6: doctor's default engine from `paths`; `doctor.test.ts` +1 | | `c91c9ce9` | 7: the editor `test` script | | `5da692f7` | `changelog:` `[Unreleased]` above `[0.10.0]` in `editor/CHANGELOG.md` (items 1, 3, 4, 5) | | _this_ | `plans:` this record; FACTS (the bucket-card paragraph closed, the boot-pass bullet amended) | **Gates** (logs `w1-*.log`): - **tsc** (`pnpm -r --no-bail --workspace-concurrency=1 exec tsc --noEmit`) was clean on the full tree before the code commits (`w1-tsc1.log`). The commits split that tree by file, and none depends on another. - **Unit tests** (`w1-units1.log`, on `c91c9ce9`): - **common 2,119/2,119.** That is +5: registry +1, streamCommand +3, doctor +1. jobKinds grew inside an existing test. So the base at `bf6904e8` is 2,114, not the prompt's 2,112. - **editor unit 85/85**, **`test:scripts` 185 + 1 skipped of 186**, **mcp 269/269**. - **Editor build** `pnpm --filter editor exec next build` ok (`w1-build1.log`, compiled in 16 s). The export build was not run: nothing under `export/` changed. - **e2e**, all queued and detached. The lock was free each time. `export/public` was linked per path, with no dangling links. Mid-slice, an added worktree moved this one's block to 4311/4310. | run | specs | passed | failed | time | |---|---|---|---|---| | A | `retry-bucket transcript-source cookies-mode sites-homepage jobs-retry diagnostics-retry-log queues cancel ops-api exclude-from-counts availability`, `--repeat-each=3` (186) | **109** | 77 | 10.7 min | | A2 | `transcript-source diagnostics-retry-log sites-homepage retry-bucket queues`, `--repeat-each=3` | **57** | **0** | 7.2 min | | B (bite) | `diagnostics-retry-log sites-homepage` on `main`'s six source files | 2 | 4 | 2.1 min | - **Run A was killed by the machine, not the code.** At 13:00:39 the kernel OOM killer ran (user journal: `session.slice: The kernel OOM killer killed some processes`). The live editor's parakeet worker held 2.5 GB. After test 110 every request got `ERR_CONNECTION_REFUSED`. - Tests 1–109 were the whole first repetition, all 11 specs 62/62, and the second through `queues.spec:40`. - A2 re-ran, ×3, everything the second and third repetitions had not reached, plus `transcript-source` (**9/9** there; **3/3** in A's first repetition). - `queues` (`cancels a queued job without disturbing…`) and `cancel` exercise the changed cancel path; `ops-api` polls metas; `exclude-from-counts` and `availability` render the two grids. - Checked after every run: the worktree has no `homepage/out`, and `homepage/public` holds no ignored data. - **Numbers tool:** none. **They bite:** - **Unit** (`w1-bite-units.log`, `main`'s files swapped in, then restored by a trap): - item 5: `main`'s registry, streamCommand and shutdownCancel fail 4 of 10. That is the registry test, both sidecar tests, and the shutdown test (no `beginShutdown` there). - **The naive fix** (`onCancel` always persists, `cancel()` always marks first) fails the shutdown test, 1 of 10, which is the guard for the boot pass. - The new registry with `main`'s `onCancel` (writes nothing) fails both sidecar tests, 2 of 3. - item 4: `main`'s `jobKinds.ts` fails "added kinds carry their pinned label", 1 of 4. - item 6: `main`'s `doctor.ts` fails the new test, 1 of 9. - **e2e** (run B, `w1-e2e-bite.log`; `main`'s `DiagnosticsStage`, `JobLane`, `HomepageBuildButtons`, `registry`, `streamCommand`, `shutdownCancel`): - both Diagnostics tests fail at the empty-list wait (`missing metadata empty`, `availability needs_auth empty`), and the grid had dropped the card; - the Build homepage test fails at the sidecar poll (`Expected "cancelled"`, `Received "queued"`); - the re-render test fails at the new site's link; - the two untouched `sites-homepage` tests pass. - **Item 2 cannot be made to bite on demand:** it is a race between a previous spec's in-flight regen and this one. The fix is structural (no stale `snapshot.json` can satisfy the wait), and `transcript-source` passed 12 of 12 across A and A2. - **Item 7** is a script, not a test. **Found and left** - **`shutdownCancel.ts` was touched** (item 5, above), outside the prompt's Owns list. - **The hub's lanes do not refresh `/sites` when they end** (`HubBuildButtons`, not this slice's file). Nothing there reads the hub's `export/out` build time the way the homepage line does, so there is nothing stale to show. - **The publish kinds are still unlabelled on `/jobs`** (`build-export`, `build-deploy`, `deploy-export`, `build-index`, `build-stats`, `archive-*`). - **Run A's OOM:** the machine has 15 GB, and the live parakeet worker plus several worktrees' servers crowd it. A full-suite gate tonight may meet the same killer. - **`shuttingDown` is one-way and process-wide** (review L2; the gaps predate this slice): - Once `beginShutdown()` has run, no queued cancel writes a sidecar. That includes an operator's, or a remote worker requester's, landing during Next's graceful close (`server.close` waits for in-flight requests). This is the old behaviour. - A job enqueued in that window is never reaped. If it starts, its child can outlive the exit, and its meta stays `running`, which the boot pass leaves alone. - The flag is the natural hook for a follow-up in which `enqueue` refuses, or at least does not start, a job once `shuttingDown` is set. It was not considered for this slice (the review's question), and it is not done. - **A job RUNNING at a graceful restart ends `cancelled` on disk, with no `cancelReason`** (review L3). Its `.finally` still writes while Next closes, so on `/jobs` a restart reads like an operator's cancel. This is unchanged by this slice, whose "nothing is written" is about queued jobs only. - **`onSettled` does not fire for a lane a newer launch replaced** (review nit). Build homepage followed at once by Deploy homepage: the build ends unseen, and "built " stays stale until AutoRefresh (5 s by default) or a reload. The prop's comment says so. - **The e2e harness reset now writes `cancelled` metas** (review nit). `invalidate-cache` cancels queued jobs, which write their sidecar asynchronously during `resetData`'s `rm`. It is the same class as a running job's terminal write, and `rm`'s `maxRetries` already absorbs it. ### Slice W1, fix round (2026-09-28) Review: **SHIP AFTER FIXES** (`w1-review.md`). One should-fix, L1 taken, L2, L3 and the two nits recorded above. `main` was not merged. - **Should-fix:** the item-5 changelog bullet claimed `/jobs` listed an evicted, cancelled job as queued again. It never did: such a job opened no log, so once evicted it is not listed, and a non-terminal meta reads "archived". - The real consequences were the boot pass re-queueing it, and `/api/media/fetch-window/` (MCP `fetch_clip`) reporting a cancelled fetch as still `queued`. - The bullet now says that. So do this record's item 5 and the `sites-homepage.spec` comment. - `cd0781f5`'s commit message repeats the old claim, and it is left as it is. - **L1:** the chain is `serialWriter(write)`, exported for its test. It is `last = last.then(write, write)` plus a no-op `last.catch`: - a writer that rejects no longer stalls every later write; - the last write rejecting is not an unhandled rejection. - `writeJobMeta` never rejects, so this is defence only. - `streamCommand.test.ts` +1: four writes, the first slow and the second and fourth rejecting, must run one at a time, in order, with no unhandled rejection. - The test fails on each of three weaker variants: `.then(write)` stops after write 2; without the catch, the runner reports an `unhandledRejection`; unchained writes interleave. - **Found while re-gating: `common/node_modules` had been replaced** at 13:25:02, after the hand-back and not by this slice, with a standalone (non-workspace) install carrying `@types/react` 19.3.0. - `tsc` then failed in `export` (`PlayerProvider.tsx:1186`: two unrelated `Ref` types). - The stray tree was moved to `$T/w1-stray-common-node_modules-1325` (530 MB, not deleted), and `pnpm install --frozen-lockfile --offline` relinked the workspace ("Already up to date", 3.4 s). - The sibling worktrees' `common/node_modules` are workspace symlinks, as this one is again. | sha | what | |---|---| | `60ac51a7` | `changelog:` the item-5 bullet, the record's item 5 and the spec comment say what the stale `queued` actually did | | `c3afe68d` | `jobs:` `serialWriter`: `then(write, write)` plus a no-op catch; `streamCommand.test.ts` +1 | | _this_ | `plans:` this fix round; the chain described as defensive; L2, L3 and the nits recorded | **Gates** (`w1-fix-gates2.log`, on the relinked tree): - tsc clean. - **common 2,120/2,120** (+1, the serialWriter test) and **editor unit 85/85**. - The first attempt (`w1-fix-gates.log`) ran on the stray tree: tsc failed. Its common run was stopped after the tree moved out from under it, and it has no result. - No e2e and no build: the changed spec's diff is a comment, and the rest is common code under unit tests, plus docs. #### Brought to main (2026-09-30) Release 13 was reviewed SHIP on 2026-09-28 and never merged: `main` went on through releases 14, 15 and 16 and the 0.11.0 cut, to `7a77536b`, 253 commits past this branch's fork (`441bdbb2`). The branch merged `main` (a merge, not a rebase, so the reviewed shas stand) at **`fd1b5a52`**. Nothing of W1's was deleted or superseded on `main`, so nothing yielded; every W1 change is in the merged tree as reviewed. **Conflicts and how each was resolved:** - **`editor/CHANGELOG.md`, the one conflict.** The cut renamed the old `[Unreleased]` to `[0.11.0] - 2026-09-30`, and `main` has no `[Unreleased]`. W1's four bullets now sit under a new `## [Unreleased]` above `## [0.11.0]`, and `[0.11.0]` is `main`'s, byte for byte. Checked by eye: none of W1's bullets is inside a released section, and each appears once. - **Merged without a conflict, both sides kept** (each re-read on the merged tree): - `common/bin/doctor.ts`. `main` added the source block (release 12 R), the stored-icon line (release 14 HP), the out-of-process stall check (DS), stagit and its render cache (SG) and the drive-health line (DT). None of these touches the worker loop, where L7's one change (the default engine from `paths.whisperBin` / `paths.parakeetBin`) sits unchanged. - `common/bin/doctor.test.ts`: `main`'s new Paths fields and tests, and W1's `whisperBin` and `parakeetBin` in the test Paths and its L7 test. - `editor/app/sites/components/HomepageBuildButtons.tsx`: `main` added one paragraph (release 12's source-mirror note under the buttons). W1's `useRouter`, `settled()` and `onSettled` prop are as reviewed. `JobLane.tsx` is unchanged on `main`; the `listed` checkbox (HS) lives in `SiteForm.tsx`, which W1 does not touch. - `editor/package.json`: DS's `start` (`UV_THREADPOOL_SIZE=${UV_THREADPOOL_SIZE:-16}`) and W1's `test` beside it. - `plans/FACTS.md`: W1's two paragraphs (the Diagnostics cards closed, a queued cancel's sidecar) land where they did; `main`'s additions are elsewhere and about nothing W1 changed. - **Checked, nothing to reconcile:** `common/jobs/**` (`jobKinds`, `registry`, `streamCommand`, `shutdownCancel`) is unchanged on `main` since the fork. DS's `needsMedia` work is in `buildStats.ts` and the records, not in `jobKinds.ts`. `DiagnosticsStage.tsx` and the two specs W1 changed (`transcript-source`, `sites-homepage`) are unchanged on `main`; `editor/e2e/helpers.ts` gained HS's `listed`, which no W1 spec uses. **Re-gates on the merged tree** (logs `$T/w1-*.log` in the job's `tmp/`): - **tsc** (all workspaces) clean, 113 s. The first pass failed in `export` only, on a generated `export/.next/dev/types/validator.ts` naming `app/use-with-ai/page.js`, the page DX removed. The gitignored `.next/dev/types` of `export` and `editor` were deleted; neither is a source file. | Suite | Result | |---|---| | common | **2,387/2,387**, 95 s: `main`'s 2,381 plus W1's 6 (registry +1, streamCommand +4, doctor +1) | | editor unit | **101/101** three ways: the rules' `pnpm exec tsx --test "app/**/*.test.ts"` in `editor/`, W1's `pnpm --filter editor test` from the root, and `pnpm test` in `editor/`. The script runs the rules' glob, so it runs the same files. | | `test:scripts` | **194 passed, 2 skipped (196)**, as at DS: the `LIVE=1` archive check, and UT's post-build check skipping this worktree's older umtool build | | mcp | **271/271** | - **Build:** the editor's `next build`, with the primary's `transcripts/` linked in and capped at 5 GB with no swap: 139 s wall (compiled in 44 s, TypeScript 85 s, under a load average of about 15), max RSS 1,626,484 KB, exit 0; 79 traces, 0 entries under `transcripts/`. The link was removed after the build, and nothing ran through it. No export, homepage or umtool file changed. - **e2e** (editor, detached and queued; `export/public` relinked per path first, the three dangling links `main` no longer makes removed): | Run | Specs | Result | |---|---|---| | 1 | W1's three (`transcript-source`, `diagnostics-retry-log`, `sites-homepage`); the two grids' (`availability`, `exclude-from-counts`, `retry-bucket`); `/sites` (`sites-crud`, HS's); the `/jobs` list and kind chips (`jobs`, `jobs-filters`); the cancel path (`queues`, `cancel`); `ops-api` (the homepage kinds as verbs) — `$T/w1-specs.txt` | **73 passed, 1 failed, 7.7 min**, no wait for the queue. The failure: `sites-crud.spec.ts:25`, "Saved" not visible in 5 s after **Create site**, the run's first `/sites/new` save. | | 2 | `transcript-source`, `diagnostics-retry-log`, `sites-homepage`, `sites-crud`, `--repeat-each=3` — `$T/w1-specs-repeat.txt` | **72 passed, 0 failed, 4.6 min**, after 2.6 min in the queue behind W3's suite: `sites-crud.spec.ts:25` passed in each of the three repetitions, and `transcript-source` 9/9. | Run 1's failure is the load-timeout class `release-7.md` recorded for `sites-crud.spec.ts:148` ("Saved" not visible in 5 s, under load), on a spec and a form W1 does not touch; it passed 3 of 3 in run 2. - **Numbers tool:** none. **The second join: W3 landed first.** `main` moved to `4a186547` (the merge of `r13/build-image`, slice W3 brought to main) while the gates above ran, so the branch merged `main` again, at **`bcba6bdf`**. Two conflicts, both resolved by keeping both sides whole: - **`editor/CHANGELOG.md`:** `[Unreleased]` holds W3's four bullets as `main` has them, then W1's four. `[0.11.0]` and everything below it are `main`'s. - **`plans/release-13.md`:** the Record keeps W3's sections as `main` has them (as shipped, W3b, W3c, its Brought to main) and puts W1's after them, before the Rollout: the merging slice's sections after `main`'s, as release 15 SS's merge did. `git diff main` on the file adds W1's lines and removes none. - **`common/bin/doctor.ts` and `doctor.test.ts` merged with no conflict:** W3's build-image section (between umtool and source publish) and its probe helpers, and W1's L7 line in the worker loop of the tools section. W3's `run()` helper passes `buildImage: noEngine`, so W1's L7 test, which calls it, never asks the machine's container engine. - `git diff main --stat` after the merge is W1's 19 files and nothing else. Re-gates at `bcba6bdf`'s tree (`w1-gates2.log`): **tsc** clean, 87 s; **doctor 19/19** (W3's 18 and W1's L7 test); **common 2,402/2,402**, 98 s (`main`'s 2,396 plus W1's 6); **editor unit 101/101**. No e2e and no build: what W3 added is `Dockerfile.build`, `common/publish/build.ts`, the doctor and their tests, which W3's own re-gate built and ran, and none of it is a file W1 changed or a path W1's specs drive. | sha | what | |---|---| | `fd1b5a52` | the merge of `main` `7a77536b`; `editor/CHANGELOG.md` resolved as above | | `2d701ef9` | `plans:` this subsection, to the first join's gates | | `bcba6bdf` | the merge of `main` `4a186547` (W3); the changelog and the Record kept both sides | | _this_ | `plans:` the second join and its re-gates | ### Slice W2, as shipped — the export, homepage and docs lows (2026-09-28) Branch `r13/lows-export` off `main` `bf6904e8`, fast-forwarded to `main` `441bdbb2` (this plan) before any change; worktree `/home/user/Projects/r13-lows-export`, port block #14 (export e2e 4420, homepage e2e 4440, origin B / hub A 6010 / 6011). One Opus implementer; scratch files `w2-*` in the job's `tmp/overnight`. Six lows left by release 11, each small; one extra file was needed (`common/lib/envVars.test.ts`, below). **1. Charts reach `--chart-6`.** `common/components/charts/ChartView.tsx` coloured its series, its pie slices' config and its pie `Cell`s with its own `var(--chart-${(i % 5) + 1})`, so the sixth series wore `--chart-1` and the slot release 11 added was reached only by the hub's cross-site chart. All three now take `seriesColor(i)` (`common/lib/homepageChart.ts`): `--chart-1..6`, then the golden-angle hues the hub chart already uses. Series 1–5 are unchanged. The consumers are `SearchChartPanel` (the export's and the hub's search chart view) and `ChartCard` (the editor's `/sites//charts` dashboard); both pass ChartView their data unchanged, so the new test covers them. The homepage and umtool do not import ChartView. `ChartView.test.ts` (new, 2 tests) renders ChartView server-side and reads the `ChartContainer`'s `