commit f69d0f40654b1638e23f986fd73d2737fbdf784e
parent 5069f8994ccf2532dd70e8b6ac1542e684b2bd76
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Mon, 28 Sep 2026 02:14:54 -0400
plans: slice O6 checkpoint A as shipped — doctor, run, mcp, every bin a subcommand, ports and env vars declared once, PUBLISH.md; the [Unreleased] bullet
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Diffstat:
2 files changed, 226 insertions(+), 0 deletions(-)
diff --git a/editor/CHANGELOG.md b/editor/CHANGELOG.md
@@ -1,5 +1,8 @@
# Changelog
+## [Unreleased]
+- **`archilyzer` checks the machine, runs one operation offline, starts the MCP server, and is one command from the repo root.** `pnpm archilyzer <command>` is the short form (`pnpm archilyzer --help` lists them all). `pnpm archilyzer doctor` is a read-only report: Node, the checkout, the corpus and whether each channel's media is reachable, `settings.json`, every tool the paths name plus each enabled worker's engine and model, umtool's report-pipeline tools, and this checkout's ports; it exits 1 only for something the machine is set up to do and cannot. `pnpm archilyzer run <operation> <channel> [ids…]` runs diarization, either attribution pass or digest over one channel as the editor's job does (a job record and log under `.jobs/`, the same summary line, the same refusal for an unmounted drive); sync, the metadata scan, downloads and transcription are refused with the reason, because they run on the editor's paced download queue and worker pool. `pnpm archilyzer mcp` starts the MCP server, so it can be registered as `-- pnpm -C "$PWD" archilyzer mcp`. Every other script in `common/bin/` is a subcommand too (`duplicates`, `posts fetch`, `digest plan`, `verify transcripts`, …), and export's `detect:duplicates` script is now `archilyzer duplicates`. Every environment variable is listed, by audience, in the new `ENVIRONMENT.md`, and `DEPLOY_CLOUDFLARE.md` and `DEPLOY_DOCKER.md` are now one `PUBLISH.md`. Settings no longer calls the Docker build mode a follow-up.
+
## [0.9.4] - 2026-09-28
- **On the Dark ground the sidebar's Archilyzer mark has a thin outline.** Its slate tile now has a 1-pixel ring just outside it, following its rounded corners, in the colour of the mark's unlit lines, so the tile's edge shows against the dark page. Light and Sepia are unchanged, and so is the favicon.
diff --git a/plans/release-11.md b/plans/release-11.md
@@ -36,6 +36,229 @@ merges `main` once O1–O5 have landed, then does its `E2E_` rename. Then one in
## Record
+### Slice O6, as shipped — one-core Phase 4 slice 3 (2026-09-28) — checkpoint A
+
+Branch `r11/phase-4-s3` off `main` `2162db92` (no later `main` to merge before the first commit),
+worktree `/home/user/Projects/r11-phase-4-s3`, port block #6, one Opus implementer. The spec is
+[`one-core.md`](one-core.md) "Phase 4", items 2 (the rest) and 3. Checkpoint A is everything but
+the `E2E_` prefix cleanup, which is checkpoint B, after O1–O5 land.
+
+**The CLI's last subcommands.**
+- **`archilyzer doctor [--json]`** (`common/bin/doctor.ts`). One read-only report over what was
+ spread across `paths.ts`, umtool's doctor and `scripts/worktree.mjs`:
+ - workspace: node against next's `>=20.9.0`, the checkout, `node_modules`, the path overrides set;
+ - corpus: channel count, each channel's media through `inspectChannelMedia`, the LMDB index by
+ `stat` only;
+ - settings: `settings.json` present, a JSON object, loads (`settingsFromFile`, new in
+ `settings.ts`: getSettings' body for a named file);
+ - tools: every binary `paths.ts` names, plus each enabled local worker's engine, `parakeet-cli`
+ and model (looked up on PATH, not run);
+ - umtool's report-pipeline table (read from `umtool/lib/tools.mjs` by a runtime import of the
+ file; common does not depend on umtool);
+ - the port block (asked of `scripts/worktree.mjs ports`), each port free or in use by a TCP
+ connect.
+ - A FAIL exits 1. It is something the machine is configured to do and cannot: a settings file
+ that is not a JSON object (every process silently reads defaults), an enabled worker's engine or
+ model missing BESIDE a corpus, yt-dlp/ffmpeg/ffprobe missing beside a corpus, a binary an env
+ override names explicitly, node too old, no `node_modules`. Everything else warns or notes. A
+ clone with no corpus is not broken, and an engine missing there is a warning.
+ - Strictly read-only: no LMDB open, no mkdir, no settings write, no bind. The one process-state
+ change is a `chdir` around umtool's table (it resolves `facecrop.py` from the cwd), restored
+ at once.
+- **The probe exists once.** `common/lib/toolProbe.mjs` is umtool's `probeOne` lifted out (plain
+ ESM, exported); `umtool/lib/tools.mjs` keeps its table and calls it. `probeTools()` output was
+ byte-identical before and after (`o6-umtool-probe-{before,after}.json`, `cmp`).
+- **`archilyzer run <operation> <channel> [ids…] [--lane local|remote]`**
+ (`common/bin/run-operation.ts`) calls `runOperationChannelJob`, the body the /channels
+ buttons, the stage cards and the lane runners call. So a run is a real `backfill-channel` /
+ `digest-channel-*` job: its record and log under `.jobs/`, the same summary line, the channel
+ snapshot flushed at the end, and `runManagedFunction`'s media guard (a refusal before any record
+ exists).
+ - **Runs:** diarization, attribution-diarized, attribution-text, digest (the four registry
+ operations).
+ - **Refused with a sentence read off the descriptor:** sync, metadata-scan, download (the
+ per-platform download queue paces every request to one source inside the editor), and
+ transcription (the editor's worker pool).
+ - **Refused up front:** an unknown operation (with both lists), an unknown channel, an
+ operation switched off in settings, a paused lane (the batch would idle-wait for a resume
+ only the editor can give), and `--lane` off digest. Ids with no data dir are named.
+ - Ctrl-C cancels the job. Exit 0 done, 1 failed or refused, 2 usage, 130 cancelled.
+ - Different from the editor's run, and said in the file's header: it does not see the editor's
+ transcription activity, so the backfill lane's yield-to-transcription does not apply.
+ - To carry ids, `runDigestChannelJob` gains `ids` (as the backfill job has; into
+ `spec.params`), `runOperationChannelJob` passes `ids` to both runners, and the editor's
+ digest replay forwards them through `digestChannelAction`, so a replayed scoped run cannot
+ widen to the channel.
+- **`archilyzer mcp [args…]`** (`common/bin/mcp.ts`) starts the MCP server as `pnpm --filter
+ yt-dlp-transcript-mcp exec tsx src/index.ts` does: mcp's tsx, cwd `mcp/`, env passed through. It
+ is a child process, not an import. Nothing is printed on stdout.
+- **`pnpm archilyzer <command>`** is a new root script for the long form. pnpm prints its `$ …`
+ line on stderr, so `-- pnpm -C "$PWD" archilyzer mcp` is a clean MCP registration (initialize
+ handshake checked through it).
+- **Every bin is a subcommand.** `_cli.ts` gains PASSTHROUGH rows. They are matched on the leading
+ words only, and everything after the path is handed on verbatim.
+ - In-process: `build stats|templates|archives`, `docs env|files`, and `brand media` (with its
+ argv).
+ - As children with their own flags (`_spawnBin.ts`): `duplicates` (with the 8 GB heap its script
+ had), `posts fetch|check`, `diarize backfill`, `digest plan|validate`, `reconcile video-dirs`,
+ `verify transcripts`, `transcribe` (`transform.ts`) and `migrate channel-priority`.
+ - A test fails when a file in `common/bin/` has no row.
+- **package.json.**
+ - Kept: the scripts the publish pipeline runs by name: export's `build:index`, `build:stats`,
+ `build:templates`, `build:archives`, `compose:site` and `compose:hub` (`build.ts`'s steps and
+ docker Phase A), and homepage's `build:index` and `compose`. They keep their names and heaps
+ and now call `archilyzer <row>`. Each was checked with `pnpm run <script> --help` (it prints
+ its row), and `build:data` + `compose:site` + `compose:hub` were smoke-run on a scratch corpus
+ (all exit 0).
+ - Removed: export's `detect:duplicates` (no referrer; it is now `archilyzer duplicates`).
+ - Kept, with the reason: root `build:index` / `sync:tick`. They are one-line CLI aliases, and
+ `sync:tick` is the documented cron line (SCHEDULED_SYNC.md, the editor's sync console copy,
+ SETTINGS.md), so removing it would silently break an operator's crontab.
+ - `docker/build-site.sh` and `publish-site.sh` still parse (`sh -n`) and name `build site`,
+ which exists.
+
+**Ports exist once.** `common/lib/ports.mjs` is the table (name, base, what for), with
+`OFFSET_STEP`, `portsForOffset` and `portFor`.
+- `scripts/worktree.mjs` imports it, and gains the three e2e ports it never offset: `HUB_PORT`,
+ `ORIGIN_B_PORT`, `HUB_A_PORT`.
+- `sync tick`'s default URL and the doctor read it.
+- Two places cannot import a module, and `ports.test.ts` reads them as text and fails on a
+ mismatch or an undeclared port:
+ - a package.json script's `${NAME:-N}` (a shell line);
+ - the `--ports NAME:N` list handed to queue-lock (its syntax is shared with older checkouts).
+- The playwright configs and e2e helpers still spell their fallbacks, held by the same test. B
+ points them at the module, since it edits them anyway (homepage's is O2's tonight).
+- Not in the table: the client default URLs (`mcp/src/fetchClip.ts`, umtool's tag route,
+ `archilyzer-ops.mjs`) and the container's ports.
+
+**Config and docs.**
+- **Env vars, declared once.** `common/lib/envVars.ts` lists every variable the apps' code
+ reads, by audience:
+ - `paths` (exactly what getPaths reads);
+ - `runtime`;
+ - `port` (generated from ports.mjs);
+ - `internal` (set by the pipeline);
+ - `docker` (the `ARCHILYZER_*` set);
+ - `test`.
+- `ENVIRONMENT.md` is generated from it (`archilyzer docs env [--check]`). `envVars.test.ts`
+ holds the list to the code in both directions: every `process.env.X` / `env.X` read under
+ common/, editor/, export/, homepage/, mcp/src and scripts/ is declared, and every entry is
+ still mentioned. umtool's own knobs stay in umtool/docs until Phase 5.
+- **PUBLISH.md absorbs DEPLOY_CLOUDFLARE.md and DEPLOY_DOCKER.md.** It covers what gets
+ published, the editor / `pnpm ops` / CLI table, Pages, previews, R2, the cost-abuse defenses,
+ containers and the MCP registration. Both old files are removed. Every link to them now names
+ PUBLISH.md, except in plans/ and the released changelog bullets: README, SETUP, AGENTS,
+ RUNNING_IN_DOCKER, r2-proxy ×3, homepage/content/README.md, and build.ts's R2 refusal plus
+ three comments.
+- **Stale claims corrected.**
+ - README's and SETUP's hand-kept env tables are now a pointer to ENVIRONMENT.md and `doctor`.
+ - SETUP said `pnpm e2e` runs on port 3001; it is 3011.
+ - CONTRIBUTING's "CLI shims" named a `retry-failures.ts` that does not exist; the section is
+ now "The archilyzer CLI".
+ - RUNNING_IN_DOCKER named whisper.cpp as the only engine; it now names parakeet.cpp beside it.
+ - **Settings → Build pipeline still called Docker "a follow-up" that "falls back to a basic
+ build"**, and its option read "(follow-up)". So did `settingsSchema.ts`'s description
+ (SETTINGS.md regenerated) and its BuildMode comment. All now say what the pipeline does.
+ - "Transcode as a stage": no doc claims one any more. The remaining "transcode" rows are
+ ffmpeg's audio extraction.
+ - `PARALLEL_TRANSCRIBE_LIMIT` was already in no doc. The only mention is the released
+ changelog bullet that records its removal.
+- **The two stale hub hints were already fixed.** `SettingsForm.tsx`'s `homepageUrl` hint and
+ `homepage.ts`'s comments name `archilyzer-hub`, since release 7's review fix `bc2d9fb6`.
+ STATE's note is stale; nothing to do.
+
+| sha | what |
+|---|---|
+| `eeeef338` | `common:` `lib/ports.mjs` (+ `ports.test.ts` 4), `worktree.mjs` and `sync-tick` read it; `lib/toolProbe.mjs`, umtool's `tools.mjs` calls it |
+| `9be897c6` | `common:` `lib/envVars.ts` (+ `envVars.test.ts` 6), `bin/env-docs.ts`, `ENVIRONMENT.md`, `docs env` row |
+| `5c16a75c` | `common:` `archilyzer doctor` (+ `doctor.test.ts` 8), `settingsFromFile` |
+| `5da3e9e1` | `common:` the digest channel job's `ids`; the dispatcher passes ids; the editor's digest replay forwards them |
+| `3bfe2ab2` | `common:` `run`, `mcp`, passthrough rows, `_spawnBin.ts`, every bin a row (+ `run-operation.test.ts` 6, `mcp.test.ts` 1, `_cli.test.ts` +3) |
+| `e7de1d96` | `export, homepage:` the pipeline's scripts call the CLI; `detect:duplicates` removed |
+| `a132c213` | `editor, common:` root `pnpm archilyzer`; the Docker build-mode copy (Settings, schema, SETTINGS.md); build.ts names PUBLISH.md |
+| `58fd7d72` | `docs:` PUBLISH.md; the two deploy docs removed; README/SETUP/CONTRIBUTING/AGENTS/RUNNING_IN_DOCKER/r2-proxy/homepage README |
+| `1a65926b` | `common:` the mcp test's exit race and the run tests' timeouts (found by the bite checks) |
+| _this_ | `plans:` this record; the `[Unreleased]` bullet in `editor/CHANGELOG.md` |
+
+**Gates**, all from the worktree root. The logs are `o6-*.log` in the job's scratch dir.
+- **tsc** (`pnpm -r --no-bail --workspace-concurrency=1 exec tsc --noEmit`) was clean four times:
+ 91 s before `eeeef338`, 58 s before `5da3e9e1`, 66 s before `a132c213`, and 95 s on
+ `1a65926b`. The three commit groups in between are prefix-closed subsets of a checked tree: no
+ file in a commit imports one from a later commit. `58fd7d72` is docs only.
+- **common 2,066/2,066** (2,038 + 28: ports 4, envVars 6, doctor 8, run 6, mcp 1, `_cli` 3), 47 s.
+- **editor unit 85/85**, 4 s.
+- **`test:scripts` 174 + 1 skip**, 9 s.
+- **mcp 269/269**, 18 s.
+- **`next build`:**
+ - editor ok, 36 s;
+ - umtool ok, 17 s (`tools.mjs` now imports `yt-dlp-transcript-common/lib/toolProbe.mjs`);
+ - export ok, 27 s (no dangling `export/public` links).
+- **e2e:**
+ - **Editor** (`o6-specs.txt`: `digest.spec.ts backfill.spec.ts jobs-retry.spec.ts`, which
+ also matches `availability-backfill.spec.ts`), no queue wait: **39 passed, 0 failed, 3.2
+ min**.
+ - **umtool** `dashboard.spec.ts` (`SONG_DIR=~/reports/quartering-uh-song/data`, through
+ `wt run`): run 1 was **6 passed, 1 failed, 1.1 min**. The failure was "the projects panel
+ says exactly what /browse's header says": `page.goto` aborted during the cold `next dev`
+ compile (35 s). It does not touch the tool probe. The two probe tests (the tools panel, and
+ `umtool doctor` exiting 1 on a bad path) passed. Rerun: **7 passed, 25.8 s**, after 19 s in
+ the queue.
+ - Nothing else an e2e exercises changed: the editor suite never runs the export build scripts
+ to completion (`deploy-page.spec.ts:130` says so), hence the scratch smoke-run.
+- **The CLI by hand, on scratch corpora only:**
+ - `run diarization chan` exited 0 in 2 s, with the summary line and two job records (the run
+ and the snapshot).
+ - `run sync chan` exited 2 with the sentence.
+ - `mcp --local <dir>` answered initialize, both directly and via `pnpm -C … archilyzer mcp`.
+ - `doctor` on the worktree: no failures. Parakeet workers ×3 (engine, cli, model), every
+ binary, umtool's table (python absent: info), block #6 with every port free.
+- **Nothing ran against the primary's `transcripts/`.**
+
+**They bite** (each mutation in a scratch copy of the tip, `o6-bite.log`):
+- ports: an export dev default drifting to 3009, and an undeclared `NEW_STUB_PORT` in a
+ playwright config, each fail 1.
+- envVars:
+ - an undeclared read in `workerToken.ts` fails 1;
+ - a declared var nothing reads fails 1;
+ - a hand-edited ENVIRONMENT.md fails 1;
+ - a new override in `paths.ts` fails 2.
+- doctor:
+ - a `mkdirSync` before returning fails 3 (the tree snapshots);
+ - a non-JSON settings file as a warning fails 1;
+ - a corpus that does not need the media tools fails 1.
+- run:
+ - the dispatcher dropping `ids` fails 1 ("ids scope the run");
+ - external ops not refused fails 2;
+ - a paused lane not refused gives pass 2, cancelled 4, exit 1. The held job leaves only
+ unref'd poll timers and node cancels the rest.
+- cli: `verify transcripts`' row removed fails 1 (every bin reachable); passthrough disabled
+ fails 1.
+- mcp: the row renamed fails 1. It hung before `1a65926b`, which the bite found.
+
+**Found and left.**
+- **AGENTS.md says `defaults()` returns `workers: []`**, so zero workers means auto-transcribe
+ does nothing. That is true of `defaults()`, but a settings file with no `workers` key, or no
+ file at all, is read through `defaultWorkersFromApps`, which synthesizes two enabled whisper
+ workers. The doctor reports what the reader yields: "worker whisper.cpp #1/#2 is enabled". Not
+ changed.
+- **The released editor changelog bullets still link `../DEPLOY_CLOUDFLARE.md` /
+ `../DEPLOY_DOCKER.md`.** They are dated records, so they were left as plans/ are. On `/changelog`
+ those links now 404. Question for the reviewer: amend the targets, or leave history alone?
+- **`archilyzer transcribe` is the old `transform.ts` shim.** It is a whisper batch with this
+ process's own worker pool, which is exactly what `run transcription` refuses. It is surfaced
+ with that caveat in its usage. Deleting it is a product call.
+- **No spec replays a digest job**, so the replay forwarding `ids` is type-checked only.
+- `mcp/README.md`'s and AGENTS.md's `claude mcp add` examples were left alone (O1 owns the first;
+ the prompt said leave both). The new form is in README and PUBLISH.md.
+- PUBLISH.md's homepage row names only the CLI. O4 adds `/sites` and `pnpm ops` verbs for it
+ tonight; the parent can fill the row at merge.
+- `homepage/content/docs/*` are hand-derived copies by design. Only the drift table in
+ `homepage/content/README.md` was updated.
+- **Optional, not done:** SETUP.md absorbing SCHEDULED_SYNC.md and WORKTREES.md; PLAN.md's phase
+ table becoming a pointer to STATE.md.
+- **Checkpoint B** (the `E2E_` prefix cleanup, and the playwright configs importing ports.mjs) is
+ next, after O1–O5 land on `main`.
+
## Rollout
Nothing is rolled out tonight. The morning runbook lists what is owed: the :3001 editor restart,
