Archilyzer · Source

archilyzer

Archilyzer
git clone https://archilyzer.pages.dev/source/archilyzer.git
Log | Files | Refs | README | LICENSE

commit 9bb68cbe45947302742400047f23ef34b3274360
parent cc3b64f76c0d1f3dd57d32f67f01fb50ffd7ff65
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date:   Mon, 28 Sep 2026 02:50:11 -0400

plans: slice O6 checkpoint A review fixes recorded — S1 the mention test, S2 the mode is a label, Q1/Q2/Q4/Q5, L1–L7; the changelog bullet says the mode is a label

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Diffstat:
Meditor/CHANGELOG.md | 2+-
Mplans/release-11.md | 85+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------
2 files changed, 67 insertions(+), 20 deletions(-)

diff --git a/editor/CHANGELOG.md b/editor/CHANGELOG.md @@ -6,7 +6,7 @@ - **"Persist source video" or a whole-recording fetch that cannot get the source no longer marks the video's download failed.** When YouTube's subtitles came down but the source video did not, the video page said "Download failed" over a transcript that is fine. The download now keeps the subtitle pass's result and records only the failed media attempt, with yt-dlp's reason. The run itself now ends failed with that reason; it used to end done with no file, so `fetch_clip` could only say the job "finished but named no file". A partial source file is left for a retry to resume, and the run's log names it. - **A bucket's retry keeps its log when it empties the bucket.** On a channel's Download stage, "Download with cookies", the partial-download resume and the missing-transcript retry could lose their run log part-way: the video they fetched left the bucket, the page refreshed, and the card disappeared with the log in it. The card now stays, with its log and its button disabled, until the page is reloaded. The Transcribe stage's "Fetch audio" button does the same. (A Diagnostics card still disappears, log and all, when its retry empties it.) - **A release cut whose commit fails still refreshes the pages.** When the changelog's new heading was written but the commit after it failed, the Cut release form and `pnpm ops cut-release` answered as if nothing had happened and no page showed the new heading until a reload. Both now refresh the changelog pages, and `pnpm ops cut-release` says the file was written. Every refused cut's answer says whether anything was written (`untouched`), and a cut of both changelogs that stopped half-way names the one already cut as well as the failure. -- **`archilyzer` checks the machine, runs one operation offline, starts the MCP server, and is one command from the repo root.** `pnpm archilyzer <command>` is the short form (`pnpm archilyzer --help` lists them all). `pnpm archilyzer doctor` is a read-only report: Node, the checkout, the corpus and whether each channel's media is reachable, `settings.json`, every tool the paths name plus each enabled worker's engine and model, umtool's report-pipeline tools, and this checkout's ports; it exits 1 only for something the machine is set up to do and cannot. `pnpm archilyzer run <operation> <channel> [ids…]` runs diarization, either attribution pass or digest over one channel as the editor's job does (a job record and log under `.jobs/`, the same summary line, the same refusal for an unmounted drive); sync, the metadata scan, downloads and transcription are refused with the reason, because they run on the editor's paced download queue and worker pool. `pnpm archilyzer mcp` starts the MCP server, so it can be registered as `-- pnpm -C "$PWD" archilyzer mcp`. Every other script in `common/bin/` is a subcommand too (`duplicates`, `posts fetch`, `digest plan`, `verify transcripts`, …), and export's `detect:duplicates` script is now `archilyzer duplicates`. Every environment variable is listed, by audience, in the new `ENVIRONMENT.md`, and `DEPLOY_CLOUDFLARE.md` and `DEPLOY_DOCKER.md` are now one `PUBLISH.md`. Settings no longer calls the Docker build mode a follow-up. +- **`archilyzer` checks the machine, runs one operation offline, starts the MCP server, and is one command from the repo root.** `pnpm archilyzer <command>` is the short form (`pnpm archilyzer --help` lists them all). `pnpm archilyzer doctor` is a read-only report: Node, the checkout, the corpus and whether each channel's media is reachable, `settings.json`, every tool the paths name plus each enabled worker's engine and model, umtool's report-pipeline tools, and this checkout's ports; it exits 1 only for something the machine is set up to do and cannot. `pnpm archilyzer run <operation> <channel> [ids…]` runs diarization, either attribution pass or digest over one channel as the editor's job does (a job record and log under `.jobs/`, the same summary line, the same refusal for an unmounted drive); sync, the metadata scan, downloads and transcription are refused with the reason, because they run on the editor's paced download queue and worker pool. `pnpm archilyzer mcp` starts the MCP server, so it can be registered as `-- pnpm -C "$PWD" archilyzer mcp`. Every other script in `common/bin/` is a subcommand too (`duplicates`, `posts fetch`, `digest plan`, `verify transcripts`, …), and export's `detect:duplicates` script is now `archilyzer duplicates`. Every environment variable is listed, by audience, in the new `ENVIRONMENT.md`, and `DEPLOY_CLOUDFLARE.md` and `DEPLOY_DOCKER.md` are now one `PUBLISH.md`. Settings and `/sites` no longer call the Docker build mode a follow-up, and say what it is: a label. **Build all sites** builds in containers whenever a container engine answers, whichever mode is set. ## [0.9.4] - 2026-09-28 - **On the Dark ground the sidebar's Archilyzer mark has a thin outline.** Its slate tile now has a 1-pixel ring just outside it, following its rounded corners, in the colour of the mark's unlit lines, so the tile's edge shows against the dark page. Light and Sepia are unchanged, and so is the favicon. diff --git a/plans/release-11.md b/plans/release-11.md @@ -393,6 +393,7 @@ mounted is fine as a rule, keep `untouched` optional tonight). - **They bite** (`o3-bite3.log`): with the old order in `sourceFetchFailure`, the table test fails on the new row; with `persistKept` counting every return, the new persistKept test fails; the other 8 in the two files pass. + ### Slice O6, as shipped — one-core Phase 4 slice 3 (2026-09-28) — checkpoint A Branch `r11/phase-4-s3` off `main` `2162db92` (no later `main` to merge before the first commit), @@ -458,7 +459,8 @@ the `E2E_` prefix cleanup, which is checkpoint B, after O1–O5 land. argv). - As children with their own flags (`_spawnBin.ts`): `duplicates` (with the 8 GB heap its script had), `posts fetch|check`, `diarize backfill`, `digest plan|validate`, `reconcile video-dirs`, - `verify transcripts`, `transcribe` (`transform.ts`) and `migrate channel-priority`. + `verify transcripts`, `transcribe` (`transform.ts`; deleted with its row in the review + round, `9424d357`) and `migrate channel-priority`. - A test fails when a file in `common/bin/` has no row. - **package.json.** - Kept: the scripts the publish pipeline runs by name: export's `build:index`, `build:stats`, @@ -500,7 +502,10 @@ the `E2E_` prefix cleanup, which is checkpoint B, after O1–O5 land. - `ENVIRONMENT.md` is generated from it (`archilyzer docs env [--check]`). `envVars.test.ts` holds the list to the code in both directions: every `process.env.X` / `env.X` read under common/, editor/, export/, homepage/, mcp/src and scripts/ is declared, and every entry is - still mentioned. umtool's own knobs stay in umtool/docs until Phase 5. + still named somewhere outside the list (a mention, not a proven read). *(Review S1: as first + shipped, that second test counted `envVars.ts` itself as a mention and could not fail; fixed in + `8bea5844`, which also checks every `ARCHILYZER_*` name in docker/, the Dockerfiles and the + compose files is declared.)* umtool's own knobs stay in umtool/docs until Phase 5. - **PUBLISH.md absorbs DEPLOY_CLOUDFLARE.md and DEPLOY_DOCKER.md.** It covers what gets published, the editor / `pnpm ops` / CLI table, Pages, previews, R2, the cost-abuse defenses, containers and the MCP registration. Both old files are removed. Every link to them now names @@ -515,7 +520,11 @@ the `E2E_` prefix cleanup, which is checkpoint B, after O1–O5 land. - RUNNING_IN_DOCKER named whisper.cpp as the only engine; it now names parakeet.cpp beside it. - **Settings → Build pipeline still called Docker "a follow-up" that "falls back to a basic build"**, and its option read "(follow-up)". So did `settingsSchema.ts`'s description - (SETTINGS.md regenerated) and its BuildMode comment. All now say what the pipeline does. + (SETTINGS.md regenerated) and its BuildMode comment. *(Review S2: the first rewrite + overstated the mode as a switch — no build reads `buildPipeline.mode`. In `366553eb` the + Settings hints, the schema texts, /sites' toggle note, the Build all paragraph and lane + subtitle and the per-site panel note all say the mode is a label and Build all uses + containers whenever `docker version` answers; `deploy-page.spec.ts` in step.)* - "Transcode as a stage": no doc claims one any more. The remaining "transcode" rows are ffmpeg's audio extraction. - `PARALLEL_TRANSCRIBE_LIMIT` was already in no doc. The only mention is the released @@ -565,7 +574,7 @@ the `E2E_` prefix cleanup, which is checkpoint B, after O1–O5 land. - **The CLI by hand, on scratch corpora only:** - `run diarization chan` exited 0 in 2 s, with the summary line and two job records (the run and the snapshot). - - `run sync chan` exited 2 with the sentence. + - `run sync chan` exited 2 with the sentence (1 since review L6: a refusal, not a usage error). - `mcp --local <dir>` answered initialize, both directly and via `pnpm -C … archilyzer mcp`. - `doctor` on the worktree: no failures. Parakeet workers ×3 (engine, cli, model), every binary, umtool's table (python absent: info), block #6 with every port free. @@ -576,7 +585,10 @@ the `E2E_` prefix cleanup, which is checkpoint B, after O1–O5 land. playwright config, each fail 1. - envVars: - an undeclared read in `workerToken.ts` fails 1; - - a declared var nothing reads fails 1; + - a declared var nothing reads: as first shipped this "failed 1" only because ENVIRONMENT.md + went stale — the mention test itself passed (review S1). After `8bea5844`, with + ENVIRONMENT.md regenerated, the mention test fails 1; + - an undeclared `ARCHILYZER_NEW_KNOB` in `docker/entrypoint.sh` fails 1 (review L5); - a hand-edited ENVIRONMENT.md fails 1; - a new override in `paths.ts` fails 2. - doctor: @@ -586,36 +598,71 @@ the `E2E_` prefix cleanup, which is checkpoint B, after O1–O5 land. - run: - the dispatcher dropping `ids` fails 1 ("ids scope the run"); - external ops not refused fails 2; + - ids none of which is on disk, not refused, fails 1 (review L6); - a paused lane not refused gives pass 2, cancelled 4, exit 1. The held job leaves only unref'd poll timers and node cancels the rest. - cli: `verify transcripts`' row removed fails 1 (every bin reachable); passthrough disabled fails 1. - mcp: the row renamed fails 1. It hung before `1a65926b`, which the bite found. -**Found and left.** -- **AGENTS.md says `defaults()` returns `workers: []`**, so zero workers means auto-transcribe - does nothing. That is true of `defaults()`, but a settings file with no `workers` key, or no - file at all, is read through `defaultWorkersFromApps`, which synthesizes two enabled whisper - workers. The doctor reports what the reader yields: "worker whisper.cpp #1/#2 is enabled". Not - changed. -- **The released editor changelog bullets still link `../DEPLOY_CLOUDFLARE.md` / - `../DEPLOY_DOCKER.md`.** They are dated records, so they were left as plans/ are. On `/changelog` - those links now 404. Question for the reviewer: amend the targets, or leave history alone? -- **`archilyzer transcribe` is the old `transform.ts` shim.** It is a whisper batch with this - process's own worker pool, which is exactly what `run transcription` refuses. It is surfaced - with that caveat in its usage. Deleting it is a product call. +**Found and left** (after the review round): +- **For the operator: should Build all / Build & deploy all honour `buildPipeline.mode`?** Today + they use containers whenever `docker version` answers and the mode is a label (review S2). The + parent ruled copy-only tonight; honouring it (basic → serial host even when docker answers) is + a behaviour change for its own slice. +- **`run` does not see the editor's lanes** (review L1). Beside the editor's lane on the same + channel it does the same videos twice — wasted CPU, not damage (atomic writes). Said in the + header and the usage line; a `.jobs/` check for a running job of the same kind and slug is not + implemented. +- **Until :3001 restarts, its digest replay predates `ids`** (review L2): Retry on an ids-scoped + `archilyzer run digest` job from the old editor widens to the whole channel (money, on the + metered lane). Morning runbook: restart before retrying a CLI digest job. +- **The doctor's default engine binary comes from `app.defaultBin()`** (the global `getPaths()`), + not the injected paths (review L7). The same in production; wrong only under injected test + paths. Left: a fix would re-derive each app's default beside the registry. - **No spec replays a digest job**, so the replay forwarding `ids` is type-checked only. - `mcp/README.md`'s and AGENTS.md's `claude mcp add` examples were left alone (O1 owns the first; the prompt said leave both). The new form is in README and PUBLISH.md. -- PUBLISH.md's homepage row names only the CLI. O4 adds `/sites` and `pnpm ops` verbs for it - tonight; the parent can fill the row at merge. - `homepage/content/docs/*` are hand-derived copies by design. Only the drift table in `homepage/content/README.md` was updated. +- Resolved in the review round: AGENTS.md's, the entrypoint comment's and RUNNING_IN_DOCKER's + "no workers → zero workers" claim (Q5); the two released changelog links (Q1, retargeted, + words unchanged); `archilyzer transcribe` (Q2, deleted); PUBLISH.md's homepage row (Q4). - **Optional, not done:** SETUP.md absorbing SCHEDULED_SYNC.md and WORKTREES.md; PLAN.md's phase table becoming a pointer to STATE.md. - **Checkpoint B** (the `E2E_` prefix cleanup, and the playwright configs importing ports.mjs) is next, after O1–O5 land on `main`. +**Review fixes (checkpoint A)** — the review (`o6-review.md`) was SHIP AFTER FIXES: no blocker, +two should-fixes, seven lows; the parent ruled on the questions. First `main` `c1d4790a` (O4 + O3) +was merged: conflicts only in `editor/CHANGELOG.md` (one `[Unreleased]`: O4's and O3's bullets, +then O6's) and this file (O4's and O3's records, then this one). + +| sha | what | +|---|---| +| `fcdaea12` | merge `main` `c1d4790a` | +| `8bea5844` | S1: the mention test leaves `envVars.ts` out and can fail; L5: docker/'s `ARCHILYZER_*` names must be declared (+1 test) | +| `366553eb` | S2: the build-mode copy says the mode is a label (Settings, schema + SETTINGS.md, /sites toggle, Build all, per-site panel; `deploy-page.spec.ts`) | +| `ef84f535` | Q1: the two released changelog links → PUBLISH.md anchors, words unchanged | +| `9424d357` | Q2: `transform.ts` + the `transcribe` row deleted; L6: usage 2 / refusal 1, all-absent ids refused (+1 test); L1 in the header and usage | +| `bec01bf9` | Q5: the seeded-worker claim (AGENTS.md, the entrypoint comment, RUNNING_IN_DOCKER); L4: WORKTREES.md → ENVIRONMENT.md#ports, `pnpm dev:*` in ports.mjs, umtool's config comment | +| `36db91e2` | S2 / Q4 / L3 in PUBLISH.md | +| _this_ | `plans:` this record | + +Gates on the merged tree, from the worktree root (`o6-gateR.log`, `o6-e2e-editorR.log`): +- **tsc clean** before `8bea5844` (on the merge: 102 s; again with S1+S2: 76 s) and before + `9424d357` (33 s, incremental). `ef84f535`, `bec01bf9` and `36db91e2` change no TypeScript + but `ports.mjs`' description strings. +- **common 2,081/2,081** (main's merged tests plus this round's 2), 45 s. **editor unit 85/85**. + **`test:scripts` 175 + 1 skip** (O4's +1). **mcp 269/269**. +- **`docs env --check`, `settings example --check`, `docs files --check`: all exit 0.** +- `docker/entrypoint.sh` still parses (`sh -n`). +- **Editor e2e** `deploy-page.spec.ts digest.spec.ts backfill.spec.ts jobs-retry.spec.ts` + (`o6-specsR.txt`; `backfill.spec.ts` also matches `availability-backfill.spec.ts`), no queue wait: **43 passed, 0 failed, 2.9 min** (39 before + deploy-page.spec's 4, the build-mode sentence among them). +- **They bite** (`o6-bite.log`, "review round"): a declared `ZZ_NOBODY_READS_ME` with + ENVIRONMENT.md regenerated fails exactly the mention test (1); an undeclared + `ARCHILYZER_NEW_KNOB` in the entrypoint fails 1; all-absent ids not refused fails 1. + ## Rollout Nothing is rolled out tonight. The morning runbook lists what is owed: the :3001 editor restart,