Archilyzer · Source

archilyzer

Archilyzer
git clone https://archilyzer.pages.dev/source/archilyzer.git
Log | Files | Refs | README | LICENSE

commit 52ed923ea1d8431e09f4d790633537cd73d8a84f
parent 5d1a28630eb51cf216d81fa4f104e58dc9f64349
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date:   Mon, 27 Jul 2026 00:40:01 -0400

Settings: a Digest section that actually exists

saveSettingsAction has had a complete digest branch since Stage B1, gated
on a hidden `digestFormPresent` marker that NOTHING in the repo emitted —
so no digest knob was reachable from a browser, while digestActions.ts and
DigestStage.tsx both told the operator to "enable it in Settings → Digest".
DIGEST_TIMESTAMP_MODE_OPTIONS was exported "for the settings UI" and unused.

SettingsForm gains the fieldset that feeds it: local engine, sections,
timestamp mode, prompt variant label, metered lane + its engine, long-tail
cutoff, spend cap, and a per-engine block (model / num_ctx / temperature /
base URL / binary) serialized to digestAppsJson in the WorkersField idiom.
timestampMode and promptVariant are now READ from the form; both still fall
back to the current value when absent, so a form that omits them cannot
silently reset the prompt shape.

Two traps worth naming:

- The option lists come from lib/digest.ts, NOT settings.ts's re-exports of
  them. settings.ts opens with `import fs from "node:fs"`, so importing
  them through it drags node:fs into the client bundle and the page dies at
  runtime. Same split digestApps.ts/execa already observes.
- `<input type="number">` carries an IMPLICIT step of 1, so a decimal spend
  cap failed constraint validation and the browser blocked the submit
  SILENTLY — no error, no request, the save simply never happened. Field
  takes a `step` now.

e2e: the fieldset round-trips every value (including per-app numCtx through
the hidden JSON), and an unrelated save does not reset timestampMode or
promptVariant — the exact silent-reset bug this block was pre-emptively
fixed for. The second test reloads between saves rather than submitting
twice from one render: React 19 resets <form action> inputs to that
render's defaultValue, so a same-render re-submit would re-post the
pre-save values and prove nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Diffstat:
Meditor/app/settings/actions.ts | 24++++++++++++++++--------
Aeditor/app/settings/components/DigestAppsField.tsx | 185+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Meditor/app/settings/components/SettingsForm.tsx | 177++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
Meditor/app/settings/page.tsx | 7++++++-
Meditor/e2e/settings.spec.ts | 89+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
5 files changed, 472 insertions(+), 10 deletions(-)

diff --git a/editor/app/settings/actions.ts b/editor/app/settings/actions.ts @@ -21,6 +21,7 @@ import { type SocialLink, } from "yt-dlp-transcript-common/lib/settings"; import { DEFAULT_TRANSCRIPTION_APP_ID } from "yt-dlp-transcript-common/lib/transcriptionApps"; +import { isDigestTimestampMode } from "yt-dlp-transcript-common/lib/digest"; import { DEFAULT_COOKIE_MODE, isCookieMode, @@ -220,6 +221,9 @@ export async function saveSettingsAction( // FormData: without it, a submit from any form that lacks the digest fields // would read remoteEnabled as false and silently reset the block. const digestFormPresent = formData.get("digestFormPresent") === "1"; + const digestTimestampModeRaw = String( + formData.get("digestTimestampMode") ?? "", + ).trim(); const digestSettings = ( digestFormPresent ? { @@ -241,14 +245,18 @@ export async function saveSettingsAction( String(formData.get("digestSpendCapUsd") ?? "").trim(), ), sections: digestSectionsRaw.length > 0 ? digestSectionsRaw : dD.sections, - // Prompt SHAPE. Carried through from the current value rather than - // read from the form, because the form has no fields for them yet — - // and a field this block rebuilds but does not carry is a field that - // silently resets on the next save. Both are freshness-affecting - // (see digestPromptVariant), so a silent reset would invalidate every - // digest generated under the non-default shape. - timestampMode: dD.timestampMode, - promptVariant: dD.promptVariant, + // Prompt SHAPE — both freshness-affecting (see digestPromptVariant), + // so a silent reset here would invalidate every digest generated + // under a non-default shape. Each still FALLS BACK to the current + // value rather than to the default when the field is absent: a form + // that rebuilds this block but omits a field is exactly how the reset + // bug happens, and the fallback is what makes omission harmless. + timestampMode: isDigestTimestampMode(digestTimestampModeRaw) + ? digestTimestampModeRaw + : dD.timestampMode, + promptVariant: formData.has("digestPromptVariant") + ? String(formData.get("digestPromptVariant") ?? "").trim() + : dD.promptVariant, } : dD ) as SiteSettings["digest"]; diff --git a/editor/app/settings/components/DigestAppsField.tsx b/editor/app/settings/components/DigestAppsField.tsx @@ -0,0 +1,185 @@ +"use client"; + +// Per-engine digest configuration, serialized to a hidden JSON field — the +// WorkersField idiom, for the same reason: `settings.digest.apps` is a keyed map +// of optional fields, and a flat set of form inputs cannot express "this app +// overrides only the model" without inventing a naming scheme per app. +// +// Only the fields an engine actually supports are rendered (DigestApp.fields), +// so the metered lane never shows a `num_ctx` box it would ignore. +// +// A blank input means "unset" and is OMITTED from the JSON rather than written +// as "" or 0 — sanitizeDigestApps drops empty values anyway, but writing them +// would make an untouched app look configured, and `numCtx: 0` in particular +// would read as a real (invalid) context size rather than as absence. + +import { useState } from "react"; +import type { DigestAppDescriptor } from "yt-dlp-transcript-common/lib/digestApps"; +import type { DigestAppConfig } from "yt-dlp-transcript-common/lib/digest"; +import { DEFAULT_DIGEST_NUM_CTX } from "yt-dlp-transcript-common/lib/digest"; + +type Row = { + bin: string; + baseUrl: string; + model: string; + numCtx: string; + temperature: string; +}; + +function toRow(cfg: DigestAppConfig | undefined): Row { + return { + bin: cfg?.bin ?? "", + baseUrl: cfg?.baseUrl ?? "", + model: cfg?.model ?? "", + numCtx: cfg?.numCtx != null ? String(cfg.numCtx) : "", + temperature: cfg?.temperature != null ? String(cfg.temperature) : "", + }; +} + +function serialize(rows: Record<string, Row>): string { + const out: Record<string, DigestAppConfig> = {}; + for (const [id, row] of Object.entries(rows)) { + const cfg: DigestAppConfig = {}; + if (row.bin.trim()) cfg.bin = row.bin.trim(); + if (row.baseUrl.trim()) cfg.baseUrl = row.baseUrl.trim(); + if (row.model.trim()) cfg.model = row.model.trim(); + const n = Number.parseInt(row.numCtx.trim(), 10); + if (Number.isFinite(n) && n > 0) cfg.numCtx = n; + const t = Number.parseFloat(row.temperature.trim()); + if (Number.isFinite(t) && t >= 0) cfg.temperature = t; + if (Object.keys(cfg).length > 0) out[id] = cfg; + } + return JSON.stringify(out); +} + +export function DigestAppsField({ + apps, + initial, + name, +}: { + apps: DigestAppDescriptor[]; + initial: Record<string, DigestAppConfig>; + name: string; +}) { + const [rows, setRows] = useState<Record<string, Row>>(() => + Object.fromEntries(apps.map((a) => [a.id, toRow(initial[a.id])])), + ); + + const set = (id: string, key: keyof Row, value: string): void => + setRows((prev) => ({ ...prev, [id]: { ...prev[id], [key]: value } })); + + return ( + <div className="flex flex-col gap-3"> + {apps.map((app) => { + const row = rows[app.id] ?? toRow(undefined); + return ( + <div + key={app.id} + aria-label={`digest engine ${app.id}`} + className="flex flex-col gap-2 rounded border border-border p-2" + > + <div className="flex items-baseline gap-2"> + <span className="text-sm font-medium">{app.label}</span> + <code className="text-xs text-muted-foreground">{app.id}</code> + <span className="text-xs text-muted-foreground"> + {app.metered ? "metered" : "local"} + </span> + </div> + {app.fields.model && ( + <Cell + label="Model" + id={app.id} + field="model" + value={row.model} + onChange={set} + placeholder={app.defaultModel} + hint={`Blank uses ${app.defaultModel}. Freshness compares what you ask for, so an alias resolving to a full tag is not a model change.`} + /> + )} + {app.fields.numCtx && ( + <Cell + label="Context window (num_ctx)" + id={app.id} + field="numCtx" + value={row.numCtx} + onChange={set} + placeholder={String(DEFAULT_DIGEST_NUM_CTX)} + type="number" + hint={`Blank uses ${DEFAULT_DIGEST_NUM_CTX}, the size the bake-off measured best. The transcript slice is sized to this automatically — ollama truncates an over-long input SILENTLY, so the two must move together. Changing it changes the recorded identity and re-runs the corpus.`} + /> + )} + {app.fields.temperature && ( + <Cell + label="Temperature" + id={app.id} + field="temperature" + value={row.temperature} + onChange={set} + placeholder="0" + hint="0 for a structured extraction task." + /> + )} + {app.fields.baseUrl && ( + <Cell + label="Base URL" + id={app.id} + field="baseUrl" + value={row.baseUrl} + onChange={set} + placeholder="http://127.0.0.1:11434" + hint="Blank uses OLLAMA_URL, then the local default." + /> + )} + {app.fields.bin && ( + <Cell + label="Binary" + id={app.id} + field="bin" + value={row.bin} + onChange={set} + placeholder="claude" + hint="Blank uses CLAUDE_BIN, then PATH." + /> + )} + </div> + ); + })} + <input type="hidden" name={name} value={serialize(rows)} readOnly /> + </div> + ); +} + +function Cell({ + label, + id, + field, + value, + onChange, + placeholder, + hint, + type = "text", +}: { + label: string; + id: string; + field: keyof Row; + value: string; + onChange: (id: string, key: keyof Row, value: string) => void; + placeholder?: string; + hint?: string; + type?: string; +}) { + return ( + <label className="flex flex-col gap-1 text-sm"> + <span className="font-medium">{label}</span> + <input + type={type} + value={value} + placeholder={placeholder} + aria-label={`${label} for ${id}`} + onChange={(e) => onChange(id, field, e.target.value)} + className="rounded border border-border bg-card px-2 py-1 text-sm" + /> + {hint && <span className="text-xs text-muted-foreground">{hint}</span>} + </label> + ); +} diff --git a/editor/app/settings/components/SettingsForm.tsx b/editor/app/settings/components/SettingsForm.tsx @@ -6,11 +6,22 @@ import { type SaveResult, } from "../actions"; import type { SiteSettings } from "yt-dlp-transcript-common/lib/settings"; +// From the CLIENT-SAFE digest module, NOT settings.ts's re-exports of them: +// settings.ts opens with `import fs from "node:fs"`, so pulling the option +// lists through it drags node:fs into the client bundle and the page dies at +// runtime. Same rule the digest modules are already split along. +import { + DEFAULT_DIGEST_TIMESTAMP_MODE, + DIGEST_SECTION_KINDS as DIGEST_SECTION_OPTIONS, + DIGEST_TIMESTAMP_MODES as DIGEST_TIMESTAMP_MODE_OPTIONS, +} from "yt-dlp-transcript-common/lib/digest"; import { DOWNLOAD_FORMAT_LABELS, DOWNLOAD_FORMAT_PRESETS, } from "yt-dlp-transcript-common/ytdlp/downloadFormat"; import type { TranscriptionAppDescriptor } from "yt-dlp-transcript-common/lib/transcriptionApps"; +import type { DigestAppDescriptor } from "yt-dlp-transcript-common/lib/digestApps"; +import { DigestAppsField } from "./DigestAppsField"; import { SocialLinksField, toSocialRow, @@ -21,9 +32,12 @@ import { WorkersField } from "./WorkersField"; type Props = { initial: SiteSettings; apps: TranscriptionAppDescriptor[]; + // Built on the server: digestApps.ts reaches process.env and imports execa, so + // it must never end up in the client bundle. Same reason as `apps`. + digestApps: DigestAppDescriptor[]; }; -export function SettingsForm({ initial, apps }: Props) { +export function SettingsForm({ initial, apps, digestApps }: Props) { const [state, formAction] = useActionState<SaveResult | undefined, FormData>( saveSettingsAction, undefined, @@ -403,6 +417,160 @@ export function SettingsForm({ initial, apps }: Props) { /> </fieldset> <fieldset className="flex flex-col gap-3 border border-border rounded p-3"> + <legend className="px-1 text-sm font-medium">Digest</legend> + {/* + THE MARKER THAT ARMS THE WHOLE BLOCK. Unchecked checkboxes are simply + absent from a FormData, so without it a submit from any other form + would read `digestRemoteEnabled` as false and silently reset this + section. saveSettingsAction gates every field below on it. + */} + <input type="hidden" name="digestFormPresent" value="1" readOnly /> + <p className="text-xs text-muted-foreground"> + AI chapters and topic tags generated from existing transcripts by a + local model, written to <code>ai-digest.json</code> beside each one. + Run a sweep from a channel&apos;s <strong>Digest</strong> stage. Every + section records the exact engine, model, prompt version and prompt + shape that produced it, and a re-run regenerates only what no longer + matches — so changing anything here is what makes the next run redo + work, and leaving it alone is what makes a re-run nearly free. + </p> + <label className="flex flex-col gap-1 text-sm"> + <span className="font-medium">Local engine</span> + <select + name="digestLocalAppId" + defaultValue={initial.digest.localAppId} + className="rounded border border-border bg-card px-2 py-1 text-sm" + > + {digestApps + .filter((a) => !a.metered) + .map((a) => ( + <option key={a.id} value={a.id}> + {a.label} + </option> + ))} + </select> + <span className="text-xs text-muted-foreground"> + The lane that carries the corpus. Runs on your own hardware; nothing + leaves the machine. + </span> + </label> + <label className="flex flex-col gap-1 text-sm"> + <span className="font-medium">Sections to generate</span> + <span className="flex flex-wrap gap-3"> + {DIGEST_SECTION_OPTIONS.map((section) => ( + <label key={section} className="flex items-center gap-1 text-sm"> + <input + type="checkbox" + name="digestSections" + value={section} + defaultChecked={initial.digest.sections.includes(section)} + /> + {section} + </label> + ))} + </span> + <span className="text-xs text-muted-foreground"> + Chapters alone is the default: tags roughly double the model calls + for a smaller payoff. Unchecking everything falls back to chapters + rather than generating nothing. + </span> + </label> + <label className="flex flex-col gap-1 text-sm"> + <span className="font-medium">Timestamp mode</span> + <select + name="digestTimestampMode" + defaultValue={initial.digest.timestampMode} + className="rounded border border-border bg-card px-2 py-1 text-sm" + > + {DIGEST_TIMESTAMP_MODE_OPTIONS.map((mode) => ( + <option key={mode} value={mode}> + {mode} + {mode === DEFAULT_DIGEST_TIMESTAMP_MODE ? " (recommended)" : ""} + </option> + ))} + </select> + <span className="text-xs text-muted-foreground"> + How each chunk&apos;s transcript markers are numbered.{" "} + <strong>chunk-local</strong> re-bases every chunk to 00:00:00 and + adds the offset back before any guard runs; on the long tail it cut + wasted chunks from 29.4% to 11.8% and the worst coverage gap from + 1:05:16 to 24:13, because the model stops having to hold a large + offset. <strong>absolute</strong> is kept for comparison. This + changes the recorded identity, so switching it re-runs the corpus + instead of silently mixing two shapes. + </span> + </label> + <Field + label="Prompt variant label" + name="digestPromptVariant" + defaultValue={initial.digest.promptVariant} + hint="Free-text label for a non-default prompt shape, recorded in every section's provenance (trimmed, max 40 chars). Setting or changing it invalidates digests generated under a different label — which is exactly what makes a bake-off round re-run its sample instead of skipping it as fresh. Leave blank unless you are running one." + /> + <details className="text-sm"> + <summary className="cursor-pointer font-medium"> + Per-engine configuration + </summary> + <div className="mt-2"> + <DigestAppsField + apps={digestApps} + initial={initial.digest.apps} + name="digestAppsJson" + /> + </div> + </details> + <label className="flex items-start gap-2 text-sm"> + <input + type="checkbox" + name="digestRemoteEnabled" + defaultChecked={initial.digest.remoteEnabled} + className="mt-1" + /> + <span className="flex flex-col gap-1"> + <span className="font-medium"> + Enable the metered (paid) digest lane + </span> + <span className="text-xs text-muted-foreground"> + Off by default, and deliberately: this lane sends transcripts to a + paid API. It exists for the long tail of very long videos, which + is a small fraction of the corpus by count and a large one by + tokens. With it off, the metered option is disabled in every + channel&apos;s Digest stage rather than failing after the fact. + </span> + </span> + </label> + <label className="flex flex-col gap-1 text-sm"> + <span className="font-medium">Metered engine</span> + <select + name="digestRemoteAppId" + defaultValue={initial.digest.remoteAppId} + className="rounded border border-border bg-card px-2 py-1 text-sm" + > + {digestApps + .filter((a) => a.metered) + .map((a) => ( + <option key={a.id} value={a.id}> + {a.label} + </option> + ))} + </select> + </label> + <Field + label="Long-tail cutoff (seconds)" + name="digestLongTailSeconds" + defaultValue={String(initial.digest.longTailSeconds)} + type="number" + hint="Videos at least this long are what the metered lane takes when you run it. Default 14400 (4 h)." + /> + <Field + label="Spend cap (USD per job)" + name="digestSpendCapUsd" + defaultValue={String(initial.digest.spendCapUsd)} + type="number" + step="0.01" + hint="Hard ceiling on cumulative metered spend within one job; the lane parks itself when it is reached. 0 means no cap. Only ever consulted for a metered engine." + /> + </fieldset> + <fieldset className="flex flex-col gap-3 border border-border rounded p-3"> <legend className="px-1 text-sm font-medium">Social links</legend> <p className="text-xs text-muted-foreground"> Default social links shown in every site&apos;s footer. Each site can @@ -449,6 +617,7 @@ function Field({ hint, required, type = "text", + step, }: { label: string; name: string; @@ -456,6 +625,11 @@ function Field({ hint?: string; required?: boolean; type?: string; + // `type="number"` carries an IMPLICIT step of 1, so a decimal value fails + // constraint validation and the browser blocks the submit SILENTLY — no + // error, no request, the save just never happens. Any numeric field that + // accepts fractions must pass a step. + step?: string; }) { return ( <label className="flex flex-col gap-1 text-sm"> @@ -465,6 +639,7 @@ function Field({ name={name} defaultValue={defaultValue} required={required} + step={step} className="rounded border border-border bg-card px-2 py-1 text-sm" /> {hint && <span className="text-xs text-muted-foreground">{hint}</span>} diff --git a/editor/app/settings/page.tsx b/editor/app/settings/page.tsx @@ -2,6 +2,7 @@ import type { Metadata } from "next"; import { getPaths } from "yt-dlp-transcript-common/lib/paths"; import { getSettings } from "yt-dlp-transcript-common/lib/settings"; import { listTranscriptionApps } from "yt-dlp-transcript-common/lib/transcriptionApps"; +import { listDigestApps } from "yt-dlp-transcript-common/lib/digestApps"; import { readXSessionStatus } from "yt-dlp-transcript-common/social/xSessionBroker"; import { SettingsForm } from "./components/SettingsForm"; import { XSessionSection } from "./components/XSessionSection"; @@ -42,7 +43,11 @@ export default async function SettingsPage() { Used by the export build to title the static site. </p> </div> - <SettingsForm initial={settings} apps={listTranscriptionApps()} /> + <SettingsForm + initial={settings} + apps={listTranscriptionApps()} + digestApps={listDigestApps()} + /> </section> <section className="flex flex-col gap-3 border-t border-border pt-6"> diff --git a/editor/e2e/settings.spec.ts b/editor/e2e/settings.spec.ts @@ -165,3 +165,92 @@ test("saves build pipeline settings (mode, concurrency, image)", async ({ .getByRole("button", { name: "Docker" }), ).toHaveAttribute("aria-pressed", "true"); }); + +// --------------------------------------------------------------------------- +// Digest +// --------------------------------------------------------------------------- +// +// Every digest knob used to be unreachable from a browser: saveSettingsAction +// had a full digest branch gated on a `digestFormPresent` marker that NOTHING in +// the repo emitted, while two places told the operator to "enable it in +// Settings → Digest" — a section that did not exist. + +test("digest settings round-trip through the form", async ({ page }) => { + await page.goto("/settings"); + + await page.getByLabel("Timestamp mode").selectOption("absolute"); + await page.getByLabel(/prompt variant label/i).fill("bakeoff-r3"); + await page.getByLabel(/spend cap/i).fill("12.5"); + await page.getByLabel(/long-tail cutoff/i).fill("7200"); + await page + .getByRole("checkbox", { name: /enable the metered/i }) + .check(); + // Per-app numCtx rides in the hidden digestAppsJson payload. + await page.getByRole("group", { name: "Digest" }).getByText( + "Per-engine configuration", + ).click(); + await page.getByLabel("Context window (num_ctx) for ollama-direct").fill("4096"); + + await page.getByRole("button", { name: /save settings/i }).click(); + await expect( + page.getByRole("status").filter({ hasText: "Saved" }), + ).toBeVisible(); + + const saved = await readJson<{ + digest?: { + timestampMode: string; + promptVariant: string; + spendCapUsd: number; + longTailSeconds: number; + remoteEnabled: boolean; + sections: string[]; + apps: Record<string, { numCtx?: number }>; + }; + }>("test-settings.json"); + expect(saved.digest?.timestampMode).toBe("absolute"); + expect(saved.digest?.promptVariant).toBe("bakeoff-r3"); + expect(saved.digest?.spendCapUsd).toBe(12.5); + expect(saved.digest?.longTailSeconds).toBe(7200); + expect(saved.digest?.remoteEnabled).toBe(true); + expect(saved.digest?.apps["ollama-direct"]?.numCtx).toBe(4096); + // Never left empty — an empty section list would generate nothing. + expect(saved.digest?.sections.length).toBeGreaterThan(0); +}); + +test("an unrelated settings save does not reset the digest prompt shape", async ({ + page, +}) => { + // THE BUG THIS EXISTS FOR. The digest branch REBUILDS the whole block on every + // save, so any freshness-affecting field it fails to carry through is silently + // reset — and timestampMode/promptVariant are folded into the recorded + // identity, so a reset would invalidate every digest generated under the + // non-default shape while looking like a no-op. + await page.goto("/settings"); + await page.getByLabel("Timestamp mode").selectOption("absolute"); + await page.getByLabel(/prompt variant label/i).fill("keepme"); + await page.getByRole("button", { name: /save settings/i }).click(); + await expect( + page.getByRole("status").filter({ hasText: "Saved" }), + ).toBeVisible(); + + // Reload rather than submitting straight again: React 19 resets <form action> + // inputs to the defaultValue of the render they were in, so a second submit + // from the same render would re-post the digest values from BEFORE the first + // save and prove nothing. A reload is also the real scenario — the operator + // sets the digest up, comes back later, and changes something unrelated. + await page.reload(); + await expect(page.getByLabel("Timestamp mode")).toHaveValue("absolute"); + await page.getByLabel(/admin title/i).fill("Unrelated Change"); + await page.getByRole("button", { name: /save settings/i }).click(); + await expect( + page.getByRole("status").filter({ hasText: "Saved" }), + ).toBeVisible(); + + const saved = await readJson<{ + adminTitle: string; + digest?: { timestampMode: string; promptVariant: string }; + }>("test-settings.json"); + expect(saved.adminTitle).toBe("Unrelated Change"); + expect(saved.digest?.timestampMode).toBe("absolute"); + expect(saved.digest?.promptVariant).toBe("keepme"); +});