commit 9993290b532327fcff20c626f4e84437d91bfc99
parent bdc6ffd2a3362fee0b2f35694d3e53d8a5d162cc
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Wed, 29 Jul 2026 12:23:14 -0400
Two bugs the live sweep found that no test would have
Ran the orchestrator for real, scoped to one small channel (`teamrcn`, 7 videos,
0.27 audio-hours) against ollama and the real corpus. It armed with its scope
persisted, launched the per-channel job, generated 7 digests with correct
provenance, refreshed the snapshot and ended clean. A second process then
resumed from settings alone, found nothing to do, launched ZERO jobs and exited
— which is the whole of the B2 claim: a restart re-derives from disk and re-does
nothing.
It also surfaced two things:
**Every channel's progress bar would have stalled one short, forever.** The run
reported `current: 7, target: 8`. The eighth directory has no transcript, so
`runDigestBatch` correctly drops it before it is ever a candidate — but
`countMissingDigests` counted it, sizing the bar against work the batch will
never do. Corpus-wide that is 2,987 videos, so most channels would never read as
finished, and over 81 days "never finishes" is indistinguishable from wedged.
`countMissingDigests` now applies the same eligibility the batch does, which is
the agreement digestTarget.ts's header exists to enforce.
**Stopping a sweep left its scope behind.** The write was guarded on
`sweepEnabled`, so stopping an already-disarmed sweep skipped it entirely and
`sweepChannels` kept the old value — after which the next "Start Digest Sweep"
would silently cover only some weeks-old channel list and report itself done.
Verified fixed: settings now read `sweepEnabled=false sweepChannels=[]`.
Neither was reachable from a unit test or the e2e suite; both needed the thing
to actually run.
Measured in passing: 41.4 s of wall clock for 989 audio-seconds — ~151 s per
audio-hour, uncontended, on 2.4-minute videos. Consistent with the recorded
"short videos cost ~2x per audio-hour" against the 90 s corpus average, and NOT
a substitute for the contended-vs-uncontended measurement that still matters.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Diffstat:
3 files changed, 49 insertions(+), 2 deletions(-)
diff --git a/common/controller/digestBatch.ts b/common/controller/digestBatch.ts
@@ -546,6 +546,21 @@ export async function countMissingDigests(
: await readdir(dataDir).catch(() => [] as string[]);
let missing = 0;
for (const id of dirs) {
+ // SAME ELIGIBILITY AS THE BATCH, and it has to be. A video with no
+ // normalized transcript is a transcription problem, not a digest one:
+ // runDigestBatch drops it before it is ever a candidate, so counting it here
+ // sizes the progress bar against work the batch will never do and the bar
+ // stalls one short of complete — for the rest of the run.
+ //
+ // Measured: `teamrcn` reported target 8, current 7. The eighth directory is
+ // a channel-id-named folder with no transcript. Corpus-wide that is 2,987
+ // videos, i.e. most channels would never show as finished, which over an
+ // 81-day sweep is indistinguishable from wedged. This is exactly the
+ // counter-disagreement digestTarget.ts's header was written about.
+ const duration = await readDurationFast(
+ path.join(dataDir, id, CUES_JSON_FILENAME),
+ );
+ if (duration === null || duration <= 0) continue;
const record = await loadDigest(path.join(dataDir, id));
const fresh = sections.every((section) =>
isSectionFresh(record, section, target),
diff --git a/common/controller/digestSweep.ts b/common/controller/digestSweep.ts
@@ -377,11 +377,25 @@ export async function resumeDigestSweepIfEnabled(
// resurrect a sweep the operator had just stopped.
export async function stopDigestSweep(): Promise<boolean> {
const settings = getSettings();
- if (settings.digest.sweepEnabled) {
+ // Note the second clause: guarding on `sweepEnabled` alone means a stop on an
+ // already-disarmed sweep skips the write entirely and leaves the SCOPE behind
+ // — which is how a stale scope survives to narrow the next sweep. Measured:
+ // after a stop, settings still read sweepChannels ["teamrcn"].
+ if (
+ settings.digest.sweepEnabled ||
+ settings.digest.sweepChannels.length > 0
+ ) {
// Awaited so a restart cannot resurrect a sweep the operator just stopped.
await writeSettings({
...settings,
- digest: { ...settings.digest, sweepEnabled: false },
+ digest: {
+ ...settings.digest,
+ sweepEnabled: false,
+ // Cleared too. A leftover scope is worse than no scope: the next
+ // "Start Digest Sweep" would silently cover only the channels of a
+ // sweep somebody stopped weeks ago, and report itself finished.
+ sweepChannels: [],
+ },
});
}
const live = getSingleton().live;
diff --git a/plans/FACTS.md b/plans/FACTS.md
@@ -773,6 +773,7 @@ every one of the 21 is accounted for below. Do not chase these.
| `settings.spec` social-links | 1 | fails on base (known placeholder collision) |
| `site-scope.spec` dashboard stats | 1 | fails on base |
| `cut-release.spec` (both) | 2 | expected with a dirty working tree — the spec rewrites the real `editor/CHANGELOG.md` and makes a git commit |
+| `scheduler.spec:28` — queues due channels | 1 | fails on base (verified 2026-07-29 against `bbaa887`) |
| `sync-break-on-existing-tree.spec` | 1 | **passes in isolation** — order-dependent flake, not a base failure |
| `undownloaded.spec:53` | 1 | **passes in isolation** (all 4 in 22.6 s) — the CPU-contention flake already recorded below, seen again 2026-07-27 |
@@ -829,6 +830,23 @@ A frontmatter parser for Phase 1.5 must be hand-rolled (three scalar/list field
---
+**Re-verified 2026-07-29** after the backfill-readiness work: **368 passed / 22
+failed of 390** — one better than the previous run. All 22 are in the table
+above. Two were isolated rather than assumed:
+
+| Spec | Isolation result |
+| --- | --- |
+| `duplicate-shorts.spec:328` | **was a real regression, now fixed.** New review buttons carried `aria-label="confirm duplicate cluster <id>"`, and `getByLabel` matches by SUBSTRING, so the card's own `duplicate cluster <id>` locator resolved to 3 elements. Renamed; 3/3 green. **Third occurrence of this collision in this repo.** |
+| `scheduler.spec:28` | fails on base too — verified by checking out `bbaa887` and re-running. Add it to the base-failure set. |
+
+Common tests **374** (was 356). Export playwright **150/150**. `tsc` clean in all
+three packages.
+
+**`cut-release.spec` mutates the repo, and it did again**: it rewrote
+`editor/CHANGELOG.md` AND created a real `Release editor 9.9.10` commit. Reset
+after the run. Budget for this every time the full suite is run with a dirty or
+in-progress changelog.
+
## Smoke test results (2026-07-25)
Run against a real 176-minute podcast transcript (`UndertheTeaVT/data/5JRFDQ7TtZ8`, 3877
