commit c3890608162a0f674ecb611034f333fc35200d28
parent dcc6501471c9d7f0cc6c65b79cd9df408732b774
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date: Sat, 10 Oct 2026 00:46:29 -0400
common: release 20 D3 — the en → 0 cues caption bug is closed; an index test for the fall-through, recorded in FACTS and STATE
An empty en-orig before an en with text, and an empty en before an en-US with
text, through the real buildIndex. FACTS gains "The caption-track rule" and a
superseded note on the old "fixture must carry timing tags" line; STATE closes
the bug and names what is left (the stats cache does not see the caption-track
pass's re-reads).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Diffstat:
3 files changed, 63 insertions(+), 0 deletions(-)
diff --git a/common/controller/buildIndexCaptionTrack.test.ts b/common/controller/buildIndexCaptionTrack.test.ts
@@ -154,3 +154,31 @@ test("an index built under the old rule is re-read once, for exactly the records
const again = await runIndex();
assert.equal(again.some((l) => /Caption track/.test(l)), false, again.join("\n"));
});
+
+// THE 2026-10-01 BUG, CLOSED (release 20 D3): a preferred track that parses to
+// no cues never hides a later one with text — in either order of the two
+// tracks the bug was about, and past `en` to a regional track. Added to the
+// index built above, so the records are read under the current rule.
+test("a preferred English track with no cues falls through to the next one with text", async () => {
+ const EMPTY_VTT = "WEBVTT\nKind: captions\nLanguage: en\n\n";
+ const ORIG_EMPTY = "OrigEmpty001"; // en-orig empty, en with text
+ const EN_EMPTY = "EnEmptyUS001"; // en empty, no en-orig, en-US with text
+ for (const [i, id] of [ORIG_EMPTY, EN_EMPTY].entries()) {
+ writeJson(path.join(dirOf(id), "metadata.info.json"), {
+ id,
+ title: `Video ${id}`,
+ upload_date: `2026070${i + 1}`,
+ duration: 30,
+ webpage_url: `https://www.youtube.com/watch?v=${id}`,
+ extractor_key: "Youtube",
+ });
+ }
+ writeFileSync(path.join(dirOf(ORIG_EMPTY), "transcript.en-orig.vtt"), EMPTY_VTT);
+ writeFileSync(path.join(dirOf(ORIG_EMPTY), "transcript.en.vtt"), ROLLING);
+ writeFileSync(path.join(dirOf(EN_EMPTY), "transcript.en.vtt"), EMPTY_VTT);
+ writeFileSync(path.join(dirOf(EN_EMPTY), "transcript.en-US.vtt"), CUE_BLOCKS);
+ await runIndex();
+ assert.equal(cuesOf(ORIG_EMPTY)?.length, 3);
+ assert.equal(cuesOf(EN_EMPTY)?.length, 7);
+ assert.equal(cuesOf(EN_EMPTY)?.[0].text.startsWith("welcome back everyone"), true);
+});
diff --git a/plans/FACTS.md b/plans/FACTS.md
@@ -7913,6 +7913,9 @@ on anchors elsewhere in this file:
zero cues. `maybeMissingBuild.test.ts`'s VTT is such a file, which is harmless there.
- The same rule makes a video whose English track is a *manual* caption (no inline tags) index
as 0 cues. That is rare: 0 in 3,000 sampled of the-quartering, 6 of chibi-reviews.
+ > **Superseded by `200a3105` (2026-10-06; verified 2026-10-10, release 20 D3):** `parseVtt` reads a document
+ > with no timing tag as plain cue blocks (every line of a cue is its text), so a manual caption and the
+ > livestream `en` shape parse; see "The caption-track rule" at the end of this file.
- **Measured before the fix,** on the whole-pool stats of 2026-09-28T20:40Z:
- 49,798 transcripts shown, of about 77,000 on disk;
- 24,710 records with `hasTranscript` but no `transcribedDate`;
@@ -9084,3 +9087,32 @@ The record is [`landed-2026-10.md`](landed-2026-10.md). Every anchor below was r
- **/sites** lists every site's articles (`umtool/lib/articles/sites.ts`): published ids unioned with draft
report dirs, each with its open notes, its source and its linked video. An article row's `kind` is the REPORT's
kind (`factcheck|sweep`), not a project kind.
+
+## The caption-track rule (verified 2026-10-10, release 20 D3, against `585be292`)
+
+The `en` → 0 cues bug (filed 2026-10-01: the index preferred `en` over `en-orig`, and some livestream VODs' `en`
+parsed to 0 cues, indexing the video textless) is CLOSED by `200a3105` (transcripts/en-track-fallback).
+- **One rule, by name then by content** (`common/lib/videoStatus.ts`, `CAPTION_TRACK_RULE_VERSION` 1): English
+ tracks ranked operator pin (`transcript-pin.json` → `transcript.en.vtt` first) > `en-orig` > `en` > regional
+ (`en-US`, …) > `en-en-*`, ties by name (`englishVttsByPreference`); the cues are the first track in that order
+ that parses to at least one cue (`readEnglishVttCues`); all empty → the most preferred, with no cues.
+ `resolvePrimaryVtt` is the name half only (the editor's "primary" label and change detection).
+- **Every cue reader goes through it:** the index (`buildIndex.ts` → `readEnglishVttCues`), normalize
+ (`normalizeTranscript.ts`), report compose (`composeReports.ts`), the editor's track reader
+ (`captionTracks-server.ts`); umtool's `report-to-video/cues.mjs` carries a copy that `captionTrack.test.ts` holds
+ equal. `buildIndex.ts`'s other `parseVtt` call is the non-English/live-chat subtitle tracks.
+- **`parseVtt` reads two shapes** (`common/lib/vtt.ts`), decided per document by the presence of an inline
+ `<hh:mm:ss.mmm>` tag: rolling (keep the tagged line) or cue blocks (every line, all tags stripped, entities
+ decoded). The fixture of the second is `common/lib/__fixtures__/vtt-cue-blocks.vtt`.
+- **An index built under an older rule is re-read once** for the records the rule can reach (two or more English
+ VTTs, or stored cues empty), logged per channel (`Caption track v1: <slug>: N re-read, M now read different text,
+ K had no text and now do.`), then `meta.captionTrackRule` is recorded.
+- **Tests:** `lib/captionTrack.test.ts` (order, pin, the fall-through, umtool's copy), `lib/vtt.test.ts` (both
+ shapes), `controller/buildIndexCaptionTrack.test.ts` (through the real `buildIndex`: `en-orig` beside a served
+ cue-block `en`, a lone cue-block `en`, the one-shot re-read, and — release 20 D3 — an empty `en-orig` before an
+ `en` with text and an empty `en` before an `en-US` with text).
+- **Not closed by it (found 2026-10-10, not fixed):** the stats cache (see "The stats cache key") keys a video's
+ stat on its `mtimes` record, and the caption-track pass rewrites a record's cues with an UNCHANGED `mtimes`
+ record (`buildIndex.ts`, the `mtimes.put` after processing writes the same `metaMs`/`transcriptMs`/…). So a
+ record the pass took from no text to text keeps `hasTranscript: false` and its old `cueCount` in the stats until
+ something else moves its record.
diff --git a/plans/STATE.md b/plans/STATE.md
@@ -22,6 +22,9 @@ first. Nothing of release 18 is live.
[`release-20.md`](release-20.md) (data model); Track B release 19 B1–B6 (machine safety and tooling); Track C docs
and plans (OPERATING.md and `archilyzer docs cli`, doc fixes, homepage docs, FACTS). Integration branch
`r19/integration`, branched from `4cffda3f`.
+- **The `en` → 0 cues caption bug (filed 2026-10-01) is CLOSED** by `200a3105` (en-orig first, empty tracks fall
+ through, cue-block VTTs parse, a one-shot index re-read); verified against the tree and an index fixture by release
+ 20 D3 (2026-10-10; FACTS, "The caption-track rule"). Left open: the stats cache does not see that pass's re-reads.
**Previously (2026-10-06): release 18 — publishing as queueable stages — is complete on `r18/integration`** (record:
[`release-18.md`](release-18.md): slices S1 the stage contract, stamps, lock, bundles and CLI; S2 deploy hardening; S3
