Archilyzer · Source

archilyzer

Archilyzer
git clone https://archilyzer.pages.dev/source/archilyzer.git
Log | Files | Refs | README | LICENSE

commit 9788a19ee9d6cfe57810f78116e88318f2e10ddd
parent 5ccb1a8f113ba57f8bb50fef3e48fb66cf1dc6ed
Author: I Mean I'm Just Saying <imeanimjustsaying@kiwifarms.st>
Date:   Fri, 19 Jun 2026 20:58:15 -0400

Make channel video-list filters combinable (intersection) + add Partial filter

The filter chips above a channel's video list were single-select and had no
way to filter partial downloads. Each chip is now an independent toggle and
selecting several narrows to videos matching all of them (intersection). A new
Partial chip surfaces videos with a leftover .part download, so Transcribed +
Partial finds already-transcribed videos still carrying an orphaned
audio.<ext>.part (e.g. cornbreadman's transcribed-but-partial cruft) —
previously unreachable because a video's status is a single mutually-exclusive
value where partial_download outranks transcribed.

The Transcribed and Partial chips key off the row's independent transcribed /
partial flags rather than the status enum so combinations are meaningful;
"Transcribed" alone now also includes transcribed videos that carry a .part or
failed-transcoding marker (they are transcribed). The active set mirrors to the
URL as a comma-joined ?filter=a,b via history.replaceState — not router.replace,
which races the global AutoRefresh router.refresh() and gets dropped — so
reload/share preserves it without an RSC refetch per toggle. Filtering stays
entirely client-side; no snapshot/server changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Diffstat:
Meditor/CHANGELOG.md | 1+
Meditor/app/channels/[slug]/components/VideoListPane.tsx | 72++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------
Meditor/app/channels/[slug]/lib/videoRows.ts | 102+++++++++++++++++++++++++++++++++++++++++++++++++++++--------------------------
Meditor/app/channels/[slug]/page.tsx | 11++++++-----
Aeditor/e2e/video-filter-combine.spec.ts | 81+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
5 files changed, 211 insertions(+), 56 deletions(-)

diff --git a/editor/CHANGELOG.md b/editor/CHANGELOG.md @@ -1,6 +1,7 @@ # Changelog ## [Unreleased] +- **The channel video-list filters are now combinable (intersection), with a new Partial filter.** The filter chips above a channel's video list used to be single-select — clicking one replaced the last — and there was no way to filter for partial downloads at all. Each chip is now an independent toggle, and selecting several narrows to videos matching **all** of them (an intersection). A new **Partial** chip surfaces videos with a leftover `.part` download. The headline use: **Transcribed + Partial** finds videos that are already transcribed but still carry an orphaned `audio.<ext>.part` (e.g. cornbreadman's, where the transcript is done but the partial lingers as cruft) — previously unreachable because a video's status is a single mutually-exclusive value where `partial_download` outranks `transcribed`. To make combinations meaningful the **Transcribed** and **Partial** chips now key off the row's independent `transcribed` / `partial` flags rather than that status enum, so "Transcribed" alone now also includes transcribed videos that happen to carry a `.part` or a failed-transcoding marker (they *are* transcribed). The active set is mirrored to the URL as a comma-joined `?filter=a,b` via `history.replaceState` (so reload/share preserves it without an RSC refetch per toggle, and without racing the global auto-refresh), and the **All** chip clears the selection. See `editor/app/channels/[slug]/lib/videoRows.ts` and `editor/app/channels/[slug]/components/VideoListPane.tsx`. - **Bookmarks can be reordered on the management page, and the order carries to the compact menu.** Both the `/jobs/bookmarks` management list and the compact one-click menu atop `/jobs` and `/jobs/active` render bookmarks in stored order, but there was no way to change it — new bookmarks just landed on top. Each row on the management page now has **↑ / ↓** buttons that move the bookmark one slot (disabled at the ends), persisting the new order immediately to `transcripts/.bookmarks/bookmarks.json`. Because both views read the same array in order, reordering on the management page is reflected in the compact quick-run menu too, so you can put your most-used job first. See `moveBookmark` in `common/jobs/bookmarks.ts`, `moveBookmarkAction` in `editor/app/jobs/bookmarkActions.ts`, and `editor/app/jobs/components/BookmarksList.tsx`. - **"Retry partial downloads" no longer skips partials that carry an audio-check snapshot.** A transcribe channel's retry-bucket prefilter decided a video was "already complete" by looking for any `audio.*` file not ending in `.part` — which wrongly matched the audio-integrity snapshots (`audio.<ext>.part.good` / `.part.testing`) and sidecars (`audio.info.json`, `audio.live_chat.json`, `audio.*.tmp-*`) left in a partial video's dir. So a genuine `audio.<ext>.part` that happened to sit next to a `.part.good` snapshot got prefiltered out (`Prefilter: 0 missing destination files, N already complete` → `Nothing to fetch`), even though that same snapshot is excluded when the video is placed in the **Partial downloads** bucket. The prefilter (`destinationExists`) now reuses the same `isRealAudioFile` predicate the bucket uses, so the two agree and genuine partials resume. See `common/ytdlp/runYtdlp.ts` and `common/lib/videoStatus.ts`. - **One-click "Drain all" to spin work down before a server restart.** The `/jobs/active` header gained a **Drain all** button that, in a single confirmed action, **drains every running job** (lets in-flight sub-operations finish, starts no new ones) and **cancels every queued job** — so you can wind the queue down gracefully before restarting the server instead of draining/cancelling each job row by hand. It reuses the existing per-job drain path (`requestDrain` drains running jobs and cancels queued ones), so semantics match the per-row **Drain**/**Cancel** buttons exactly; non-drainable running kinds are marked draining and finish on their own. A confirm prompt guards the bulk action. See `editor/app/jobs/components/DrainAllButton.tsx` and `drainAllAction` in `editor/app/jobs/actions.ts`. diff --git a/editor/app/channels/[slug]/components/VideoListPane.tsx b/editor/app/channels/[slug]/components/VideoListPane.tsx @@ -5,7 +5,7 @@ import { useRouter, useSearchParams } from "next/navigation"; import { useMemo, useState, useTransition } from "react"; import type { StreamActionResult } from "yt-dlp-transcript-common/jobs/streamCommand"; import type { VideoRow, VideoFilter } from "../lib/videoRows"; -import { filterRows } from "../lib/videoRows"; +import { filterRows, serializeFilters } from "../lib/videoRows"; import { QueueControl } from "../../../components/QueueControl"; import { bulkClearFailedMarkersAction, @@ -41,19 +41,23 @@ type Props = { slug: string; rows: VideoRow[]; selectedVideoId: string | null; - initialFilter: VideoFilter; + initialFilters: VideoFilter[]; initialQuery: string; defaultTranscribeQueue: string; defaultDownloadQueue: string; existingQueues: string[]; }; +// Toggleable filter chips. They combine as an intersection (a row must match +// every active chip), so e.g. Transcribed + Partial finds already-transcribed +// videos that still carry a leftover .part download. "All" (rendered +// separately) is the cleared state, not a chip in this list. const FILTER_OPTIONS: { value: VideoFilter; label: string }[] = [ - { value: "all", label: "All" }, { value: "needs_action", label: "Needs action" }, { value: "failed", label: "Failed" }, { value: "no_audio", label: "No audio" }, { value: "downloaded_no_transcript", label: "No transcript" }, + { value: "partial", label: "Partial" }, { value: "untranscribable", label: "Untranscribable" }, { value: "running", label: "Running" }, { value: "transcribed", label: "Transcribed" }, @@ -77,7 +81,7 @@ export function VideoListPane({ slug, rows, selectedVideoId, - initialFilter, + initialFilters, initialQuery, defaultTranscribeQueue, defaultDownloadQueue, @@ -85,7 +89,9 @@ export function VideoListPane({ }: Props) { const router = useRouter(); const searchParams = useSearchParams(); - const [filter, setFilter] = useState<VideoFilter>(initialFilter); + const [filters, setFilters] = useState<Set<VideoFilter>>( + () => new Set(initialFilters), + ); const [query, setQuery] = useState(initialQuery); const [selected, setSelected] = useState<Set<string>>(new Set()); const [pending, startTransition] = useTransition(); @@ -98,11 +104,11 @@ export function VideoListPane({ const [deleteConfirm, setDeleteConfirm] = useState(""); const visibleRows = useMemo(() => { - const filtered = filterRows(rows, filter); + const filtered = filterRows(rows, filters); const q = query.trim().toLowerCase(); if (!q) return filtered; return filtered.filter((r) => r.id.toLowerCase().includes(q)); - }, [rows, filter, query]); + }, [rows, filters, query]); const allSelected = visibleRows.length > 0 && visibleRows.every((r) => selected.has(r.id)); @@ -130,15 +136,33 @@ export function VideoListPane({ }); } - function changeFilter(value: VideoFilter) { - setFilter(value); - const params = new URLSearchParams(searchParams.toString()); - if (value === "all") params.delete("filter"); - else params.set("filter", value); - const q = params.toString(); - router.replace(q ? `/channels/${slug}?${q}` : `/channels/${slug}`, { - scroll: false, - }); + // Filtering is entirely client-side, so we only need the URL to mirror the + // active filters for reload/share — not a navigation. Use history.replaceState + // (Next's useSearchParams still reflects it) instead of router.replace: it + // avoids an RSC refetch per chip toggle and, crucially, won't be dropped when + // it races the global AutoRefresh router.refresh() transition. + function syncFiltersToUrl(next: Set<VideoFilter>) { + const url = new URL(window.location.href); + const serialized = serializeFilters(next); + if (serialized) url.searchParams.set("filter", serialized); + else url.searchParams.delete("filter"); + window.history.replaceState(null, "", url.toString()); + } + + // Toggle one chip in/out of the active set (intersection filtering). + function toggleFilter(value: VideoFilter) { + const next = new Set(filters); + if (next.has(value)) next.delete(value); + else next.add(value); + setFilters(next); + syncFiltersToUrl(next); + } + + // The "All" chip clears every active filter. + function clearFilters() { + const next = new Set<VideoFilter>(); + setFilters(next); + syncFiltersToUrl(next); } function changeQuery(value: string) { @@ -304,14 +328,26 @@ export function VideoListPane({ className="w-full rounded border border-zinc-300 dark:border-zinc-700 bg-white dark:bg-zinc-900 px-2 py-1.5 text-sm" /> <div className="flex flex-wrap gap-1.5"> + <button + type="button" + aria-pressed={filters.size === 0} + onClick={clearFilters} + className={`text-xs rounded-full border px-2.5 py-1 ${ + filters.size === 0 + ? "border-blue-300 dark:border-blue-700 bg-blue-50 dark:bg-blue-950 text-blue-900 dark:text-blue-100" + : "border-zinc-200 dark:border-zinc-800 bg-white dark:bg-zinc-900 hover:bg-zinc-100 dark:hover:bg-zinc-800" + }`} + > + All + </button> {FILTER_OPTIONS.map((opt) => { - const active = filter === opt.value; + const active = filters.has(opt.value); return ( <button key={opt.value} type="button" aria-pressed={active} - onClick={() => changeFilter(opt.value)} + onClick={() => toggleFilter(opt.value)} className={`text-xs rounded-full border px-2.5 py-1 ${ active ? "border-blue-300 dark:border-blue-700 bg-blue-50 dark:bg-blue-950 text-blue-900 dark:text-blue-100" diff --git a/editor/app/channels/[slug]/lib/videoRows.ts b/editor/app/channels/[slug]/lib/videoRows.ts @@ -33,60 +33,96 @@ export type VideoRow = { status: VideoRowStatus; }; +// "all" is not a real filter — it's the empty selection. The rest are the +// toggleable chips; multiple can be active at once and combine as an +// intersection (a row must match every selected filter). The two flag-based +// cases (transcribed, partial) are intentionally NOT keyed off the +// mutually-exclusive `status` enum so they can be meaningfully combined — e.g. +// "transcribed AND partial" surfaces an already-transcribed video that still +// has a leftover .part download. export type VideoFilter = - | "all" | "needs_action" | "failed" | "no_audio" | "downloaded_no_transcript" | "transcribed" + | "partial" | "untranscribable" | "running"; -export function filterRows(rows: VideoRow[], filter: VideoFilter): VideoRow[] { +const VIDEO_FILTERS: readonly VideoFilter[] = [ + "needs_action", + "failed", + "no_audio", + "downloaded_no_transcript", + "transcribed", + "partial", + "untranscribable", + "running", +]; + +function matchesFilter(r: VideoRow, filter: VideoFilter): boolean { switch (filter) { - case "all": - return rows; case "needs_action": - return rows.filter( - (r) => - (r.status !== "transcribed" && - r.status !== "untranscribable" && - !r.excluded) || - r.failedTranscription || - r.failedTranscoding, + return ( + (r.status !== "transcribed" && + r.status !== "untranscribable" && + !r.excluded) || + r.failedTranscription || + r.failedTranscoding ); case "failed": - return rows.filter((r) => r.failedTranscription || r.failedTranscoding); + return r.failedTranscription || r.failedTranscoding; case "no_audio": - return rows.filter( - (r) => - r.status === "no_audio" || - r.status === "not_downloaded" || - r.status === "corrupt_source", + return ( + r.status === "no_audio" || + r.status === "not_downloaded" || + r.status === "corrupt_source" ); case "downloaded_no_transcript": - return rows.filter((r) => r.status === "downloaded_no_transcript"); + return r.status === "downloaded_no_transcript"; case "transcribed": - return rows.filter((r) => r.status === "transcribed"); + return r.transcribed; + case "partial": + return r.partial; case "untranscribable": - return rows.filter((r) => r.untranscribable); + return r.untranscribable; case "running": - return rows.filter((r) => r.running); + return r.running; } } -export function parseFilter(raw: string | undefined): VideoFilter { - switch (raw) { - case "needs_action": - case "failed": - case "no_audio": - case "downloaded_no_transcript": - case "transcribed": - case "untranscribable": - case "running": - return raw; - default: - return "all"; +// Intersection: a row is visible only if it matches EVERY selected filter. An +// empty set means no constraint (the "All" chip) — every row is returned. +export function filterRows( + rows: VideoRow[], + filters: Set<VideoFilter>, +): VideoRow[] { + if (filters.size === 0) return rows; + return rows.filter((r) => + [...filters].every((filter) => matchesFilter(r, filter)), + ); +} + +// Parse the comma-separated `?filter=` value into a set of valid filters, +// ignoring unknown/legacy tokens (including "all"). A single legacy +// `?filter=transcribed` parses to a one-element set, so old links still work. +export function parseFilters(raw: string | undefined): Set<VideoFilter> { + const out = new Set<VideoFilter>(); + if (!raw) return out; + for (const token of raw.split(",")) { + const t = token.trim(); + if ((VIDEO_FILTERS as readonly string[]).includes(t)) { + out.add(t as VideoFilter); + } } + return out; +} + +// Serialize a filter set back into the `?filter=` value, or null when empty +// (so the param is dropped). Order follows VIDEO_FILTERS for stable URLs. +export function serializeFilters(filters: Set<VideoFilter>): string | null { + if (filters.size === 0) return null; + const ordered = VIDEO_FILTERS.filter((f) => filters.has(f)); + return ordered.join(","); } diff --git a/editor/app/channels/[slug]/page.tsx b/editor/app/channels/[slug]/page.tsx @@ -44,7 +44,7 @@ import { TranscribeStage } from "./components/stages/TranscribeStage"; import { VideoListPane } from "./components/VideoListPane"; import { VideoListPaneSection } from "./components/VideoListPaneSection"; import { VideoPanel, type VideoFile } from "./videos/[id]/components/VideoPanel"; -import { parseFilter, filterRows } from "./lib/videoRows"; +import { parseFilters, filterRows, serializeFilters } from "./lib/videoRows"; import { computeVideoRows, readDataDirVideoIds, @@ -122,7 +122,7 @@ export default async function ChannelDetailPage({ const selectedVideoId = typeof rawVideo === "string" && rawVideo.length > 0 ? rawVideo : null; const rawFilter = typeof sp.filter === "string" ? sp.filter : undefined; - const filter = parseFilter(rawFilter); + const filters = parseFilters(rawFilter); const queryRaw = typeof sp.q === "string" ? sp.q : ""; const paths = getPaths(); const config = await readChannelConfig(paths, slug); @@ -320,7 +320,7 @@ export default async function ChannelDetailPage({ runningJobs, excludedIds: excludedDownloadIds, }); - const filteredRows = filterRows(rows, filter); + const filteredRows = filterRows(rows, filters); const q = queryRaw.trim().toLowerCase(); const orderedRows = q ? filteredRows.filter((r) => r.id.toLowerCase().includes(q)) @@ -331,7 +331,8 @@ export default async function ChannelDetailPage({ const buildVideoHref = (videoId: string): string => { const params = new URLSearchParams(); - if (filter !== "all") params.set("filter", filter); + const filterParam = serializeFilters(filters); + if (filterParam) params.set("filter", filterParam); if (queryRaw) params.set("q", queryRaw); params.set("video", videoId); return `/channels/${slug}?${params.toString()}`; @@ -405,7 +406,7 @@ export default async function ChannelDetailPage({ slug={slug} rows={rows} selectedVideoId={selectedVideoId} - initialFilter={filter} + initialFilters={[...filters]} initialQuery={queryRaw} defaultTranscribeQueue={TRANSCRIPTION_QUEUE} defaultDownloadQueue={platformDefaultQueueKey} diff --git a/editor/e2e/video-filter-combine.spec.ts b/editor/e2e/video-filter-combine.spec.ts @@ -0,0 +1,81 @@ +// The channel video-list filter chips are multi-select and combine as an +// intersection: selecting Transcribed + Partial shows only videos that are +// BOTH transcribed and still carry a leftover .part download. This is how you +// find an already-transcribed video whose audio download was left partial. + +import { rename, writeFile } from "node:fs/promises"; +import { test, expect } from "@playwright/test"; +import { resetData, resolvePath } from "./helpers"; + +const CHANNEL = "test-transcribe"; +const DATA = `test-transcripts/channels/${CHANNEL}/data`; + +// A non-empty whisper transcript so the video reads as transcribed (and isn't +// flagged untranscribable for being empty). +const WHISPER = JSON.stringify({ transcription: [{ text: "hello" }] }); + +async function seedThreeStates() { + await resetData("one-transcribe-channel-with-audio"); + // vidA: transcribed AND partial — has a whisper transcript and a leftover + // .part, with no finalized audio. + await writeFile(resolvePath(`${DATA}/vidA/transcript.json`), WHISPER); + await rename( + resolvePath(`${DATA}/vidA/audio.m4a`), + resolvePath(`${DATA}/vidA/audio.m4a.part`), + ); + // vidB: transcribed only — keeps its finalized audio. + await writeFile(resolvePath(`${DATA}/vidB/transcript.json`), WHISPER); + // vidC: partial only — a .part and no transcript. + await rename( + resolvePath(`${DATA}/vidC/audio.m4a`), + resolvePath(`${DATA}/vidC/audio.m4a.part`), + ); +} + +test("Transcribed + Partial filters intersect to the both-states video", async ({ + page, +}) => { + await seedThreeStates(); + await page.goto(`/channels/${CHANNEL}`); + + const list = page.getByLabel("videos", { exact: true }); + // All three visible by default. + await expect(list.getByLabel(/^open vid[ABC]$/)).toHaveCount(3); + + // Transcribed alone: vidA + vidB (vidA is transcribed even though its status + // is partial_download). + await page.getByRole("button", { name: "Transcribed", exact: true }).click(); + await expect(list.getByLabel("open vidA")).toBeVisible(); + await expect(list.getByLabel("open vidB")).toBeVisible(); + await expect(list.getByLabel("open vidC")).toBeHidden(); + + // Add Partial: intersection is vidA only. + await page.getByRole("button", { name: "Partial", exact: true }).click(); + await expect(list.getByLabel("open vidA")).toBeVisible(); + await expect(list.getByLabel("open vidB")).toBeHidden(); + await expect(list.getByLabel("open vidC")).toBeHidden(); + + // URL carries both filters, and a reload preserves the selection. + await expect + .poll(() => + (new URL(page.url()).searchParams.get("filter") ?? "").split(",").sort(), + ) + .toEqual(["partial", "transcribed"]); + await page.reload(); + const listAfter = page.getByLabel("videos", { exact: true }); + await expect(listAfter.getByLabel("open vidA")).toBeVisible(); + await expect(listAfter.getByLabel(/^open vid[ABC]$/)).toHaveCount(1); + await expect( + page.getByRole("button", { name: "Transcribed", exact: true }), + ).toHaveAttribute("aria-pressed", "true"); + await expect(page.getByRole("button", { name: "Partial", exact: true })).toHaveAttribute( + "aria-pressed", + "true", + ); + + // All clears back to the full list. + await page.getByRole("button", { name: "All", exact: true }).click(); + await expect(page.getByLabel("videos", { exact: true }).getByLabel(/^open vid[ABC]$/)).toHaveCount( + 3, + ); +});